MTL Consumers

Posted on December 30, 2009 by Amit Banerjee

A previous post of mine talked about SQL Server memory architecture in a nutshell. Let’s find out what are the common consumers of the much talked about MemToLeave region and what you should be looking for in-case you run into MemToLeave issues.

Major Consumers of MTL

Use of Linked Servers – You can find out the linked servers that being used in your environment using the sysservers system catalog.
Use of XML documents – You would have to find out if any queries or procedures perform any kind of XML data manipulation or use sp_xml_preparedocument.
Extended Stored Procedures or sp_OAcreate calls
1. Extended stored procedure usage can be identified by inspecting the SQL Server Errorlog and searching for the word using. The first XSP calls is logged in the Errorlog in the following manner: Using ‘<dll name>’ version ‘<version>’ to execute extended stored procedure ‘<XSP name>’.
2. If you are using sp_OAcreate, then this information would be logged in the SQL Errorlog for the first invocation of sp_OAcreate using the same pattern mentioned above. The only difference would be that the DLL name would be odsole70.dll.
Query Plans which are larger than 8 KB in size
1. If you are using SQL Server 2000, query syscacheobjects and search for entries which have values greater than 1 for the pagesused column.
2. If you are using SQL Server 2005, search for plans greater than 8KB using the DMV sys.dm_exec_cached_plans. Inspect the size_in_bytes column value and search for entries which have values greater than 8192 bytes.
Use of SQLCLR (Applicable to SQL Server 2005 and above) – Check the memory usage for SQLCLR clerk using DBCC MEMORYSTATUS.
Backups using larger MAXTRANSFERSIZE parameter – In such a case find out BACKUP DATABASE/LOG commands and inspect the MAXTRANSFERSIZE parameter value. The lowest size that can be provided is 65536 bytes.
Using Network Packet Size higher than 8192 bytes. You can find all connections on SQL Server 2005 that use more than 8KB of connection using the following DMV: select * from sys.dm_exec_connections:

select * from sys.dm_exec_connections
where net_packet_size > 8192

How to find the largest free block in the MTL

Posted on December 30, 2009 by Amit Banerjee

Often we need to find the largest free block when troubleshooting “MemToLeave” issues with SQL Server Memory. Since, this is related to virtual address space and we didn’t expose any method to query the process address space on SQL Server 2000, you would have to rely on a tool called VMSTAT to find out the largest free block. This tool is used by CSS to troubleshoot MTL issues. However in SQL Server 2005, we added a new DMV which would inform you the status of the virtual address space for SQL. The query below can be used to find the largest contiguous block on the system.

If you are getting the following error:

WARNING: Failed to reserve contiguous memory of Size= 2162688 (*This value is always in bytes)

If the query below is run during the problem period, then you will find one of the following conditions to be true:

1. The value returned by the query is larger than the value in the error (once you have done the necessary conversions) – This means that when the error occurred, the MemToLeave region of SQL Server was being used by some component due to which a current allocation request failed. This is due to a virtual address space crunch.

2. The value returned by the query is lower than the requested size – This means that you either have a virtual address space fragmentation issue or some component has used up all the available MemToLeave region. (A lot of my colleagues believe that MTL is a misnomer but I will continue to use these throughout the post as it makes referencing the memory region simpler.) In such a case, it would always be advisable to get a Microsoft CSS Engineer to setup data collection tools in place for the next problem occurrence and analyze the existing logs for any tell-tale signs.

Query (for 64-bit plaform)

SELECT convert(varchar,getdate(),120) as [Timestamp], max(region_size_in_bytes)/1024 [Total max contiguous block size in KB] from sys.dm_os_virtual_address_dump where region_state = 0x0000000000010000 --MEM_FREE

Query (for 32-bit plaform)

SELECT convert(varchar,getdate(),120) as [Timestamp], max (region_size_in_bytes)/1024 [Total max contiguous block size in KB] from sys.dm_os_virtual_address_dump where region_state = 0x00010000 -- MEM_FREE

For more information on this DMV, read about it on sys.dm_os_virtual_address_dump

NOTE: The above query is very very performance intensive and shouldn’t be run very frequently on a production system. On a 64-bit system, frequent use of this query can actually degrade performance severely due to the large virtual address space of 64-bit systems.

SQL Server Memory Basics

Posted on December 30, 2009 by Amit Banerjee

I will give a brief account of how SQL Server memory is organized (in a nutshell). There is much more to it under the covers! 🙂

SQL Server Memory space is organized in the following manner:

Each instance of Microsoft SQL Server 2000 has an address space with two main components, each of which has several subcomponents:

There are two main areas of memory within SQL Server’s address space, the buffer pool (BPool) and a second memory pool sometimes called the “MemToLeave” area. The contents of the SQL Server buffer pool include cached table data, workspace memory used during query execution for in-memory sorts or hashes, most cached stored procedure and query plans, memory for locks and other internal structures, and the majority of other miscellaneous memory needs of the SQL Server. The Bpool handles memory allocations for 8K and below. All allocation requirements above 8K is satisfied from the MTL region.

Executable code

The number and size of the executable files and dynamic link libraries (DLLs) used by an instance of SQL Server varies over time. In addition to the executable files and DLLs used by Open Data Services, the SQL Server engine, and server Net-Libraries, the following components load in their own DLLs, and these DLLs can allocate memory themselves:
Distributed queries can load an OLE DB Provider DLL on the server running the instance of SQL Server.
Extended stored procedures are implemented as DLLs that are loaded into the address space of the instance of SQL Server.
The OLE Automation system stored procedures are used to create instances of OLE Automation objects. Each class of OLE Automation object loads its own code into the address space of the instance of SQL Server.

Memory pool

The memory pool is the main unit of memory for an instance of SQL Server. Almost all data structures that use memory in an instance of SQL Server are allocated in the memory pool. The main types of objects allocated in the memory pool are:

System-level data structures

These are data structures that hold data global to the instance, such as database descriptors and the lock table.

Buffer cache

This is the pool of buffer pages into which data pages are read.

Procedure cache

This is a pool of pages containing the execution plans for all Transact-SQL statements currently executing in the instance.

Log caches

Each log has a cache of buffer pages used to read and write log pages. The log caches are managed separately from the buffer cache to reduce the synchronization between log and data buffers. This results in fast, robust code.

Connection context

Each connection has a set of data structures that record the current state of the connection. These data structures hold items such as parameter values for queries and stored procedures, cursor positioning information, and tables currently being referenced.

The high level memory architecture remains the same in SQL Server 2005 and 2008. The difference is that we introduced a change called Memory Clerks which were responsible for memory allocation requests for requirements for a particular object/cache store. The allocators changed to Single Page Allocator (any allocation required below 8KB) and Multi Page Allocator (any allocation above 8KB). With the help of SPA and MPA, we are now able to track Buffer Pool and non-Buffer Pool memory requests in a more structured manner when components within SQL Server are requesting for memory. However, if your DLL/CLR code hosted within SQL Server memory is calling VirtualAlloc() directly, then tracking those requires extensive data collection and analysis.

Useful Articles

How to track SQL Server 2000 memory usage using DBCC Memorystatus

How to track SQL Server 2005 memory usage using DBCC Memorystatus

How to insert BLOB data into a SQL Server database table using T-SQL

Posted on December 30, 2009 by Amit Banerjee

One of the methods that you can use to insert BLOB data into a SQL Server database table is:

CREATE TABLE BLOB_TABLE (BLOBName varchar(100),BLOBData varbinary(MAX))

GO

INSERT INTO BLOB_TABLE (BLOBName, BLOBData)

SELECT 'First test file', BulkColumn

FROM OPENROWSET(Bulk 'C:\temp\picture1.jpg', SINGLE_BLOB) AS BLOB

GO

OPENROWSET has the functionality of letting you import BLOB data by returning the contents of the Binary Data as a single rowset varbinary(max) output.

SQL Server 2005 setup failing with 29538 error

Posted on December 30, 2009 by Amit Banerjee

There are numerous occasions when we find that the SQL Server 2005 setup has failed due to the following error:

Product : Database Services (MSSQLSERVER)
Product Version (Previous): 3042
Product Version (Final) :
Status : Failure
Log File : C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Hotfix\SQL9_Hotfix_KB955706_sqlrun_sql.msp.log
Error Number : 29538
Error Description : MSP Error: 29538 SQL Server Setup did not have the administrator permissions required to rename a file: C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\mssqlsystemresource1.ldf. To continue, verify that the file exists, and either grant administrator permissions to the account currently running Setup or log in with an administrator account. Then run SQL Server Setup again.

This can happen when we are trying to install SQL Server Service Packs, Hotfixes, Cumulative Updates or GDRs. Now what you need to do is find if any of the following are true:

1. The SQL Server service account doesn’t have sufficient privileges on the DATA folder of SQL Server

2. It is a recommendation that the MSSQLSYSTEMRESOURCE files (MDF and LDF) be in the same location as the master database. This issue is documented under KB947989.

3. Another point that you would want to check is if you have any services to re-start the SQL service if it is stopped. SQL Server Setup performs setup in two phases: the first phase being replacement of binaries and the second phase being running the configuration scripts for updating the metadata. During the first phase, the SQL Server service is stopped. In case, you have a service which restarts the SQL service at that point, then you would run into this issue.

The easiest way to verify this would be to check the status of the SQL service from the Services snap-in right after the failure and it should show up as started. Or if you are command prompt savvy Smile

, then you could use "sc query mssqlserver" (provided it’s a default instance) to check the state. If it’s a named instance, then the service name would change from mssqlserver to mssql$<instance name>

Sample output:

SERVICE_NAME: mssqlserver
        TYPE               : 10 WIN32_OWN_PROCESS
        STATE              : 4 RUNNING
                                (STOPPABLE, PAUSABLE, ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0 (0x0)
        SERVICE_EXIT_CODE : 0 (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

Two of the known service restart rules applied by other non-Microsoft services are:

1. Cisco security agent has a Service Restart rule which causes the SQL server service to restart once it is stopped.

2. Altiris has a similar rule.

The job of these services is to start services that were stopped. So when SQL setup stopped SQL Server service these services started SQL service which ended up holding a handle on this mssqlsystemresource1.ldf.