Tools Tips and Tricks: Round-up


Last month I ran a blog series on the different Tools that CSS uses for certain troubleshooting scenarios SQL Server related issues. There were 12 posts that covered the use of different tools like Debug Diagnostic, XPerf, tools from Sysinternals, SQLDIAG, SQL Nexus and some debugging tips.

Now it is time to do a SELECT [PostURL], [Summary] FROM [TroubleshootingSQL].[tblBlogSeries] WHERE [Series] = ‘Tools Tips and Tricks’…..

Post#1: Tools Tips and Tricks #1: Process MonitorSome best practices to be followed when capturing Process Monitor traces to ensure that the data collection doesn’t causes additional performance issues.

Post#2: Tools Tips and Tricks #2: SQL Express RANU instances Explains how to connect to SQL Express Run As User Instances (RANU) using existing tools.

Post#3: Tools Tips and Tricks #3: Custom Rowsets using SQL NexusAnother quick tip on using SQL Nexus to import outputs of .sql scripts used to capture diagnostic data using Rowset Importer.

Post#4: Tools Tips and Tricks #4: RML Utilities – Some helpful tips on the use of RML Utilities which is used by SQL Nexus under the hood for importing SQL Profiler traces into a SQL Server database.

Post#5: Tools Tips and Tricks #5: SQLDIAG and RANU – Explains how to capture diagnostic data for SQL Express Run As User Instances (RANU) using SQLDIAG.

Post#6: Tools Tips and Tricks #6: Custom Reports in SQL NexusA quick tip on how to create custom reports for SQL Nexus a tool widely used within CSS for analyzing performance diagnostic data collected from a SQL Server instance.

Post#7: Tools Tips and Tricks #7: PsExec as parent and ProcMon as childExplains how to use PsExec (a tool from Sysinternals) to launch a process remotely and capture data. In this example, I have used Process Monitor as the remote process.

Post#8: Tools Tips and Tricks #8 – Debug Diagnostic and Crash Rules – A walkthrough on using Debug Diagnostic tool for capturing crash dumps and analyzing them using the Crash Analysis rule.

Post#9: Tools Tips and Tricks #9: PSSDIAG Configuration Manager – Explains how to configure PSSDIAG collection using Configuration Manager GUI with a few tips and tricks on tweaking the XML configuration file.

Post#10: Tools Tips and Tricks #10: Caching PDB files locally – Explains how to cache symbol files locally using CDB.exe.

Post#11: Tools Tips and Tricks #11: Debug Diag and Memory leaks – A walkthrough on configuring Debug Diag for tracking memory leaks for a program which can be extended to tracking non-BPool allocations for SQL Server.

Post#12: Tools Tips and Tricks #12: XPerf, Memory usage and much more – A walkthrough on how to use XPerf Heap allocation tracking for identifying memory consumers for a program. Can be extended to SQL Server Out-of-Memory (OOM) issues for non-BPool memory crunch.

Webcast Material for Virtual Tech Days


In May, I had done a webcast on “Understanding Performance Bottlenecks using Performance Dashboard”. The presentation material is now available on the SQLServerFAQ MSDN blog and the webcast videos are available for download on MSDN. Refer the links below for the presentation deck and webcast video download link.

Managing and Optimizing Resources for SQL Server [PPT | Webcast Download] – Balmukund Lakhani [Blog | @Blakhani]
Optimizing and Tuning Full Text Search for SQL Server [PPT | Webcast Download] – Sudarshan Narasimhan [Blogs @ SQLServerFAQ]
Understanding Performance Bottlenecks using Performance Dashboard [PPT | Webcast Download | QnA]
Cool Tools to have for SQL Server DBA [Webcast Download] – Pradeep Adiga [Blog | @PradeepAdiga]
Learn Underappreciated Features of SQL Server to Improve Productivity [Webcast Download] – Nakul Vachhrajani [Blog]

 

del.icio.us

Tags: ,,,

Tools Tips and Tricks #12: XPerf, Memory usage and much more


This is the last post for the series Tools Tips and Tricks as May draws to a close. Today I shall talk about another tool that we use called XPerf for performance troubleshooting. Though this is not a common tool which is used on a regular basis by the SQL CSS team. But when we do decide to use this for very specific scenarios, the usefulness of this tool cannot be put in words. I had talked about using Debug Diag for monitoring memory usage and tracking down allocations right upto the function call. There is another way to track heap allocations which is what I shall be talking about today. I shall use the same MemAllocApp that I had used last week. I start off the Xperf monitoring using the following commands:

xperf -on PROC_THREAD+LOADER -BufferSize 1024 -MinBuffers 16 -MaxBuffers 16
xperf -start HeapSession -heap -Pids 9532 -BufferSize 1024 -MinBuffers 128 -MaxBuffers 128 -stackwalk HeapAlloc+HeapRealloc+HeapCreate

Now once I have collected the XPerf data, you can use the following command to stop the data collection:

xperf -stop HeapSession -stop –d F:\MemAlloc.etl

Once that is done, you should have an ETL file in the specified location by the –d parameter. Since, I am interested in the functions in the functions which were allocating the maximum amount of memory, I will use the following command to generate a summary report for the heap allocations traced by XPerf using the command below:

xperf -i "F:\MemAlloc.etl" -o "F:\MemAlloc.txt" -symbols -a heap -stacks -top 5

/* Output of MemAlloc.txt file */

Results for process MemAllocApp.exe (9532):

———————————————————————

GLOBAL ALLOCATIONS:
Alloc       :         100,     512000.0 KB
Realloc     :           0
Outstanding :         100,     512000.0 KB

———————————————————————

TOP 1:
Alloc       :         100,     512000.0 KB
Realloc     :           0
Outstanding :         100,     512000.0 KB

———————————————————————

MemAllocApp.exe!fn_allocatememory
MemAllocApp.exe!wmain
MemAllocApp.exe!__tmainCRTStartup
MemAllocApp.exe!wmainCRTStartup
kernel32.dll!BaseThreadInitThunk
ntdll.dll!RtlUserThreadStart

Alloc       :         100,     512000.0 KB
Realloc     :           0
Outstanding :         100,     512000.0 KB

 

As you can see from the above output, the function fn_allocatememory was responsible for 100 allocations worth 512KB each. With just the use of a single command I was able to figure out the reason behind my outstanding allocations for my EXE. Troubleshooting SQL Server outstanding memory allocations for heaps may not be as easy as this but it definitely saves time in having to look and dig out the allocations from the a memory dump.

This method is quite useful when you have a very large ETL file which you need to analyze. You can even configure a Circular Buffer for capturing data appending the following command for your HeapSession tracing commands:

-BufferSize 1024 -MaxBuffers 1024 -MaxFile 1024 -FileMode Circular

Note: Make sure that you set your _NT_SYMBOL_PATH environment variable correctly if you want the function calls to be resolved correctly.

Hope you enjoyed this series of Tools Tips and Tricks as much as I had fun in posting the various methods that I use to collect diagnostic data while troubleshooting SQL performance related issues.

References:
Using Actions to process Heap Data
Enabling Data Capture using XPerf
XPerf Options