Debug: Handle Leaks

The intent of these articles is not to regurgitate the WinDbg / NTSD help files but rather introduce real problems, what they are and how to solve them using the debuggers.

What is the handle?

To an application, a “handle” is some instance of a device, file, or some other SYSTem object or resource. The Application WILL create an instance of the Resource by calling a function such as CreateFile or RegOpenKey , and USE the handle is absent. calls to these functions to perform operations on the resource.

In general, you never really care what this handle value is as it really is of no value to your application aside from passing it into functions that perform operations on it. So, what exactly is the handle and what does it represent?

In order to get a clearer picture, let’s go over a simple CreateFile call.

If YOu debug the the CreateFile , You Will to notice eventually it GETS to NtCreateFile .

NtCreateFile results in a kernel call. How the call is Actually performed varies on the System, it could be a sysenter Instruction for this search or a Software generated interrupt int 2eh . Either case, a call is Made into the kernel and the system Dispatches, the call to the correct driver. When this occurs, an object is created in the kernel which represents the requested resource, in this case, a file. If YOu look no Parameters to NtCreateFile , posted, you will notice that the first Parameter to NtCreateFile is where the handle value will be returned. Let’s take a look at what was the first parameter.

As we can see, the first parameter was an address, most likely to some local variable in the function. Upon return, we can now check the value it points to. According to the documentation, it’s a file handle.

The returned value is 7c4h, which is NOT a pointer to any memory in your application. It is also not even a pointer in kernel memory. To further investigate, let’s find what information we can gather from the handle. There is a debugger command that DisplayS handle information This Command known called !handle Let’s use this command on the Value and See what WE Get.

So, we are told that it is indeed a file type object, we are told the attributes and even granted access to the handle. What we aren’t told in this information is the file that this handle points to. To me, that would seem to be the most valuable information that you could gather. There is a reason why we do not see this information, but before I tell you that, we must continue to figure out how the system even found what it did in the first place.

What does the value of a handle represent?

To show you this, I need to hook up the kernel debugger. I have started to debug another process on a Windows 2000 professional machine since I do not have it set up for my Windows XP box. I started Notepad and attempted to get handles in notepad A exe.

Tip: In Windows XP/2003, more information is available to The SYSTem as well as displayed in the debugger. For example, !handle on a thread will Display the thread entry function Windows 2000 Will not do this. This can be very useful There is also the command dt which was Shown earlier in these Tutorials Windows 2000 may have problems displaying STRUCTURES that are IF You want to find the MATCHING thread to a handle, especially if the thread handle was Leaked and the thread is no longer around. always displayed on Windows 2003/XP the (referring system STRUCTURES like NT’s _EPROCESS ). This is either due to the way in DBGS and PDBs are Handled when debugging Windows 2000, or the information is simply missing. These and many other Benefits make debugging on Windows XP/2003 more desirable than Windows 2000.

So, I selected “a file in Notepad and set break POINts is on CreateFile . After I selected a file, the Breakpoint hit I Simply set another Breakpoint on the return ADDress. Now, I dump EAX information to get the handle of the file I Selected.

This doesn’t really tell me much aside from really, it’s a file. Now, I’m going to let you in on the first secret. These handles are process specific. This value, “120h”, is only known from within this process space. If this handle value was sent to another process, it would either not exist or most likely be a handle to a different object. These handles are not like window handles, their scope is within a process.

Handles are also representations of kernel mode objects. This means that every process has a handle table located in kernel mode and each entry points to a kernel memory location. This is secret number two. This is why I hooked up the kernel debugger. Now, let’s break into the kernel and we will attempt to find this handle.

The first thing I do in the kernel debugger is! Process 0 0 To list all PROCESSES Once all processes are listed, I Will then use the !handle command The syntax is A little different though I Will need to specify the process in order to list the correct handle.

The bold NUmber of The first is THE PROCESS object. It’s basically a structure in Memory that the contains information and locations of the data specific to THAT PROCESS Since handles are PROCESS Specific WE Need to tell the !handlecommand what PROCESS to look in.. the “ff “I use simply sets all the bit fields to give me all the information it possibly can. The help will tell you what each bit represents if you want to be picky about the displayed information. I always just use” ff “so I don ‘ t need to bother with setting exact flags and I know I should get all available information.

The second bold address is the handle table for that process. This memory location specifies all entries to the handles used by that process. Notice that it currently has 74 handles open. The last bold is the object’s memory location itself. This is where all the data is being pulled from. Notice that the “Name” property gives us the actual location and file name. How come we couldn’t see this in the user mode debugger?

Displaying Handle Information In User Mode

Obviously, this table is Located in The kernel so it can not be seen directly from USer mode NT does provide APIs that can be Used to query these and handles incorporations from the kernel if you have read my, the QuickView: System Explorer utility I wrote actually does this and WILL display the handle information for the PROCESSES that it can Access The API that does this is called NtQueryObject .

The downfall of this API is that certain objects may hang when you attempt to query them. This is because those objects, for example, are opened with SYNC access and some of them really mean business. There are pipes opened on the system that if you attempt to query them will hang indefinitely. In order to prevent this, the debugger as well as the application I wrote attempt to prevent this by not querying objects that have the potential to cause deadlock. In my application though, I not only check for the SYNC flag to be set, I found some common access masks That would not hang and allow those to be queried.

There is Another the Utility called on the Handle.exe that Can be Downloaded from that to the DOES display all information. So, how does IT do that without deadlocking? They have their own kernel Driver, and since all MEMory in system space is accessible by any driver also Running in System SpacE, it’s Quite simple. As shown, the kernel object’s Memory location CAN be found and simply read directly without HAVING to acquire a SYSTem lock which deadlocks The the NtQueryObject call. I have been thinking to add a driver to future versions of QuickView to get around the deadlocking issue and display more detailed information.

Multiple Handles

You have seen the above example on my Windows 2000 machine. I have opened / TripItinerary.txt file. So, what happens if I open it again with another instance of Notepad? Let’s try this and see what happens.

Inside the user mode debugger:

In the kernel debugger, I found the process and list the handle:

I can also list the handle in the original Notepad process:

In this case, not only did they get their own handle, but they got their own memory location in the kernel. This isn’t always the case with all objects. Sometimes the kernel object can be shared across processes. If the same process opens the file, generally they get two handles but one kernel object as well. You can experiment by using the utility I wrote or SysInternals handle the application to display handle information and sort them by kernel object or handle #. This can help to get you acquainted with handles.

Why did the user mode debugger show a HandleCount of 2 and a PointerCount of 3?

If you noticed, there was a “HandleCount” 2 and “PointerCount” 3 when we used the user mode debugger to show handle information. This is because it did not adjust itself when displaying the information. In order for the debugger to get the handle information, it must duplicate the handle a using DuplicateHandle . The duplication Increases The the HandleCount by 1 and the PointerCount by 2. If the debugger Would decrement by 1 and by 2, it could display the correct information but it chooses Not to. Let’s check this out and see.

What I have first done is looked at the handle ObjectHeader : fcd32430 “in the kernel.

As we can see, the handle count is 1 and the pointer count is 1. I then set “ba r1” breakpoints on the handle count address and the pointer count address. “BA” means Break if the address is accessed. “R” means if it’s accessed through reading or write. The 1 on “r1” simply means 1 byte.

Once I have done this and hit “g” to let the kernel continue, I will now simply type “! Handle 120 ff” in the user mode debugger. This will cause the user mode debugger to access this object to find the information. Let’s see when these pointers are incremented.

As WE CAN see, Our first call calls ObReferenceObjectByHandle This adds a reference to The pointer Let’s see what our handle count says now.

As we can see, our pointer count has incremented to 2. Let’s see what happens next.

As WE CAN see, this now calls ObpIncrementHandleCount and the our handle count is now also. Notice that Both calls are the within NtDuplicateObject which was called DuplicateHandle from the user-mode API. So, the duplicate object will increment the REFERENCE pointer count to this kernel MEMory and INCREMENT the number of handles that are pointing to this kernel object. This is an instance where two processes are referencing the same kernel object. So, where does the third pointer reference come in?

There it is, as I mentioned before, the debugger WILL need to call NtQueryObject in order to Ask the SYSTem to tell it information about an object. the The NtQueryObject API before attempting to read the memory will reference it by again calling ObReferenceObjectByHandle This CAUSES pointer count to Increase to 3 However, NtQueryObject does not need to create Another handle instance or duplicate any handles. Once it’s able to get a pointer reference to the object, it simply queries the Information and releases the REFERENCE count.

Another process could possibly release its handle or pointer count during this operation and could even display only the debugger’s references to the object. The returned information to the debugger will be 1 extra handle reference (the one it created), and 2 extra pointer references ( the one it created when Duplicating the handle and the one created by NtQueryObject The debugger, AS mentioned before, could always subtract this to display only relevant information, since after the NtQueryObect call one pointer reference goes away and to read the object information). it will also close its duplicated handle. That will let the object dereference to its original state (or new state if another process also now references it).

The BTW, the The DuplicateHandle API takes a PROCESS handle from which to duplicate the handle So, you must be able to call OpenProcess with duplicate privileges in order to duplicate handles in other PROCESSES, just in case you were Wondering.

Debugging Handle Leaks

So, now that we have been acquainted with handles, how do we find leaks? Aside from the usual applications such as “Bounds Checker”, we can find them ourselves. The first thing to do is to check the application at a good time to get a baseline for handles. Task Manager has an option to display a “handles” column. This is a good place to start. Next, you should then perform operations with the application and of course, see if the handles grow or shrink. When the handles grow, you should not immediately think there’s a handle leak. You must know to determine expected behavior from unexpected.

For example, if you had a server application and you were checking network connections. You notice that as you connect a lot of clients to the application, the number of network connections on the server grew. This would be expected, obviously, so it would need to be determined if it’s more than expected. Also, when all the clients disconnected, it would then be expected that all the network connections were cleaned up. So, if you know your handle count grows on purpose but selecting another option should then decrease the handle count, that should be tested.

Step One: Determine A Leak

If you’ve been monitoring the application, determine if the handle count is expected or not. See if it’s growing steadily over time, fast or slow. Once you have determined this is truly a leak or are unsure at this point, it may be time to move onto the next step which would be to identify the handles being leaked. If possible, it would also be great to attempt to determine the pattern for the leak. An example would be every time a certain menu item is selected or a file is opened. This would help to narrow down reproduction steps and limit the area of ??the source that appears to be the location causing the problem.

Step Two: Determine The Type And Object Information

The best ways to do this would be to use a tool such as “Handle” from SysInternals and/or the debugger. I created a simple program that leaked handles very fast. I ran the application and looked in Task Manager. I found that over 65,000 handles were leaked. So, the first thing I want to do is determine what type of handle leaked. This way, I can limit down what APIs I want to look at in my application to find the leak.

As we can see in this exaggerated example, the handle that appears to be leaking is a “Key”, which is a registry handle. Some/place in this application is opening the registry. The first thing now to do is attempt to identify the location and if there are more opens on a particular key than others. If we can get the key information, we would then be able to sort out which key was opened the most if there is one, and then we can limit our search in the application.

With the “Key”, we get lucky The debugger shows us the key being opened and HKCU / Software appears to be being opened the most We have identified it is a key object for, which MeanS POINts of the creation would be RegOpenKey and RegOpenKeyEx . We have also now identified the actual key being opened, HKCU / Software. Step 3:

Step Three: Browse Source and Debug Application

We now have the type, the APIs, and Luckily enough even being opened on the exact Resource So now, WE CAN do things like look through the source for locations opening this key in or even SETTING break POINts is in the application on RegOpenKey / RegOpenKeyEx to get stack traces. We could then keep track of the allocated handles and determine which ones were getting closed. Bounds checker could also be used as a tool in aiding this process.

Another technique which can be done but may be overkill in most cases would be to use the memory leak trick stated in the “Heap” tutorial. Wrapper functions can be written around the APIs creating the handles and you could add the handle to a linked list . Upon free, you could search the linked list and remove the handle. This operation would allow the handle itself to still be returned so you do not have to have a wrapper for all functions. The only downfall is that it would be slower since you would need to traverse the list on a close.

Also, remember that the above are examples, almost pseudo code. Critical sections would be used when needed and implementations can vary. The purpose of a global list is so a debugger extension (tutorial part 4) could be written to traverse the linked list and debug symbols could be used to find the location of this global to walk the list. This may be overkill for most problems and the implementation would probably only benefit if it was pre-built into all the builds without necessary implementation all the time (such as #define MyOpenKey RegOpenKey for Retail builds).

Smaller Leaks

Not all handle leaks will appear like the above. For example, you may notice an operation is causing one or two handle leaks every time it’s called. You could then break into the application, get a handle count for all values, then perform the leak and get the next snapshot. Once that occurs, you could then cause the problem again, but this time set breakpoints on locations that would create the objects that you have seen increasing.

Other Tips

Here are some other tips when dealing with handles.

Invalid thread wait for exit looping

I have noticed to Many Applications use a Sleep() / GetExitCodeThread() loop combination to wait for a thread to exit. For example:

The MSDN states The problem with this is that the thread could return STILL_ACTIVE (value 259) as its return value. While that may BE true, provided that’s not the reason then I would worry about this loop. A problem that I named FOUND of was that some Applications WERE Using MFC’s CThread LIBRARIES and attempting to do this loop of The problem is that AfxBeginThread() SENDS the thread handle to The created thread uses AfxEndThread() to exit, Which then closes the handle. If you have not Duplicated this handle, that handle is now invalid.

In general, this may not be a problem as the loop would still exit when it got the failure to use the handle. The problem occurs when another object was now created using the old thread handle! Remember, these handles are re-used by the system once they are freed. That means that another object could be assigned to the handle in the time it takes to call the function again. This object could be anything and the call may fail with the invalid object. It could also be another, the newly created thread which would now not let this loop exit!

So the first rules are, even outside of the Afx* functions, then if you’re going to close the handle elsewhere, be sure to duplicate it. Secondly, You do not need to loop since the thread handle becomes signaled when the thread Exits.

So, the above could have been better solved by the series of code modeled above.

. DMP files missing handle information?

If you have a DMP file and you USE !handle sometimes YOu May Receive an error. This is because if you have a. DMP file, You Can not CALL NtQueryObject to Get the handle information anymore. In that case, the debugger needed to have queried all objects and saved the information into the. DMP while creating it. Some. dump flags do not do this. What I have found is that almost all NTSD / CDB / WinDbg versions never save handle information when using. dump / f x.dmp (full dump). They have a separate option, / mh though that does .. dump / mh x2.dmp but it’s not a full dump, it’s a “minidump”. What I have done in the past is to create two dumps using both options.

This is not necessary if you have a newer debugger downloaded from Microsoft’s site though. There is a new option, / math at creates a full dump with handle information .. dump/ma x3.dmp and that is all you need. This is the flag I would recommend to create all user mode dumps since you never know if you may need to look at handle information.

Conclusion

Handles are yet another part of Windows which we must learn to work with and “handle” them correctly. When checking applications you write for memory leaks and other problems, always be sure to check the handle count! Hopefully, this article helped you learn what handles are and how to investigate them.