Windows Debugging API - Part 1

I recently started exploring the Debugging APIs available in Microsoft Windows. Using these APIs, one can write their own software to write a debugger, process tracer/analyzer and what not. My goal is to write a full-fledged debugger using only Win32 APIs by the end of this learning stage. In today's post, I will explain what I have learned so far - some of the Debugging APIs themselves and a sample implementation of a so called process tracer, for lack of a better name, which uses those APIs.

The debugging APIs available in the Windows OS have been remained largely unchanged for many years now. Articles written way back in the 90s and 2000s are relevant even today. Scroll down to the end of this post to find some references that I have used so far. Before going in to the details of the APIs themselves, I will enumerate the tasks that a debugger should be able to perform in order to satisfy the user's requirements.

1. First and foremost, the debugger must enable the user to fully control the execution of the process to be debugged, called the target process from here on. This means that the user must be able to start and stop execution of the target by setting breakpoints, single stepping, break-all threads immediately and so on.
2. The user must also be able to control the data that the target process uses - register contents(even the EIP register to alter control flow), stack and heap contents, global variables and so on.
3. Provide at least a disassembly of the code that the target process is executing whenever the user wishes to examine it.
4. It must provide all details about the target process during its execution - threads, child processes, register and memory contents, loaded modules(DLLs), address in memory of the modules, current stack layout and so on.

The Windows debugging APIs consists of:

• WaitForDebugEvent: The caller is blocked for the amount of time indicated or until a debug event occurs in the target process. When this function returns TRUE, a debugging event has occured and at this point the target process has been suspended completely. The debugger(caller) is free to process the debug event and update its own UI and do other stuff.
• ContinueDebugEvent: Once the debugger(caller) has processed the debug event, it has to call this function in order to let the target process continue execution.
• DebugActiveProcess: This enables the debugger to attach to an already running process in the system.
• DebugActiveProcessStop: DebugActiveProcess() function's counterpart. This is used to detach the debugger from the target process. Once this is done, the target process is no longer under the control of the debugger and the debugger will not receive any debug events.

There are other debugging APIs which I have not used so far, so I will explain them in a future post. They are: DebugSetProcessKillOnExit(), DebugBreakProcess() and CheckRemoteDebuggerPresent(). Important APIs that are not specific to debugging but are very necessary for accomplishing the tasks of a debugger:

• CreateProcess(): This is used to create the target process as a new process with the DEBUG_PROCESS or the DEBUG_ONLY_THIS_PROCESS flag which enables the calling thread in the debugger to receive debug events from this target process.
• SuspendThread() and ResumeThread(): Suspend and resume a thread in the target process.
• TerminateThread() and TerminateProcess(): To terminate a thread or the target process itself.
• ReadProcessMemory() and WriteProcessMemory(): To read and write to the memory contents of the target process. These are essential because there is no direct way to access the target process's memory like using a pointer in the debugger. This is because the target process, obviously, is created in a different virtual address space than the debugger.

Before going into the details of the APIs, I will give a brief of the various events that a debugger will receive when it is attached to a target process. A DEBUG_EVENT structure is sent along with each debug event. This event has the target PID, thread ID where this debug event occurred and an associated structure inside that is filled with information related to the corresponding event. These structures contain very useful information about the target process. The debug events sent are:

• CREATE_PROCESS_DEBUG_EVENT: This event is the very first event sent to the debugger when it creates a new target process or attaches to an existing process. The associated structure is CREATE_PROCESS_DEBUG_INFO. Important fields are:
  • hProcess, hThread: handle to target process and the first thread.
  • hFile: Handle to the memory area where the executable binary(of the target process) is mapped into memory. Use this to read information about the target binary for may be disassembly generation purposes. You must close this handle once you are finished using it.
  • lpBaseOfImage: The starting address of the executable image mapped into memory. For PE binaries, you will see that this points to the DOS header with "MZ" at the start.
• CREATE_THREAD_DEBUG_EVENT: Event sent whenever a thread is created in the target process. The associated structure is CREATE_THREAD_DEBUG_INFO. The hThread member in this gives a handle to the thread.
• LOAD_DLL_DEBUG_EVENT: This is sent when a DLL is loaded into the target process. hFile is the handle to the loaded DLL and can be used to obtain the image name of the DLL and must be closed after use. lpBaseOfDll gives the address at which the DLL is loaded in the target process's address space.
• OUTPUT_DEBUG_STRING_EVENT: This is a special event that is sent when the target process a call to the OutputDebugString() function. This is used especially for the target process to be able to communicate to the debugger. The associated OUTPUT_DEBUG_STRING_INFO structure contains information needed to retrieve the string value passed to the OutputDebugString() function. lpDebugStringData gives the starting address of the string and nDebugStringLength gives the number of characters in the string. fUnicode specifies whether the string is a wide-char string or not. ReadProcessMemory() must be used to actually read the string value from the target process's address space. Keep in mind that if it is a unicode string then you will have to read length*sizeof(WCHAR) bytes in order to read the full string.
• EXIT_PROCESS_DEBUG_EVENT and EXIT_THREAD_DEBUG_EVENT: Counterparts of the respective create events. Keep in mind that you will have to remember the PID of the target process so that when you receive the EXIT_PROCESS_DEBUG_EVENT with this PID, you will know that the target process has exited.
• UNLOAD_DLL_DEBUG_EVENT: Sent when a DLL is unloaded in the target process. The only member in the associated UNLOAD_DLL_DEBUG_INFO structure is lpBaseOfDll which is the same as before. See that there is no information about which DLL is being unloaded. The debugger has to remember this information by mapping the base of DLL to the DLL name when it gets the LOAD_DLL_DEBUG_EVENT and can use this information when it receives the unload DLL event.
• EXCEPTION_DEBUG_EVENT:Sent to the debugger whenever an exception occurs in the target process. The structure EXCEPTION_DEBUG_INFO has information about the exception itself. First chance exception means the exception has not yet been sent to the target process yet. The debugger is the first one to receive and can make use of it before it reaches the target process. This is useful when the debugger sets breakpoints which result in the breakpoint exception. In this case, the debugger handles the breakpoint exception and continues the target process. The target won't even know that an exception occurred in this case. If the debugger cannot handle the exception then it is passed onto the target process. The exception debug information structure also contains the exception code which indicates the kind of exception that occurred. There is a lot more detail to be written regarding the exception debug event and I will reserve that for the second part of this post.

Now, there are two starting points for debugging a target process - first is to create the new process as a child of the debugger using the CreateProcess() API and second, attach to a process that is already running using the DebugActiveProcess() API. Likewise, there are two ways to end a debug session - terminate the target using TerminateProcess() API or simply detach from the target using DebugActiveProcessStop() API and let it continue executing. Going into the specifics now...

Creating a new target process
You must specify the DEBUG_PROCESS or the DEBUG_ONLY_THIS_PROCESS flag while creating the new target process so that the debugger has full access to the target process. When this function returns, you will get the PID of the newly created target process and a handle to its main thread and the target process itself. It is better to close these handles now since we get these again later when the debugger receives the CREATE_PROCESS_DEBUG_EVENT. Once you the create the target process the following set of events happen from the debugger's perspective:

Attaching to an active target process
When DebugActiveProcess() function is called, the target process is first suspended by Windows. Then the following events are sent to the debugger: CREATE_PROCESS_DEBUG_EVENT and CREATE_THREAD_DEBUG_EVENT for the main process and thread, CREATE_THREAD_DEBUG_EVENT for all other threads currently in the target process and LOAD_DLL_DEBUG_EVENT for all loaded DLLs. Windows sends one EXCEPTION_DEBUG_EVENT with exception code 0x80000003(breakpoint) before resuming execution of the target process. So once the debugger continues this exception, the target process is resumed and now any debug event may be sent to the debugger as and when they are generated.

Receiving debug events
Debug events are sent to the debugger, specifically the thread within the debugger that called either CreateProcess() or DebugActiveProcess(), once it is attached to the target process. In order receive these events, the debugger must call the WaitForDebugEvent() API. This blocks the calling thread until a debug event is sent or the specified time runs out. If a debug event is sent within the specified time out, then a DEBUG_EVENT structure is sent along as described earlier. Calling ContinueDebugEvent() resumes execution of the target process.

Closing a debug session
Similar to starting a debugging session, there are two ways to end it.
First is to terminate the target process and the second is to simply detach the debugger from the target process and let it continue execution. Termination of a process is via the TerminateProcess() API. Many people recommend not to use this because it immediately terminates the target process without invoking any cleanup code in the target. However, if the user wants to stop debugging, the target process must be terminated this way because giving it a chance to cleanup first is the total opposite of wanting to terminate the process. Think about it - if you are debugging a malware you would not want to give it a chance to cleanup, would you?

Coming back to TerminateProcess() now. There is one thing you should absolutely remember not to do: do not exit the debug thread as soon as you call TerminateProcess. Doing so keeps the target process in an infinite wait state and it cannot even be killed. See the appendix for an explanation. Once TerminateProcess() has been called, the target process is suspended and Windows sends UNLOAD_DLL_DEBUG_EVENT for all loaded DLLs, EXIT_THREAD_DEBUG_EVENT for all threads and finally a EXIT_PROCESS_DEBUG_EVENT. Once all these events are processed, Windows calls CloseHandle and other clean-up code for any resources held by the target process and then proceeds to remove the process from the system.

Second, detaching from the target process is by calling the DebugActiveProcessStop() API. This simply causes the debugger to stop receiving any further debug events and closes all the debugger's open handles to the target process. From now on, the target process continues executing without an attached debugger. Any exception that occurs is handled by Windows the same way as for any other process.

Using these APIs
I wrote a small GUI program called ProcessTracer that receives debug events and displays them to the user. Source code for this can be found in Github here: ProcessTracer git repo.

Appendix

Terminating the target process
When TerminateProcess() is called, the target process is suspended just like when a debug event occurs. Windows then sends multiple UNLOAD_DLL_DEBUG_EVENTs and EXIT_THREAD_DEBUG_EVENTs followed by a final EXIT_PROCESS_DEBUG_EVENT. All this happens without the target process being resumed at all. So if the debugger does not process these events by calling ContinueDebugEvent() then the target process is in a wait-forever state within a kernel thread. Since this is in a kernel mode wait state it cannot be killed. See this article for an explanation of un-killable processes.