Posts mit dem Label Exception Handling werden angezeigt. Alle Posts anzeigen
Posts mit dem Label Exception Handling werden angezeigt. Alle Posts anzeigen

Mittwoch, 30. März 2011

Compiler internals - What happens with try/catch/throw in MSVC

Hello everyone

After i have seen that a lot of people are thinking that try - catch is a concept which is completely analyzed during compile time and therefore wont have big impact at runtime i thought i might shed some light on how the Microsoft compiler (cl.exe) is acting with try, catch and throw.

First off we will have a look on how a throw statement is interpreted by the compiler. Lets have a look at the following code:
int main()
{
 try
 {
  throw 2;
 }
 catch(...)
 {
 }
}

For now we are only interested in the throw 2. When the compiler hits the throw statement it actually has no clue if the exception its now converting is handled by an exception handler (and it doesnt care). The throw statement will be converted into a call to _CxxThrowException (exported by MSVCR100.dll (or any other version)). That function is a built in function in the compiler. You can call it yourself if you like ;). The first parameter of that function is a pointer to the object thrown. Therefore it gets clear, that the code above definitely expands to the following:
int main()
{
 try
 {
  int throwObj = 2;
  throw throwObj;
 }
 catch(...)
 {
 }
}

The second parameter of _CxxThrowException holds a pointer to a _ThrowInfo object. _ThrowInfo is also a built in type of the compiler. Its a struct holding various information about the type of exception that was thrown. It looks like that:
typedef const struct _s__ThrowInfo
{
 unsigned int attributes;
 _PMFN pmfnUnwind;
 int (__cdecl*pForwardCompat)(...);
 _CatchableTypeArray *pCatachableTypeArray;
} _ThrowInfo;

Here the important thing is the _CatchableTypeArray. It holds a set of runtime type informations of the types that are catchable within this throw. In our case thats pretty simple. The only catchable type is typeid(int). Lets say you have a class derived from std::exception called my_exception. If you now throw an object of type my_exception you will have two entries in pCatchableTypeArray. One of them is typeid(my_exception) and the other is typeid(std::exception).

The compiler now fills the _ThrowInfo object as a global variable (and all the other objects needed). In the above case this is done the following way:
_TypeDescriptor tDescInt = typeid(int);

_CatchableType tcatchInt = 
{
 0,
 &tDescInt,
 0,
 0,
 0,
 0,
 NULL,
};

_CatchableTypeArray tcatchArrInt = 
{
 1,
 &tcatchInt,
};

_ThrowInfo tiMain1 = 
{
 0,
 NULL,
 NULL,
 &tcatchArrInt
};

You see that thats pretty a lot of information stored just for the throw 2. So finally Our above code expands to:
_TypeDescriptor tDescInt = typeid(int);

_CatchableType tcatchInt = 
{
 0,
 &tDescInt,
 0,
 0,
 0,
 0,
 NULL,
};

_CatchableTypeArray tcatchArrInt = 
{
 1,
 &tcatchInt,
};

_ThrowInfo tiMain1 = 
{
 0,
 NULL,
 NULL,
 &tcatchArrInt
};

int main()
{
 try
 {
  int throwObj = 2;
  _CxxThrowException(&throwObj, &tiMain1);
 }
 catch(...)
 {
 }
}

Inside _CxxThrowException now the following happens: RaiseException is called. But first the neccessary parameters are created. The exception code for an exception thrown by _CxxThrowException is 0xE06D7363. It also passes 3 parameters to RaiseException. A magic number, the pointer to the object thrown and the pointer to the _ThrowInfo. Resulting in the following pseudo code:
__declspec(noreturn) void __stdcall __CxxThrowException(void* pObj, _ThrowInfo* pInfo)
{
 struct { unsigned int magic; void* object, _ThrowInfo* info } Params;
 Params throwParams = 
 {
  0x19930520,
  pObj,
  pInfo
 }

 RaiseException(0xE06D7363, 1, 3, (const ULONG_PTR*)&throwParams);
}

Now we basically know how throw is handled by the compiler and we also see that in the end what you will notice is something like if you have encountered an access violation as they are also invoked by RaiseException.

Ok, if we now go further and inspect the try and catch there should be a bell ringing like crazy and it should be yelling "Wait!! You say that the throw gets transformed into a call to RaiseException like its for access violations, 0 divides and so on?! But they cannot be catched with try-catch!". And yes, you are right, they cant and thats way try - catch in fact gets transformed to a __try __except but in a special form. In code it would look somehow like that (its not real code, just theory):
unsigned long __stdcall mainHandler1(LPEXCEPTION_POINTERS info)
{
 if(info->ExceptionRecord->ExceptionCode != 0xE06D7363)
  return EXCEPTION_CONTINUE_SEARCH;

 if(WeHaveAHandlerForThisTypeSomeWhere(info->ExceptionRecord))
  return EXCEPTION_EXECUTE_HANDLER;

 return EXCEPTON_CONTINUE_SEARCH;
}

/* The stuff with _ThrowInfo comes here, omitted for readability */

int main()
{
 __try
 {
  int throwObj = 2;
  _CxxThrowException(&throwObj, &tiMain1);
 }
 __except(mainHandler1(GetExceptionInfo())
 {
 }
}

But thats not all! Somewhere we need to store which types of exceptions we can catch using our catch-statement. In fact the catch(int) gets transformed into an own function (actually only a function chunk where the runtime jumps using jmp not a real function called with call) which looks like that (now its really pseudocode because i cannot really translate it to C as it misses some information which would blow up the whole thing)
_s_FuncInfo* info = mainCatchBlockInfo1;
__asm { mov eax, info } // Its used for the following function as argument and passed through eax
goto CxxFrameHandler3;

The _s_FuncInfo is now again a structure that is built in to the compiler. It would make the article to big to explain everything like i did for the _ThrowInfo. In short it holds information for every type that can be caught in the current block. This consists (beneath other stuff) of runtime type information for every type and for each of them also the address of the actual code that is inside the catch-block.

Ok, now what is CxxFrameHandler3 doing? This is pretty simple:
1. It rejects exceptions that dont have 0xE06D7363 as code (which stands for C++ exceptions).
2. It searches through the _s_FuncInfo structure to find a type witch matches with one of the types it gets from the exception objects _CatchableTypeArray.
3. If it gets a match it indicates that there is a handler read
4. If there is not match it instructs the OS to search in the next frame

To finish the catch-part all we now need is the actual handler code. This code also is transformed into a function chunk (not a complete function). It actually is transformed into the chunk that ends a function. In code it would look like that:
// execute handler code
return addressWhereToContinueAfterCatch;

The operating system gets the address where it should jump to when it has set up again the original context and performs that jump. An example:
catch(...)
{
}

MessageBox(0, L"Ello!", L"", MB_OK);

Gets translated into the following assembler code:
.text:00401088 $LN16:
.text:00401088                 mov     eax, offset $LN9
.text:0040108D                 retn
.text:0040108E ; ---------------------------------------------------------------------------
.text:0040108E
.text:0040108E $LN9:                                   ; DATA XREF: _main:$LN16 o
.text:0040108E                 push    0               ; uType
.text:00401090                 push    offset Caption  ; lpCaption
.text:00401095                 push    offset Text     ; "Ello!"
.text:0040109A                 push    0               ; hWnd
.text:0040109C                 call    ds:__imp__MessageBoxW@16 ; MessageBoxW(x,x,x,x)

You see that it returns $LN9 in eax which is the address of the call to MessageBox. And $LN16 is the address of the catch block which is referenced in the _s_FuncInfo somewhere.

All that remains now is the try part. Here its no longer the compiler that can "decide" how to do things because now its the operating system that says how it works.

Inside the Thread Information Block the first field (fs:[0]) holds a pointer to a linked list of exception handlers (in our case its the address of the part where it goes to CxxFrameHandler3). Now what try does is it adds the catch-block to the linked list. After the RaiseException call we arrive in the function KiUserExceptionDispatcher. This function does a lot of work but in the end the important thing is that it loads the current linked list from the TIB using FS:[0] and loops through it to find a handler that says that it could handle the exception and calls its handler. If you want to browse through the currently attached handlers you do the following:
struct LinkedExceptionFrame
{
 LinkedExceptionFrame* pPrevious;
 void* pFunction;
};

LinkedExceptionFrame* pCur = NULL;
__asm
{
 mov eax, fs:[0]
 mov pCur, eax
}
while((DWORD)pCur != 0xFFFFFFFF)
{
 std::cout << pCur->pFunction << std::endl;
 pCur = pCur->pPrevious;
}

Now we have all the basic concepts we need to understand that try/catch/throw is not as trivial as most people think and that most things are actually handled at runtime (though a huge amount of additional data and function overhead is made to catch the correct type of exception). There is way more we could talk about (for example: What if we have parts of our frame protected by try-catch and others not or if we even have more than one try-catch-block and so on. But i think so far the most important things are said!

Some tips if you like to browse through it using a disassembler and a debugger:
Use Release build but disable any kind of code optimization. So you dont have all the register checks at the beginning and the end of function calls but your code is not getting rearranged by the optimizer so you can better compare it to the source. And its a good thing to disable Dynamic Base (ASLR) in the linker options (under Advanced).

So far
Yanick

Samstag, 26. März 2011

Exception Handling - Inform your users!

Hello everyone!

This article will show you methods to inform the user that something has happened in the program which caused an exception that was not handled by the program. It will also teach you how to use the values you will get and display some more information the user can then submit to you.

First we should know what happens if an exception is thrown. The operating system now aims to find a handler which tells what to do now. In order to find such a handler the system has a fixed order to search. It will start with the vectored exception handlers (added for example using AddVectoredExceptionHandler). If no handler was found there it will continue and in each frame of the stack it will examine all the frame based exception handlers (try - catch, __try - __except). If no handler suited the exception thrown it will then have a look if a top level exception handler is registered (using SetUnhandledExceptionFilter). If that fails too you will get a window "XY has stopped working" with the commonly known options. There is actually one exception from that behavior: When a debugger is attached. In this case the debugger is automatically installed as the top level exception handler and you cannot change it. Using various settings you can even intercept all of the above mentioned handlers and directly pass every exception to the debugger.

Well, now how could we get informed that an exception happened that was not properly handled? Using AddVectoredExceptionHandler is not a good idea because every exception that happened will be interpreted as if it was not handled. Thats not what we try to achieve. Installing a frame on top of every thread that gets started and handling every type of exception there is possible but to much work. Also threads started by external components maybe dont act like that and exceptions caused by them wont be recognized.

The response is using SetUnhandledExceptionFilter. This function lets us register a function that is called if all other handlers failed. As this is the last resort we can be sure that no one "cared" about that exception and that it will lead to an unwanted program termination if nothing is done now. The usage of that function is very easy. All it wants is a pointer to a function which has a special format. It then returns a pointer to the function that was installed before our call so we could reset it if we dont need to handle the exceptions anymore. So to test that we make a simple program:
#include <Windows.h>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 MessageBox(0, "An exception occured which wasnt handled!", "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

The return value of our function indicates what the operating system should do next. Returning EXCEPTION_CONTINUE_HANDLER instructs it to execute the handler for that exception. On the top level this actually means that it will directly call ExitProcess. If you run the above code (please make sure you are not attaching any debugger) you will get the messagebox (unless you are in kernel mode) and the application will terminate.

Now lets experiment with that. First we will look if the function also gets called if we surround the bad code with try - catch(...). First maybe think if you can figure out the response yourself.
#include <Windows.h>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 MessageBox(0, "An exception occured which wasnt handled!", "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 try
 {
  char* ptr = NULL;
  *ptr = 'a';
 }
 catch(...)
 {

 }
}

After running the code we know: It gets called! Maybe that is surprising you. If so have a look at the following code:
#include <Windows.h>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 MessageBox(0, "An exception occured which wasnt handled!", "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 __try
 {
  char* ptr = NULL;
  *ptr = 'a';
 }
 __except(EXCEPTION_EXECUTE_HANDLER)
 {

 }
}

Wait, now it is not called?! But why? The answer is pretty simple: There are two main types of exceptions. The first type are exceptions that are thrown using the keyword 'throw'. They are called C++-exceptions. These exceptions get caught using the try-catch statement. The other type are exceptions that are thrown by the operating system in response of faults that happened. These are called SEH-exceptions (structed exception handling). To catch such an exception there must be a frame which catches exceptions using __try - __except. We see the EXCEPTION_EXECUTE_HANDLER here again. In this case the executed "handler" is the part in the scope after __except.

As dereferencing the NULL-Pointer and assigning a value to it in user mode is not allowed the operating system indicates a fault and throws a SEH-exception. Those exceptions cannot be handled by try - catch statements. Therefore in the first code the exception is passed to our function. But in the second code the exception is caught and the program can continue so our function is not called.

Well, now our messagebox does not really contain a lot of information. A user which sees that message wont have an idea what to do and if he submits it to you this wont help at all. So providing some more information would be helpful. So we'll use the parameter our exception handler receives to find additional information about the exception which we can show to the user. The most important things one should now are the following two:
What happened?
Where did it happen?

The answer to both questions lies inside the exceptionInfo parameter. And here is how we can access it:
#include <Windows.h>
#include <iostream>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: 0x%08X\nAddress: 0x%08X", 
  exceptionInfo->ExceptionRecord->ExceptionCode,
  exceptionInfo->ExceptionRecord->ExceptionAddress
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

I think this is pretty self-explaining. Though there is one major problem with the exception address. If our binary uses ASLR (Address Space Layout Randomization) its base address will start at a random address at every launch. Therefore the address is completely useless for us as we dont know where the binary starts. To solve that problem we could print the start and end address for every module loaded into the message box which would allow us to exactly determine in which module at which offset the exception happened or we just print the start of the main module and the offset from there. The second version is less informative because if the exception didnt happen in the main module we again have an offset which is not really useful. But anyway, we will use the second approach as we focus on our executable.

#include <Windows.h>
#include <iostream>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: 0x%08X\nOffset: 0x%08X\nCodebase: 0x%08X", 
  exceptionInfo->ExceptionRecord->ExceptionCode,
  (DWORD)exceptionInfo->ExceptionRecord->ExceptionAddress - codeBase,
  codeBase
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

Now we have an offset which can be used to determine which instruction caused the exception. If you start google and search for "Exception code 0xC0000005" you will find that this stands for an access violation. Wouldnt it be nice to show EXCEPTION_ACCESS_VIOLATION instead of 0xC0000005? There is a simple way doing that using a little macro that converts the predefined codes into a friendlier form:
#include <Windows.h>
#include <iostream>

#define EXCEPTION_CASE(code) \
 case code: \
  exceptionString = #code; \
  break

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 const char* exceptionString = NULL;
 switch(exceptionInfo->ExceptionRecord->ExceptionCode)
 {
 EXCEPTION_CASE(EXCEPTION_ACCESS_VIOLATION);
 EXCEPTION_CASE(EXCEPTION_DATATYPE_MISALIGNMENT);
 EXCEPTION_CASE(EXCEPTION_BREAKPOINT);
 EXCEPTION_CASE(EXCEPTION_SINGLE_STEP);
 EXCEPTION_CASE(EXCEPTION_ARRAY_BOUNDS_EXCEEDED);
 EXCEPTION_CASE(EXCEPTION_FLT_DENORMAL_OPERAND);
 // add more cases...

 default:
  exceptionString = "Unknown exception";
  break;
 }

 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: %s (0x%08X)\nOffset: 0x%08X\nCodebase: 0x%08X", 
  exceptionString,
  exceptionInfo->ExceptionRecord->ExceptionCode,
  (DWORD)exceptionInfo->ExceptionRecord->ExceptionAddress - codeBase,
  codeBase
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

Now the user sees a better readable name for the exception code. Have a look at this article on MSDN to get a list with all defined exception codes that commonly occur: Article

Now lets change the main function to the following:
int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 throw "An error!";
}

Even if you defined all cases from the MSDN-article here you will get "Unknown exception (weird hex-number)". What does that number stand for? Maybe its a pointer to the string we have thrown? And beside that, why do we get such a huge offset? Our executable isn't that big! To answer the second question first: throw will actually be replaced by the compiler to a internal function call to CxxThrowException which then again calls RaiseException. The offset you get is the offset to RaiseException as this is considered the source of the exception and RaiseException lies inside the ntdll.dll which is loaded in high memory regions. See the following extract from IDA Pro:
.text:7DE80000 ; File Name   : C:\Windows\System32\ntdll.dll
.text:7DE80000 ; Format      : Portable executable for 80386 (PE)
.text:7DE80000 ; Imagebase   : 7DE70000

To answer the first question try throwing different things. Throw an integer, throw nothing (just throw;), throw another string, ... . You will notice that the hex-number will always be the same (0xE06D7363). So obviously it has nothing to do with the content thrown. No, it denotes that the thrown exception was a C++ exception (remember, these are exceptions thrown by the programmer using throw). So every time you use throw you will get the code mentioned above. So we can extend our switch with that static code:
#include <Windows.h>
#include <iostream>

#define EXCEPTION_CASE(code) \
 case code: \
  exceptionString = #code; \
  break

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 const char* exceptionString = NULL;
 switch(exceptionInfo->ExceptionRecord->ExceptionCode)
 {
 EXCEPTION_CASE(EXCEPTION_ACCESS_VIOLATION);
 EXCEPTION_CASE(EXCEPTION_DATATYPE_MISALIGNMENT);
 EXCEPTION_CASE(EXCEPTION_BREAKPOINT);
 EXCEPTION_CASE(EXCEPTION_SINGLE_STEP);
 EXCEPTION_CASE(EXCEPTION_ARRAY_BOUNDS_EXCEEDED);
 EXCEPTION_CASE(EXCEPTION_FLT_DENORMAL_OPERAND);
 // add more cases...

 case 0xE06D7363:
  exceptionString = "C++ exception (using throw)";
  break;

 default:
  exceptionString = "Unknown exception";
  break;
 }

 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: %s (0x%08X)\nOffset: 0x%08X\nCodebase: 0x%08X", 
  exceptionString,
  exceptionInfo->ExceptionRecord->ExceptionCode,
  (DWORD)exceptionInfo->ExceptionRecord->ExceptionAddress - codeBase,
  codeBase
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 throw "An error!";
}

So far those are the most important things you should now about displaying generic information about every type of exception. In upcoming articles we will have a look on how we can give more information which will require us to perform individual actions for each type of exception.

Thanks for reading and happy commenting
Yanick