Working with GDI+ - Part 2

Welcome back!

In the last part we have seen how to initialize GDI+ and draw some basic shapes. In this part we wanna draw some images to the window! I have chosen an image from a site where you can get free images:
http://www.freedigitalphotos.net/images/Blues_and_Violets_g333-Blue_Wallpaper_p59652.html

Drawing images is a very simple task. You have to do the following steps:
1. Create a Gdiplus::Image object
2. Draw it

You see, nothing complex here!

As in the previous part I'm loading my image in the startup function and I dispose of it in the shutdown function. And then i draw it in the draw function. To load you can use one Image::FromFile or Image::FromStream. To emulate a Image::FromMemory you can either implement your own IStream based on a MemoryStream or you can copy it to a HGLOBAL allocated large enough and then use the CreateStreamOnHGlobal function.

Gdiplus::Image* backgroundImage = nullptr;

void startupGDIPlus(ULONG_PTR* pToken) {
	// code omitted to start Gdiplus

	backgroundImage = Gdiplus::Image::FromFile(L"ID-10059652.jpg");
	status = backgroundImage->GetLastStatus();
}

void shutdownGDIPlus(ULONG_PTR token) {
	delete backgroundImage;
	Gdiplus::GdiplusShutdown(token);
}

void drawGDI(HDC hDc) {
	Gdiplus::Graphics* g = Gdiplus::Graphics::FromHDC(hDc);
	g->SetSmoothingMode(Gdiplus::SmoothingModeAntiAlias);

	g->DrawImage(backgroundImage, 10, 10);

	g->Flush();
	delete g;
}

For DrawImage you have a lot of possible parameters. In general they can be grouped the following way:

1. Parameters with "src" refer to the portion of the image you want to draw. They allow you to select only a part of the image
2. Parameters that are also position or size related but with no "src"-prefix refer to the part of the window that the portion selected with "src" should be drawn to. The "src"-portion is stretched to fit into this area.
3. The GraphicsUnit parameter lets you choose how the previous two parameter groups should be measured. Usually (and default) it uses Pixels.
4. The ImageAttributes can define more sophisticated operations on the image. We will have a look on them later.

Now lets modify our image a bit. In a first step Id like to add a transform to the image that rotates it 45° clock wise. Thus i create a Gdiplus::Matrix for identity (no transform) and one for rotation and before i draw the image I call SetTransform to apply the transform to the next drawing calls.

Gdiplus::Matrix* matrixIdentity = nullptr;
Gdiplus::Matrix* matrixRotation = nullptr;

void startupGDIPlus(ULONG_PTR* pToken) {
	// previous code omitted

	matrixIdentity = new Gdiplus::Matrix();
	matrixRotation = new Gdiplus::Matrix();
	matrixRotation->RotateAt(45.0f, Gdiplus::PointF(210, 210));
}

void drawGDI(HDC hDc) {
	Gdiplus::Graphics* g = Gdiplus::Graphics::FromHDC(hDc);
	g->SetSmoothingMode(Gdiplus::SmoothingModeAntiAlias);

	g->SetTransform(matrixRotation);
	g->DrawImage(backgroundImage, 10, 10);
	g->SetTransform(matrixIdentity);

	g->Flush();
	delete g;
}

The first question I already see: Why RotateAt with 210/210?! Now thats just math! The behavior of a rotation matrix is to rotate around the origin which is the top left corner of the window in my case. So that means our whole image gets rotate around the top left corner of the window. That is not what we want (or at least not what I want). The rotation should be at the center of the image. My image is 400x400 pixels so its center is at 200/200 and I have to add the offset 10/10 because I draw the image at 10/10. Now i created a matrix that rotates around the center of the image. You get a result like that:

A common request is to draw images that have an alpha channel and contain transparency. GDI+ supports transparency without any further adjustments needed. So I created a PNG of my image and set the color of the border to be transparent. Its not optimal because the border changes color a bit but you see clearly that it works.

There are transparent parts as well as semi transparent parts.

You can obtain very cool effects with the ImageAttributes parameter. You crate an ImageAttribute like any other object in GDI+. Then you can add several effects to the object. The first one I like to explain is the SetOutputChannel-function. It allows you to specify one of the 4 channels in CMYK (cyan, magenta, yellow, black) that is used to calculate a grayscale picture where the shade of gray equals the intensity of that channel. An example:

imageAttribs->SetOutputChannel(Gdiplus::ColorChannelFlagsM);

and the result:

The second parameter id like to explain is the color matrix. The color matrix is a 5x5 matrix that gets applied to the color of the image. It is 5x5 because it uses homogeneous coordinates and a color vector has 4 components (r, g, b, a) in that order. So lets start with the identity matrix:

	Gdiplus::ColorMatrix clrMatrix = {
		{
			{ 1, 0, 0, 0, 0 },
			{ 0, 1, 0, 0, 0 },
			{ 0, 0, 1, 0, 0 },
			{ 0, 0, 0, 1, 0 },
			{ 0, 0, 0, 0, 1 }
		}
	};

	imageAttribs->SetColorMatrix(&clrMatrix);

As expected there is no effect on the image, because its the identity matrix. To understand which component is creating which result you have to understand how matrix multiplication works. I try to visualize it a bit more:

{ a, b, c, d, e }      
{ f, g, h, i, j }      
{ k, l, m, n, o } * { red, green, blue, alpha, 1 } 
{ p, q, r, s, t }   
{ u, v, w, x, y }

  { a * red + f * green + k * blue + p * alpha + u }
  { b * red + g * green + l * blue + q * alpha + v }
= { c * red + h * green + m * blue + r * alpha + w }
  { d * red + i * green + n * blue + s * alpha + x }
  { e * red + j * green + o * blue + t * alpha + y }

From the fact that the last component must be 1 its clear that e j and o must always be 0 and y must be 1. It also gets clear that the center diagonal is used for scaling the color. So lets try the following matrix:

			{ 1, 0, 0, 0, 0 },
			{ 0, 0, 0, 0, 0 },
			{ 0, 0, 0, 0, 0 },
			{ 0, 0, 0, 1, 0 },
			{ 0, 0, 0, 0, 1 }

What do we expect? For the first component we only have 1 * red because all other are 0, for the second component we have 0 because all matrix elements on the second row are 0, same for the third. So the resulting color should be: (red, 0, 0, alpha). Thats the output:

As expected, only the red channel was shown. Success! If we take a look at the matrix equation we see that u, v and w can be used to add something to a color component. Colors are in range 0 to 1 so lets intensify our red channel a bit:

			{ 1, 0, 0, 0, 0 },
			{ 0, 1, 0, 0, 0 },
			{ 0, 0, 1, 0, 0 },
			{ 0, 0, 0, 1, 0 },
			{ 0.5f, 0, 0, 0, 1 }

So this should add 0.5 to the red channel (dont worry, if its bigger than 1 it just stays white, GDI+ clamps the color). The result:

Now lets think of something else: Lets have a look at the red component of the resulting color. Its calculated the following way:

finalRed = { a * inputRed + f * inputGreen + k * inputBlue + p * inputAlpha + u }

Now what if we set a to 0 and k to 1 and f to 0. We get 0 * inputRed + 0 * inputGreen + 1 * inputBlue. So we map the blue channel also to the red channel. So lets have a look at the following matrix:

			{ 1, 0, 0, 0, 0 },
			{ 0, 1, 0, 0, 0 },
			{ 0.5f, 0, 0.5f, 0, 0 },
			{ 0, 0, 0, 1, 0 },
			{ 0, 0, 0, 0, 1 }

Interesting is the third line, it stands for the blue channel. What we do is taking the average of the red and blue channel together as the blue channel. We get a very interesting result:

Well thats it for the image part. You can explore the other functions of the ImageAttribute pretty easy.

See you soon in the next part, text rendering!

Greetings
Yanick

Working with GDI+ - Part 1

Hello everyone

If you are using nothing but the plain Win32-API you might have a lot of struggles when you want to perform more complicated drawing actions in your window or control. Loading compressed image formats like PNG or JPG requires a lot of work and your code pretty soon starts to get bloated.For that reason Microsoft added a new feature to Windows with XP which is called GDI+. If you need it on an older platform you can install it using Windows Update back till Windows 2000. Now whats GDI+? Pretty simple: Its a library containing a lot (and by that I mean A LOT) of features that help you working with everything related to windows graphics. As the name suggests it supersedes the traditional GDI.

Well, enough talk now, lets get some code! In our first step we just want to draw a simple shape using GDI+. I have created a simple framework that creates a window and runs the message loop, ill omit these because there is nothing special there, its just the regular code for creating a window and running its message loop. This leads me to the following main function:

#include <gdiplus.h>

#pragma comment(lib, "Gdiplus.lib")

BOOL WINAPI WinMain(HINSTANCE, HINSTANCE, LPSTR, INT) {
	HWND hWindow = createWindow(L"GDI+", WndProc);
	ULONG_PTR gdiToken = 0;

	startupGDIPlus(&gdiToken);
	
	runWindowLoop(hWindow);

	shutdownGDIPlus(gdiToken);
	return TRUE;
}

As you can see you have to start GDI+ before you are using it and you have to shut it down when you are finished using it. The token acts as an indicator which call to startup shutdown should process. These two functions perform the following tasks:

Gdiplus::Pen* redPen = nullptr;

void startupGDIPlus(ULONG_PTR* pToken) {
	Gdiplus::GdiplusStartupInput startInput;
	
	Gdiplus::Status status = Gdiplus::GdiplusStartup(pToken, &startInput, nullptr);
	if(status != Gdiplus::Ok) {
		// do error handling
	}

	redPen = new Gdiplus::Pen(Gdiplus::Color::IndianRed, 3.0f);
}

void shutdownGDIPlus(ULONG_PTR token) {
	delete redPen;
	Gdiplus::GdiplusShutdown(token);
}

The functions are pretty easy to understand, we call GdiplusStartup to start GDI+ and GdiplusShutdown to shut it down. GdiplusStartup has two additional parameters, a GdiplusStartupInput and a GdiplusStartupOutput. With the input you can control a few settings like which version should be loaded. Just passing the default object is fair enough for most cases. The GdiplusStartupOutput parameter gets populated by the function and then contains two function pointers that have to be called if you specified in the input that the background thread should be suspended. You see: Most likely you will never use that!

You can also use the startup-function to load resources that persist over the entire lifetime of GDI+ and then delete them in shutdown. I did this for the pen I will be using later to draw my shape. It creates a pen with a stroke width of 3 and the color IndianRed.

Drawing is now done at the same place it is for regular GDI drawings - in WM_PAINT. You use the HDC from BeginPaint to and then draw the actions to that context. I created a function drawGDI which takes the HDC and then performs the painting.

void drawGDI(HDC hDc) {
	Gdiplus::Graphics* g = Gdiplus::Graphics::FromHDC(hDc);

	g->DrawArc(redPen, Gdiplus::Rect(10, 10, 100, 50), 0, 270);

	g->Flush();
	delete g;
}

All drawing is done with a Graphics object. It encapsulates all the functionality for painting. There are several ways to obtain a Graphics object. You can use FromHDC to draw to a device context, you can use FromImage to draw to an image (we will use that later) or FromHwnd to draw to the window. We use the FromHDC as we want to paint to the device context returned by BeginPaint. The next step is to actually draw something. I used the DrawArc-function to draw... well... an arc! I use the Pen created by the startup function to draw the arc at the coordinates specified by my Rect and using the angles 0 and 270 as boundaries. At the end I Flush all the draw actions so that they get executed and then delete the object. The result you get should be something like that:

We see or first problem: That line is so blocky! Is there something to change that? Of course there is! Its called anti aliasing. The concept of anti aliasing is pretty simple: Instead of saying "Here we have red, here not" the renderer creates a fade out at the edge of the arc addind several pixels around the edge which are partially transparent and fade into completely transparent. For the eye it still looks like a single line but with way less blocky appearance. We are only one function call away from having a perfectly smooth arc. After we created the graphics object we tell GDI+ to smooth the lines using AntiAlias:

void drawGDI(HDC hDc) {
	Gdiplus::Graphics* g = Gdiplus::Graphics::FromHDC(hDc);
	g->SetSmoothingMode(Gdiplus::SmoothingModeAntiAlias);

	g->DrawArc(redPen, Gdiplus::Rect(10, 10, 100, 50), 0, 270);

	g->Flush();
	delete g;
}

And thats it, you should see a nice smoothed line like the one in the picture:

Now you can play with the different functions like DrawLine, DrawRectangle and so on and we will meet for the next part when you are ready to draw some images!

Thanks for reading and see you soon
Yanick

Compiler internals - What happens with try/catch/throw in MSVC

Hello everyone

After i have seen that a lot of people are thinking that try - catch is a concept which is completely analyzed during compile time and therefore wont have big impact at runtime i thought i might shed some light on how the Microsoft compiler (cl.exe) is acting with try, catch and throw.

First off we will have a look on how a throw statement is interpreted by the compiler. Lets have a look at the following code:

int main()
{
 try
 {
  throw 2;
 }
 catch(...)
 {
 }
}

For now we are only interested in the throw 2. When the compiler hits the throw statement it actually has no clue if the exception its now converting is handled by an exception handler (and it doesnt care). The throw statement will be converted into a call to _CxxThrowException (exported by MSVCR100.dll (or any other version)). That function is a built in function in the compiler. You can call it yourself if you like ;). The first parameter of that function is a pointer to the object thrown. Therefore it gets clear, that the code above definitely expands to the following:

int main()
{
 try
 {
  int throwObj = 2;
  throw throwObj;
 }
 catch(...)
 {
 }
}

The second parameter of _CxxThrowException holds a pointer to a _ThrowInfo object. _ThrowInfo is also a built in type of the compiler. Its a struct holding various information about the type of exception that was thrown. It looks like that:

typedef const struct _s__ThrowInfo
{
 unsigned int attributes;
 _PMFN pmfnUnwind;
 int (__cdecl*pForwardCompat)(...);
 _CatchableTypeArray *pCatachableTypeArray;
} _ThrowInfo;

Here the important thing is the _CatchableTypeArray. It holds a set of runtime type informations of the types that are catchable within this throw. In our case thats pretty simple. The only catchable type is typeid(int). Lets say you have a class derived from std::exception called my_exception. If you now throw an object of type my_exception you will have two entries in pCatchableTypeArray. One of them is typeid(my_exception) and the other is typeid(std::exception).

The compiler now fills the _ThrowInfo object as a global variable (and all the other objects needed). In the above case this is done the following way:

_TypeDescriptor tDescInt = typeid(int);

_CatchableType tcatchInt = 
{
 0,
 &tDescInt,
 0,
 0,
 0,
 0,
 NULL,
};

_CatchableTypeArray tcatchArrInt = 
{
 1,
 &tcatchInt,
};

_ThrowInfo tiMain1 = 
{
 0,
 NULL,
 NULL,
 &tcatchArrInt
};

You see that thats pretty a lot of information stored just for the throw 2. So finally Our above code expands to:

_TypeDescriptor tDescInt = typeid(int);

_CatchableType tcatchInt = 
{
 0,
 &tDescInt,
 0,
 0,
 0,
 0,
 NULL,
};

_CatchableTypeArray tcatchArrInt = 
{
 1,
 &tcatchInt,
};

_ThrowInfo tiMain1 = 
{
 0,
 NULL,
 NULL,
 &tcatchArrInt
};

int main()
{
 try
 {
  int throwObj = 2;
  _CxxThrowException(&throwObj, &tiMain1);
 }
 catch(...)
 {
 }
}

Inside _CxxThrowException now the following happens: RaiseException is called. But first the neccessary parameters are created. The exception code for an exception thrown by _CxxThrowException is 0xE06D7363. It also passes 3 parameters to RaiseException. A magic number, the pointer to the object thrown and the pointer to the _ThrowInfo. Resulting in the following pseudo code:

__declspec(noreturn) void __stdcall __CxxThrowException(void* pObj, _ThrowInfo* pInfo)
{
 struct { unsigned int magic; void* object, _ThrowInfo* info } Params;
 Params throwParams = 
 {
  0x19930520,
  pObj,
  pInfo
 }

 RaiseException(0xE06D7363, 1, 3, (const ULONG_PTR*)&throwParams);
}

Now we basically know how throw is handled by the compiler and we also see that in the end what you will notice is something like if you have encountered an access violation as they are also invoked by RaiseException.

Ok, if we now go further and inspect the try and catch there should be a bell ringing like crazy and it should be yelling "Wait!! You say that the throw gets transformed into a call to RaiseException like its for access violations, 0 divides and so on?! But they cannot be catched with try-catch!". And yes, you are right, they cant and thats way try - catch in fact gets transformed to a __try __except but in a special form. In code it would look somehow like that (its not real code, just theory):

unsigned long __stdcall mainHandler1(LPEXCEPTION_POINTERS info)
{
 if(info->ExceptionRecord->ExceptionCode != 0xE06D7363)
  return EXCEPTION_CONTINUE_SEARCH;

 if(WeHaveAHandlerForThisTypeSomeWhere(info->ExceptionRecord))
  return EXCEPTION_EXECUTE_HANDLER;

 return EXCEPTON_CONTINUE_SEARCH;
}

/* The stuff with _ThrowInfo comes here, omitted for readability */

int main()
{
 __try
 {
  int throwObj = 2;
  _CxxThrowException(&throwObj, &tiMain1);
 }
 __except(mainHandler1(GetExceptionInfo())
 {
 }
}

But thats not all! Somewhere we need to store which types of exceptions we can catch using our catch-statement. In fact the catch(int) gets transformed into an own function (actually only a function chunk where the runtime jumps using jmp not a real function called with call) which looks like that (now its really pseudocode because i cannot really translate it to C as it misses some information which would blow up the whole thing)

_s_FuncInfo* info = mainCatchBlockInfo1;
__asm { mov eax, info } // Its used for the following function as argument and passed through eax
goto CxxFrameHandler3;

The _s_FuncInfo is now again a structure that is built in to the compiler. It would make the article to big to explain everything like i did for the _ThrowInfo. In short it holds information for every type that can be caught in the current block. This consists (beneath other stuff) of runtime type information for every type and for each of them also the address of the actual code that is inside the catch-block.

Ok, now what is CxxFrameHandler3 doing? This is pretty simple:
1. It rejects exceptions that dont have 0xE06D7363 as code (which stands for C++ exceptions).
2. It searches through the _s_FuncInfo structure to find a type witch matches with one of the types it gets from the exception objects _CatchableTypeArray.
3. If it gets a match it indicates that there is a handler read
4. If there is not match it instructs the OS to search in the next frame

To finish the catch-part all we now need is the actual handler code. This code also is transformed into a function chunk (not a complete function). It actually is transformed into the chunk that ends a function. In code it would look like that:

// execute handler code
return addressWhereToContinueAfterCatch;

The operating system gets the address where it should jump to when it has set up again the original context and performs that jump. An example:

catch(...)
{
}

MessageBox(0, L"Ello!", L"", MB_OK);

Gets translated into the following assembler code:

.text:00401088 $LN16:
.text:00401088                 mov     eax, offset $LN9
.text:0040108D                 retn
.text:0040108E ; ---------------------------------------------------------------------------
.text:0040108E
.text:0040108E $LN9:                                   ; DATA XREF: _main:$LN16 o
.text:0040108E                 push    0               ; uType
.text:00401090                 push    offset Caption  ; lpCaption
.text:00401095                 push    offset Text     ; "Ello!"
.text:0040109A                 push    0               ; hWnd
.text:0040109C                 call    ds:__imp__MessageBoxW@16 ; MessageBoxW(x,x,x,x)

You see that it returns $LN9 in eax which is the address of the call to MessageBox. And $LN16 is the address of the catch block which is referenced in the _s_FuncInfo somewhere.

All that remains now is the try part. Here its no longer the compiler that can "decide" how to do things because now its the operating system that says how it works.

Inside the Thread Information Block the first field (fs:[0]) holds a pointer to a linked list of exception handlers (in our case its the address of the part where it goes to CxxFrameHandler3). Now what try does is it adds the catch-block to the linked list. After the RaiseException call we arrive in the function KiUserExceptionDispatcher. This function does a lot of work but in the end the important thing is that it loads the current linked list from the TIB using FS:[0] and loops through it to find a handler that says that it could handle the exception and calls its handler. If you want to browse through the currently attached handlers you do the following:

struct LinkedExceptionFrame
{
 LinkedExceptionFrame* pPrevious;
 void* pFunction;
};

LinkedExceptionFrame* pCur = NULL;
__asm
{
 mov eax, fs:[0]
 mov pCur, eax
}
while((DWORD)pCur != 0xFFFFFFFF)
{
 std::cout << pCur->pFunction << std::endl;
 pCur = pCur->pPrevious;
}

Now we have all the basic concepts we need to understand that try/catch/throw is not as trivial as most people think and that most things are actually handled at runtime (though a huge amount of additional data and function overhead is made to catch the correct type of exception). There is way more we could talk about (for example: What if we have parts of our frame protected by try-catch and others not or if we even have more than one try-catch-block and so on. But i think so far the most important things are said!

Some tips if you like to browse through it using a disassembler and a debugger:
Use Release build but disable any kind of code optimization. So you dont have all the register checks at the beginning and the end of function calls but your code is not getting rearranged by the optimizer so you can better compare it to the source. And its a good thing to disable Dynamic Base (ASLR) in the linker options (under Advanced).

So far
Yanick

Internet Explorer 9 - Avoid JavaScript in x64

Ahoi

Even though im not using the Internet Explorer since ages now and am pretty sticked to one type of browser i still like to browse announcements of new browser (versions). It was Microsoft to release version 9 some days ago so i decided to test it out and see if there are cool new things available.

While actually the x86 version runs pretty fair ive seen a problem coming up with the x64 version. If you launch a site which makes heavy usage of javascript (and there are plenty) you should be aware that it might get pretty slow. Its parsed and executed in a way that really makes you fall asleep. Of course i first thought that this is an issue caused by my system. So i restarted everything checked windows update and dug through the settings of the IE. Sadly nothing helped to make it faster. After browsing a bit through the wide web ive seen that this is an issue that Microsoft obviously knows. They didnt implement the JIT compiler into the x64 version of the browser. This causes the javascript code made by the programmer being executed without any optimization.

Why does Microsoft release a version of a product which lacks such an important feature? It reminds me a lot of visual studio 2010 which until yet does not have any support for intellisense in CLI/C++ which is one of the most important features of .NET as no one can memorize those zillions of objects... I really hate that manner of "We release half finished products and maybe there will be updates some when..."!

What do you think?
Yanick

RtlCaptureStackBackTrace in managed world - Error, error, error!

Hello everyone

Lately ive created an API for a game which is no longer maintained by its developers which will be loaded using DLL-injection. It allows users to write plug-ins for that game. The DLL containing the Framework consists of pure managed code (C#) and is therefore loaded using the ICLRRuntimeHost::ExecuteInDefaultAppDomain method.

As its hard to debug an executable without having the source code of it ive started to create my own symbol database for functions and global objects. I planed to use that table in my stack trace to see more detailed information if i did something wrong (those who now throw in debuggers may know that the game had anti debugging mechanics implemented (as it was an online game back then) and i was to lazy to bypass them (sadly...)).

Since ages i use the function RtlCaptureStackBackTrace to easily create stack traces in unmanaged world. So i did the same in C# using DllImport. After setting framesToSkip to 0 i was pretty shocked that i only got 1 frame returned by the function which was actually RtlCaptureSBT itself. I tried several things and then used Marshal.GetLastWin32Error() to get more information on what im doing wrong. Interestingly it returned that no error happened (yes, SetLastError=true ;)). So the function said something is wrong and GetLastError said nothing is wrong, oh yeah!

So i launched IDA and stepped through the function in another .NET executable which wasn't injected to the game. On my (in this time) x86 machine what it did was calling RtlWalkFrameChain which then called RtlWalk32BitStack. That function loops through the stack frames and calls for the address of each of them the function RtlpIsPointerInDllRange from ntdll.dll which determines if a given pointer is in the accessible memory of any of the loaded modules. If it returns false RtlWalk32BitStack returns false thus RtlWalkFrameChain returns false which also causes RtlCaptureStackBackTrace to return false. But as none of these functions calls SetLastError it stays 0 also on failure.

Ok, so i started a debug run. First frame worked good and it was added to the list. Second frame was in a managed module which caused RtlpIsPointerInDllRange to return false. Now all aborted.

Well then i dug trough ntdll.dll searching references of the global variable used to indicate accessible ranges and found the function RtlpStkMarkDllRange. This function is only referenced by LdrpLoadOrMapDll which is used by LdrLoadDll (called by LoadLibrary). RtlpStkMarkDllRange is also not exported so other DLLs can not actually call it. So i searched for references to RtlpIsPointerInDLLRange (also not exported) and found that its only used by functions that perform stack walks.

So i got the feeling that somehow they forgot to include the call to RtlpStkMarkDllRange when a fully managed DLL is loaded because actually there is no reason why a .NET executable should be not valid for holding pointers to functions! Even more as StackWalk64 works as intended and returns all frames while RtlCaptureStackBackTrace isnt.

What do you think?
Yanick

Exception Handling - Inform your users! Part 2

Hello again

In the previous article (Part 1) i explained how you provide some basic information about an unhandled exception to the user and announced that in the next part we will have a look on how to get more detailed information. Now thats exactly what we will do in this article. Ill show two methods to gather more information what happened and why it happened.

Method 1: Do it manually
For this method we will create several helper functions that handle one or more types of exceptions. We start with the following code:

#include <Windows.h>
#include <iostream>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);
}

The first function we make will return a name which describes the type of exception that happened. Writing that function is pretty easy if we use macro that helps us:

#define EX_CASE(code) \
 case code:\
  return #code;

LPCSTR GetExceptionName(DWORD code)
{
 switch(code)
 {
 EX_CASE(EXCEPTION_ACCESS_VIOLATION);
 EX_CASE(EXCEPTION_DATATYPE_MISALIGNMENT);
 EX_CASE(EXCEPTION_BREAKPOINT);
 EX_CASE(EXCEPTION_SINGLE_STEP);
 EX_CASE(EXCEPTION_ARRAY_BOUNDS_EXCEEDED);
 EX_CASE(EXCEPTION_FLT_DENORMAL_OPERAND);
 EX_CASE(EXCEPTION_FLT_DIVIDE_BY_ZERO);
 EX_CASE(EXCEPTION_FLT_INEXACT_RESULT);
 EX_CASE(EXCEPTION_FLT_INVALID_OPERATION);
 EX_CASE(EXCEPTION_FLT_OVERFLOW);
 EX_CASE(EXCEPTION_FLT_STACK_CHECK);
 EX_CASE(EXCEPTION_FLT_UNDERFLOW);
 EX_CASE(EXCEPTION_INT_DIVIDE_BY_ZERO);
 EX_CASE(EXCEPTION_INT_OVERFLOW);
 EX_CASE(EXCEPTION_PRIV_INSTRUCTION);
 EX_CASE(EXCEPTION_IN_PAGE_ERROR);
 EX_CASE(EXCEPTION_ILLEGAL_INSTRUCTION);
 EX_CASE(EXCEPTION_NONCONTINUABLE_EXCEPTION);
 EX_CASE(EXCEPTION_STACK_OVERFLOW);
 EX_CASE(EXCEPTION_INVALID_DISPOSITION);
 EX_CASE(EXCEPTION_GUARD_PAGE);
 EX_CASE(EXCEPTION_INVALID_HANDLE);

 case 0xE06D7363:
  return "C++ Exception";
 
 default:
  return "Unknown exception";
 }
}

#undef EX_CASE

The next helper function will take an exception address an return in which module the exception occurred. This is done by looping through the loaded modules and comparing which one is the last with its base address before the exception location. Then we use GetModuleFileName to retrieve the name of the module.

#include <windows.h>
#include <iostream>
#include <psapi.h>

#pragma comment(lib, "psapi.lib")

// Definition of function GetExceptionName....


HMODULE GetExceptionModule(LPVOID address, LPSTR moduleName /* must support MAX_PATH characters */)
{
 HMODULE moduleList[1024];
 DWORD sizeNeeded = 0;
 if(FALSE == EnumProcessModules(GetCurrentProcess(), moduleList, 1024, &sizeNeeded) || sizeNeeded < sizeof(HMODULE))
  return NULL;

 int curModule = -1;
 for(int i = 0; i < (sizeNeeded / sizeof(HMODULE)); ++i)
 {
  if((DWORD)moduleList[i] < (DWORD)address)
  {
   if(curModule == -1)
    curModule = i;
   else
   {
    if((DWORD)moduleList[curModule] < (DWORD)moduleList[i])
     curModule = i;
   }
  }
 }

 if(curModule == -1)
  return NULL;

 if(!GetModuleFileName(moduleList[curModule], moduleName, MAX_PATH))
  return NULL;

 return moduleList[curModule];
}

We can test those two functions now and see what result we get using the following code:

// previous code...

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 char message[MAX_PATH + 255];
 char module[MAX_PATH];
 char* moduleName = NULL;
 if(GetExceptionModule(exceptionInfo->ExceptionRecord->ExceptionAddress, module))
  moduleName = module;
 else
  moduleName = "Unknown module!";

 sprintf_s(message, 
  "An exception has occured which was not handled!\nCode: %s\nModule: %s", 
  GetExceptionName(exceptionInfo->ExceptionRecord->ExceptionCode),
  moduleName
 );

 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 int b = 0;
 int a = 2 / b;
}

The messagebox now should display the name of your executable. The EXCEPTION_RECORD structure inside exceptionInfo contains an array of additional data that can be used to provide more information. Sadly those values are not defined by a standard. Its up to every compiler to do whatever it likes to. There are only 2 codes for which additional information is defined in a standardized way. EXCEPTION_ACCESS_VIOLATION and EXCEPTION_IN_PAGE_ERROR. We will now write a function that handles EXCEPTION_ACCESS_VIOLATION separately and provide a function that handles all the other codes.

void HandleAccessViolation(LPEXCEPTION_POINTERS info)
{
 char message[MAX_PATH + 512];
 char module[MAX_PATH];
 char* moduleName = NULL;
 if(GetExceptionModule(info->ExceptionRecord->ExceptionAddress, module))
  moduleName = module;
 else
  moduleName = "Unknown module!";

 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 DWORD offset = (DWORD)info->ExceptionRecord->ExceptionAddress - codeBase;

 char* accessType = NULL;
 switch(info->ExceptionRecord->ExceptionInformation[0])
 {
 case 0:
  accessType = "Read";
  break;
 case 1:
  accessType = "Write";
  break;
 case 2:
  accessType = "Execute";
  break;
 default:
  accessType = "Unknown";
  break;
 }

 sprintf_s(message, 
  "An exception has occured which was not handled!\nCode: %s\nModule: %s\n"\
  "The thread %u tried to %s memory at address 0x%08X which is inaccessible!\n"\
  "Offset: 0x%08X\nCodebase: 0x%08X",
  GetExceptionName(info->ExceptionRecord->ExceptionCode),
  moduleName,
  GetCurrentThreadId(),
  accessType,
  info->ExceptionRecord->ExceptionInformation[1],
  offset,
  codeBase
 );

 MessageBox(0, message, "Error!", MB_OK);
}

void HandleCommonException(LPEXCEPTION_POINTERS info)
{
 char message[MAX_PATH + 512];
 char module[MAX_PATH];
 char* moduleName = NULL;
 if(GetExceptionModule(info->ExceptionRecord->ExceptionAddress, module))
  moduleName = module;
 else
  moduleName = "Unknown module!";

 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 DWORD offset = (DWORD)info->ExceptionRecord->ExceptionAddress - codeBase;

 sprintf_s(message, 
  "An exception has occured which was not handled!\nCode: %s\nModule: %s\n"\
  "Offset: 0x%08X\nCodebase: 0x%08X",
  GetExceptionName(info->ExceptionRecord->ExceptionCode),
  moduleName,
  offset,
  codeBase
 );

 MessageBox(0, message, "Error!", MB_OK);
}

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 switch(exceptionInfo->ExceptionRecord->ExceptionCode)
 {
 case EXCEPTION_ACCESS_VIOLATION:
  HandleAccessViolation(exceptionInfo);
  break;
 default:
  HandleCommonException(exceptionInfo);
  break;
 }
 return EXCEPTION_EXECUTE_HANDLER;
}

Well, thats it! Now we have handled access violations and also a default handling for other exceptions. Method 2: Let windows do the thing! Windows provides a method to create a dump file from the current execution context. This will include a dump of the memory as well as other information visual studio can use to display all the information necessary. The function is called inside the exception handler. Its very easy so i wont tell much about it. It will create a .dmp file which can be opened with Visual Studio. When you start debugging (using the green arrow) you will get all the information needed. Here is the corresponding code:

#include 
#include 
#include 
#include 

#pragma comment(lib, "dbghelp.lib")

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 char timeStr[255];
 time_t t = time(NULL);
 tm* tM = localtime(&t);
 strftime(timeStr, 255, "CrashDump_%d_%m_%Y_%H_%M_%S.dmp", tM);
 HANDLE hFile = CreateFile(
  timeStr, 
  GENERIC_WRITE | GENERIC_READ,
  0,
  NULL,
  CREATE_ALWAYS,
  0,
  NULL
 );

 if(hFile != NULL)
 {
  MINIDUMP_EXCEPTION_INFORMATION info = 
  {
   GetCurrentThreadId(),
   exceptionInfo,
   TRUE
  };

  MiniDumpWriteDump(
   GetCurrentProcess(),
   GetCurrentProcessId(),
   hFile,
   MiniDumpWithIndirectlyReferencedMemory,
   &info,
   NULL,
   NULL
  );

  CloseHandle(hFile);
 }

 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 int b = 0;
 int a = 2 / b;
}

Now you have 2 good methods to retreive vital information about errors that happened.

Thanks for reading and bye
Yanick

Exception Handling - Inform your users!

Hello everyone!

This article will show you methods to inform the user that something has happened in the program which caused an exception that was not handled by the program. It will also teach you how to use the values you will get and display some more information the user can then submit to you.

First we should know what happens if an exception is thrown. The operating system now aims to find a handler which tells what to do now. In order to find such a handler the system has a fixed order to search. It will start with the vectored exception handlers (added for example using AddVectoredExceptionHandler). If no handler was found there it will continue and in each frame of the stack it will examine all the frame based exception handlers (try - catch, __try - __except). If no handler suited the exception thrown it will then have a look if a top level exception handler is registered (using SetUnhandledExceptionFilter). If that fails too you will get a window "XY has stopped working" with the commonly known options. There is actually one exception from that behavior: When a debugger is attached. In this case the debugger is automatically installed as the top level exception handler and you cannot change it. Using various settings you can even intercept all of the above mentioned handlers and directly pass every exception to the debugger.

Well, now how could we get informed that an exception happened that was not properly handled? Using AddVectoredExceptionHandler is not a good idea because every exception that happened will be interpreted as if it was not handled. Thats not what we try to achieve. Installing a frame on top of every thread that gets started and handling every type of exception there is possible but to much work. Also threads started by external components maybe dont act like that and exceptions caused by them wont be recognized.

The response is using SetUnhandledExceptionFilter. This function lets us register a function that is called if all other handlers failed. As this is the last resort we can be sure that no one "cared" about that exception and that it will lead to an unwanted program termination if nothing is done now. The usage of that function is very easy. All it wants is a pointer to a function which has a special format. It then returns a pointer to the function that was installed before our call so we could reset it if we dont need to handle the exceptions anymore. So to test that we make a simple program:

#include <Windows.h>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 MessageBox(0, "An exception occured which wasnt handled!", "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

The return value of our function indicates what the operating system should do next. Returning EXCEPTION_CONTINUE_HANDLER instructs it to execute the handler for that exception. On the top level this actually means that it will directly call ExitProcess. If you run the above code (please make sure you are not attaching any debugger) you will get the messagebox (unless you are in kernel mode) and the application will terminate.

Now lets experiment with that. First we will look if the function also gets called if we surround the bad code with try - catch(...). First maybe think if you can figure out the response yourself.

#include <Windows.h>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 MessageBox(0, "An exception occured which wasnt handled!", "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 try
 {
  char* ptr = NULL;
  *ptr = 'a';
 }
 catch(...)
 {

 }
}

After running the code we know: It gets called! Maybe that is surprising you. If so have a look at the following code:

#include <Windows.h>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 MessageBox(0, "An exception occured which wasnt handled!", "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 __try
 {
  char* ptr = NULL;
  *ptr = 'a';
 }
 __except(EXCEPTION_EXECUTE_HANDLER)
 {

 }
}

Wait, now it is not called?! But why? The answer is pretty simple: There are two main types of exceptions. The first type are exceptions that are thrown using the keyword 'throw'. They are called C++-exceptions. These exceptions get caught using the try-catch statement. The other type are exceptions that are thrown by the operating system in response of faults that happened. These are called SEH-exceptions (structed exception handling). To catch such an exception there must be a frame which catches exceptions using __try - __except. We see the EXCEPTION_EXECUTE_HANDLER here again. In this case the executed "handler" is the part in the scope after __except.

As dereferencing the NULL-Pointer and assigning a value to it in user mode is not allowed the operating system indicates a fault and throws a SEH-exception. Those exceptions cannot be handled by try - catch statements. Therefore in the first code the exception is passed to our function. But in the second code the exception is caught and the program can continue so our function is not called.

Well, now our messagebox does not really contain a lot of information. A user which sees that message wont have an idea what to do and if he submits it to you this wont help at all. So providing some more information would be helpful. So we'll use the parameter our exception handler receives to find additional information about the exception which we can show to the user. The most important things one should now are the following two:
What happened?
Where did it happen?

The answer to both questions lies inside the exceptionInfo parameter. And here is how we can access it:

#include <Windows.h>
#include <iostream>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: 0x%08X\nAddress: 0x%08X", 
  exceptionInfo->ExceptionRecord->ExceptionCode,
  exceptionInfo->ExceptionRecord->ExceptionAddress
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

I think this is pretty self-explaining. Though there is one major problem with the exception address. If our binary uses ASLR (Address Space Layout Randomization) its base address will start at a random address at every launch. Therefore the address is completely useless for us as we dont know where the binary starts. To solve that problem we could print the start and end address for every module loaded into the message box which would allow us to exactly determine in which module at which offset the exception happened or we just print the start of the main module and the offset from there. The second version is less informative because if the exception didnt happen in the main module we again have an offset which is not really useful. But anyway, we will use the second approach as we focus on our executable.

#include <Windows.h>
#include <iostream>

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: 0x%08X\nOffset: 0x%08X\nCodebase: 0x%08X", 
  exceptionInfo->ExceptionRecord->ExceptionCode,
  (DWORD)exceptionInfo->ExceptionRecord->ExceptionAddress - codeBase,
  codeBase
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

Now we have an offset which can be used to determine which instruction caused the exception. If you start google and search for "Exception code 0xC0000005" you will find that this stands for an access violation. Wouldnt it be nice to show EXCEPTION_ACCESS_VIOLATION instead of 0xC0000005? There is a simple way doing that using a little macro that converts the predefined codes into a friendlier form:

#include <Windows.h>
#include <iostream>

#define EXCEPTION_CASE(code) \
 case code: \
  exceptionString = #code; \
  break

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 const char* exceptionString = NULL;
 switch(exceptionInfo->ExceptionRecord->ExceptionCode)
 {
 EXCEPTION_CASE(EXCEPTION_ACCESS_VIOLATION);
 EXCEPTION_CASE(EXCEPTION_DATATYPE_MISALIGNMENT);
 EXCEPTION_CASE(EXCEPTION_BREAKPOINT);
 EXCEPTION_CASE(EXCEPTION_SINGLE_STEP);
 EXCEPTION_CASE(EXCEPTION_ARRAY_BOUNDS_EXCEEDED);
 EXCEPTION_CASE(EXCEPTION_FLT_DENORMAL_OPERAND);
 // add more cases...

 default:
  exceptionString = "Unknown exception";
  break;
 }

 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: %s (0x%08X)\nOffset: 0x%08X\nCodebase: 0x%08X", 
  exceptionString,
  exceptionInfo->ExceptionRecord->ExceptionCode,
  (DWORD)exceptionInfo->ExceptionRecord->ExceptionAddress - codeBase,
  codeBase
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 char* ptr = NULL;
 *ptr = 'a';
}

Now the user sees a better readable name for the exception code. Have a look at this article on MSDN to get a list with all defined exception codes that commonly occur: Article

Now lets change the main function to the following:

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 throw "An error!";
}

Even if you defined all cases from the MSDN-article here you will get "Unknown exception (weird hex-number)". What does that number stand for? Maybe its a pointer to the string we have thrown? And beside that, why do we get such a huge offset? Our executable isn't that big! To answer the second question first: throw will actually be replaced by the compiler to a internal function call to CxxThrowException which then again calls RaiseException. The offset you get is the offset to RaiseException as this is considered the source of the exception and RaiseException lies inside the ntdll.dll which is loaded in high memory regions. See the following extract from IDA Pro:

.text:7DE80000 ; File Name   : C:\Windows\System32\ntdll.dll
.text:7DE80000 ; Format      : Portable executable for 80386 (PE)
.text:7DE80000 ; Imagebase   : 7DE70000

To answer the first question try throwing different things. Throw an integer, throw nothing (just throw;), throw another string, ... . You will notice that the hex-number will always be the same (0xE06D7363). So obviously it has nothing to do with the content thrown. No, it denotes that the thrown exception was a C++ exception (remember, these are exceptions thrown by the programmer using throw). So every time you use throw you will get the code mentioned above. So we can extend our switch with that static code:

#include <Windows.h>
#include <iostream>

#define EXCEPTION_CASE(code) \
 case code: \
  exceptionString = #code; \
  break

LONG WINAPI UnhandledException(LPEXCEPTION_POINTERS exceptionInfo)
{
 const char* exceptionString = NULL;
 switch(exceptionInfo->ExceptionRecord->ExceptionCode)
 {
 EXCEPTION_CASE(EXCEPTION_ACCESS_VIOLATION);
 EXCEPTION_CASE(EXCEPTION_DATATYPE_MISALIGNMENT);
 EXCEPTION_CASE(EXCEPTION_BREAKPOINT);
 EXCEPTION_CASE(EXCEPTION_SINGLE_STEP);
 EXCEPTION_CASE(EXCEPTION_ARRAY_BOUNDS_EXCEEDED);
 EXCEPTION_CASE(EXCEPTION_FLT_DENORMAL_OPERAND);
 // add more cases...

 case 0xE06D7363:
  exceptionString = "C++ exception (using throw)";
  break;

 default:
  exceptionString = "Unknown exception";
  break;
 }

 DWORD codeBase = (DWORD)GetModuleHandle(NULL);
 char message[255];
 sprintf_s<255>(message, 
  "An exception occured which wasnt handled!\nCode: %s (0x%08X)\nOffset: 0x%08X\nCodebase: 0x%08X", 
  exceptionString,
  exceptionInfo->ExceptionRecord->ExceptionCode,
  (DWORD)exceptionInfo->ExceptionRecord->ExceptionAddress - codeBase,
  codeBase
 );
 MessageBox(0, message, "Error!", MB_OK);
 return EXCEPTION_EXECUTE_HANDLER;
}

int main()
{
 SetUnhandledExceptionFilter(UnhandledException);

 throw "An error!";
}

So far those are the most important things you should now about displaying generic information about every type of exception. In upcoming articles we will have a look on how we can give more information which will require us to perform individual actions for each type of exception.

Thanks for reading and happy commenting
Yanick