Investigating why an instance is kept alive in a .net application

Sometimes, when I want to verify .net memory management behavior, I fire up ANTS Memory Profiler and run it on a test application just to see how memory management behaves.

So today I went and created this application

class Program
{
    static void Main(string[] args)
    {
        A a = new A();
        B b = new B { Pointer = a };
        Console.WriteLine("One");
        Console.ReadLine();
        b.Pointer = null;
        b = null;
        Console.WriteLine("Two");
        Console.ReadLine();
        a = null;
        Console.WriteLine("Three");
        Console.ReadLine();
        a = new A();
        Console.WriteLine("Four");
        Console.ReadLine();
    }
}

class A
{

}

class B
{
    public A Pointer;
}

The ReadLine calls allow me to hit the “Take Memory Snapshot” in profiler. Later I examined these snapshots, specially the one between One and Two. What would you expect, how many instances are alive at that point? I’d say one, the instance of class A (referenced by a).

However, the profiler was showing, surprisingly, two. According to it, both an instance of A and an instance of B, were still very much alive. This result surprised me. Even more surprisingly, after Four there was still an instance of B around. How is this possible? There are no references to b and it is pointing to null. A bug in Memory Profiler? Hardly possible.

Right before I was going to post a question on RedGate’s forum I decided to check the IL code. With .net reflector of course. Immediately I saw the reason for that odd behavior. Can you spot it?

.method private hidebysig static void Main(string[] args) cil managed
{
    .entrypoint
    .maxstack 2
    .locals init (
        [0] class ConsoleApplication213.A a,
        [1] class ConsoleApplication213.B b,
        [2] class ConsoleApplication213.B b2)
    L_0000: nop 
    L_0001: newobj instance void ConsoleApplication213.A::.ctor()
    L_0006: stloc.0 
    L_0007: newobj instance void ConsoleApplication213.B::.ctor()
    L_000c: stloc.2 
    L_000d: ldloc.2 
    L_000e: ldloc.0 
    L_000f: stfld class ConsoleApplication213.A ConsoleApplication213.B::Pointer
    L_0014: ldloc.2 
    L_0015: stloc.1 
    L_0016: ldstr "One"
    L_001b: call void [mscorlib]System.Console::WriteLine(string)
    L_0020: nop 
    L_0021: call string [mscorlib]System.Console::ReadLine()
    L_0026: pop 
    L_0027: ldloc.1 
    L_0028: ldnull 
    L_0029: stfld class ConsoleApplication213.A ConsoleApplication213.B::Pointer
    L_002e: ldnull 
    L_002f: stloc.1 
    L_0030: ldstr "Two"
    L_0035: call void [mscorlib]System.Console::WriteLine(string)
    L_003a: nop 
    L_003b: call string [mscorlib]System.Console::ReadLine()
    L_0040: pop 
    …
}

For some reason, compiler decided to allocate two references for class B: local variables b and b2. When the instance of class B is created it is stored in b2. After the Pointer property is assigned the same reference is stored to b while b2 isn’t set to null. Then when b is set to null, the b2 is still very much alive and thus keeping the instances count for class B to 1 – even though b is null and there are no other C# variables for referencing instances of B.

Mystery solved.  The lessons learned are that compiler might generate code one wouldn’t expect and that profiling memory isn’t always black and white. Those 50 shades of gray happen as well. IOW experience and good understanding of .net is a necessity.

Automating the Righthand Dataset Visualizer build process and a refresh build

Since the last version released I become aware of an issue in the visualizer. If you were working on a project that referenced newer DevExpress assemblies the visualizer might have reported an exception due to the binary incompatibility with its references to the DevExpress assemblies - a assembly binding issue.

(If you want just the binaries you can skip to the end of the article and download them.)

The solution is to use a custom build of DevExpress assemblies. If you have their sources you can build your custom DevExpress assemblies using these useful scripts.

Then I have to use those custom built assemblies with my visualizer but I want to use them only for release build, not for debug. So I created a folder named CustomAssemblies under visualizer solution. I added a reference path to this folder to all of the visualizer projects. Which means that the MSBuild will use assemblies in this folder if they are present or ones from GAC (the original ones) if the folder is empty. Unfortunatelly the reference paths are global to the project and you can't have two different sets for two different configurations.

So the building of the release version looks like: populate CustomAssemblies folder with custom DevExpress assemblies, run MSBuild on Release configuration and at the end clear the CustomAssemblies folder so the debug version works with the original DevExpress assemblies. But there is one more obstacle. The license.licx file lists public key of the DevExpress assemblies and it doesn't match the one found in custom version. So I have to replace all occurences of the original public key with my custom version public key before the build and restore the originals after the build. Problems solved.

The actual release process involves also {SmartAssembly} which merges all assemblies into a single file and signing it with a certificate+timestamping and finally zipping the rather large result. Because I am not a masochist I decided to create a FinalBuilder project that does all of this automatically for me (except for building custom DevExpress assemblies).

Let me know if there are still problems!

Righthand.DebuggerVisualizer.Dataset.2008_v1.0.1.zip (12.67 mb)

Righthand.DebuggerVisualizer.Dataset.2010_v1.0.1.zip (12.67 mb)

Read more about Righthand DataSet Visualizer here.

Righthand DataSet Visualizer now supports Visual Studio 2010

New in 0.9.16: added support for Visual Studio 2010 and updated user interface a bit. As before, everything is merged into a single dll file which is also digitally signed now.

Thanks RedGate {smartassembly} obfuscator tool for merging everything into a single DLL (ILMerge and another 3rd tool failed in this task). So far, I can only praise {smartassembly}.

Read more about Righthand DataSet Visualizer here.

Download the newest and older versions from download section.

Enjoy, and let me know whether you miss features or if you have any other feedback, good or bad.

An annoying non-persistent memory leak in .net framework MDI

I had to investigate a memory leak in an application I am building for my client. It is a MDI application using DevExpress XtraTabbedMdiManager that provides some MDI eyecandy. Anyway, I’ve used ANTS Memory Profiler (an excellent profiler, highly recommended) for my mission. I have soon found the cause of the true memory leak, which wasn’t an actual memory leak but rather a feature of the application – it was logging events in the memory.

One thing puzzled me though. I’ve tried this scenario. Take a memory snapshot, open a MDI child form, close it and take another snapshot. There shouldn’t be any memory leak, should it? But it was. ANTS Memory Profiler reported that the form wasn’t disposed and still held in memory. So I went looking at the Object Retention Graph for the form in question only to see this image (sensitive namespace is removed in Paint.net):

leak

Is is pretty much obvious that there are two references holding my should-be-disposed form. One is from InternalAccessibleObject and the other one is from Form.PropertyStore. Neither is caused by application’s code. So, what’s going on? It turns out that this is a feature of .net framework MDI and not a real memory leak – it stores the last reference of the active MDI child form or something like that and thus the last form isn’t released. If you repeat the open form/close step the memory leak shouldn’t increase. In fact even the memory leak from the first step is cured.

Even though it is not a true memory leak it is a distractive feature when it comes to memory profiling – for any .net memory profiler, not just ANTS. I guess only experience helps you with these kind of distractions when hunting for a real memory leak.

See also this thread in ANTS Memory Profiler support forum.

.net reflector pro is awesome

.net reflector

I am sure we all know and love .net reflector originally developed by Lutz Roeder and took over by fine folks at Red Gate. If you don’t know what .net reflector is or what it does you must take a look. It is an indispensible tool for understanding how a certain assembly (i.e. from .net framework or 3rd party) works – .net reflector does that by disassembling assemblies into C#/VB/IL/whatever (whatever code is achieved through a right plugin) code you want. And based on all this information it can provide you a ton of useful data, i.e. who uses which type, what types are derived from a type, etc.

.net reflector pro

And now there is a PRO version with a kick ass feature – it lets you step through source code even for referenced assemblies without sources while debugging an application under Visual Studio 2005/2008/2010. The functionality is similar to using symbols for same purposes. But it is a lot better because you aren’t constrained to assembly vendor as vendor has to provide symbol files for you to debug them. .net reflector pro does the trick for any assembly, regardless of the origin. True, it is useless with obfuscated assemblies and it doesn’t provide comments but hey – AFAIK right now there is only Microsoft providing some, not all, symbol files. You are out of luck for other Microsoft and 3rd party assemblies in this case.

Let me tell you an example. I’ve been bugging Developer Express guys for a while to provide symbol files and they probably will, but who knows when because this task is a low priority one for them. Now, with .net reflector pro I don’t need those files anymore nor I need any other 3rd party vendor symbol files.

The question is why would I want this feature at all?

This question should be asked only by a beginner. Everybody that did some serious development knows how important is to understand how a certain feature in a certain assembly is working, specially when you are presented an exception dialog and you have to understand what happened, what went wrong. Normally if you try digging the call stack to show code some code below your methods

call stack

(double click on a method where you don’t have source code – almost all call stack is gray meaning there is no source code available) you are presented with this informative dialog:

no source code

Go ahead, if you are an assembler guy, click Show Disassembly. For mere mortals disassembler code is useless.

How to use .net reflector pro

Download early access program version from .net reflector pro forum, unzip it somewhere and run reflector.exe. By running the executable for at least once you will register the Visual Studio addin that integrates .net reflector pro into Visual Studio. Without the addin registered nothing will happen in Visual Studio.

Run Visual Studio. Take note that there is a new root menu entry - .NET Reflector. Next create a test application – mine will be a WinForms one hosting a single SimpleButton from DevExpress. I’ll implement its Click event and put a breakpoint there. No additional code.

Run the application. If you have turned on Just My Code debugging setting then you’ll be prompted by a .net reflector dialog like this:

turnoffjustmycode

You have to turn off Just My Code feature. This is the same restriction as with symbol files. Then you’ll be presented another dialog where you can select which assemblies are you interested in:

assemblyselection

Select all assemblies. Once you hit OK button .net reflector will start decompiling assemblies by using all available cores on the computer (note that there are many entries in “In progress” state running in parallel).

decompiling

This is the first application I use that actually use all of my Core i7 4+4 cores. Good job and very smart because decompiling so many assemblies even in parallel takes a around 10 minutes on my Core i7 920 computer. But don’t fear – the results are cached and next run there is almost no performance hit at the start. That’s it for configuration and single window application will finally run.

The application will present a default window with a single button. Click on the button and execution will hit a breakpoint you’ve put in button’s Click handler. Look at the call stack again - it isn’t gray anymore:

newcallstack

And double clicking on the BaseButton.OnClick call stack entry opens decompiled source code from DevExpress assembly with everything except the Edit&Continue feature.

kickass

Let me repeat. The code above comes from an assembly decompilation and it is almost as useful as normal source code in debugger. You can inspect the variables and step forth and back but, of course, you can’t change it. Or better, you can change it but it won’t be recompiled.

No more black boxes! Now, if this isn’t an awesome feature, a must must have one, I don’t really know what such a feature is.

There is a slight drawback though: pro version won’t be free, which isn’t a big surprise and not a big issue. The regular version will continue to be free but it won’t have the *feature*. Also currently, being in beta, there are some bugs here and there which should disappear by RTM I guess.

More (official) information on this Alex Davies' blog post.

.net reflector pro is awesome

.net reflector

I am sure we all know and love .net reflector originally developed by Lutz Roeder and took over by fine folks at Red Gate. If you don’t know what .net reflector is or what it does you must take a look. It is an indispensible tool for understanding how a certain assembly (i.e. from .net framework or 3rd party) works – .net reflector does that by disassembling assemblies into C#/VB/IL/whatever (whatever code is achieved through a right plugin) code you want. And based on all this information it can provide you a ton of useful data, i.e. who uses which type, what types are derived from a type, etc.

.net reflector pro

And now there is a PRO version with a kick ass feature – it lets you step through source code even for referenced assemblies without sources while debugging an application under Visual Studio 2005/2008/2010. The functionality is similar to using symbols for same purposes. But it is a lot better because you aren’t constrained to assembly vendor as vendor has to provide symbol files for you to debug them. .net reflector pro does the trick for any assembly, regardless of the origin. True, it is useless with obfuscated assemblies and it doesn’t provide comments but hey – AFAIK right now there is only Microsoft providing some, not all, symbol files. You are out of luck for other Microsoft and 3rd party assemblies in this case.

Let me tell you an example. I’ve been bugging Developer Express guys for a while to provide symbol files and they probably will, but who knows when because this task is a low priority one for them. Now, with .net reflector pro I don’t need those files anymore nor I need any other 3rd party vendor symbol files.

The question is why would I want this feature at all?

This question should be asked only by a beginner. Everybody that did some serious development knows how important is to understand how a certain feature in a certain assembly is working, specially when you are presented an exception dialog and you have to understand what happened, what went wrong. Normally if you try digging the call stack to show code some code below your methods

call stack

(double click on a method where you don’t have source code – almost all call stack is gray meaning there is no source code available) you are presented with this informative dialog:

no source code

Go ahead, if you are an assembler guy, click Show Disassembly. For mere mortals disassembler code is useless.

How to use .net reflector pro

Download early access program version from .net reflector pro forum, unzip it somewhere and run reflector.exe. By running the executable for at least once you will register the Visual Studio addin that integrates .net reflector pro into Visual Studio. Without the addin registered nothing will happen in Visual Studio.

Run Visual Studio. Take note that there is a new root menu entry - .NET Reflector. Next create a test application – mine will be a WinForms one hosting a single SimpleButton from DevExpress. I’ll implement its Click event and put a breakpoint there. No additional code.

Run the application. If you have turned on Just My Code debugging setting then you’ll be prompted by a .net reflector dialog like this:

turnoffjustmycode

You have to turn off Just My Code feature. This is the same restriction as with symbol files. Then you’ll be presented another dialog where you can select which assemblies are you interested in:

assemblyselection

Select all assemblies. Once you hit OK button .net reflector will start decompiling assemblies by using all available cores on the computer (note that there are many entries in “In progress” state running in parallel).

decompiling

This is the first application I use that actually use all of my Core i7 4+4 cores. Good job and very smart because decompiling so many assemblies even in parallel takes a around 10 minutes on my Core i7 920 computer. But don’t fear – the results are cached and next run there is almost no performance hit at the start. That’s it for configuration and single window application will finally run.

The application will present a default window with a single button. Click on the button and execution will hit a breakpoint you’ve put in button’s Click handler. Look at the call stack again - it isn’t gray anymore:

newcallstack

And double clicking on the BaseButton.OnClick call stack entry opens decompiled source code from DevExpress assembly with everything except the Edit&Continue feature.

kickass

Let me repeat. The code above comes from an assembly decompilation and it is almost as useful as normal source code in debugger. You can inspect the variables and step forth and back but, of course, you can’t change it. Or better, you can change it but it won’t be recompiled.

No more black boxes! Now, if this isn’t an awesome feature, a must must have one, I don’t really know what such a feature is.

There is a slight drawback though: pro version won’t be free, which isn’t a big surprise and not a big issue. The regular version will continue to be free but it won’t have the *feature*. Also currently, being in beta, there are some bugs here and there which should disappear by RTM I guess.

More (official) information on this Alex Davies' blog post.

ANTS Memory Profiler 5 and its expiration message

I installed ANTS Memory Profiler 5 beta version a couple of weeks ago to try it out. Unfortunately I haven’t had time to really use it but it sure looks like a great memory profiler – easy to use and very powerful. So, I’ve started it again today only to get these two funny messages:

expiration1

expiration2

 

First one is really funny. Note that a newer build is available with bug fixes and upgrading is easy.

Here is a funny splash screen as well:

ants

It is really refreshing to see un-boring messages and splash screen. That said you should give it a try – you can get a public beta version from Red-gate’s forum.