Jinxing your application

If you ever wrote a multithreading application you should understand how hard is to get it right. If you don’t understand it then your application most probably isn’t written correctly.

You’ve written a multithreaded application and now what. How can you test it whether it is written correctly or not. Unit testing won’t be of great help because there are complex currency issues that might manifest in a bug only under certain circumstances. Imagine this piece of code:

class Program
{
    static int x;
    static Random rnd = new Random();

    static void Main(string[] args)
    {
        List<Thread> threads = new List<Thread>();
        for (int i = 0; i < 4; i++)
        {
            Thread t = new Thread(Runner) { IsBackground = true };
            t.Start();
            threads.Add(t);
        }
        foreach (Thread t in threads)
        {
            t.Join();
        }
    }

    static void Runner()
    {
        for (int i = 0; i < 1000000; i++)
        {
            Thread.Sleep(rnd.Next(10));
	   int orig = x;
            x += 5;
            Debug.Assert(x == orig+5);        
} } }

I am increasing a shared static variable from multiple threads without any synchronization. Will it work? It might or might not (try it!). It depends on when different threads are accessing the variable x. One thing is for sure, this code isn’t correct and it most probably won’t work and for sure it won’t work always. That is the biggest problem with multithreading – if something works it doesn’t mean that it it is correctly written and that it will work always. More about this later. If you use unit testing to test runner method the test will pass because unit testing doesn’t test multithreading, at least not easily.

So, how does one test such code and scenarios. One way is to use a static analysis tool. The other way is to put a jinx on your application. Once your application is jinxed it will be much more prone to displaying concurrency and other multithreading errors. And that’s exactly what Jinx does. Behind the scenes it makes your application fail more often that it would fail in normal circumstances. I mean that it shows faults in the application (if any) that would otherwise remain hidden and would occur only randomly here and there (you know, your user will find it after 2 minutes of running the application) – it doesn’t fail your application for no reason, it just emphasizes your bugs.

The most interesting aspect of Jinx is its the way it works. Jinx is a sort of hypervisor. You certainly know Hyper-V hypervisor whose task is to run guest operating systems. Jinx’s task is to make a clone of your OS and debugged application within and run multiple versions of it under various conditions. This is done so that any multithreading error is more likely to appear. Just by running more versions of the same application the error is more likely to manifest itself. But Jinx throws all sort of other jinxes to your application as well. That’s the shallow explanation. You’ll find more on official overview page and FAQ page.

So, let’s jinx the code above. Note that if you don’t want to you don’t need any modification of an existing code. Jinx can be set to do its task on any newly run application. However, some fine control is a better way to go. Here are the simple steps for selective jinxing:

  1. Reference jinxinterface assembly. It contains a single static JinxInteface class with a bunch of static methods and serves as a communication bridge between application and Jinx.
  2. Add jinx.cs file to your project. Again it contains a single static Jinx class that is a wrapper around JinxInteface class mentioned above. Its purpose is mostly to apply Conditional("DEBUG") over methods so they won’t get executed for non-debug version of the application.
  3. Call Jinx.RegisterApplication(); method at start of the application. This way you’ll let Jinx know that your application should use some jinxing.
  4. Replace Debug.Assert with Jinx.Assert. The only difference with both asserts is that the latter send statistical information to Jinx.

After the changes the code should look like this:

class Program
{
    static int x;
    static Random rnd = new Random();

    static void Main(string[] args)
    {
        Jinx.RegisterApplication();
        List<Thread> threads = new List<Thread>();
        for (int i = 0; i < 4; i++)
        {
            Thread t = new Thread(Runner) { IsBackground = true };
            t.Start();
            threads.Add(t);
        }
        foreach (Thread t in threads)
        {
            t.Join();
        }
    }

    static void Runner()
    {
        for (int i = 0; i < 1000000; i++)
        {
            Thread.Sleep(rnd.Next(10));
            int orig = x;
            x += 5;
            Jinx.Assert(x == orig + 5);
        }
    }
}

Before any analyzing takes place Jinx should be enabled and set – this is a system wide option. There are two ways to open the Jinx console – either through Tool/Jinx Visual Studio menu item or directly from All Programs via start menu. Either way you need Administrator privileges. Enabling is easy, just check Enable Jinx checkbox and that’s it. As per what programs are analyzed I’ll use “Analyze the most recent program I have registered.” option.

jinx

You can adjust some strategy settings on Strategy tab, I’ll skip this as it is an advanced option. And you can see the statistics on Status tab. Let’s run the application now. Jinx will kick in and the CPU will get under heavy load and the system might shutter due to Jinx running versions of the application in parallel. But the assert failure pops up almost immediately – the bug was caught for sure. If you run the application without jinxs there error would manifest much more later if at all and note that the test code in question is an extreme example. Here is the status tab page after a couple of errors caught:

status

The asserts observed counter is increased thanks to Jinx.Assert method call. Jinx analyzer isn’t limited to asserts and it might catch other type of errors as well according to PetraVM guys. Jinx is much more than this simple example.

So far I’ve run few examples like this and Jinx performed well and I think Jinx is a good weapon against multithreading bugs. However there might be a problem for testing. Since Jinx is a hypervisor on its own it won’t get along with other hypervisors such as Hyper-V. In other words forget about running Jinx on guest OS. A dedicated machine is required. Perhaps this issue will change in the future.

Also be careful when experimenting with Jinx as it is in beta phase right now (you can apply for testing over here). Running a beta hypervisor might result in a BSOD and all the consequences from BSOD such as non bootable Windows after. Which happened when I was writing this post. Perhaps this post is jinxed as well :-). Humor aside, I am sure guys behind Jinx will make it rock solid for the RTM. They obviously know very well the hypervisor craft.

Happy jinxing your applications!

Things that you didn’t know

I am working on a some sort of file cache system lately and I’ve encountered a bizarre error. When a certain file representing an image had to be deleted it wouldn’t let it – an IOException was thrown stating:

The process cannot access the file '[FILEPATH]' because it is being used by another process.

The odd thing is that I’ve used this file only for retrieve images from and I had no idea why would it be locked. Eventually I’ve traced the lock down to this line:

Image result = Image.FromFile(path);

Would you say that the line above would lock a file? I wouldn’t. But it does for some obscure reason. Thus switching to Image.FromStream did the trick:

using (FileStream stream = File.OpenRead(path))
{
    return Image.FromStream(stream);
}

It is more lines but it works as expected.

IMPORTANT UPDATE: The workaround described above only seems to work - it crashes randomly. By reading documentation (who does that anyway? - thanks Dario for the hint Smile) it states that the stream has to open for the lifetime of the Image. Ouch. So here is the second workaround that works for sure - I copy the data to a MemoryStream and create image from there. It is even more lines but this time it has to work (there is a Stream.CopyTo method in forthcomming .net 4.0).

using (FileStream stream = File.OpenRead(path))
{
    MemoryStream imageDataStream = new MemoryStream(Convert.ToInt32(stream.Length));
    byte[] buffer = new byte[4096];
    int read;
    while ((read = stream.Read(buffer, 0, buffer.Length)) != 0)
    {
        imageDataStream.Write(buffer, 0, read);
    }
    imageDataStream.Position = 0;
    return Image.FromStream(imageDataStream);
}

What’s new in the BCL in .net 4.0 beta 2

Check out this post about what’s new in the BCL in .net 4.0. beta 2. As you can see there is a good amount of timesaving functionality.

My favorites are Stream.CopyTo, String.Concat/Join overloads that take IEnumerable<T> (no more casting IList to Array!) and of course Enum.HasFlag which I already implemented it my way.

I wonder though why they created Environment.Is64BitProcess and Is64BitOperatingSystem. At the present time it makes sense but Microsoft is already talking about 128 bit Windows OS. We’ll get, I suppose, a couple of years after a 128 bit Windows is released new properties: Environment.Is128BitProcess and Is128BitOperatingSystem and years after that perhaps 256 bit versions of them.

So, wouldn’t it be easier to have an enum like:

enum Bitness
{
    16Bit,
    32Bit,
    64Bit
}

And when 128 bit version comes out a new value of 128Bit might be added. So we might have Environment.ProcessBits and Environment.OSBits or something that would return Bitness enum, would be future proof and easier to use than inspecting all IsXXXBitXXX methods.

Or perhaps, if adding a new value to an enum poses some kind of problem, just return an int instead of an enum representing the number of bits.

Visual Studio 2010 beta 2 and .net 4.0 beta 2 available on MSDN

Both Visual Studio 2010 beta 2 and .net 4.0 beta 2 are available for MSDN subscribers and on Wednesday for everybody. Perhaps an important feature is that a “go live” license is available.

Release has been announced for March 22th next year. I guess RTM will be available before the year’s end for MSDN subscribers – judging from the Visual Studio 2008 timeline.

Here is a bounch of useful beta 2 launch blog posts:

About TFS 2010

Hanselman’s post

Jason Zander’s post

What’s new for the Task Parallel Libary

ScottGu’s post

and of course Somma’s post

Just don’t forget that this is beta 2 release and you should be cautious – installing it in a virtual machine rather then production one is always a good practice when dealing with early builds.

Dealing with iterations over null lists in LINQ to Objects

Problem

If you used LINQ to Objects you have certainly come across iterations over null values resulting in an ArgumentNullException being thrown at you.

See this example:

int[] b = null;
var query = from i in b select i;

In the case above the error is obvious but if the list comes as an argument of a method or some more complicated way it is hard to spot it. Sure, the obvious solution is an easy one. A simple if statement before will do it:

if (b != null)
    var query = from i in b select i;

What about nested loops? Like this

class Tubo
{
public List&lt;string> Strings;
}
...
IList&lt;Tubo> tubos = ...;
var query = from t in tubos
from s in t.Strings
select s;

Sure, we could add guardian clauses like the if before:

if (tubos != null)
     var query = from t in tubos
        where t.Strings != null
        from s in t.Strings
        select s;

The problem with this approach is that it gets cluttered and it complicates the flow, specially the first if.

Solution

So, here is my proposal and I am sure it has been proposed before (I just couldn’t find it on Google, err, I mean internet).

public static class Extension
{
public static IList&lt;T> Safe&lt;T>(this IList&lt;T> source)
{
if (source == null)
return new T[0];
else return source;
}
}

The extension method makes sure that query always gets a non-null list by making an empty one if it is null. The trick with extension method is that they can be invoked on null values which are passed as this argument.

And the last nested LINQ to Objects loop would look like:

var query = from t in tubos.Safe()
             from s in t.Strings.Safe()
             select s;

I am not sure that the extension method name Safe is adequate or not but it sure does help in code readability.

What do you say?

.net reflector pro is awesome

.net reflector

I am sure we all know and love .net reflector originally developed by Lutz Roeder and took over by fine folks at Red Gate. If you don’t know what .net reflector is or what it does you must take a look. It is an indispensible tool for understanding how a certain assembly (i.e. from .net framework or 3rd party) works – .net reflector does that by disassembling assemblies into C#/VB/IL/whatever (whatever code is achieved through a right plugin) code you want. And based on all this information it can provide you a ton of useful data, i.e. who uses which type, what types are derived from a type, etc.

.net reflector pro

And now there is a PRO version with a kick ass feature – it lets you step through source code even for referenced assemblies without sources while debugging an application under Visual Studio 2005/2008/2010. The functionality is similar to using symbols for same purposes. But it is a lot better because you aren’t constrained to assembly vendor as vendor has to provide symbol files for you to debug them. .net reflector pro does the trick for any assembly, regardless of the origin. True, it is useless with obfuscated assemblies and it doesn’t provide comments but hey – AFAIK right now there is only Microsoft providing some, not all, symbol files. You are out of luck for other Microsoft and 3rd party assemblies in this case.

Let me tell you an example. I’ve been bugging Developer Express guys for a while to provide symbol files and they probably will, but who knows when because this task is a low priority one for them. Now, with .net reflector pro I don’t need those files anymore nor I need any other 3rd party vendor symbol files.

The question is why would I want this feature at all?

This question should be asked only by a beginner. Everybody that did some serious development knows how important is to understand how a certain feature in a certain assembly is working, specially when you are presented an exception dialog and you have to understand what happened, what went wrong. Normally if you try digging the call stack to show code some code below your methods

call stack

(double click on a method where you don’t have source code – almost all call stack is gray meaning there is no source code available) you are presented with this informative dialog:

no source code

Go ahead, if you are an assembler guy, click Show Disassembly. For mere mortals disassembler code is useless.

How to use .net reflector pro

Download early access program version from .net reflector pro forum, unzip it somewhere and run reflector.exe. By running the executable for at least once you will register the Visual Studio addin that integrates .net reflector pro into Visual Studio. Without the addin registered nothing will happen in Visual Studio.

Run Visual Studio. Take note that there is a new root menu entry - .NET Reflector. Next create a test application – mine will be a WinForms one hosting a single SimpleButton from DevExpress. I’ll implement its Click event and put a breakpoint there. No additional code.

Run the application. If you have turned on Just My Code debugging setting then you’ll be prompted by a .net reflector dialog like this:

turnoffjustmycode

You have to turn off Just My Code feature. This is the same restriction as with symbol files. Then you’ll be presented another dialog where you can select which assemblies are you interested in:

assemblyselection

Select all assemblies. Once you hit OK button .net reflector will start decompiling assemblies by using all available cores on the computer (note that there are many entries in “In progress” state running in parallel).

decompiling

This is the first application I use that actually use all of my Core i7 4+4 cores. Good job and very smart because decompiling so many assemblies even in parallel takes a around 10 minutes on my Core i7 920 computer. But don’t fear – the results are cached and next run there is almost no performance hit at the start. That’s it for configuration and single window application will finally run.

The application will present a default window with a single button. Click on the button and execution will hit a breakpoint you’ve put in button’s Click handler. Look at the call stack again - it isn’t gray anymore:

newcallstack

And double clicking on the BaseButton.OnClick call stack entry opens decompiled source code from DevExpress assembly with everything except the Edit&Continue feature.

kickass

Let me repeat. The code above comes from an assembly decompilation and it is almost as useful as normal source code in debugger. You can inspect the variables and step forth and back but, of course, you can’t change it. Or better, you can change it but it won’t be recompiled.

No more black boxes! Now, if this isn’t an awesome feature, a must must have one, I don’t really know what such a feature is.

There is a slight drawback though: pro version won’t be free, which isn’t a big surprise and not a big issue. The regular version will continue to be free but it won’t have the *feature*. Also currently, being in beta, there are some bugs here and there which should disappear by RTM I guess.

More (official) information on this Alex Davies' blog post.

The slides and code from my “Making asp.net mvc applications strong typed” presentation

Yesterday I held a presentation (as the part of Bleeding Edge 2009 conference) on how to make ASP.NET MVC applications strong typed by using CodeSmith and CodeRush (actually by using its free DXCore part). Attendees were great and the presentation went well. Attached are the slides in Slovene and source code in C#.

If you are interested in the topic you might read my previous blog posts as well:

Thanks everybody for attending the presentation.