When .net assemblies aren’t properly versioned odd errors happen

Today I’ve upgraded a SignalR library from 1.x to 2.0 in an MVC 5 (server) and WPF (client) projects I am developing. The upgrade itself went smooth but afterwards the client started yielding odd errors and refusing to connect anymore. It was yielding an exception saying something like Transport Connection time out (I can’t remember the exact message). Since it was a rather simple upgrade this error made no sense. Google didn’t help much either.

So I went debugging.

Investigation

First step is to create a simple repro sample – a solution with a basic server and a console client (see attached zip).

Second step is to turn on SignalR tracing on client:

hubConnection.TraceLevel = TraceLevels.All;
hubConnection.TraceWriter = Console.Out;

Armed with these two steps I was able to get an error that made a bit more sense although it was even more bizzare.

18:43:53.9073548 - null - ChangeState(Disconnected, Connecting)
18:43:55.4913444 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - WS Connecting to: ws://localhost:53705/signalr/connect?transport=webSockets&connectionToken=0dQ3scw2aJ7RbXuUsTfa4wlY%2FJlMzW%2FVL1sVb%2FyswewEO8n4qAth7Erpt0ga0laLV%2B8BCD833lZZy1MblOe0HuQs0O1mYi%2FfiiLSHc5%2F%2B1wk6tUsbFxMKVQFwRO9BvSW&connectionData=[{"Name":"formsHub"}] 18:43:55.5800651 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - WS: OnMessage({"C":"d-5AB67EA2-B,0|C,0|D,1|E,0","S":1,"M":[]})
18:43:55.5940751 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - OnError(System.MissingMethodException: Method not found: 'System.Collections.Generic.IEnumerator`1 Newtonsoft.Json.Linq.JArray.GetEnumerator()'. at Microsoft.AspNet.SignalR.Client.Transports.TransportHelper.ProcessResponse(IConnection connection, String response, Boolean& shouldReconnect, Boolean& disconnected, Action onInitialized) at Microsoft.AspNet.SignalR.Client.Transports.WebSocketTransport.OnMessage(String message) at Microsoft.AspNet.SignalR.WebSockets.WebSocketHandler.d__e.MoveNext())
18:43:55.6110864 - b8fd1874-483d-4d89-8a69-c2cfe33cf946 - WS: OnClose()

It says something like – I can’t find a method in the Newtonsoft.Json assembly. That’s totally weird as this kind of error should be handled at compile time.

I went looking into Newtonsoft.Json assembly referenced by SignalR 2.0. It correctly references version 5.0.8. The assembly in package folder has the method it was mentioned as missing. Mystery. One thing that caught my eye  while .net reflecting the assembly was its version. While the file version is 5.0.8. the assembly version is, oddly, 4.5.0.0. How come?

Fourth step was looking into modules loaded by my application. They are listed through Visual Studio-Debug/Windows/Modules. Amazingly, Visual Studio was listing Newtonsoft.Json v4.05.8.15203 from … GAC. Not from my package. And by looking at its date it was clear that it was an older version (4.8.2012). Heck, I didn’t even know it was in GAC. I didn’t put it, some other app had to.

 

From previous Googling I remembered an explanation somewhere that the missing GetEnumerator method was introduced in version 5.0.5. So this was probably it – .net was loading an older version from GAC - that was really missing that method. At this point I knew what was wrong.

Just in case, to make sure, I checked whether the assembly really was in GAC:

C:\Windows\system32>gacutil /l newtonsoft.json
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.18020
Copyright (c) Microsoft Corporation.  All rights reserved.

The Global Assembly Cache contains the following assemblies:
  newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchi
tecture=MSIL

Number of items = 1

It was.

Remedy

The proper way would be to have a properly versioned Newtonsoft.Json assembly (those are strong key signed as well). That’s why there is versioning and .net wouldn’t make this confusion. I really wonder why James uses the same assembly version for assemblies that should really have different signatures. Hopefully he will fix this problem in next versions. But until then there is …

… the next best way is to force .net to not load it from GAC.  If the assembly is removed from GAC, .net won’t find the wrong version anymore. See, .net always tries GAC first and only if it doesn’t find a suitable assembly it will go looking elsewhere, like into the folder where your application is. Perhaps I could configure it to load the proper assembly through .config file but this solution is better – it’ll solve the problem unless an older newtonsoft.json is reinstalled again in GAC. Hm.

Removing assembly from GAC should be a rather simple process. Just fire

GACUTIL /U Newtonsoft.Json

and it should be removed. But not this time. Instead I got this warning:

C:\Windows\system32>gacutil /u newtonsoft.json
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.18020
Copyright (c) Microsoft Corporation.  All rights reserved.


Assembly: newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, proces
sorArchitecture=MSIL
Unable to uninstall: assembly is required by one or more applications
Pending references:
              SCHEME: <WINDOWS_INSTALLER>  ID: <MSI>  DESCRIPTION : <Windows Installer>
Number of assemblies uninstalled = 0
Number of failures = 0

Basically it is saying that some installed application did install it into GAC and now it is preventing me from removing it from GAC. Uf, nasty. To make it worse, it doesn’t say which application. This article lists the way to “unreference it” to be able to remove it from GAC. Basically you have to remove its entry from the registry and then execute gacutil removal command again.

 

After registry modification it succeeded.

C:\Windows\system32>gacutil /u newtonsoft.json
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.18020
Copyright (c) Microsoft Corporation.  All rights reserved.


Assembly: newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, proces
sorArchitecture=MSIL
Uninstalled: newtonsoft.json, Version=4.5.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, pro
cessorArchitecture=MSIL
Number of assemblies uninstalled = 1
Number of failures = 0

The aftermath

Removing that old Newtonsoft.Json from registry did the trick. SignalR works like charm again. However, I had to manually remove a component of an unidentified application. And this removal might cause problems with that application in the future.

I really hope that Newtonsoft.Json gets proper versioning to avoid situations like this. This should be a lesson to every library developer out there – do use proper version numbering!.

SignalRTest.zip (4.69 mb)

UPDATES:

12.11.2013: James, the author, is looking at the possible solution. See https://json.codeplex.com/discussions/465332#post1122165.

Making JSON data strong typed with TypeScript and CodeSmith

The situation

Imagine a scenario where you have a JSON service that returns data and you'd like to have it strongly typed on javascript powered client side.

Let's say there is an ASP.NET MVC application that goes by the name MvcApplication33 (yes, the 33rd in a row of test applications) . There are two models in there in two files under Models folder:

namespace MvcApplication33.Models
{
    public class Customer
    {
        public string Name { get; set; }
        public string Surname { get; set; }
        public int Age { get; set; }
        public Order[] Orders { get; set; }
    }
}

namespace MvcApplication33.Models
{
    public class Order
    {
        public double Amount { get; set; }
        public string Category { get; set; }
        public bool IsActive { get; set; }
    }
}

There is also a ApiController derived CustomersController like:

namespace MvcApplication33.Controllers
{
    public class CustomersController: ApiController
    {
        public IEnumerable<Customer> GetCustomers()
        {
            return new Customer[]{ new Customer
            {
                Name = "Tubo",
                Age = 22,
                Orders = new[]{ new Order { Amount = 54, IsActive = true, Category = "waste"} }
            }};
        }
    }
}

It simply returns an Customer array consisting of a single customer with a single order. The javascript, well jQuery, code that gets this data would look like:

$(function () {
        $.getJSON("/api/customers", null, function (d) {
            var jsonCustomer = d;
        });
    });

The problem

While the code above works there is a drawback (mostly for people coming from strong typed languages): on the client side there is no metadata information at design time whatsoever. One is left with dynamic constructs. TypeScript addresses this with interfaces. One could write matching TypeScript interfaces for C# types like:

interface ICustomer {
    Name: string;
    Surname: string;
    Age: number;
    Orders: IOrder[];
}
interface IOrder {
    Amount: number;
    Category: string;
    IsActive: bool;
}

and then rewrite retrieval like

$(function () {
    $.getJSON("/api/customers", null,
        d =>
        {
            var customer = <ICustomer[]>d;
        });
});

This way customer becomes an instance of a type that implements ICustomer which means properties are now strong typed and intellisense works. Of course this is only TypeScript design time sugar which doesn't reflect in generated javascript code but that's enough to prevent a zillion of typing and other "easy to catch at design time" errors.

There is one problem though. Typing interfaces is boring, error prone and there are synchronization issues between manually typed ones and their C# originals.

The solution

There is already all metadata information for our interfaces in C# code. Hence I created a CodeSmith template that automatically creates TypeScript interfaces based on their C# originals by parsing C# code. When C# code changes the template can be rećrun to recreate interfaces. Almost one click error-less operation.

Here is how it works: the template parses all C# files in a given folder and subfolders and generates matching TypeScript interfaces with corresponding namesapaces (using TypeScript modules). The template output for the given problem would look like:

// Autogenerated using CodeSmith and JsonGenerator
// © Righthand, 2013
module MvcApplication33.Models {
export interface ICustomer {
    Name: string;
    Surname: string;
    Age: number;
    Orders: IOrder[];
}
export interface IOrder {
    Amount: number;
    Category: string;
    IsActive: bool;
}

    export module Subform {
        export interface ISubclass {
            Tubo: bool;
        }

    }
}

Just for testing nested interfaces there is also interface ISubclass from original file located in Models subfolder named (Subform).

 

The TypeScript file that uses the mentioned autogenerated interfaces should reference autogenerated interfaces file using <reference> directive. Here is a sample TypeScript test:

/// <reference path="typings/jquery/jquery.d.ts" />
/// <reference path="../CodeSmith/JsonInterfaces.ts" />
module KnockoutTest {
    $(function () {
        $.getJSON("/api/customers", null,
            d =>
            {
                var customers = <MvcApplication33.Models.ICustomer[]>d;
                alert(customer[0].Name);
            });
    });
}

How to

The template comes in two parts. An actual CodeSmith template (JsonInterfaces.cst) and a .net assembly (KnockoutGenerator.Extractor.Parser.dll - name should give you a hint where all this is going in a next blog post) which is used to extract metadata from C# sources. The assembly code could be a part of CodeSmith template but it is easier to develop (read: debug) code within Visual Studio. The assembly in turn uses NRefactory (free C# parser and much more) which (v4.x) is a part of CodeSmith, so no additional files required.

Perhaps the easiest way to use the template is to include these two files in a project and use CodeSmith from within Visual Studio. The template requires a single property: FolderWithModels which is rather self explanatory. Excerpts from attached sample project:

The CodeSmith project content located in Project1.csp:

<?xml version="1.0" encoding="utf-8"?>
<codeSmith xmlns="http://www.codesmithtools.com/schema/csp.xsd">
  <propertySets>
    <propertySet name="JsonInterfaces" output="JsonInterfaces.ts" template="JsonInterfaces.cst">
      <property name="FolderWithModels">..\Models</property>
    </propertySet>
  </propertySets>
</codeSmith>

The project structure. I tend to put CodeSmith related stuff in CodeSmith folder. Feel free to arrange it otherwise.

folder structure

Right click on Project1.csp and Generate Outputs should (re)generate JsonInterfaces.ts. Open the output file and if Web Essentials and TypeScript are installed it should (re)generate final JavaScript file.

The project itself won't show any meaningful output (for now) but you can use JSON output in a strongly typed way.

Have fun and read next post which will talk about further enhancements for knockoutjs.

You can download my sample project (without NuGet packages, just sources) which includes everything you need (subfolder CodeSmith).

MvcApplication33.zip (566.78 kb)

NOTE: The CodeSmith template could be rewritten to T-4 (using the same support assembly) however, one should make sure that NRefactory assemblies are there otherwise parser won't work.