Making JSON data strong typed with TypeScript and CodeSmith

The situation

Imagine a scenario where you have a JSON service that returns data and you'd like to have it strongly typed on javascript powered client side.

Let's say there is an ASP.NET MVC application that goes by the name MvcApplication33 (yes, the 33rd in a row of test applications) . There are two models in there in two files under Models folder:

namespace MvcApplication33.Models
{
    public class Customer
    {
        public string Name { get; set; }
        public string Surname { get; set; }
        public int Age { get; set; }
        public Order[] Orders { get; set; }
    }
}

namespace MvcApplication33.Models
{
    public class Order
    {
        public double Amount { get; set; }
        public string Category { get; set; }
        public bool IsActive { get; set; }
    }
}

There is also a ApiController derived CustomersController like:

namespace MvcApplication33.Controllers
{
    public class CustomersController: ApiController
    {
        public IEnumerable<Customer> GetCustomers()
        {
            return new Customer[]{ new Customer
            {
                Name = "Tubo",
                Age = 22,
                Orders = new[]{ new Order { Amount = 54, IsActive = true, Category = "waste"} }
            }};
        }
    }
}

It simply returns an Customer array consisting of a single customer with a single order. The javascript, well jQuery, code that gets this data would look like:

$(function () {
        $.getJSON("/api/customers", null, function (d) {
            var jsonCustomer = d;
        });
    });

The problem

While the code above works there is a drawback (mostly for people coming from strong typed languages): on the client side there is no metadata information at design time whatsoever. One is left with dynamic constructs. TypeScript addresses this with interfaces. One could write matching TypeScript interfaces for C# types like:

interface ICustomer {
    Name: string;
    Surname: string;
    Age: number;
    Orders: IOrder[];
}
interface IOrder {
    Amount: number;
    Category: string;
    IsActive: bool;
}

and then rewrite retrieval like

$(function () {
    $.getJSON("/api/customers", null,
        d =>
        {
            var customer = <ICustomer[]>d;
        });
});

This way customer becomes an instance of a type that implements ICustomer which means properties are now strong typed and intellisense works. Of course this is only TypeScript design time sugar which doesn't reflect in generated javascript code but that's enough to prevent a zillion of typing and other "easy to catch at design time" errors.

There is one problem though. Typing interfaces is boring, error prone and there are synchronization issues between manually typed ones and their C# originals.

The solution

There is already all metadata information for our interfaces in C# code. Hence I created a CodeSmith template that automatically creates TypeScript interfaces based on their C# originals by parsing C# code. When C# code changes the template can be rećrun to recreate interfaces. Almost one click error-less operation.

Here is how it works: the template parses all C# files in a given folder and subfolders and generates matching TypeScript interfaces with corresponding namesapaces (using TypeScript modules). The template output for the given problem would look like:

// Autogenerated using CodeSmith and JsonGenerator
// © Righthand, 2013
module MvcApplication33.Models {
export interface ICustomer {
    Name: string;
    Surname: string;
    Age: number;
    Orders: IOrder[];
}
export interface IOrder {
    Amount: number;
    Category: string;
    IsActive: bool;
}

    export module Subform {
        export interface ISubclass {
            Tubo: bool;
        }

    }
}

Just for testing nested interfaces there is also interface ISubclass from original file located in Models subfolder named (Subform).

 

The TypeScript file that uses the mentioned autogenerated interfaces should reference autogenerated interfaces file using <reference> directive. Here is a sample TypeScript test:

/// <reference path="typings/jquery/jquery.d.ts" />
/// <reference path="../CodeSmith/JsonInterfaces.ts" />
module KnockoutTest {
    $(function () {
        $.getJSON("/api/customers", null,
            d =>
            {
                var customers = <MvcApplication33.Models.ICustomer[]>d;
                alert(customer[0].Name);
            });
    });
}

How to

The template comes in two parts. An actual CodeSmith template (JsonInterfaces.cst) and a .net assembly (KnockoutGenerator.Extractor.Parser.dll - name should give you a hint where all this is going in a next blog post) which is used to extract metadata from C# sources. The assembly code could be a part of CodeSmith template but it is easier to develop (read: debug) code within Visual Studio. The assembly in turn uses NRefactory (free C# parser and much more) which (v4.x) is a part of CodeSmith, so no additional files required.

Perhaps the easiest way to use the template is to include these two files in a project and use CodeSmith from within Visual Studio. The template requires a single property: FolderWithModels which is rather self explanatory. Excerpts from attached sample project:

The CodeSmith project content located in Project1.csp:

<?xml version="1.0" encoding="utf-8"?>
<codeSmith xmlns="http://www.codesmithtools.com/schema/csp.xsd">
  <propertySets>
    <propertySet name="JsonInterfaces" output="JsonInterfaces.ts" template="JsonInterfaces.cst">
      <property name="FolderWithModels">..\Models</property>
    </propertySet>
  </propertySets>
</codeSmith>

The project structure. I tend to put CodeSmith related stuff in CodeSmith folder. Feel free to arrange it otherwise.

folder structure

Right click on Project1.csp and Generate Outputs should (re)generate JsonInterfaces.ts. Open the output file and if Web Essentials and TypeScript are installed it should (re)generate final JavaScript file.

The project itself won't show any meaningful output (for now) but you can use JSON output in a strongly typed way.

Have fun and read next post which will talk about further enhancements for knockoutjs.

You can download my sample project (without NuGet packages, just sources) which includes everything you need (subfolder CodeSmith).

MvcApplication33.zip (566.78 kb)

NOTE: The CodeSmith template could be rewritten to T-4 (using the same support assembly) however, one should make sure that NRefactory assemblies are there otherwise parser won't work.

Integrating MvcMiniProfiler and LLBLGenPro

MvcMiniProfiler is a lean and mean mini profiler for MVC 3 that shows the profiling results on each page displayed at runtime. Besides the custom steps you can seed everywhere in your code it supports database profiling as well. Out of the box are supported ado.net, Linq to SQL and Entity Framework. LLBLGenPro, my favorite ORM, isn’t supported though and it won’t work just like that.

Luckily, it turns out, it requires just a little effort to integrate MvcMiniProfiler into LLBLGenPro.

How does MvcMiniProfiler database profiling works

The way it works is that it wraps DbConnection, DbCommand and other Db[Stuff] and thus records the execution time by tracking their inner workings. Here is an example for MvcMiniProfiler documentation about how to start:

public static DbConnection GetOpenConnection()
{
    var cnn = CreateRealConnection(); // A SqlConnection, SqliteConnection ... or whatever

    // wrap the connection with a profiling connection that tracks timings 
    return MvcMiniProfiler.Data.ProfiledDbConnection.Get(cnn, MiniProfiler.Current);
}

If client calls DbConnection.CreateCommand on an ProfiledDbConnection instance returned from previous method it will get a wrapped whatever command original connection returns and so on. There is also a way to manually create DbCommand through ProfiledDbCommand constructor.

The support for Linq To SQL and Entity Framework is done in a similar manner.

This gets us to the point, why can’t I just use the same approach with LLBLGenPro?

Integrate MvcMiniProfiler with LLBLGenPro – why doesn’t work with same approach

The major problem with LLBLGenPro and MvcMiniProfiler integration is that LLBLGenPro doesn’t use DbConnection.CreateCommand method to create commands from existing connection. Instead it creates an instance of proper DbCommand derived class and assigns a connection to it. Thus it won’t work because it would try to assign a ProfiledDbConnection to a i.e. SqlCommand class.

So a bit more work is required to match them.

The code for adapter scenario

1. Create a DynamicQueryEngine derived class. Note: this class is database specific, thus if you work with i.e. SQL Server you’ll find it in SD.LLBLGen.Pro.DQE.SqlServer.NET20.dll assembly.

public class ProfilingDynamicQueryEngine : DynamicQueryEngine
{
    protected override DbCommand CreateCommand()
    {
         DbCommand cmd = base.CreateCommand();
         ProfiledDbCommand pCmd = new ProfiledDbCommand(cmd, null, MiniProfiler.Current);
         return pCmd;
    }
}

Here the DbCommand creation is overriden. Note that I wrap the original cmd and pass a current MiniProfiler instance as arguments to ProfiledDbCommand constructor, while I pass a null for the connection instance because it will be assigned later.

2. Derive from DataAccessAdapter class. Note: this class is generated from a template and you’ll find it in DBSpecificLayer project generated by LLBLGenPro.

public class DataAccessAdapterEx: DataAccessAdapter
{    
    protected override System.Data.Common.DbConnection CreateNewPhysicalConnection(string connectionString)
    {
        DbConnection conn = base.CreateNewPhysicalConnection(connectionString);
        // return ProfiledDbConnection.Get(conn); Pre MvcMiniProfiler 1.9
        return new ProfiledDbConnection(conn, MiniProfiler.Current);
} protected override DynamicQueryEngineBase CreateDynamicQueryEngine() { return PostProcessNewDynamicQueryEngine(new ProfilingDynamicQueryEngine()); } }

Within CreateDynamicQueryEngine I pass the class I’ve created in step #1. CreateNewPhysicalConnection will return a wrapped connection.

Instead of using DataAccessAdapter you should use the one created in step #2 - DataAccessAdapterEx. That’s it.

Conclusion

As it turns out, integrating MvcMiniProfiler with LLBLGenPro is quite easy. And the required coding might be added to LLBLGenPro templates by modifying them, so you won’t have to manually add the same code each time.

Let me know if you have feedback.

Update 19.9.2011: Updated the code because MvcMiniProiler introduced a breaking change in v1.9 (instead of ProfiledDbConnection.Get static method a constructor has to be used - thanks for swift response from David from LLBLGenPro support team)

SIOL (IPTV provider) managed to upset its customers once more

I have an IPTV provided by biggest national IPTV/IS provider. The current service is poor at best. We have FTTH (the slowest is 20Mb - that's awesome) but really lousy (~4Mb at best) MPEG2(!) encoded streams. The most obvious consequence of such encoding is pixelation when there is a faster change on the picture and poor quality in general. To make things worse there is a crappy, yet somewhat functional UI that lets you select channels and read EPG. It mostly works (except for occasional ASP.NET 1.1 server too busy error) and its speed is decent. IMO it mostly lacks two features:

  • one can't browse other channel's EPG (electronic program guide) without actually changing channel (IOW watch a channel and read other channel's EPG). I once asked SIOL about this and the response was: "Check out the teletext - you can't perform this stunt in teletext either". So much about understanding the technology progress.
  • one can't mark favorite shows and have an option of notification before the show

Now, both features are minor, yet SIOL isn't capable of implementing them. Since the UI is built around ASP.NET 1.1 (as seen on occasional errors) it shouldn't take more than a week to implement the two features. But they didn't change the UI for years now. Except for upgrading the progress icon to an animated gif.

Then, one day, they started a SIOL TV+ UI. (note the + char). It features a newly organized and richer UI, plus VOD and a recorder through UI (storage is provided on SIOL's server). Sounds great, right? Not at all. Here is the list of consequences:

  1. You have to buy a newer STB (or get one for free if you sign with SIOL for 2 years)
  2. UI is slow as hell. I mean really really slow at the point where it is unusable.
  3. UI is cluttered as hell - it is really bad and a total mess.
  4. It doesn't have favorite channels group.
  5. Recorder doesn't work and even when it will it will be useless: it is enabled only for a couple of Slovene channels (due to legal issues) and those shows will be persisted only for at most 2 days (the total place for storage per user is 6 hrs in total) . It won't be free as well since it uses SIOL's storage. There are other problems with recorder not listed but the question is why they bother at all with such crap service?
  6. VOD: after you pay for the movie you have 24 hours to watch it. The library is poor, at least the last time I've checked it.

True, I didn't check it lately. But for a reason. There were reports that lately people who checked out the new UI couldn't go back to the old, functional, one. The two UIs could coexist until recently.

And now, actually 10 days from now, SIOL is disabling the old UI for good and forcing everybody to new great new one - with or without the new STB. That's indeed a great move, kill the customer's will to watch TV, way to go! On the other side I'll watch less TV and spend more time with other activities I guess.

ASPxGridView, MS Ajax and XYDataSource

If you use [DevEx] ASPxGridView within MS Ajax' UpdatePanel (ASPxGridView.EnableCallBacks="False") you should be aware that you should perform DataBind() method within OnInit method (Init event). Otherwise editing just won't work, or better, it works, just the modifications aren't persisted. It took me some time to pinpoint the problem as ASPxGridView worked just fine outside UpdatePanel. I had to put it within UpdatePanel because I needed to refresh other controls as well (otherwise grid refreshes just itself).

Note that when ASPxGridView is hosted in UpdatePanel the nice error reporting feature (see the picture below) won't work either - instead you'll get script error reported by browser. IOW you have to handle errors by yourself.

image

Cool automatic error feedback

See also this support thread.

The time of &quot;Vote for XY product&quot; is here again

asp.netPRO started voting process for asp.netPRO Readers' Choice Awards 2008 as they do every year. And all of the 3rd party component vendors are asking for your vote, as usual. Nothing wrong there. But there are two things that I noticed during my voting. First, immediately on the top of first page, I saw this product:

image

This refactor tool was around at the time of Visual Studio 2002/2003 and died soon after, or better it froze. The web site is still there and you can even download it for free (Order tab). It was a great product at that time, offering refactoring when nobody else did. However the product is dead as the dodo for many years now. So I wonder, why is such product even listed as a choice?

The other odd thing is the categories. For example, Component Set category lists a lot of different sets. I mean different like apples and hard disks. How can one compare those? Or Utility category where the difference is even huger - looks like basically everything that didn't fit in other categories landed here. Go ahead and compare Aqtime profiler with dtSearch with LLBLGen Pro. Which is better? I think the products with larger client base will get more votes simply because it is natural that you would vote for product you own in such case.

What gives? The results in at least the two categories mentioned can't be relevant if you ask me, or better, they'll just reflect the client base size of each product.

telerik supporting Mono?

Looks like telerik is going to support Mono. This news is certainly interesting for Mono guys as it might give wings to Mono. Which is a good thing, indeed. However, the support (if it is going to happen) will probably be limited to asp.net components - they don't even mention WinForms at this point. I assume that running WinForms controls on Mono is a different story because WinForms controls are still pretty much tied to Windows API for performance reasons. I wonder if the situation will change with adoption of WPF where there is no need to hook into Windows API directly.

UPDATE: This news is not new news but old news (2 years).

Slovenia/1st European Silverlight Challenge

SLODUG has launched the Slovene website of the 1st European Silverlight Challenge. You can read more about it in this post (in Slovene) or go directly to the website (also in Slovene).

Believe in your talent! Take part in the “European Silverlight Challenge” Competition – Dare to participate in the European Silverlight Development Competition. Win the recognition that goes with first place and, of course, fabulous prizes! (or 'swag' as our UK friends would say)

Note that words "fabulous prizes" are actually, thanks to the generous sponsors, an understatement.

So, hurry up, read the rules, apply and submit a silverlight application before the 28th January 2008.

Custom embedding using CodeRush

It happened that I had to embed plenty of asp.net elements into tables, i.e:

<table> <tr> <td> <asp:Label ... /> <asp:TextBox ... /> </td> <td> <asp:Label ... /> <asp:TextBox ... /> </td> ... </tr> </table>

Basically I had to put every Label/TextBox pair into its own table. The outcome has to be something like this:

<table> <tr> <td> <table> <tr> <td> <asp:Label ... /> </td> <td> <asp:TextBox ... /> </td> </tr> </table> </td> <td> <table> <tr> <td> <asp:Label ... /> </td> <td> <asp:TextBox ... /> </td> </tr> </table> ... </tr> </table>

And there were plenty of such pairs. So I was facing tedious manual repetitive work every decent developer tends to avoid at all costs. My first though was that it might be done with [CodeRush]. After all it has embedding functionality. Not only it comes with predefined set of embeddings - [CodeRush] allows you to easily create your own embeddings easily. But first let take a look how embeddings are accessible. There are two main ways. First way is through context menu in Visual Studio editor after selecting one or more lines:

image

As you can see there are plenty of embeddings out of the box. There is another way to trigger embedding functionality: through shortcuts (i.e. CTRL+3 in my VS triggers method embedding - embeds the method where caret is located in #region [MethodName] ... #endregion block. Actually I use this embedding heavily. Note also that embedding availability is based on language of the file where caret is positioned (i.e different embeddings are available for asp.net files and c# or vb.net files) and on the context (when invoking through shortcuts). But back to my problem. I actually needed two embeddings that would do:

  1. Wrap selection into <table><tr>...</tr></table> block
  2. Wrap selection into <td>...</td> block

Here is the how you add those two embeddings.

Click DevExpress/Options... menu to open Options window, expand the tree on the left to click on Embedding node.

image

Since these embeddings will work on asp.net files pick HTML as Language in combobox on the bottom of the form.

image

Click on the big green plus sign to add first custom embedding.

image

Assign name Tabloid and caption the same. Next select proper embedding style by clicking the first icon in Style row.

image

 

The final step is to type lines <table><td> above the special line Selected Text and </tr></table> lines below this special line. These lines will form embedding header and footer. That's it. It should look like this:

image

Repeat the same steps for the table cell embedding (named Celloid with caption Table Cell in my case):

image

We are done. Now let's try it on my example. First select all lines that have to be embedded into table (Label and Textbox in my case) and right click to show context menu.

image

There they are my two embeddings - FYI there are no predefined ones thus only mine are showing up. Here is the result after Tabloid embedding:

image

Do the same with Celloid embedding.

image

The result is a good one, too. Exactly what I was looking for.

image

So far this is a great time saver and annoying typing repellent. But [CodeRush] can be of more help. It is able to fire embeddings (and other stuff) through custom defined shortcuts. Mouse is great however it is not as fast as keyboard is.

Lets create the shortcuts. The goal is to bind key t to Tabloid and key d to Celloid embeddings. Open [CodeRush] options again but this time select Shortcuts tree node.

image

Click on New Keyboard Shortcut icon

image

In the Key 1 field type t (shortcut)in Command combo box pick Embed (action) and as Parameter type the name of embedding which is Tabloid in my case.

The shortcut is almost ready and the only thing left is to specify the context where the shortcut fires. If you don't set the context the embedding will fire each time you type key t wherever you are. This might be not the desired behavior. Set that the following conditions that have to be met: Selection should span Multiple Lines (one or more lines) and it should contain Whole Line. Also, the shortcut should fire only when editing in HTML View:

image

That's it. When you select one or more lines in HTML code (which includes ASP.NET) and then press key t the action Embed/Tabloid will be fired and your lines will be captured to a table.

Almost same steps are required to create Celloid shortcut on key d. Except for the context settings. Since context is the same in both cases a copy/paste operation can be performed on context. Right click anywhere in the Tabloid's context window and pick Copy Context.

image

Go back to Celloid, right click anywhere in the context window and pick Paste Context. Context is then copied and you avoid re-clicking all of the conditions (there might be plenty of them).

That's really it. I can go now through asp.net code, selecting lines and pressing t or d to embed elements.

Trust me, if you have many pages with many embeddings to perform you'll love this embedding feature of [CodeRush]. Now you can create all sorts of embeddings you want. Enjoy.

Thanks to Mark and Dustin for guidance.

Peculiar problem involving Windows 2003, VMWare Server, SQL Server 2005 and networking

Over the weekend I've built my new server - yes, the content you are reading right now is served from it. Perhaps more about this new server in another post. Back to the point. Host OS on the server is Windows 2003 R2 x32 and there is also a SQL Server 2005 running there. I've had x64 bit before but it is just too much trouble running it since the drivers and support situation. So, if you don't need more than 4GB of RAM then I don't see a compelling reason to go with 64 bits. Why I say "host OS"? Because I am running VMWare Server on top of it. And there are two guest OS running inside virtual machines: Another Windows 2003 that serves web content (it uses SQL Server located on host) and Windows Home Server that takes care of backups. So, after I've installed everything I fired up my web virtual machine and take a took a look at my blog - it was a no go. Virtual machine was working fine, just the Community Server wasn't running. After turning off custom exception handling I ended with a asp.net exception reporting page which I was looking for. However the error was an odd one. It stated that connection with SQL Server (running on host) has timed out. Hm. I investigated further by creating a test application that reads a table from database. Running on my workstation it read the data just fine. But when running from within web server it read just first n rows (i.e. 20) and then it timed out, always at the same row - which was really puzzling. The same symptom appeared to any SQL Server client running within guest OS. So it was obviously a problem related to VMWare Server. Yet, if I've turned off Windows Firewall on the host my application worked even on guest OS - this fact deceived me to thinking that the problem is firewall related (perhaps it was in a way) - after half an hour testing with any possible firewall configuration I gave up. Since I knew it has something to do with VMWare Server I then started searching their forums. And soon enough I've found a solution (at the bottom of the thread):

Disable TCP Offload on the host

While the solution talks about disabling TCP Offload Engine on Windows, I've disabled TCP Offload LargeSend (it sounded enough similar to me and a good candidate for my problem) and it worked like a charm. This is how it looks on my computer:

image

Perhaps networking now consumes 0.00000000001% more of my CPU but at least it works fine. I am not sure whether this is a bug or not, I'll contact VMWare anyway. Funny, the building and installing my server took less than troubleshooting this problem.