Righthand’s DataSet Debugger Visualizer supports VS2013

Highlights from new version to 1.0.11.

  • added VS2013 version
  • added a "separated assembly" versions. Until now I was using RedGate's Smart Assembly to pack all referenced assemblies into a single DLL file for easier management and distribution. However, this black magic might cause problems in certain situations (Visual Studio add-ins screams for problems). Thus I've added another set that features assemblies in separate files. The bottom line, if you have problems or you want to be on the safe side, use the later set.

Go nuts!

NT Konferenca 2012 lectures schedule for mobile devices

I’ve created a simple ASP MVC 4/jQuery Mobile UI web site that helps you with NT Konferenca 2012 lectures schedule if you have a mobile device (now, who doesn’t?). Note that it isn’t a well thought, thoroughly and carefully crafted site. Instead it is a quick and simple one, actually it took me an hour (thanks to wonderful tools – VS2010, NuGet and technologies - .net 4.0, OData, WCF Data Service, jQuery Mobile UI, ASP MVC 4 and so forth).

Feel free to browse the lectures schedule through ntk.rthand.com. I might add additional features in next days – feedback me if you’d like to see a feature.

Updated Righthand's utilities for managing Intel Rapid Storage Technology

I’ve updated my tools for managing IRST and dedicated page as well. Command line utility now displays arrays, volumes, disks, RAID support and ports.

It is also in line with the latest IRST 10.8.0.1003 drivers .

Noteworthy change is that the utilities require .net 4.0 – see the dedicated page for installation instructions and other info.

Expect much more in the future.

Here is a dedicated page to the tools.

Realtime graph for WPF

You know the Task Manager’s CPU Usage History graph that shows CPU utilization over time? Try that type of the graph in WPF and you’ll realize that it isn’t as easy as it should be due to the WPF’s performance for real time graphs. Which sucks due to the way WPF works. Even with 3rd party you’ll have hard time to find a graph fast enough to cope with even more than few hundred samples.

Hence I’ve decide to build my own real time graph. It is based on Direct2D because Direct2D is much faster when it comes to presenting the graphical result. However, merging Direct2D into WPF is, again, not an easy task as it should be (hint to MS – there should be native WPF support for hosting Direct2D output). Anyway, I found the article Using Direct2D with WPF on CodePlex. It comes with sources which I use. Those sources require Windows API CodePack(WACP), a bug ridden set of managed API’s to various, otherwise unsupported, features (MS went with this approach (if it works, it works, if it doesn’t fix it by yourself) instead of providing an official support). The WACP binaries I provide are compiled from 1.1. sources with some bug fixed regarding DirectCompute.

Here is a snapshot from a graph showing a sinusoide as a product of many samples (the attached example contains required code and binaries).

realtimechart

I utilize a combination of Direct2D and samples optimization which yields smooth result even with million and more samples.

 

Required code

Create a class that implements IGraphItem interface. This class will hold a single sample through Time and Value property. Time typically represents a time elapsed since the start and it is expressed in millisecond while Value is a double and holds whatever value.

public class RealtimeGraphItem : IGraphItem
{    
    public int Time { get; set; }
    public double Value { get; set; }
}

Configure RealtimeGraphControl like

xmlns:rg="clr-namespace:Righthand.RealtimeGraph;assembly=Righthand.RealtimeGraph"
...
<rg:RealtimeGraphControl x:Name="Graph" AxisColor="Blue" AxisWidth="1"      VerticalLinesInterval="5" VerticalLabelsStep="2"  SpanX="100"
      MaxY="250" MinY="50"  HorizontalLinesInterval="10"/>

where SpanX is time span expressed in seconds telling the graph how many seconds are shown.

Finally create a BindingList<RealtimeGraphItem> source and bind it to Graph.SeriesSource property (BindingList is required as it has the event ListChanged that is used to trigger rendering update of the graph). And that’s it. When you add new items to the source the graph will automatically reflects the new state.

 

Requirements

.net 4.0

Dowload

Attached are binaries and the sources for the Example project.

Let me know what you think.

RealtimeGraph_1_0.zip (1.31 mb)

Fixing combination of NuGet and Team Foundation in workgroup configuration: 401 Unauthorized

The problem

A lot of users of Visual Studio 2010 (SP1), Team Foundation Server in workgroup and NuGet faced a very annoying problem – often we’d get 401-Unauthorized when installing/uninstalling/updating a NuGet package. Apparently it happens only in this combination (not sure if my host OS – Windows 7 plays any role in it) and not consistently. But when it starts the only way to get rid of errors is to restart Visual Studio.

The only workaround so far was to:

  1. Go Offline with TFS
  2. Manually make files writable (csproj, packages configuration, etc.) or uncheck them before #1
  3. Close Visual Studio
  4. Open Visual Studio
  5. Do NuGet
  6. Close Visual Studio
  7. Open Visual Studio
  8. Go Online with TFS

The steps above were mandatory for every batch of NuGet operations. Which is a huge pain and absurdly great annoyance with, otherwise excellent, NuGet. Needless to say I was among the people facing this issue. And I get so annoyed that I decided to make a choice at that point: either ditch NuGet or fix it myself (NuGet is an open source project).

Being a developer I opted for second choice of course. Was there really a choice? Anyway, here is how it went my 24hrs of debugging and 15s fixing. If you just want to see the solution feel free to skip to the Solution below.

Debugging

1. I downloaded NuGet sources.

2. When opening NuGet solution I quickly find out that I was missing Visual Studio 2010 SDK (because NuGet is an Visual Studio extension) so I downloaded the one I’ve found on the Internet. And it didn’t install saying something about prerequisites not installed. Ah, one needs Visual Studio 2010 SP1 SDK. Get it here. Somebody, please let know Visual Studio Extensibility Developer Center that they are listing the outdated SDK.

3. I set NuGet.VsExtension as Start Up project and fired up the debugger. Which opens another instance of Visual Studio 2010 where I’ve crafted up a sample solution used for reproducing the dreadful 401. I was able to reproduce it often but not always.

4. It took me some time to get familiar with NuGet sources. After that I invested some time to speed up the problem detection (as soon as possible the better) by modifying pieces of NuGet sources and after many trials and errors I’ve found that I have to dig deeper, into the bowels of Team Foundation System Client code.

5. I fired up my preferred tool for debugging assemblies for which I don’t have sources – .net reflector. It works better than using MS source symbols and it works for every assembly. It doesn’t work perfectly but that’s due to assembly optimizations and other black magic issues but it works well enough. Armed with decompiled TFS client assemblies I dug deeper and deeper. But couldn’t find an obvious fault.

6. I brought up a new weapon: Microsoft Network Monitor to analyse the network traffic. After all TFS communication is through HTTP/SOAP. There I’ve found the first clue to the root of the problem. Normally TFS client would send a request that would be refused by server with response saying that NTLM authentication is required. The client would re-send request with NTLM authentication and everything would work. But when the problem occurs the client just doesn’t respond to NTLM challenge – instead it just throws 401 unauthorized exception without even trying to authenticate against the server. I had no idea why it sometimes work and sometimes not.

Successful communication
Successful communication

Unsuccessful communication
Unsuccessful communication

7. At this point I was thinking of enabling System.Net tracing to get more useful info if possible. Immediately I faced a problem. The only way to enable System.Net is through app.config file but not in code. See, I couldn’t use app.config file because I was debugging a library and library’s app.config file is simply ignored. I’ve looked for a way to enable tracing programmatically in code, which is possible for user tracing scenarios, but not for System.Net. Bad luck, but there is nothing that can’t be fixed with a bit of reflection, like this:

private static void InitLogging()
{
    TextWriterTraceListener listener = new TextWriterTraceListener(@"D:\temp\ts.log");
    Type type = typeof(HttpWebRequest).Assembly.GetType("System.Net.Logging");
    MethodInfo initl = type.GetMethod("InitializeLogging", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic);
    initl.Invoke(null, null);

    foreach (string s in new string[] { "s_WebTraceSource", "s_HttpListenerTraceSource", "s_SocketsTraceSource", "s_CacheTraceSource" })
    {
        FieldInfo webTsFi = type.GetField(s, BindingFlags.Static | BindingFlags.NonPublic);
        TraceSource webTs = (TraceSource)webTsFi.GetValue(null);
        webTs.Switch.Level = SourceLevels.Verbose;
        webTs.Listeners.Add(listener);
    }
    FieldInfo le = type.GetField("s_LoggingEnabled", BindingFlags.Static | BindingFlags.NonPublic);
    le.SetValue(null, true);
}

And voila, the thing started to spit a ton of information into file D:\temp\ts.log. But again, it only showed the symptom but not the cause (trace parts after first request, note that unsuccessful one doesn’t even try to NTLM authenticate):

System.Net Information: 0 : [10488] Associating HttpWebRequest#51488348 with ConnectStream#13361802
System.Net Information: 0 : [10488] Associating HttpWebRequest#51488348 with HttpWebResponse#7364733
System.Net Information: 0 : [10488] AcquireDefaultCredential(package = NTLM, intent  = Outbound)
System.Net Information: 0 : [10488] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = HTTP/TFS, inFlags = Delegate, MutualAuth, Connection)
System.Net Information: 0 : [10488] InitializeSecurityContext(In-Buffers count=0, Out-Buffer length=40, returned code=ContinueNeeded).
System.Net Warning: 0 : [10488] HttpWebRequest#51488348::() - Resubmitting request.

Successful communication

System.Net Information: 0 : [10488] Associating HttpWebRequest#51488348 with ConnectStream#13361802
System.Net Information: 0 : [10488] Associating HttpWebRequest#51488348 with HttpWebResponse#7364733

Unsuccessful communication

8. At this point I concentrated on debugging System.Net.HttpWebRequest class as re-submitting is not done at TFS client level. After even more trial and errors I was finally able to pinpoint the root of the evil.

The root of the problem

The decision whether to or not to try NTLM authentication is based on which internet zone OS thinks the request target is. In other words if OS says that your TFS server is outside intranet then HttpWebRequest won’t bother with NTLM authentication at all. It is that simple. The decision lies within PresentationCore’s (!) internal CustomCredentialPolicy.InternetSecurityManager class which delegates the question about the internet zone to OS and returns the result to HttpWebRequest. For some reason at some point it starts to return Internet instead of Intranet. I am not sure exactly why, but I have a remedy. A dramatically simple one which doesn’t even involve modifications to NuGet (no need to wait for a NuGet fix!).

The solution

Open Internet Explorer browser, go to Internet Options/Security, select Local Intranet icon, click Sites button

image

On Local Intranet dialog click Advanced

image

and add your TFS server to the Websites list, like I did with mine (replace TFS with the name of your server)

image

Restart Visual Studio any enjoy NuGet from a new perspective!

This solution apparently solves all of the issues I had with the dreaded 401. Let me know if it works for you as well.

Considerations

The problem might not be related to NuGet at all but rather to PresentationCore (NuGet is a WPF application) which gets confusing results from OS through some interop. NuGet/Visual Studio is just a combination that triggers the otherwise sleeping problem.

Integrating MvcMiniProfiler and LLBLGenPro

MvcMiniProfiler is a lean and mean mini profiler for MVC 3 that shows the profiling results on each page displayed at runtime. Besides the custom steps you can seed everywhere in your code it supports database profiling as well. Out of the box are supported ado.net, Linq to SQL and Entity Framework. LLBLGenPro, my favorite ORM, isn’t supported though and it won’t work just like that.

Luckily, it turns out, it requires just a little effort to integrate MvcMiniProfiler into LLBLGenPro.

How does MvcMiniProfiler database profiling works

The way it works is that it wraps DbConnection, DbCommand and other Db[Stuff] and thus records the execution time by tracking their inner workings. Here is an example for MvcMiniProfiler documentation about how to start:

public static DbConnection GetOpenConnection()
{
    var cnn = CreateRealConnection(); // A SqlConnection, SqliteConnection ... or whatever

    // wrap the connection with a profiling connection that tracks timings 
    return MvcMiniProfiler.Data.ProfiledDbConnection.Get(cnn, MiniProfiler.Current);
}

If client calls DbConnection.CreateCommand on an ProfiledDbConnection instance returned from previous method it will get a wrapped whatever command original connection returns and so on. There is also a way to manually create DbCommand through ProfiledDbCommand constructor.

The support for Linq To SQL and Entity Framework is done in a similar manner.

This gets us to the point, why can’t I just use the same approach with LLBLGenPro?

Integrate MvcMiniProfiler with LLBLGenPro – why doesn’t work with same approach

The major problem with LLBLGenPro and MvcMiniProfiler integration is that LLBLGenPro doesn’t use DbConnection.CreateCommand method to create commands from existing connection. Instead it creates an instance of proper DbCommand derived class and assigns a connection to it. Thus it won’t work because it would try to assign a ProfiledDbConnection to a i.e. SqlCommand class.

So a bit more work is required to match them.

The code for adapter scenario

1. Create a DynamicQueryEngine derived class. Note: this class is database specific, thus if you work with i.e. SQL Server you’ll find it in SD.LLBLGen.Pro.DQE.SqlServer.NET20.dll assembly.

public class ProfilingDynamicQueryEngine : DynamicQueryEngine
{
    protected override DbCommand CreateCommand()
    {
         DbCommand cmd = base.CreateCommand();
         ProfiledDbCommand pCmd = new ProfiledDbCommand(cmd, null, MiniProfiler.Current);
         return pCmd;
    }
}

Here the DbCommand creation is overriden. Note that I wrap the original cmd and pass a current MiniProfiler instance as arguments to ProfiledDbCommand constructor, while I pass a null for the connection instance because it will be assigned later.

2. Derive from DataAccessAdapter class. Note: this class is generated from a template and you’ll find it in DBSpecificLayer project generated by LLBLGenPro.

public class DataAccessAdapterEx: DataAccessAdapter
{    
    protected override System.Data.Common.DbConnection CreateNewPhysicalConnection(string connectionString)
    {
        DbConnection conn = base.CreateNewPhysicalConnection(connectionString);
        // return ProfiledDbConnection.Get(conn); Pre MvcMiniProfiler 1.9
        return new ProfiledDbConnection(conn, MiniProfiler.Current);
} protected override DynamicQueryEngineBase CreateDynamicQueryEngine() { return PostProcessNewDynamicQueryEngine(new ProfilingDynamicQueryEngine()); } }

Within CreateDynamicQueryEngine I pass the class I’ve created in step #1. CreateNewPhysicalConnection will return a wrapped connection.

Instead of using DataAccessAdapter you should use the one created in step #2 - DataAccessAdapterEx. That’s it.

Conclusion

As it turns out, integrating MvcMiniProfiler with LLBLGenPro is quite easy. And the required coding might be added to LLBLGenPro templates by modifying them, so you won’t have to manually add the same code each time.

Let me know if you have feedback.

Update 19.9.2011: Updated the code because MvcMiniProiler introduced a breaking change in v1.9 (instead of ProfiledDbConnection.Get static method a constructor has to be used - thanks for swift response from David from LLBLGenPro support team)

Getting HRESULT: 0x80131515 when running Righthand DataSet Visualizer?

Are you getting a HRESULT: 0x80131515 when invoking Righthand DataSet Visualizer from Visual Studio like this:

image_thumb11[4]

The problem is that OS marked the visualizer assembly as unsecure since it originated from the Internet. The solution to the problem is an easy one.

Locate the Righthand.DebuggerVisualizer.Dataset.2010.dll within File Explorer, right click to get Properties and click on Unblock button:

HRESULT: 0x80131515 error dialog

Happy DataSet/DataTable visualization!

What’s new in C# 4.0 presentation at NTK 2010 in Portorož

Tomorrow I am talking about new features in C# 4.0 at NT Konferenca in Portorož, Slovenia and I’ll be there through all day. So if you want to hear what features were added to C# in its latest incarnation you are welcome to attend the presentation taking place in Emerald 1, 14:45 - 16:00.

Later I’ll participate in MVP Panel in MSTech (Pečina), 16:30 - 17:30 and as a SLODUG lead I’ll be present at SloUG meeting in Sunset, 17:30 as well.

If you just want to talk about something .net-ish or just want to say hi, feel free to find me as I’ll be lurking around during my free time.

BTW, the official NT Konferenca twitter tag is #ntk10.

Visual Studio 2010 and .net 4.0 are being released today

Today is the day. See Soma’s announcement and prepare your browsers pointed to MSDN Subscriber Downloads. The goods should be available for download starting at 10AM PST which translates to 19:00 for Slovenia (you might check out other local times).

Also watch the Visual Studio 2010 launch event live here.

Some useful links: