Porting a .net application to Linux/Mono/MonoDevelop

I have a small .net application that I wanted to run on Linux since it is a deamon/service sort of app. Basically is an app that peridically retrieves data from a REST service and stores it to a PostgreSQL database using LLBLGenPro 5.0 CTP2. So I created a .net 4.6.1 console application that uses HttpClient/ModernHttpClient package, NLog package for logging and System.Collections.Immutable package for immutable collecitons obviously. That was on Windows using Visual Studio 2015. After I added all the necessary code it run perfectly well on Windows.

Then I tried to open the same solution on Linux. I cloned the repository and opened it in MonoDevelop 5.10/Mono 4.2.1 (both latest). And it didn’t work out of the box. I stumbled over several issues I will describe here.

.net version

MonoDevelop doesn’t work with anything over .net 4.5 for the time being due to the MSBuild thing – at least that was the complaint.

Hence I downgraded my project to .net 4.5. No problems since I didn’t use anything .net 4.6ish.

System.Collection.Immutable

It won’t install System.Collection.Immutable package because System.Linq package (which System.Collection.Immutable depends on for some reason) won’t install unless NuGet Package Manager is 3.0 or newer. MonoDevelop uses NuGet Package Manager 2.8.7. While this is only a NuGet Package installation issue, it is annoying. First, because System.Collection.Immutable doesn’t really require System.Linq package on Mono/.net 4.5 and next becuse it isn’t smooth as it should be. Funny thing is that at NuGet web page it says that System.Collection.Immutable requires NuGet 2.8.6 or higher. I guess it should be 3.0 or higher since a package that it depends on requires it.

Hence I dropped System.Collection.Immutable package which wasn’t difficult because I wasn’t using it extensively. Hopefully MonoDevelop catches up with NuGet and that is fixed.

HttpClient/ModernHttpClient

ModernHttpClient speeds up communication since it uses native networking libraries. However it won’t work on Linux.

Dropped it since it doesn’t affect the functionallity at all.

NLog

I had configure NLog though nlog.config file. And it works on Windows perfectly. Hower when run on Linux I didn’t get any output whatsoever. Just like there was no configuration file. As a Windows guy I was a bit puzzled why. Since NLog is open source, I downloaded the code and tried debugging. Sadly MonoDevelop didn’t allow me to step into NLog code for some reason but luckily old school “Console.WriteLine” approach worked as usual. After some strategically placed Console.WriteLine calls within NLog library I found the reason, which might be apparent to a Linux guy. Bloody file name casing. I was using nlog.config and Windows doesn’t care about file name casing.

On Linux file name casing does and thus changing the name to NLog.config did the trick.

Linux specific addition

If app wants to gracefully handle SIGTERM signal (like when somebody/something tries to kill it) it might use UnixSignalWaiter NuGet package. While only at version 0.0.1 it seems to work just fine. It is also safe to use it on non-Linux systems where it does nothing (one might also check whether the feature is supported).

Conclusion

If .NET Core has proper database support (PostgreSQL in my case, but more more importantly LLBLGenPro doesn’t work due to lack of ado.net support – I don’t know about you but for me the proper ado.net stack on server is one of the most important aspects) I’d try to use it as well. But since it doesn’t, well, someday in the future I guess.

Anyway, after learning all the problems and the workarounds above I feel more confident in my Mono/Linux skills and the application compiles fine and works as expected. I also think that Xamarin/mono guys are doing really great job. Kudos.

Did I mention that I’m using Docker to run the application? I guess that’s a topic for a future post.

Integrating MvcMiniProfiler and LLBLGenPro

MvcMiniProfiler is a lean and mean mini profiler for MVC 3 that shows the profiling results on each page displayed at runtime. Besides the custom steps you can seed everywhere in your code it supports database profiling as well. Out of the box are supported ado.net, Linq to SQL and Entity Framework. LLBLGenPro, my favorite ORM, isn’t supported though and it won’t work just like that.

Luckily, it turns out, it requires just a little effort to integrate MvcMiniProfiler into LLBLGenPro.

How does MvcMiniProfiler database profiling works

The way it works is that it wraps DbConnection, DbCommand and other Db[Stuff] and thus records the execution time by tracking their inner workings. Here is an example for MvcMiniProfiler documentation about how to start:

public static DbConnection GetOpenConnection()
{
    var cnn = CreateRealConnection(); // A SqlConnection, SqliteConnection ... or whatever

    // wrap the connection with a profiling connection that tracks timings 
    return MvcMiniProfiler.Data.ProfiledDbConnection.Get(cnn, MiniProfiler.Current);
}

If client calls DbConnection.CreateCommand on an ProfiledDbConnection instance returned from previous method it will get a wrapped whatever command original connection returns and so on. There is also a way to manually create DbCommand through ProfiledDbCommand constructor.

The support for Linq To SQL and Entity Framework is done in a similar manner.

This gets us to the point, why can’t I just use the same approach with LLBLGenPro?

Integrate MvcMiniProfiler with LLBLGenPro – why doesn’t work with same approach

The major problem with LLBLGenPro and MvcMiniProfiler integration is that LLBLGenPro doesn’t use DbConnection.CreateCommand method to create commands from existing connection. Instead it creates an instance of proper DbCommand derived class and assigns a connection to it. Thus it won’t work because it would try to assign a ProfiledDbConnection to a i.e. SqlCommand class.

So a bit more work is required to match them.

The code for adapter scenario

1. Create a DynamicQueryEngine derived class. Note: this class is database specific, thus if you work with i.e. SQL Server you’ll find it in SD.LLBLGen.Pro.DQE.SqlServer.NET20.dll assembly.

public class ProfilingDynamicQueryEngine : DynamicQueryEngine
{
    protected override DbCommand CreateCommand()
    {
         DbCommand cmd = base.CreateCommand();
         ProfiledDbCommand pCmd = new ProfiledDbCommand(cmd, null, MiniProfiler.Current);
         return pCmd;
    }
}

Here the DbCommand creation is overriden. Note that I wrap the original cmd and pass a current MiniProfiler instance as arguments to ProfiledDbCommand constructor, while I pass a null for the connection instance because it will be assigned later.

2. Derive from DataAccessAdapter class. Note: this class is generated from a template and you’ll find it in DBSpecificLayer project generated by LLBLGenPro.

public class DataAccessAdapterEx: DataAccessAdapter
{    
    protected override System.Data.Common.DbConnection CreateNewPhysicalConnection(string connectionString)
    {
        DbConnection conn = base.CreateNewPhysicalConnection(connectionString);
        // return ProfiledDbConnection.Get(conn); Pre MvcMiniProfiler 1.9
        return new ProfiledDbConnection(conn, MiniProfiler.Current);
} protected override DynamicQueryEngineBase CreateDynamicQueryEngine() { return PostProcessNewDynamicQueryEngine(new ProfilingDynamicQueryEngine()); } }

Within CreateDynamicQueryEngine I pass the class I’ve created in step #1. CreateNewPhysicalConnection will return a wrapped connection.

Instead of using DataAccessAdapter you should use the one created in step #2 - DataAccessAdapterEx. That’s it.

Conclusion

As it turns out, integrating MvcMiniProfiler with LLBLGenPro is quite easy. And the required coding might be added to LLBLGenPro templates by modifying them, so you won’t have to manually add the same code each time.

Let me know if you have feedback.

Update 19.9.2011: Updated the code because MvcMiniProiler introduced a breaking change in v1.9 (instead of ProfiledDbConnection.Get static method a constructor has to be used - thanks for swift response from David from LLBLGenPro support team)

Entity Framework leaves the door open to a SQL Injection attack?

I couldn’t believe it when I read Julie Lerman’s post about EF and SQL Injection. She discovered that Entity Framework doesn’t use parameterized queries always!

I had to try it for myself on Northwind database:

var query = from c in entities.Customers
                        where c.ContactName == "Tubo"
                        select c;

produces this SQL statement (courtesy of SQL Profiler):

SELECT
[Extent1].[CustomerID] AS [CustomerID],
[Extent1].[CompanyName] AS [CompanyName],
[Extent1].[ContactName] AS [ContactName],
[Extent1].[ContactTitle] AS [ContactTitle],
[Extent1].[Address] AS [Address],
[Extent1].[City] AS [City],
[Extent1].[Region] AS [Region],
[Extent1].[PostalCode] AS [PostalCode],
[Extent1].[Country] AS [Country],
[Extent1].[Phone] AS [Phone],
[Extent1].[Fax] AS [Fax]
FROM [dbo].[Customers] AS [Extent1]
WHERE N'Tubo' = [Extent1].[ContactName]

See the Tubo condition expressed like a string? Speechless. Granted, if you specify a variable instead a string within the EF query then a parameter is used.

So I tried a classic SQL Injection attack, like this:

var query = from c in entities.Customers
                        where c.ContactName == "Tubo'; UPDATE [Northwind].Categories SET Description='SQL Injection' WHERE CategoryID=9; --'"
                        select c;

which produces this condition (select columns trimmed)

WHERE N'Tubo''; UPDATE [Northwind].Categories SET Description=''SQL Injection'' WHERE CategoryID=9; --''' = [Extent1].[ContactName]

Luckily the attack fails due to duplication of single quotes. So there is a protection against SQL attacks afterall.

However, I am not feeling safe with this approach. Attackers are smart and sooner or later they might find a way across string-replacement defense strategy.

But the real questions is: why leave the door open to a possible attack in first place? Why not always use parameters as there is no way around them? What good is to risk a SQL injection attack?

Update: A possible scenario where one could exploit this feature, granting there is a way around single-quote-replacement, is with add-ins. Imagine you have an add-in based application where each add-in is allowed to perform EF selects on database (enforced by your application). One could fake a select with all sort of T-SQL code nested within.

Is true belief a good attribute of an architect

Recently I've came across this forum thread regarding the future of [DevEx]' eXpress Persistent Objects for .NET (XPO) ORM product. Customers (actual ones, not potential) are asking whether XPO will someday support n-tier development. They are happily using XPO but they miss a vital part, which is n-tier development. Note that XPO never advertised n-tier support and nobody is arguing that. Perhaps this comment from a customer's post, where the situation is described:

"As Trevor said, I am one whose core is 100% XPO. The future of my application is tied to XPO. Was this a bad decision? Maybe, but probably not. However, XPO lacks in areas (n-tier) where I really need it to start to shine."

Later on Oliver from [DevEx] comments that [DevEx] is planning to add, among various features, a multi-tier support. No other details, no dates, no nothing:

"The architectural changes we are planning will result in the potential to support a variety of layered application architectures, including multi-tier ones. So this is good news for you and everybody else waiting for this kind of functionality."

And crowd goes cheering and applauding. Can this be defined as a true belief? From the point of [DevEx] such a comment in current situation makes sense (I won't question whether XPO is good or bad and if it is going n-tier or not at this point). What I fail to understand is the crowd. How can it be that you use a product that misses a very important feature for you (from the beginning!) and you are prepared to wait for years to get it? We aren't talking about a cosmetic feature. No, we are talking about core functionality. And you are happy that somebody is planning someday to deliver that feature, which might suite you or not, without any obligation to deliver (again, I am not questioning [DevEx]).

Generally speaking, is such a belief and faith that the feature will be what you've asked for, a good attribute of an architect? Would you choose a product that doesn't support a core functionality and it might never support it. Knowing that competition is stiff and you have plenty of choices out there that support this feature and more, you are a bad architect if you don't look around. At least one I wouldn't trust. This is simply bad practice. If you are basing a core part of your application on something, it should work from the start. Gambling on future is not an option.

LINQ to SQL showstoppers

I have to say that I like LINQ to SQL, heck, I even do presentations about it. So, it is a cute ORM. What is that good about it features LINQ. However, there are some serious showstoppers in there. One are more serious than the others. I won't deal with what's good about LINQ to SQL, rather I'll dive into showstoppers as I see them. So, let's take a look at them:

  1. Support for SQL Server flavors only, not even Katmai (SQL Server 2008).
    This might or might be a showstopper for you. One of the great advantages of ORM products is that they are abstracted from database, meaning the same code works with different supported databases. You probably won't write an application that is totally abstracted and you'll have to write few database dependant stuff but it is still true for the majority of the code. So, why would I bother learning an ORM that deals with SQL Server only when I have plenty of other ORMs that supports a bunch of different databases? Why would I bother to tie my application to SQL Server only when I could have an application that works with all the major databases out there?
  2. Poor n-tier support
    A good ORM would let you send all the modified entities to the server tier with a few of method calls and store changes to the database with few other method calls. But not LINQ to SQL. Oh, no. Doing n-tier with LINQ to SQL requires a fair amount of coding and lot of manual work that should be done by LINQ to SQL itself. IOW LINQ to SQL is very client-server oriented. Looks like n-tier was added afterwards.
  3. Designer that doesn't sync with database
    Imagine you have a Context with myriad of entity types on it. That's very messy by itself since you can't isolate a certain set of entity types you might be interested in but you always have to look at all of them. But that's not the real issue here. The real problem arises when database metadata changes (right, you always define your database structure in stone before you start coding and you don't modify it ever) - perhaps this is a bad project management, but it does change through the life of the project. And when it does, LINQ to SQL won't help you modifying the Context. Nope, you are on your own. So, you can recreate the context manually and reapplying all the modifications done to it (you are certainly looking forward to this) which is not only boring, but error prone, too. Or you can manually sync the Context with database which is not that boring but error prone and tedious. So, this one is a huge showstopper if you ask me.
    Luckily there is [CodeSmith] with its PLINQ templates that supports database synchronization. While not a perfect solution it solves many problems. Furthermore it allows you to modify the templates to your requirements - IMO the [CodeSmith] path is the only path if you are serious about LINQ to SQL.
  4. No source code
    [MS] is probably releasing LINQ to SQL source code, but even if they does, you won't be able to fix problems as you can't modify it in any way. Not a big problem.
  5. Slow development cycle (tied to .net 3.5)
    If you need a fix quickly you will have problems as LINQ to SQL is part of .net 3.5 framework. And knowing the speed of [MS] applying bugfixes it might take years before official update is available.
  6. No direct execution against database
    Imagine you have a task that has to update all item prices for 10% for whatever reason (it acually happened last month in Slovenia for bread and milk items). All orders have to be fetched from the database to the application, each order has to be properly update in application and all of them have to be persisted back to database. Right, you can't execute the operation directly against database from the code. OK, you can issue a SQL statement or use a stored procedure but the former breaks strong typing (SQL statement is a string, compiler won't help you catching the errors) while the later is unnecessary burden.
  7. There is plenty of better competitors out there
    Apart from the (obviously) LINQ advantage there is plenty of better and proven ORM products out there. Even the LINQ, major LINQ to SQL's advantage will soon be integrated by the competition as some of them already feature LINQ (to Entities) while others will soon.

Again, don't take me wrong. LINQ to SQL has plenty of good features. It is just that it has too much of show stoppers for me and knowing problems of certain technology before using it is usually a valuable knowledge. So, what can you do? Pick an alternative of course, either one of commercial or free 3rd parties or wait for Entity Framework which will see light next year, sometime after .net 3.5 RTM.

What will I do? Actually nothing. I am very happy with [LGP] that is much superior to LINQ to SQL except for the LINQ. Also [LGP] is getting LINQ support supposedly this year so it shouldn't be long before I am be able to happily LINQing with ORM.

And let me know if there are other problems you foresee with LINQ to SQL.

Speaking at CodeCamp in Zagreb, Croatia

CodeCamp2007Banner I am pretty busy lately and also have been attending CEE MVP Open Days 2007 summit in beautiful Cavtat right next Dubrovnik in beautiful warm weather. Nevertheless I've found enough time to (re)deliver LINQ to SQL presentation and co-moderate ORM - hero to zero round table on Saturday, 29th of September at CodeCamp. The event will happen in [MS] Hrvatska building located in Zagreb, Croatia.

I have to say that this is my first engagement in non-native and non-English language and I am curios whether attendees will understand me at all. See you in Zagreb!

LLBLGen Pro got even better

Frans released a new major version of his [LGP] ORM. Basically, a top notch ORM product got even better. Make sure you check the what's new list - and if you are looking for an (better) ORM, don't hesitate to give it a try, you won't regret it.

The only feature I miss is LINQ support which is supposed to appear in the next version. Actually, including LINQ support isn't that useful right now as Visual Studio 2008 aka Orcas isn't supposed to ship until very late 2007/beginning of 2008. Frans is actually wise to wait a little bit more with LINQ and instead concentrate on actual improvements which help developers right now. That said I am really waiting to see LINQ for [LGP] when VS 2008 gets released. I suspect LINQ will make [LGP] (and any other ORM product out there) code much much more readable that it is now (limited by current .net languages).

Keep up the excellent work, Frans.