Damian Hickey

Mostly software and .NET related. Mostly.

Our Open Source policy at eVision

There is a problem within our industry, particularly around .NET, known as the "no-contrib culture". That is, companies take and benefit from OSS but don't given anything back and also prevent their employees from doing so. A while ago I blogged about establishing an Open Source Software policy at a proprietary software company that happens to consume and depend on a significant amount of open source libraries. eVision was a typical company in that the standard employment contract had that catch-all clause that they own everything one does on a computer.

With the proliferation of OSS, the fact the platform we predominately build on is going OSS, and the fact that OSS is a viable business strategy (even within proprietary businesses), this was a somewhat short-sighted. Indeed, several employees were simply ignoring this; a situation which was in neither party's interest.

Thus I am delighted to announce that we have established a policy that we believe strikes the right balance and have made it available on our github organisation site

In layman's terms, this means that our employees are free to create any sort of open source outside of business hours (as long as it doesn't compete with our business), are free to contribute to open source we depend on at any time, and they own the copyright to that work (or whatever the terms are of the project they contribute to). The only real stipulation is that the project's licence must allow us to use it in our commercial software.

We hope this will have the effect of encouraging contribution to the platform we depend on strengthening it to our mutual benefit. We also hope that engagement with the open source community will have a positive learning effect for our engineers.

I personally hope that more organisations and companies adopt our policy and make it known publicly. Thus, we've released this policy under a creative commons licence. Disclaimer: you should have your lawyer check it over!

To all developers out there, when considering whom to work for I would strongly suggest that you should only look for organisations that have this or a similar policy in place. These are mature organisations that care for the platform and ecosystem they and their staff build upon.

Any feedback or questions, please contact me dhickey at gmail.com. Thanks to Adam RalphFrans Bouma and Christian Horsdal for their feedback.

On that note, eVision are looking for talented engineers (and more) to work on some challenging, distributed and large scale problems. :)


Establishing OSS policies in a proprietary software company

My company produces proprietary software but consumes a lot of FOSS libraries and components. Until recently, our standard contracts were of the 'we own everything you do' type. Obviously that created some difficulties for someone like myself given that I'm active in the .NET OSS community. The irony being that if I signed it, I wouldn't be able to support the very OSS libraries I maintain and that the company uses.

We'd like to establish a corporate policy where developers can contribute back and, possibly, create new OSS. I believe this is strategically a good move for several reasons, including:

 A) It supports the software that we've built our business on, and without which, we might not even exist
 B) It exposes our developers to more code and ways of working, thereby helping to improve their craft.
 C) It's an attractive policy to the type of developer that we are interested in attracting as we grow.
 
At the same time though, it is critically important for us to protect the proprietary aspect of our software.
 
Our legal department and I are not experienced in establishing such policy and procedures. I'd like to get in touch with someone from a company who has enacted something like this, so if you know anyone, I'd be very grateful if you could introduce me (dhickey@gmail.com, @randompunter).

I hope to publicly share what we come up with such that other companies can learn and implement similar policies.

Protecting a Self-Hosted Nancy API with Microsoft.Owin.Security.ActiveDirectory

This post is the Nancy version of Vittorio's 'Protecting a Self-Hosted API with Microsoft.Owin.Security.ActiveDirectory'. Since I'm lazy, I may even be lifting some parts of it verbatim, if you don't mind, Vittorrio ;)

I'm going to skip the intro to Owin etc, if you are reading this, you probably know all about it by now. (Except for you Phillip; Get yout s**t together man!). This tutorial will be using Nancy.MSOwinSecurity that I introduced in the previous post.

What I am going to show you is how you can set up a super simple Nancy HTTP API in a console app and how you can easily secure it with Windows Azure AD with the exact same code you use when programming against IIS Express/full IIS.

Here's what we are going to do:

  • Create a minimal self-hosted Nancy Http API in a console app.
  • Add middleware for validating JWT tokens from AAD.
  • Create a console app client to poke our API and challenge the user for credentials.

Create a Minimal Self-Hosted Nancy Http API

Let's start by creating blank solution. I'm a bit unimaginative so I am calling this 'NancyAAD'. Note: this entire solution will be targeting .NET 4.5. .NET 4.0 is not supported. Next, we add a console application, 'NancyAAD.Server', that is going to host our Nancy HTTP API. To this project, we're going to add a number of nuget packages, in this order:

  1. Nancy.Owin - This will bring in Nancy and the adapter that allows Nancy to be hosted in an OWIN pipeline.
  2. Microsoft.Owin.Hosting - Provides default infrastructure types for hosting and running OWIN-based applications. There are other hosting implementations out there, if you feel the need to explore.
  3. Microsoft.Owin.Host.HttpListener - OWIN server built on the .NET Framework's HttpListener class.
  4. Microsoft.Owin.Security.ActiveDirectory - The middleware through which we will secure our Nancy HTTP API. This package will pull in a bunch of other packages. We'll be using this later.
  5. Nancy.MSOwinSecurity - This will provide integration between Nancy's context and modules and Microsoft.Owin.Security.*.

Your NancyAAD.Server packages.config should look similar to this:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Owin" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Host.HttpListener" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Hosting" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.ActiveDirectory" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.Jwt" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.OAuth" version="2.0.2" targetFramework="net45" />
  <package id="Nancy" version="0.21.1" targetFramework="net45" />
  <package id="Nancy.MSOwinSecurity" version="1.0.0" targetFramework="net45" />
  <package id="Nancy.Owin" version="0.21.1" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="4.5.11" targetFramework="net45" />
  <package id="Owin" version="1.0" targetFramework="net45" />
  <package id="System.IdentityModel.Tokens.Jwt" version="1.0.0" targetFramework="net45" />
</packages>

Next we'll define our Nancy module and a very simple GET endpoint that we'll want to secure later:

namespace NancyAAD.Server
{
    using Nancy;

    public class ValuesModule : NancyModule
    {
        public ValuesModule()
        {
            Get["/values"] = _ => new[] { "value1", "value2" };
        }
    }
}

That done, let's add the Startup class for hosting our Nancy application in OWIN:

namespace NancyAAD.Server
{
    using Owin;

    public class Startup
    {
        public void Configuration(IAppBuilder app)
        {
            app.UseNancy();
        }
    }
}

This says that Nancy is handling all requests from the root. Nice and simple. 

Lastly, we need to host the OWIN application itself. In Program.cs, we write the following:

namespace NancyAAD.Server
{
    using System;
    using System.Net.Http;
    using System.Net.Http.Headers;
    using Microsoft.Owin.Hosting;

    internal class Program
    {
        public static void Main(string[] args)
        {
            using (WebApp.Start<Startup>("http://localhost:9000/"))
            {
                Console.ForegroundColor = ConsoleColor.Blue;
                Console.WriteLine("Nancy listening at http://localhost:9000/");

                // Test call
                var client = new HttpClient();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                var response = client.GetAsync("http://localhost:9000/values").Result;
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine(response.Content.ReadAsStringAsync().Result);

                Console.ReadLine();
            }
        }
    }
}

The call to WebApp.Start initializes a new server, which listens at the specified address. The rest of the method calls the Nancy HTTP API to double check that we did everything correctly.

There is one subtle difference here compared to Vittorio's post - we are defining an Accept header "application/json". The reason is that because Nancy supports view engines where it's default behaviour when returning a model for a request with no accept header, is to try to resolve a view. This would result in an exception for us as we haven't defined any views.

F5'ing the app should result in:

Exactly the same as Vittorio's screen shot. Nice.

Before moving on to the next task, remove the test call from the Main method and change it to the following:

public static void Main(string[] args)
{
	using (WebApp.Start<Startup>("http://localhost:9000/"))
	{
		Console.ForegroundColor = ConsoleColor.Blue;
		Console.WriteLine("Nancy listening at http://localhost:9000/");
		Console.WriteLine("Press ENTER to terminate");
		Console.ReadLine(); 
	}
}

Secure the Nancy HTTP API with Windows Azure AD

Here comes the raison d’être of the entire post. We have already added the Microsoft.Owin.Security.ActiveDirectory nuget package, now let's add it in the right place into the OWIN pipeline. Note, the order things are defined in the pipeline are important, so we must add this before Nancy so requests are piped through it before hitting the Nancy handlers. 

Our OWIN pipeline with the middleware added now looks like:

public void Configuration(IAppBuilder app)
{
	app.UseWindowsAzureActiveDirectoryBearerAuthentication(
		    new WindowsAzureActiveDirectoryBearerAuthenticationOptions
		    {
		        Audience = "https://contoso7.onmicrosoft.com/RichAPI",
		        Tenant = "contoso7.onmicrosoft.com"
		    })
	      .UseNancy();
}

The new code is the "UserWindowsAzure....". You’ll notice that instead of relying on app.config settings, values are passed directly to the middleware. This is another beauty of OWIN, you can now use can keep such settings wherever you deem most appropriate.

Now our pipeline includes the right middleware: if we receive a JWT, we’ll validate it and if it checks out we’ll pass the corresponding ClaimsPrincipal to the handler delegate. Very good, but not good enough. Let’s modify the module class to mandate that all callers must present a valid token from the tenant of choice: 

public class ValuesModule : NancyModule
{
	public ValuesModule()
	{
		this.RequiresMSOwinAuthentication();
		Get["/values"] = _ =>
		{
			ClaimsPrincipal claimsPrincipal = Context.GetMSOwinUser();
			Console.WriteLine("==>I have been called by {0}", claimsPrincipal.FindFirst(ClaimTypes.Upn));
			return new[] {"value1", "value2"};
		};
	}
}

The key statement here is "this.RequiresMSOwinAuthentication" - this ensures that a request hitting this module must have a valid authenticated user, otherwise an unauthorized HTTP status code is returned.

The second key statement, "Context.GetAuthenticationManager().User", retrieves the user from IAuthenticationManager where we write a claim to the console, just for demonstration purposes. The UPN claim is among the ones that Windows Azure AD sends in JWTs.

Believe it or not, that’s all we had to do to secure the Nancy HTTP API: we just had to add the code in bold :)

Create a Client App and Test the Nancy HTTP API

Add a new console app project called NancyAAD.Client (see, I'm not really imaginative!). To this project, install the 'Microsoft.WindowsAzure.ActiveDirectory.Authentication' nuget package. This package contains the main assembly for the Windows Azure Authentication Library (AAL) and provides easy to use authentication functionality for .NET client apps. Your packages.config should look like;

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.WindowsAzure.ActiveDirectory.Authentication" version="0.7.0" targetFramework="net45" />
</packages>

Next up, the Main method should looks like this:

namespace NancyAAD.Client
{
    using System;
    using System.Net.Http;
    using System.Net.Http.Headers;
    using Microsoft.WindowsAzure.ActiveDirectory.Authentication;

    internal class Program
    {
        [STAThread]
        public static void Main(string[] args)
        {
            Console.ForegroundColor = ConsoleColor.Green;
            Console.WriteLine("Client ready.");
            Console.WriteLine("Press any key to invoke the service");
            Console.WriteLine("Press ESC to terminate");
            ConsoleKeyInfo consoleKeyInfo;

            var authenticationContext = new AuthenticationContext("https://login.windows.net/contoso7.onmicrosoft.com");

            do
            {
                consoleKeyInfo = Console.ReadKey(true);
                // get the access token
                AuthenticationResult authenticationResult = authenticationContext.AcquireToken(
                    "https://contoso7.onmicrosoft.com/RichAPI",
                    "be182811-9d0b-45b2-9ffa-52ede2a12230",
                    "http://whatevah");
                // invoke the Nancy API
                var httpClient = new HttpClient();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", authenticationResult.AccessToken);
                HttpResponseMessage response = httpClient.GetAsync("http://localhost:9000/values").Result;
                // display the result
                if (response.IsSuccessStatusCode)
                {
                    string result = response.Content.ReadAsStringAsync().Result;
                    Console.WriteLine("==> Successfully invoked the service");
                    Console.WriteLine(result);
                }
            } while (consoleKeyInfo.Key != ConsoleKey.Escape);
        }
    }
}

This is a very classic ADAL client and is virtually identical to the one in Vittorio's post. It simply acquires the token, attaches it to the Authorization header before making the call. The one difference is the Accept header is defined, as explained previously.

Starting both projects and triggering a call from the client should result in:

Sign in as any user in your tenant, and you’ll get to something like the screen below:

Ta-dah!

Now go forth and secure your enteprise(!) Nancy Apps!

Full code for this is up on Github

Introducing Nancy.MSOwinSecurity

Under the Katana Project, Microsoft has released a suite of security related middleware including cookie and oauth based providers such as Google, Twitter etc. Nancy.MSOwinSecurity allows your Nancy application to work and interact with the Microsoft.Owin.Security.* middleware by giving you access to Microsoft.Owin.Security.IAuthenticationManager. From there you can access the CurrentUser as a ClaimsPrincipal, as well as perform various operations such as SignIn(), SignOut(), Challenge() etc.

How to use

Install the nuget package:

install-package Nancy.MSOwinSecurity

Getting the authentication manager and current user from the context:

public class MyModule : NancyModule
{
    public MyModule()
    {
        Get["/"] = _ =>
        {
            IAuthenticationManager authenticationManager = Context.GetAuthenticationManager();
            //ClaimsPrincipal user = authenticationManager.User;
            //authenticationManager.SignIn(..);
            //authenticationManager.SignOut(..);
            //authenticationManager.AuthenticateAsync(..);
            //authenticationManager.Challenge(..);
        };
    }
}

Securing a nancy module:

public class MyModule : NancyModule
{
    public MyModule()
    {
        this.RequiresMSOwinAuthentication();
        Get["/"] = _ => {...}
    }
}

Securing an individual route:

public class MyModule : NancyModule
{
    public MyModule()
    {

        Get["/"] = _ => 
        {
            this.RequiresMSOwinAuthentication();
            ....
        }
    }
}

Getting the current user (just a helper extension around IAuthenticationManager.User):

public class MyModule : NancyModule
{
    public MyModule()
    {

        Get["/"] = _ => 
        {
            ClaimsPrincipal = Context.GetMSOwinUser();
            ....
        }
    }
}

Authorizing the user at module level:

public class MyModule : NancyModule
{
    public MyModule()
    {
        this.RequiresSecurityClaims(claims => claims.Any(
            claim.ClaimType = ClaimTypes.Country &&
            claim.Value.Equals("IE", StringComparision.Ordinal)));
        Get["/"] = _ => 
        {
           ....
        }
    }
}

Authorizing the user at route level:

public class MyModule : NancyModule
{
    public MyModule()
    {

        Get["/"] = _ => 
        {
            this.RequiresSecurityClaims(claims => claims.Any(
                claim.ClaimType = ClaimTypes.Country &&
                claim.Value.Equals("IE", StringComparision.Ordinal)));
            ...
        }
    }
}

Personal note: this nancy extension package would integrate much better if we had extenstion properties in c# :(

Note: This package doesn't replace nor integrate with the exisiting nancy authentication and authorization infrasctructure (IUserIdentity). We're currently considering the best way to proceed on this.

Uniqueness in a DDD-CQRS-ES application

A while back I blogged that set based based concerns, i.e. duplicates, that when analyzed aren't really a problem. An example of this could a be a duplicate product in a catalog. The worst that can happen is the user sees the item twice when browsing / searching. Everything still works - the customer can still purchase it, you can still ship it. It's low risk, low impact and life goes on.

There are situations though where global uniqueness is a real requirement, for example, choosing a username during a new user registration process. The problem here is that if a duplicate occurs you may have a security issue, or maybe neither user can log in. It depends on your application of course, but this may be considered high risk.

The solution to this is to use the reservation pattern where we 'reserve' the username from some fully consistent store before creating the user. When the user is successfully created, we then confirm the reservation, via a process manager. Reservations have an expiry so if for some reason the user is not created, i.e. they abandoned the process, the username is eventually released for someone else to use. The worst thing that can happen is that a reserved/unconfirmed username is unavailable to other users for whatever duration you decide to set as an expiry.

A word of caution though - this pattern introduces a single point of failure. If the reservation system is not available, new users won't be able to sign up. Therefore this should only be considered if you *absolutely* need to have uniqueness (which is far less often than you think). Of course, you'll keep this component separate from your authentication and application so existing users can still log in, if it does go down :)

From a disaster recovery perspective, a username reservation system should be rebuildable from domain events.

In the end, it's all about risk analysis.

RavenDB and OWIN Webinar

Yesterday I did a webinar for RavenDB 3's Mystery Feature #2 - OWIN support. I'd like to thank Oren, Daniel and the HR team for giving me the opportunity to work on this. 

If you notice any typos, or things aren't as smooth as they should be, it's because it was my first webinar and I only starting preparing 45 mins before hand.

( It's tough to listen to my own voice :s )

NDC-London Talk on Advanced Nancy

On wednesday, 4th of December, I'll be delivering a talk at NDC London on some of the more advanced aspects of Nancy FX. The outline of the talk (to be finalized) looks something like this:

  • Dependency injection
  • Authentication and authorization
  • Content Negotiation
  • Validation
  • Extending and overriding
  • Logging and Diagnostics
  • Error handling and Custom Status Code Handling
  • Combining Nancy with Signalr via OWIN

It appears I am competing with Jon Skeet, Dan North and few other heavy hitters (as well as beer o'clock) so I expect this to be a more intimate affair.

Hope to see you there.

Introducing Owin.Testing

Owin.Testing is discontinued. Use Microsoft.Owin.Testing instead.

Continuing in my mini-series of introductions, Owin.Testing (nuget, github) is an OSS helper library to host your OWIN application and, by using Owin.HttpMessageHandler, make in-memory testing of OWIN applications simple, fast, and environment dependency free. The method by which your application is invoked is using an HttpClient - a similar mechansim from which your users will be interacting with your application. Cookies support is supported by default (and can be turned off).

An example best demonstrates how this works:

//Arrange
OwinTestServer owinTestServer = OwinTestServer.Create(builder => new Startup().Configuration(builder));
HttpClient httpClient = owinTestServer.CreateHttpClient();

//Act
HttpResponseMessage response = await httpClient.GetAsync("http://localhost");

//Assert
response.StatusCode.Should().Be(HttpStatusCode.OK);
  1. Line #1 is how we create the test server. The Startup class is the class that defines your OWIN application.
  2. Line #2 is how we create the HttpClient that is internally configured to invoke your OWIN app's root application delegate.
  3. Line #3 & #4 is how we invoke and assert our application. (The assertion here uses the excellent FluentAssertions library).

And that's it! Fast, in-memory, CI friendly OWIN testing :)

A note on Owin.Testing history - I originally started this based of code from Gate. Later, Microsoft.Owin.Testing appeared which is similar, but has since been delisted. I have used some of the code from that. As far as I know at this point, there are no plans to revive Microsoft.Owin.Testing. So for the time being, I'll be continuing to maintain this.

A word NEventStore concurrency

NEventStore implements optomistic concurrency for Commits based on the StreamId and CommitSequence (and BucketId in v5). The exact implementation of which depends on the persistence engine. For instance, for SQL storage engines it is enforced by a primary key. This means if you open the same stream twice concurrently, add a commit, and close, you will get a ConcurrencyException:

IStoreEvents store = Wireup
	.Init()
	.UsingInMemoryPersistence()
	.Build();

Guid streamId = Guid.NewGuid();
using (var stream1 = store.OpenStream(streamId, 0, int.MaxValue))
using (var stream2 = store.OpenStream(streamId, 0, int.MaxValue))
{
	stream1.Add(new EventMessage{ Body = "test"});
	stream2.Add(new EventMessage{ Body = "test"});
	stream1.CommitChanges(Guid.NewGuid());
	stream2.CommitChanges(Guid.NewGuid()); // throws ConcurrencyException
}

Making Visual Studio 2012 extensions work with VS 2013 Preview

I 've been taking the new VS preview for a spin, mainly because of the new task aware call stack. Unfortunately, many of the extensions in the Visual Studio gallery are not "compatible". Well, many of them probably are compatible as VS2013 does not have major architectural changes from VS2012. Not being able to live without StopOnFirstBuildError, and Hide Main Menu here is a trick to make them compatible again. Warning: may void your warranty!

  1. Download the VSIX 
  2. Open the VSIX in 7-zip 
  3. Right-Click and edit 'extension.vsixmanifest':



  4. Add entry for VS2013:



  5. Close notepad. 7-Zip will prompt you to save the archive.
  6. Install :)