Damian Hickey

Mostly software and .NET related. Mostly.

Decrapify your Visual Studio installation

Fed up with the Visual Studio spamming your Programs and Features list? Make your voice heard over here.

In mean time, a lot of MSIs are silently uninstallable so you can use PC Decrapifier to remove them in one go.

Getting contextual state into a RavenDB's document store listener

Scenario: In multi-thread message driven application, I want to add contextual metadata to each and every modified and stored document. I would like to encapsulate this feature in such a way as so that it is done in one place and does not burden the event handlers with the responsibility.

An event handler may be supplied by seperate assembly and a contrived one that creates documents from events may look something like this:

public class MyEventHandler : IHandle<MyEvent>()
{
    private readonly IDocumentStore _store;
    
    public MyEventHandler(IDocumentStore store)
    {
        _store = store;
    }
    
    public void Handle(MyEvent @event)
    {
        using(var session = _store.OpenSession())
        {
            var doc = new MyDocument { // Populate properties from message };
            session.Store(doc);
            session.SaveChanges();
        }
    }
}

This handler is typically invoked at the point where the message enters this system in the main application and then

public void OnMessage<T>(Message<T> message)
{
    var handlers = _handlerResolver.ResolveAll<T>();
    foreach(var handler in handlers)
    {
        handler.Handle(message.Body);
    }
}

This method is reentrant and may (will) be called from more than one thread as the same time.

I would like to add contextual information, here message.Id, to each document that is created or modified as a result handling this message. Do do so I need to intercept the document etity instance just before it is saved. This is achieved through implementing the IDocumentStoreListener interface which looks like...

public interface IDocumentStoreListener
{
    void AfterStore(string key, object entityInstance, RavenJObject metadata);
    bool BeforeStore(string key, object entityInstance, RavenJObject metadata);
}

DocumentStoreListeners are registered before initializing the document store and like the DocumentStore, the listener's lifecyle is singleton with respect to the main application. Therefore it must be reentrant and thread safe.

var store = new DocumentStore { // Uri = etc };
store.RegisterListener(new EventIdDocumentStoreListener);
store.Initialize();

...where my implementation may look something like this (just BeforeStore() implemention shown for berevity):

public class EventIdDocumentStoreListener : IDocumentStoreListener
{
    public const string PropertyName = "LastMessageId";
    
    public bool BeforeStore(string key, object entityInstance, RavenJObject metadata)
    {
        RavenJToken token = RavenJToken.FromObject(messageId);
        if (metadata[PropertyName] == null)
        {
            metadata.Add(PropertyName, token);
        }
        else
        {
            metadata[PropertyName] = token;
        }
        return false;
    }
}

See the problem here? I have no obvious mechanism to supply the messageId into this method from which the RavenJToken is then constructed from. The solution I have arrived at is to use thread local storage, specifically .NET4's ThreadLocal<T>. This will allow me to store messageId value in a place that is specific to the running thread. I set it in the OnMessage dispatcher and pick it up in the listener. This ThreadLocal instance is also singleton with respect to the main application.

Now our listener will depend on a ThreadLocal:

public class EventIdDocumentStoreListener : IDocumentStoreListener
{
    public const string PropertyName = "LastMessageId";
    private readonly ThreadLocal<Guid> messageIdThreadLocal;
    
    public EventIdDocumentStoreListener(ThreadLocal<Guid> messageIdThreadLocal)
    {
        _messageIdThreadLocal = messageIdThreadLocal;
    }
    
    bool BeforeStore(string key, object entityInstance, RavenJObject metadata)
    {
            RavenJToken token = RavenJToken.FromObject(_messageIdThreadLocal.Value);
            if (metaData["LastMessageId"] == null)
            {
                metaData.Add(PropertyName, token);
            }
            else
            {
                metaData["LastMessageId"] = token;
            }
        }
    }
}

And our OnMessage dispatcher can set the message ID for the specific thread before calling the handlers:

public void OnMessage<T>(Message<T> message)
{
    var handlers = _handlerResolver.ResolveAll<T>();
    _messageIdThreadLocal.Value = message.Id;
    foreach(var handler in handlers)
    {
        handler.Handle(message.Body);
    }
}

So that is how I supply contextual state to a listeners in a safe and transparent manner.

Notes
- This code above is to demonstrate using ThreadLocal to pass contextual state into a RavenDB IDocumentStoreListener. It does not represent a production quality solution :)
- This solution does not work in an asp.net / wcf environment. Modifying above to do that is left as an exercise to the reader (hint: HttpContext.Items)

Getting a document's metadata in RavenDB via client API without loading the document

In an upcoming build of RavenDB (current is 588) you will be able get document's metadata wiithout having to loading it:

JsonDocumentMetadata docMetaData = store.DatabaseCommands.Head(string key);
...where docMetaData will be null when the document does not exist.

If all you want to do is check if it exists, one can use an extension method (guards omitted for berevity):

public static cass IDocumentStoreExtensions
{
	public static bool Exists(this IDocumentStore documentStore, string key)
	{
		return documentStore.DatabaseCommands.Head(key) != null;
	}
}

Questions to ask whenever set based business concerns come up in a DDD-CQRS-ES based application

One of the biggest brain speed bumps for people who come from a centralized data-model first \ RDMBS world is the issue of duplicates. "What if we have duplicate names?" "What if we have duplicate accounts?" etc. This can be of special concern in distributed systems, where your aggregate roots may be created in different places.

Before I engage in any sort of implementation, I have several business questions that must be answered first:

  1. What is the business impact of something like this happening?
  2. Under what exact circumstances / business operations could this issue occur?
  3. What are the chances of this happening? Or, how big is the window of opportunity for this to occur?
  4. Is it possible to mitigate the chances of it happening through design of the business operations?
  5. Can it be automatically detected?
  6. How quickly can it be detected after the issue occurs?
  7. Can it be compensated (fixed) after occurring?
  8. Can the compensation be automated and transparent?

99% of the time, it turns out that set based concerns, aren't.

Putting exception message into NLog's Chainsaw log4j xml output

It doesn't appear possible to get exception infomation into the chainsaw target renderer, so I got around this by extending the Log4JXmlEventLayout and butchering the xml.

Chalk this one up as a nasty hack, it's not pretty.

[Target("ChainsawExtended")]
public class ChainsawExtendedTarget : ChainsawTarget
{
    private readonly Log4JXmlEventLayoutExtended _layout = new Log4JXmlEventLayoutExtended();

    public override Layout Layout
    {
        get { return _layout; }
        set { }
    }

    private class Log4JXmlEventLayoutExtended : Log4JXmlEventLayout
    {
        protected override string GetFormattedMessage(LogEventInfo logEvent)
        {
            string s = base.GetFormattedMessage(logEvent);
            if(logEvent.Exception == null)
            {
                return s;
            }
            s = s.Replace("<log4j:event", "<log4j:event xmlns:log4j=\"http://nlog-project.org/dummynamespace/\"");
            var element = XDocument.Parse(s);
            var messageElement = element.Descendants().Where(e => e.Name.LocalName == "message").Single();
            messageElement.Value += Environment.NewLine + logEvent.Exception;
            s = element.ToString();
            s = s.Replace(" xmlns:log4j=\"http://nlog-project.org/dummynamespace/\"", string.Empty);
            return s;
        }
    }
}

Ninject's InjectAttribute on an interface member

Say you interface where you wish to property / method inject a service when activating implementors:

public interface IServiceConsumer
{
	[Inject]
	ISomeService Service { get; set; }

	[Inject]
	void SetService(ISomeService service);
}

The problem is that implementors of this interface will not 'inherit' these [Inject] attributes, by design. Thus Ninject won't see the attribute and won't inject when activating.

To solve this I've created a custom injection heuristic that inspects a member type's interface map to see if the member of concern:

  1. implements an interface method that ithat has the desired attribute.
  2. is a interface property set_ method whose corresponding property has the desired attribute.

 It also works with your own custom injection attribute if you are using one with via the T type paramater.

public class InterfaceInjectAttributeInjectionHeuristic<T> : NinjectComponent, IInjectionHeuristic
	where T:Attribute
{
	public bool ShouldInject(MemberInfo member)
	{
		foreach (Type @interface in member.DeclaringType.GetInterfaces())
		{
			InterfaceMapping interfaceMapping = member.ReflectedType.GetInterfaceMap(@interface);
			int index = Array.IndexOf(interfaceMapping.TargetMethods, member);
			if (index < 0)
			{
				continue;
			}
			MethodInfo interfaceMethod = interfaceMapping.InterfaceMethods[index];
			Contract.Assume(interfaceMethod != null);
			member = GetProperty(interfaceMethod) ?? interfaceMethod;
			return member.GetCustomAttributes(true).Any(a => a.GetType() == typeof(T));
		}
		return false;
	}

	static MemberInfo GetProperty(MethodInfo method)
	{
		bool takesArg = method.GetParameters().Length == 1;
		bool hasReturn = method.ReturnType != typeof(void);
		if (takesArg == hasReturn)
		{
			return null;
		}
		return takesArg ? method.DeclaringType.GetProperties().Where(prop => prop.GetSetMethod() == method).FirstOrDefault() : null;
	}
}

Now whether this is actually  a good idea or not, I'll leave it you to decide, but it helped me solve a problem. Feedback always welcome.

For completeness here are my tests:

using Ninject;
using Ninject.Selection.Heuristics;
using Xunit;

public class InterfaceInjectAttributeInjectionHeuristicTests
{
	public class Given_a_kernel_resolved_component
	{
		private readonly MyComponent _sut;

		public Given_a_kernel_resolved_component()
		{
			var standardKernel = new StandardKernel();
			standardKernel.Components.Add<IInjectionHeuristic, InterfaceInjectAttributeInjectionHeuristic<InjectAttribute>>();
			standardKernel.Bind<ISomeService>().To<SomeServiceImpl>();
			_sut = standardKernel.Get<MyComponent>();
		}

		[Fact]
		public void Then_Service_should_not_be_null()
		{
			Assert.NotNull(_sut.Service);
		}

		[Fact]
		public void Then_SetSerervice_should_be_called()
		{
			Assert.True(_sut.SetServiceCalled);
		}
	}
}

public interface ISomeService {}

public interface IServiceConsumer
{
	[Inject]
	ISomeService Service { get; set; }

	[Inject]
	void SetService(ISomeService service);
}

public class SomeServiceImpl : ISomeService { }

public class MyComponent : IServiceConsumer
{
	public ISomeService Service { get; set; }

	public void SetService(ISomeService service)
	{
		SetServiceCalled = service != null;
	}

	public bool SetServiceCalled { get; private set; }
}

CA1810 - A reason to not disable this rule

Run this code and you'll see why the rule is there in the first place.

 

using System;
using System.Diagnostics;
using System.Windows;

internal class Program
{
	private static void Main(string[] args)
	{
		const int upper = 1000000;
		var stopwatch = new Stopwatch();
		stopwatch.Start();
		for (int i = 0; i < upper; i++)
		{
			string name = WithoutStaticCtor.DependencyProperty.Name;
		}
		stopwatch.Stop();
		long withoutStaticCtorTicks = stopwatch.ElapsedTicks;
		stopwatch.Restart();
		for (int i = 0; i < upper; i++)
		{
			string name = WithStaticCtor.DependencyProperty.Name;
		}
		stopwatch.Stop();
		long withStaticCtorTicks = stopwatch.ElapsedTicks;
		Console.WriteLine(string.Format("With static ctor: {0} ticks", withStaticCtorTicks));
		Console.WriteLine(string.Format("Without static ctor: {0} ticks", withoutStaticCtorTicks));
		Console.WriteLine(string.Format("Without static ctor is over {0} times faster!", withStaticCtorTicks / withoutStaticCtorTicks));
		Console.ReadLine();
	}
}

public class WithStaticCtor : DependencyObject
{
	public static readonly DependencyProperty DependencyProperty = DependencyProperty.Register("Prop", typeof(int), typeof(WithStaticCtor));

	static WithStaticCtor()
	{
		int i = 1;
	}
}

public class WithoutStaticCtor : DependencyObject
{
	public static readonly DependencyProperty DependencyProperty = DependencyProperty.Register("Prop", typeof(int), typeof(WithoutStaticCtor));
}

The performance penalty for a single access to a static field on a class with a static contructor s almost immeasurable, but for something like WPF, it could add up.

Beatport ditches flash, finally!

.. and it only took them 4 years after this conversation.

from: Customer Support comm****@beatport.com via factorylabs.com
reply-to: comm***@beatport.com
to: dhi****@gmail.com
date : 30 July 2007 23:42
subject: Can you offer us some feedback?

Hello,

This is Clayton with Beatport Customer Support. I see that you haven't used your account in a month or two. We really appreciate the relationship that we had with you, and I feel that it's important to reach out and talk to our customers about what we can do to make the site better. I would love to hear any suggestions or comments that you may have. Please be as brutally honest with me as possible. Was there an issue we couldn’t resolve, or a need that we weren't able to meet? Did you find your music elsewhere? Etc.. If you still use Beatport with a secondary account, but would like to make some suggestions please feel free to do so, noting that you are still a Beatport user.

I want Beatport to be the best and I think that can only be accomplished by really listening to what our customers have to say about our site and service.

I look forward to hearing from you,

Clayton Olson
303.565.2688
Beatport Customer Service

To which I responded:

from: Damian Hickey dhi***@gmail.com
to: comm***@beatport.com
date: 1 August 2007 23:35
subject: Re: Can you offer us some feedback?
   
Hello Clayton,

Thank you for emailing me, I welcome to opportunity to pass on some constructive critisism.

There are a lot of good things about your site, 320 kbps MP3, wav's, extensive library that's regularly updated etc, but, for the sake of berevity, I'll concentrate on the parts that I feel are bad / need improving.

  • All flash site... where do I begin with this...
    • Doesn't resize to fit my screen (1920x1200 laptop, triple monitor at 1600x1200each desktop (browser only max in one monitor though)). It's a tiny green box surronded by big gray area.
    • Browser Back / Forward buttons broken. Re-implemented and a little funky in your site.
    • Can't link to anything. "Hey buddy check this track out...er..."
    • Can't bookmark anything. (I use google bookmarks to sync between machines)
    • Can't resize the tiny font. While I like small fonts, the fact that you are using flash denies the user the abilty to resize the font using the browser built-in abilty.
    • Other HTML / browser / platform things don't work - followed link colour, non-standard scroll bars (scroll functionality on my laptop touch pad doesn't work), Find-In-Page, browser username & password remembering doesn't work, rocker navigaton for back & forward broken, mouse gestures don't work (granted those last two are power user tools), etc...
    • You aren't doing yourself any favours with respect to SEO, google can't index you... (and as a business that's gotta hurt!)
    • No hackable urls. i.e. http://www.beatport.com/genres/house or http://www.beatport.com/charts/progressive/2007/07
  • No RSS feeds. I don't want the weekly email (simply don't read it). I want RSS feeds for genres, charts, labels, artists, search results etc.RSS feeds allow the user to target and subsribe to specific information they want to track.
  • More format support, OGG & Flac. (This may explode your storage requirements, I'd be happy to wait for a transcode and notification when available to download).

  • $1 wav handling fee per track is seriously excessive. You have to store the WAV somewhere and serve it somehow. Say a WAV file is 100MB. Amazon S3 storage would charge you 15c a month to store 10 of them. Amazon S3 would charge you 18c to transfer it 10 times. If 10 people a month bought the WAV, your S3 costs would be 33c. Which is $0.033 handling costs per purchase. Now that is a back of the envelope calculation and there are other factors, maybe 10c or 20c would be more appropriate (peferable 0c for FLAC).

  • Package purchased tracks as a single download. Clicking download link each time is mildly irritating. (I'd be happy with a notification when availble to download). Reward those who buy a lot with an easy download process!

  • Geographic restrictions on tracks suck rightly. Probably a licening issue, but if I can't buy it, and a lot of the time it's a impulse purchase, I'm not going to look elsewhere and that artist has just lost a sale.

  • The sound quality of the preview player is poor.

  • I would like a more social aspect. Connect me with other user who have bought similar genres, tracks, artists etc so I can discover new music.

The flash is a major inhibitor to me. It realy affects the usabilty and does affect me spending my money. I've been buying from AudioJelly for a while now for some of the above reasons - html, player that queues up previews, rss feeds etc. Though their site design isn't the best, don't have charts amongst other things.

If you want to do an application, create it in Adobe Air or Silverlight and connect it to the desktop better, otherwise, go html / ajax etc.

I hope you find these comments useful and I look forward to any improvements that may happen.

Kind regards,

Damian Hickey

Now that they've ditched flash, I may actually start buying some music from them again.

Windows 8:Your time over the next 5 years

It's satire. Smile!

SystemClock

Working on a financial application, operations and calculations are often time sensitive making for tricky unit and acceptance tests. As a result, I never use DateTime.UtcNow ( or DateTime.Now ) anywhere in my code, but instead, I use Ayende's SystemTime abstraction. For this abstraction to really work, all components and systems in your application need to support this. Two core components in my application, RavenDB and EventStore have been recently updated to include this abstraction (thanks for accepting the pull requests!) and a third, Quartz.net, already has it built in. I have an outsanding pull request for NLog to support it, but in the mean time, I am using a custom build.

If you are a library or component developer and you use DateTime.UtcNow anywhere, I strongly recommend that you add support for SystemTime.

Now that I have several SystemTime classes sitting in different libraries and frameworks I'd like one place to set them all. While one can set the exact specific time via SystemTime.UctNow, I usually like the time to move forward as usual. It's not desirable for all of my log messages to have the same timestamp for instance.

SystemClock

This is the one place to get the current SystemTime that moves forward and is also optimized (an idea I pinched from NLog):

using System;
using Beam.Common.Logging;

public class SystemClock
{
    private DateTime? _systemUtcTime;
    private DateTime _start;
    private int _lastTicks = -1;
    private DateTime _lastUtcDateTime = DateTime.MinValue;

    public DateTime UtcNow
    {
        get
        {
            int tickCount = Environment.TickCount;
            if(tickCount == _lastTicks)
            {
                return _lastUtcDateTime;
            }
            if (_systemUtcTime == null)
            {
                _lastUtcDateTime = DateTime.UtcNow;
            }
            else
            {
                var progressed = (DateTime.UtcNow - _start);
                _lastUtcDateTime = _systemUtcTime.Value + progressed;
            }
            _lastTicks = tickCount;
            return _lastUtcDateTime;
        }
    }

    public void SetSystemUtcTime(DateTime systemUtcTime)
    {
        _start = DateTime.UtcNow;
        _systemUtcTime = systemUtcTime;
        _lastTicks = -1;
    }
}

From a container perspective, SystemClock's activation scope is singleton. Now all you do is point your libraries SystemTime classes and your own to this clock at startup:

SystemClock systemClock = container.Resolve<SystemClock>();
SystemTime.UtcNow = systemClock.UtcNow
EventStore.SystemTime.UtcNow = systemClock.UtcNow;
Raven.Abstractions.SystemTime.UtcNow = systemClock.UtcNow;
Quartz.SystemTime = systemClock.UtcNow;
NLog.SystemTime = systemClock.UtcNow;
...

Then set the SytemClock's UtcNow and you're all sync'd up:

systemClock.SetUtcTime(new DateTime(1999, 12, 31, 23, 59, 59));

With this, I hope to be able to do more some interesting things such as accelerating time and jumping into the future that will allow me to similuate long running acceptance tests in shorter timeframes. More on that another time.

Further reading:

Kick the DateTime.Now addiction

SystemTime versus ISystemClock – dependencies revisited

Update: The irony is not lost on me that to effectively test this class, one must abstract over Environment.TickCount :)