Jb in a nutshell

I’ve tweeted it before, Jackson tumblered it in return, but after commenting on a StackOverflow answer where the poster got it wrong, commenting:

I always seem to turn that call around.

I felt that I needed to broaden this grand hack. If you’re like me, and apparently like a lot of coders out there, you tend to always get Type.IsAssignableFrom wrong on the first try, then you need this extension method:

public static bool IsAssignableTo (this Type self, Type type)
{
    if (self == null)
        throw new ArgumentNullException ("self");
    if (type == null)
        throw new ArgumentNullException ("type");

    return type.IsAssignableFrom (self);
}

Somehow, even if IsAssignableFrom is more «left to rightish», preserving the side of the arguments a real assignment would have, I understand IsAssignableTo faster. For what it’s worth, I’ve used it extensively while writing our .net 3.5 implementation of System.Linq.Expressions, and it served me well.

Have a great friday.

Generic collections, iterators and Linq all have in common a single interface: IEnumerable<T>. If you’ve been writing C# code lately, I bet you wrote your fair share of methods taking or returning IEnumerables.

Boo (among other marvels) has a cute syntactic shorthand for it. You can write T* instead of IEnumerable[of T]. Of course people would really like to have that in C# as well. Sure T* would be nice, but it’s already used for pointers. Some other suggestions in the stackoverflow question involve using the # suffix, or {}. On my side, I quite like the idea of re-using ~. It’s only used by the bitwise complement operator in C# (and by finalizers, thanks Scott for the heads up), and if you look at it with a bit of poetry, it kind of look like a river, a stream. And what’s an IEnumerable<T> but a stream of T?

Now discussing wishes for the next version of C# is nice and all, but I felt like hacking a bit on mcs, Mono’s C# compiler tonight, and in less than half an hour of grepping through the code, I had a working version of a patch that would enable this syntactic sugar.

Let’s take a very poorly implemented set of Linq operators:

using System;
using System.Collections.Generic;

static class Enumerable {

	public static IEnumerable<T> Concat<T> (this IEnumerable<T> source, IEnumerable<T> other)
	{
		foreach (var item in source)
			yield return item;

		foreach (var oitem in other)
			yield return oitem;
	}

	public static IEnumerable<TResult> Select<TSource, TResult> (this IEnumerable<TSource> source, Func<TSource, TResult> selector)
	{
		foreach (var item in source)
			yield return selector (item);
	}

	public static IEnumerable<TResult> SelectMany<TSource, TResult> (this IEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> selector)
	{
		foreach (var item in source)
			foreach (var sub_item in selector (item))
				yield return sub_item;
	}

	public static IEnumerable<T> Where<T> (this IEnumerable<T> source, Func<T, bool> predicate)
	{
		foreach (var item in source)
			if (predicate (item))
				yield return item;
	}
}

With my patch, you could write instead:

using System;

static class Enumerable {

	public static T~ Concat<T> (this T~ source, T~ other)
	{
		foreach (var item in source)
			yield return item;

		foreach (var oitem in other)
			yield return oitem;
	}

	public static TResult~ Select<TSource, TResult> (this TSource~ source, Func<TSource, TResult> selector)
	{
		foreach (var item in source)
			yield return selector (item);
	}

	public static TResult~ SelectMany<TSource, TResult> (this TSource~ source, Func<TSource, TResult~> selector)
	{
		foreach (var item in source)
			foreach (var sub_item in selector (item))
				yield return sub_item;
	}

	public static T~ Where<T> (this T~ source, Func<T, bool> predicate)
	{
		foreach (var item in source)
			if (predicate (item))
				yield return item;
	}
}

I let the reader decides for himself which version is the most readable. It's pretty much the same debate as the one for nullable types. Also note that this is just a toy patch, mainly for the sake of digging a bit in mcs, but it’s an interesting experiment nevertheless.

Just as everyone, I learned yesterday via Jimmy the near death experience that IronRuby is undergoing. And spent some time thinking about it.

For those of you who will stumble upon this and don’t follow this blog regularly, let me add a pinch of background. I implemented the .net 3.5 version of System.Linq.Expressions in Mono, I signed the IronRuby contributor agreement on day one, and regularly made sure it runs fine on Mono along with the DLR and IronPython. In the process of doing so, I contributed fixes and improvements to IronRuby. If I’m far for being the largest external contributor, at least I have a clue about how contributing to IronRuby works. And doesn’t work.

Also I spent the last two weeks working on making the IronRuby 1.1 test suite pass on Mono. Which may be the biggest reason I feel like I have something to say, as it involved countless hours of diving into the IronRuby code to extract simple test cases.

But now to the point. The IronRuby team currently consists of one hacker. We don’t know much about the IronPython team. And everyone who wanted to work on .net with their favorite dynamic language is freaking out. To a reason.

The good news is that the code of IronPython, IronRuby and the DLR is open source, and has recently been re-licensed under the Apache2 license. The official message is that IronRuby’s fate is now in the hands of the community.

That doesn’t sound like a bright future. So far, the community has been excluded from the development process of IronRuby. It’s impossible to contribute code to the core compiler of IronRuby, let alone to the DLR which is now part of .net 4.0. The code in github is a mere mirror of an internal TFS repository, and may or may not be up to date. And until IronRuby’s divorce with Microsoft is completely consumed, it will stay like this. So, sure we can contribute to external libraries, but that’s definitely not where the fun lies, and from now on, nor where the real work will be required.

So is the solution a fork?

It’s indeed a possibility. But for knowing first hand, IronRuby’s code source is far from being a simple piece of code to comprehend and to hack on. So what will happen in a couple of months when the buzz will have settled? Who will be able to take over the development of IronRuby? I for one am interested in helping, but I have already a lot on my plate. With Microsoft excluding the community from the core development process from day one, there’s no real knowledge of the insides of IronRuby outside of Microsoft, yet.

If Microsoft employees keep working on IronRuby occasionally, and keep the current setup, the community will still be excluded, and no one outside will contribute, besides Jimmy. IronRuby will simply fall into oblivion.

Yet I think that until Microsoft completely stops funding any work on IronRuby, they will be the only one really able to make something out of it. As Jimmy suggested, letting Microsoft know that there’s an important demand for IronRuby and IronPython, is probably the best course of action. Who knows, there’s maybe still time for Microsoft to correct its course. Special bonus points if we can get Microsoft to not only accept that there’s a demand for IronRuby, but also for accepting external contributions as it would make them move forward faster.

What about this petition (deleted since) to ask the Mono developers to take over IronRuby?

Actually that’s maybe what I’ve found the most tragic, and one of the reasons that triggered this post. It’s a bit along the line of “we don’t quite want to work on it, but we definitely want to have it, so let’s ask for someone else to steer the project”. If I can’t talk for my boss Miguel, I can say that we already have a lot on our plate, and that I don’t see us taking over IronRuby. I of course will continue to make sure it runs fine on Mono. But really, if the community wants IronRuby, it will have to do something about it itself. Either by contributing code or begging Microsoft.

A possibility of course would be for Microsoft to enter a development agreement with the Mono team as they did for Silverlight and Moonlight, and I’ll gladly volunteer to join the effort, but I don’t see that happening.

As a conclusion, and on a personal note, I find the situation extremely disappointing. In my opinion, Microsoft is making a strategic mistake. The Iron* projects were somehow the fun and hip face of the .net development (outside of Mono, of course). They had a true potential to attract developers that would traditionally stay away from Microsoft and .net.

If you’re querying a LINQ provider, and if you’re dynamically creating the query, you probably stumbled upon the PredicateBuilder class.

It gives you a simple API to get a predicate in the form of an Expression Tree that a LINQ provider expects. You simply And (or Or) them together. For instance:

Expression<Func<User, bool>> is_underage = u => u.Age < 18;
Expression<Func<User, bool>> name_starts_with_a = u => u.Name [0] == 'A'

Expression<Func<User, bool>> a_is_underage = is_underage.And (name_starts_with_a);

Easy and effective. But let’s have a look at their And implementation:

public static Expression<Func<T, bool>> And<T> (this Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
    var invokedExpr = Expression.Invoke (expr2, expr1.Parameters.Cast<Expression> ());
    return Expression.Lambda<Func<T, bool>>
        (Expression.AndAlso (expr1.Body, invokedExpr), expr1.Parameters);
}

It creates a new expression tree (they are immutable), but the right hand side of the AndAlso node is an invocation of the second expression. If it works great if you compile the expression into an delegate, most LINQ providers won’t be able to optimize this form, as they’ll expect a more simple form, without nested lambdas.

I added to Mono.Linq.Expressions my own version of a PredicateBuilder which doesn’t have this issue. Let’s examine the implementation:

public static class PredicateBuilder {

    public static Expression<Func<T, bool>> True<T> ()
    {
        return Expression.Lambda<Func<T, bool>> (Expression.Constant (true), Expression.Parameter (typeof (T)));
    }

    public static Expression<Func<T, bool>> False<T> ()
    {
        return Expression.Lambda<Func<T, bool>> (Expression.Constant (false), Expression.Parameter (typeof (T)));
    }

    public static Expression<Func<T, bool>> OrElse<T> (this Expression<Func<T, bool>> self, Expression<Func<T, bool>> expression)
    {
        return Combine (self, expression, Expression.OrElse);
    }

    public static Expression<Func<T, bool>> AndAlso<T> (this Expression<Func<T, bool>> self, Expression<Func<T, bool>> expression)
    {
        return Combine (self, expression, Expression.AndAlso);
    }

    static Expression<Func<T, bool>> Combine<T> (Expression<Func<T, bool>> self, Expression<Func<T, bool>> expression, Func<Expression, Expression, Expression> selector)
    {
        CheckSelfAndExpression (self, expression);

        var parameter = CreateParameterFrom (self);

        return Expression.Lambda<Func<T, bool>> (
            selector (
                RewriteLambdaBody (self, parameter),
                RewriteLambdaBody (expression, parameter)),
            parameter);
    }

    static Expression RewriteLambdaBody (LambdaExpression expression, ParameterExpression parameter)
    {
        return new ParameterRewriter (expression.Parameters [0], parameter).Visit (expression.Body);
    }

    class ParameterRewriter : ExpressionVisitor {

        readonly ParameterExpression candidate;
        readonly ParameterExpression replacement;

        public ParameterRewriter (ParameterExpression candidate, ParameterExpression replacement)
        {
            this.candidate = candidate;
            this.replacement = replacement;
        }

        protected override Expression VisitParameter (ParameterExpression expression)
        {
            return ReferenceEquals (expression, candidate) ? replacement : expression;
        }
    }

    static ParameterExpression CreateParameterFrom<T> (Expression<Func<T, bool>> left)
    {
        var template = left.Parameters [0];

        return Expression.Parameter (template.Type, template.Name);
    }

    static void CheckSelfAndExpression<T> (Expression<Func<T, bool>> self, Expression<Func<T, bool>> expression)
    {
        if (self == null)
            throw new ArgumentNullException ("self");
        if (expression == null)
            throw new ArgumentNullException ("expression");
    }
}

Instead of nesting lambdas, it simply rewrites a lambda and inlines the two previous expressions bodies in it. This way, you can compose predicates and make sure they’ll be usable by any LINQ provider without having to deal with it explicitely.

I also named the method AndAlso and OrElse as it’s the terminology used in the Expression Tree API for the logical And (&&) and Or (||).

What’s next for Mono.Linq.Expressions? Probably an IEqualityComparer<Expression> implementation. Any other useful feature you can think of?