Jb in a nutshell

Between .net 3.5 and .net 4.0, the Linq Expression Tree API changed quite a bit, going from a simple compiler supporting expressions only, to a full fledged compiler supporting expressions as well as statements, on top of which are implemented all DLR based language. For anyone working closely with this API, the Expression Tree spec is a great source of information, the MSDN isn’t very helpful there.

When the spec comes to describing the object model for nodes, it differentiates several kind of nodes:

The core nodes and the common nodes are those which are implemented inside System.Linq.Expressions, and common nodes are implemented on top of the core nodes, using a process that is called reducing nodes.

A reducible node is simply a node that can be compiled as a compound of core nodes. We’re not going to talk about library specific extensions, which are nodes that you would write if you were to implement, say, a LINQ provider. We’ll focus on reducible nodes.

I took some time last year to implement a few helper nodes that are not available in .net 4.0, but, according to the spec, may be included in a future release. They’re mapped to similar C# constructs, and are pretty useful it you generate complex code using expression trees. Here’s the list of custom expressions I added in Mono.Linq.Expressions, my project to complement the System.Linq.Expression namespace:

• DoWhileExpression
• ForEachExpression
• ForExpression
• UsingExpression
• WhileExpression

Even if those map to a C# statement, they’re suffixed with `Expression` for consistency with the whole System.Linq.Expressions namespace. Using them is as simple as, heh, adding a using directive:

using Mono.Linq.Expressions;

And using the factory methods on the CustomExpression class to create those new nodes:

var d = Expression.Parameter (typeof (DisposableType), "d");

var printAndDispose = Expression.Lambda<Action<DisposableType>> (
    CustomExpression.Using (
        d,
        Expression.Call (typeof (Console).GetMethod (
            "WriteLine", new [] { typeof (object) }), d)),
    d);

The Expression Tree API makes it easy to write custom reducible expressions, all you have to do, is to inherit from Expression, override CanReduce and NodeType, and implement the Reduce method:

public abstract partial class CustomExpression : Expression {

    public override ExpressionType NodeType {
        get { return ExpressionType.Extension; }
    }

    public override bool CanReduce {
        get { return true; }
    }

    /* ... */
}

If we take, for instance, our UsingExpression, mapping to the C# using statement, Reduce is implemented as follows:

public override Expression Reduce ()
{
    var end_finally = Expression.Label ("end_finally");

    return Expression.Block (
        new [] { variable },
        Expression.Assign (variable, disposable),
        Expression.TryFinally (
            body,
            Expression.Block (
                Expression.Condition (
                    Expression.NotEqual (variable, Expression.Constant (null)),
                    Expression.Block (
                        Expression.Call (
                            Expression.Convert (variable, typeof (IDisposable)),
                            typeof (IDisposable).GetMethod ("Dispose")),
                        Expression.Goto (end_finally)),
                    Expression.Goto (end_finally)),
                Expression.Label (end_finally))));
}

Just like a C# using, it will try to execute a block, and whether the block triggered an exception or not, it will call IDisposable.Dispose on the subject of the using statement.

Wasn’t that easy? Even if those nodes are pretty general, writing custom and specific nodes is a powerful and simple way to factorize code when generating code using the Expression Tree API.

In the end, I added in Mono.Linq.Expressions a CustomExpression type, our base class for well, custom expressions. We provide a CustomExpressionVisitor which extends the standard ExpressionVisitor to support our custom expressions, and the CSharpWriter has been updated to support them as well:

[Test]
public void Using ()
{
    var arg = Expression.Parameter (typeof (IDisposable), "arg");

    var lambda = Expression.Lambda<Action<IDisposable>> (
        CustomExpression.Using (
            arg,
            Expression.Call (typeof (Console).GetMethod (
                "WriteLine", new [] { typeof (object) }), arg)),
        arg);

    AssertExpression (@" 
void (IDisposable arg)
{
    using (arg)
    {
        Console.WriteLine(arg);
    }
}
", lambda);
}

Next feature for Mono.Linq.Expressions: using Mono.Reflection to turn a delegate into an expression of this delegate.

Or how to use APIs missing from the WP7 SDK.

Monday saw the official launch of Windows Phone 7, at this occasion, I downloaded the SDK to see how it compares to MonoTouch and MonoDroid. Basically, it’s based on the Compact Framework Base Class Library, on top of which is exposed the Silverlight UI API.

The Compact Framework has historically targeted devices with limited capabilities. The Compact Framework BCL is a trimmed down version of the vanilla .net BCL. One notable missing feature is runtime code generation through System.Reflection.Emit. As such, you can’t use DynamicMethods, nor can you compile assemblies on the device, and nor can you compile Expression Trees. You can expect to find missing tidbits sprinkled throughout the whole BCL. But luckily for us, there’s Mono.

The Mono BCL being licensed under the MIT/X11, you can easily take C# code from our libraries and use it inside your own applications. And that whenever our code is not tied to a runtime feature. This means that you couldn’t bring System.Reflection.Emit to WP7, but that you could probably bring XmlDocument with a bit of work.

Basically, this reminded me of what db4o did to bring support for Expression Trees to the Compact Framework using code from Mono. I figured I’d try to do the same for WP7. If you played with the WP7 SDK, you probably know that you have access to Expression Trees, and the compiler can emit them without any issue. But LambdaExpression and Expression of T don’t have their Compile method you’re used to. And WP7 is missing IQueryable and folks.

So as an example, I extracted our Expression Tree interpreter, which is used, guess what, when you can’t compile Expression Trees at runtime, and our IQueryable support. Here we are, by just referencing this System.Linq.dll, you can write code such as:

using System.Linq;

var data = new [] { 1, 2, 3, 4, 5, 6, 7, 8 };

var query = data.AsQueryable ();
query.Where (i => i % 2 == 0);

int count = query.Count ();

Or:

using System.Linq.Expressions;

var p1 = Expression.Parameter (typeof (string), "x");
var p2 = Expression.Parameter (typeof (string), "y");

Expression<Func<string, Func<string, Func<string>>>> e =
    Expression.Lambda<Func<string, Func<string, Func<string>>>> (
        Expression.Lambda<Func<string, Func<string>>> (
            Expression.Lambda<Func<string>> (
                Expression.Call (
                    typeof (string).GetMethod ("Concat", new [] { typeof (string), typeof (string) }),
                    new [] { p1, p2 }),
                new ParameterExpression [0]),
            new [] { p2 }),
        new [] { p1 });

var f = e.Compile ();
var f2 = f ("Hello ");
var f3 = f2 ("World !");

var result = f3 ();

(Crazy, I know)

You can simply take the assembly and reference it from your projects, or compile yourself the code. Let me know if this has been useful to you.

To conclude, I’d say don’t hesitate to take code from Mono to fill the voids in the WP7 SDK. Who knows if you won’t end up contributing missing parts or filing bugs afterwards.

I’ve tweeted it before, Jackson tumblered it in return, but after commenting on a StackOverflow answer where the poster got it wrong, commenting:

I always seem to turn that call around.

I felt that I needed to broaden this grand hack. If you’re like me, and apparently like a lot of coders out there, you tend to always get Type.IsAssignableFrom wrong on the first try, then you need this extension method:

public static bool IsAssignableTo (this Type self, Type type)
{
    if (self == null)
        throw new ArgumentNullException ("self");
    if (type == null)
        throw new ArgumentNullException ("type");

    return type.IsAssignableFrom (self);
}

Somehow, even if IsAssignableFrom is more «left to rightish», preserving the side of the arguments a real assignment would have, I understand IsAssignableTo faster. For what it’s worth, I’ve used it extensively while writing our .net 3.5 implementation of System.Linq.Expressions, and it served me well.

Have a great friday.

Generic collections, iterators and Linq all have in common a single interface: IEnumerable<T>. If you’ve been writing C# code lately, I bet you wrote your fair share of methods taking or returning IEnumerables.

Boo (among other marvels) has a cute syntactic shorthand for it. You can write T* instead of IEnumerable[of T]. Of course people would really like to have that in C# as well. Sure T* would be nice, but it’s already used for pointers. Some other suggestions in the stackoverflow question involve using the # suffix, or {}. On my side, I quite like the idea of re-using ~. It’s only used by the bitwise complement operator in C# (and by finalizers, thanks Scott for the heads up), and if you look at it with a bit of poetry, it kind of look like a river, a stream. And what’s an IEnumerable<T> but a stream of T?

Now discussing wishes for the next version of C# is nice and all, but I felt like hacking a bit on mcs, Mono’s C# compiler tonight, and in less than half an hour of grepping through the code, I had a working version of a patch that would enable this syntactic sugar.

Let’s take a very poorly implemented set of Linq operators:

using System;
using System.Collections.Generic;

static class Enumerable {

	public static IEnumerable<T> Concat<T> (this IEnumerable<T> source, IEnumerable<T> other)
	{
		foreach (var item in source)
			yield return item;

		foreach (var oitem in other)
			yield return oitem;
	}

	public static IEnumerable<TResult> Select<TSource, TResult> (this IEnumerable<TSource> source, Func<TSource, TResult> selector)
	{
		foreach (var item in source)
			yield return selector (item);
	}

	public static IEnumerable<TResult> SelectMany<TSource, TResult> (this IEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> selector)
	{
		foreach (var item in source)
			foreach (var sub_item in selector (item))
				yield return sub_item;
	}

	public static IEnumerable<T> Where<T> (this IEnumerable<T> source, Func<T, bool> predicate)
	{
		foreach (var item in source)
			if (predicate (item))
				yield return item;
	}
}

With my patch, you could write instead:

using System;

static class Enumerable {

	public static T~ Concat<T> (this T~ source, T~ other)
	{
		foreach (var item in source)
			yield return item;

		foreach (var oitem in other)
			yield return oitem;
	}

	public static TResult~ Select<TSource, TResult> (this TSource~ source, Func<TSource, TResult> selector)
	{
		foreach (var item in source)
			yield return selector (item);
	}

	public static TResult~ SelectMany<TSource, TResult> (this TSource~ source, Func<TSource, TResult~> selector)
	{
		foreach (var item in source)
			foreach (var sub_item in selector (item))
				yield return sub_item;
	}

	public static T~ Where<T> (this T~ source, Func<T, bool> predicate)
	{
		foreach (var item in source)
			if (predicate (item))
				yield return item;
	}
}

I let the reader decides for himself which version is the most readable. It's pretty much the same debate as the one for nullable types. Also note that this is just a toy patch, mainly for the sake of digging a bit in mcs, but it’s an interesting experiment nevertheless.