Jb in a nutshell

Just as everyone, I learned yesterday via Jimmy the near death experience that IronRuby is undergoing. And spent some time thinking about it.

For those of you who will stumble upon this and don’t follow this blog regularly, let me add a pinch of background. I implemented the .net 3.5 version of System.Linq.Expressions in Mono, I signed the IronRuby contributor agreement on day one, and regularly made sure it runs fine on Mono along with the DLR and IronPython. In the process of doing so, I contributed fixes and improvements to IronRuby. If I’m far for being the largest external contributor, at least I have a clue about how contributing to IronRuby works. And doesn’t work.

Also I spent the last two weeks working on making the IronRuby 1.1 test suite pass on Mono. Which may be the biggest reason I feel like I have something to say, as it involved countless hours of diving into the IronRuby code to extract simple test cases.

But now to the point. The IronRuby team currently consists of one hacker. We don’t know much about the IronPython team. And everyone who wanted to work on .net with their favorite dynamic language is freaking out. To a reason.

The good news is that the code of IronPython, IronRuby and the DLR is open source, and has recently been re-licensed under the Apache2 license. The official message is that IronRuby’s fate is now in the hands of the community.

That doesn’t sound like a bright future. So far, the community has been excluded from the development process of IronRuby. It’s impossible to contribute code to the core compiler of IronRuby, let alone to the DLR which is now part of .net 4.0. The code in github is a mere mirror of an internal TFS repository, and may or may not be up to date. And until IronRuby’s divorce with Microsoft is completely consumed, it will stay like this. So, sure we can contribute to external libraries, but that’s definitely not where the fun lies, and from now on, nor where the real work will be required.

So is the solution a fork?

It’s indeed a possibility. But for knowing first hand, IronRuby’s code source is far from being a simple piece of code to comprehend and to hack on. So what will happen in a couple of months when the buzz will have settled? Who will be able to take over the development of IronRuby? I for one am interested in helping, but I have already a lot on my plate. With Microsoft excluding the community from the core development process from day one, there’s no real knowledge of the insides of IronRuby outside of Microsoft, yet.

If Microsoft employees keep working on IronRuby occasionally, and keep the current setup, the community will still be excluded, and no one outside will contribute, besides Jimmy. IronRuby will simply fall into oblivion.

Yet I think that until Microsoft completely stops funding any work on IronRuby, they will be the only one really able to make something out of it. As Jimmy suggested, letting Microsoft know that there’s an important demand for IronRuby and IronPython, is probably the best course of action. Who knows, there’s maybe still time for Microsoft to correct its course. Special bonus points if we can get Microsoft to not only accept that there’s a demand for IronRuby, but also for accepting external contributions as it would make them move forward faster.

What about this petition (deleted since) to ask the Mono developers to take over IronRuby?

Actually that’s maybe what I’ve found the most tragic, and one of the reasons that triggered this post. It’s a bit along the line of “we don’t quite want to work on it, but we definitely want to have it, so let’s ask for someone else to steer the project”. If I can’t talk for my boss Miguel, I can say that we already have a lot on our plate, and that I don’t see us taking over IronRuby. I of course will continue to make sure it runs fine on Mono. But really, if the community wants IronRuby, it will have to do something about it itself. Either by contributing code or begging Microsoft.

A possibility of course would be for Microsoft to enter a development agreement with the Mono team as they did for Silverlight and Moonlight, and I’ll gladly volunteer to join the effort, but I don’t see that happening.

As a conclusion, and on a personal note, I find the situation extremely disappointing. In my opinion, Microsoft is making a strategic mistake. The Iron* projects were somehow the fun and hip face of the .net development (outside of Mono, of course). They had a true potential to attract developers that would traditionally stay away from Microsoft and .net.

Quite a number of friends pinged me about the recent release of the CCI, under the Ms-PL, and were curious about my take on it, and its effect on Cecil and its ecosystem.

First of all, there’s a bit of a story here, and I’ll write it here for those who like me, love software history. Back in the years 2003 and 2004, I was working with Thomas Gil, one of my mentor and programming hero, on one of the first static aspect weaver on .net, AspectDNG, now abandoned. I was actively researching better ways to do CIL injection.

We went from raw IL text manipulation, to Reflection and Reflection.Emit using Lutz’s ILReader library, to RAIL, until I decide to work on Cecil.

In the meantime, I’ve stumbled upon ILMerge, a tool from Mike Barnett, and mailed him to ask what powered the tool, and he put me in contact with Herman Venter, the man behind the CCI effort. I wrote Herman a couple of mails, in a terrible English, and begged him to push for a release of the CCI under a license we could use in AspectDNG. That was in March 2004.

As you can guess, it quite didn’t work out at that time, so I started working on Cecil. A few weeks after, Miguel blogged about the need of such library. He already had the Mono Linker in mind. I mailed him, got SVN access, checked in the beginning of Cecil, got Sébastien interested, etc.

I had the opportunity to be invited by Microsoft to attend an informal AOP workshop the year later, and to met with Herman, which I remember as a very nice person. I am not sure he remembers the terribly shy kid that did a terrible presentation in a terrible English. But all in all, I’m happy that five years later, my request went through.

Now the CCI release in its own CodePlex page is not really a big event, as it was already released and licensed under the Ms-PL, as it’s part of Sandcastle.

Anyway, Cecil is quite mature in its current form, it’s used by a fair number of (known) applications (please help to improve the list), and I’m currently working on two things.

The first one is a refactoring of Cecil, which vastly reduces memory consumption as well as reading/writing time. Hopefully I’ll have a beta in a month or so. We have great plans for this version of Cecil, and it’s consuming a lot of my time, more on this later.

The second one is an extensible decompiler, Cecil.Decompiler, that will greatly benefit from the Cecil refactoring. The time I dedicate to it is a bit phagocyted by the Cecil refactoring right now, but it’s certainly one of my favorite project.

The CCI is a combination of Cecil, the decompiler, and something to write a decompiled AST back, which will be the natural evolution of the Cecil decompiler. Note that the CCI decompilation/compilation process is not extensible. Now that it’s open source, you can hack it yourself, sadly, the CCI code is well, a bit messy to be polite, or not exactly a joy to read. Also you probably won’t be able to contribute back to the CCI.

Anyway, it does its job alright, and so does Cecil. Choice is always good, let’s welcome the CCI in the small family of such tools. I, for one, will surprisingly keep hacking on and with Cecil :)

To conclude on a very positive note, the fantastic thing about this release is that the CCI contains a fully managed PDB reader and writer. That’s great news as so far, we failed to get any details about this file format. This means that we can now implement a fully managed Mono.Cecil.Pdb support, and that’s just great.

UPDATE: it appears that only the PDB reader is fully managed, the PDB writer is just a wrapper over the COM stuff, just like the current implementation of Mono.Cecil.Pdb. Well, at least it’s a start.

I’ll be offline until April 20th.

As I was writing in my last entry, announcing the most awesome Cecil.Decompiler library, I took my last week off to host a CodeCamp event at my parents place in the quiet place of Ardèche, France.

I had the pleasure to host, by order of arrival,

• Romain Verdier
• Patrice Lamarche
• Vincent Bourdon
• Robert Pickering
• Mathieu Szablowski
• Sébastien Lebreton

While I was working on the core of the decompiler, I paired with Romain to work on improving our pattern matching story, to refine a basic AST to something more evolved. We still clearly need to do some work here, as we’re not completely satisfied with what we came with. Robert with his experience of F# and pattern matching then helped Romain.

Patrice worked on a NUnit addin which allow us to write tests that are executed for both optimized and non optimized code. He then joined Vincent to work on the Visual Basic language support. Vincent also wrote a ColoredConsoleFormatter, which adds some color to my tests in the console routines.

Robert started a Windows.Forms front tend in F# a while ago, but we decided to switch to C#, to lower the entry barrier for contributions. The inimitable Sébastien took the responsibility to write the front-end for the next millennium. I’ll force him to blog on it ASAP.

Mathieu took the pictures. He sadly was busy working on a presentation he had to do for a client next week.

On the next monday, Vincent told me something like: wow it’s quite depressing, I barely had a single laugh today, while we were laughing every five minutes during the last few days.

So we maybe weren’t the most productive crowd of week-end hackers, but at least we had tons of fun.

I guess that if I had a lesson to take from the event, it would have been to prepare it a little bit more, and have more fine grained tasks, but I also loved the organic way it evolved.

Thanks guys for the week-end, hopefully we’ll manage to organize more of those CodeCamp.