Shifting Into The Next Gear: Moving The Mendix Server Runtime To Scala
In my previous article I discussed how using Java for the Mendix Server Runtime did not feel satisfactory anymore and that it felt like the right time to move to another language.
This other language has become Scala.
We have been developing in Scala for over six months now and the just released version 4.0 of the Mendix Platform (Mendix Spring 2012 Release) is the first version containing Scala code. In this article I will discuss why we have chosen Scala and how Scala compares to other candidates I came across. If you are using Java and are looking for an alternative, this will be an interesting read.
Before delving into the plethora of languages currently available to find a suitable choice, I thought the requirements our new language should fulfill. Two aspects are very important to me:
- The language should be a JVM language.
- The language should be statically typed.
The new language being a JVM language is crucial, because moving incrementally from Java is a must. Dropping all our work for a minimum of six months to do a complete rewrite is not something I would be able to sell to management, and frankly, it’s not very appealing to myself either. Frequent maintenance releases have to be cranked out, which means all the fixes from these releases have to be ported to the rewrite branch which will become insanely tedious. In addition, rewriting from scratch for a good half year is simply not that much fun. It will take months before the rewrite branch will be in a state where it remotely resembles the original version. Afterwards a couple more months will have to be spend to fix each and every bug. All of that, while cool new functionality could have been written that partners and customers were actually waiting for. So: no JVM language, no cigar.
Static typing is essential to me, because I know I am only human (and my colleagues as well, as far as I know). I will make typos and I will screw up the variable types from time to time. When I do, I’d like to know as soon as possible and not when an application, running in production with 1000 concurrent users, stumbles upon an edge case for which no unit test was made. In addition, I love refactoring, but I always want to refactor safely. Of course some methods exist to make refactoring easier in dynamically typed languages, but you’re never really sure whether you did everything right until you run your program. I need to know this at compile time.
Pick me, pick me!
Let’s turn to our contestants now. Some languages out there have a lot of potential, but do not adhere to the stated requirements. A language as Smalltalk might have a lot going for it, the “roles” concept is quite elegant for instance, but it’s not a JVM language and not statically typed. Therefore, not feasible as the new Mendix Runtime language.
Looking at the JVM languages there are quite some interesting ones:
I probably forgot some, as a lot of new languages for the JVM are coming into bloom at the moment, but these seem to be creating the most buzz currently.
Of these languages JRuby and Jython, the JVM-variants of Ruby and Python, are not suitable as they are not statically typed. The same goes for Groovy and Clojure.
Groovy used to be quite popular, but does not seem to have so much momentum anymore. The creator, James Strachan, said the following a while ago: “I can honestly say if someone had shown me the “Programming in Scala” book back in 2003, I’d probably never created Groovy”.
Clojure does have some momentum right now. However, apart from being a dynamically typed language, which is a not a good fit for me, it also has strong roots in Lisp. I think this makes the leap from Java to Clojure a bit too big.
Kotlin and Ceylon are the new kids on the block. They are interesting languages, but do not seem mature enough yet. We need a language which is ready to run applications in production.
Fantom exists for quite a while now. Fantom seems like a solid language and would solve a lot of Java’s shortcomings. Although generally a statically typed language, Fantom also involves dynamic typing. The interoperability with Java is ok, but not seamless. Integration is accomplished using a Java foreign function interface (FFI).
Xtend is a language based on Xtext. Xtend generates Java instead of compiling to byte code. This language was not really on my radar when I was doing my investigation. The first version which made it more well known was the 2.1 release last November. Before that time, Xtend did not appear to have reached an adequate maturity level yet. In addition, I’m wondering if compiling to Java could lead to transformation issues. What happens if you first generate Java using Xtend and then change something in the Java code? Is this nicely reflected in Xtend?
Scala, the Scalable Wow?
Scala was a language I kept hearing good things about (the same goes for Clojure though). In 2001 Martin Odersky, who is known for his work on the Java compiler and Java Generics, started the design of Scala, which led to a first version in 2003. The current stable version is 2.9.2 which was released last April.
I will not go into the language details as there are numerous blog posts and books that cover this topic in a much more comprehensive way than I would be able to do here. I can recommend the “Scala for Java Refugees” blog posts by Daniel Spiewak and the “Programming in Scala” book (Odersky, 2010) to start with.
What made me choose Scala above the other mentioned languages? First of all, Scala fulfills my two main requirements: it’s a JVM language and statically typed. In addition, the interoperability with Java is very good. Scala can call Java classes, interfaces and methods and the other way around without needing an FFI.
The big advantage over Java is that Scala offers functional aspects, which makes the language much more powerful. This way the code is much less verbose and more readable. That solves my main frustration with Java, a lot of times I had to write too much code for something simple.
Looking back at the other contestants I think Fantom and Xtend would be the main rivals. I did not choose Fantom as the interoperability with Java seemed somewhat more complicated and the language is not fully statically typed. For the reasons I mentioned earlier I like static typing better. Xtend was not really in the picture at the time of my investigation, but in the end I think the fact Xtend generates to Java instead of compiling to byte code, would have made me choose Scala. Generating to a language which has some fundamental flaws does not sound like the way to go long term. The JVM also has some flaws, but when those would be fixed, you don’t have to potentially wait for Java fixing those as well.
Ultimately, I think, you also have to listen to your gut feeling, as comparing all languages in excruciating detail is not possible. My gut feeling placed Scala on top.
After my personal decision to go for Scala, I started playing around with the language. During that time, I discovered programming had become fun again! I’m not saying I was frustrated with Java all the time, but the times I was very happy with the code I had written had become quite sporadic. With Scala, however, every new language feature I discovered made me able to solve something a lot more elegantly. Also, the lines of code I needed were on average 50% less than when I would program in Java and the code became more readable.
Let’s have a look at some Scala code. Remember the Java example of filtering virtual attributes from the previous post?
In Scala this can be written as follows:
No more superfluous semi-colons and parentheses and far more to the point!
In the comments section of the previous post, similar code snippets were proposed as alternatives to the Java variant (although in other languages). I think they all have in common that the code is far more concise and also more readable. Although you may need some time to get familiar with the syntax when you are used to Java, I think the Scala code snippet is easier to read.
Eye-tracking experiments have shown that for program comprehension, idiomatic Scala constructions can be grasped 30% faster than their Java equivalents (Dubochet, 2009). When code takes less time to understand, it will also take less time to maintain.
Shifting into the next gear
Our search for a new language has resulted in the choice for Scala. Scala is a JVM language, statically typed and combines functional and object-oriented aspects. Its conciseness leads to increased productivity, better maintainability and more fun coding. For me, it definitely feels we’re shifting into the next gear by adopting a language with the expressiveness Scala has.
What lies ahead? We are gradually turning Java code in to Scala and will continue to do so. In a next post I plan to share my experiences so far.
With regard to the users of the Mendix platform, if enough interest exists, we could generate templates for Scala actions which can be called from microflows in your Mendix project. Another interesting direction is to facilitate the creation of third party modules in Scala. Making your own module is already possible in the Mendix Runtime, but I would like to take this to the next level.
Let me know what you think, what would you like to see?
That’s all for now, have fun with the new Mendix, hope you’ll enjoy it!
Dubochet, G. (2009). Proceedings of the 21st Working Conference on the Psychology of Programmers Interest Group, p. 174-187. Exton, Chris ; Buckley, Jim (eds.)
Limerick, Ireland: University of Limerick.
Odersky, M. Spoon, L., Venners, B. (2010). Programming in Scala Second Edition. Artima Press, Waltnut Creek, California.
Photo by John Liu