Helping you drive digital innovation
Subscribe
RSS Feed of the Mendix Blog
Thanks for Subscribing

Keep an eye out for Mendix resources coming straight to your inbox.

Shifting Into The Next Gear: Moving The Mendix Server Runtime To Scala

on May 24, 2012

Share:

In my previous article I discussed how using Java for the Mendix Server Runtime did not feel satisfactory anymore and that it felt like the right time to move to another language.
This other language has become Scala.

TachometerWe have been developing in Scala for over six months now and the just released version 4.0 of the Mendix Platform (Mendix Spring 2012 Release) is the first version containing Scala code. In this article I will discuss why we have chosen Scala and how Scala compares to other candidates I came across. If you are using Java and are looking for an alternative, this will be an interesting read.

Requirements, please!

Before delving into the plethora of languages currently available to find a suitable choice, I thought the requirements our new language should fulfill. Two aspects are very important to me:

  1. The language should be a JVM language.
  2. The language should be statically typed.

The new language being a JVM language is crucial, because moving incrementally from Java is a must. Dropping all our work for a minimum of six months to do a complete rewrite is not something I would be able to sell to management, and frankly, it’s not very appealing to myself either. Frequent maintenance releases have to be cranked out, which means all the fixes from these releases have to be ported to the rewrite branch which will become insanely tedious. In addition, rewriting from scratch for a good half year is simply not that much fun. It will take months before the rewrite branch will be in a state where it remotely resembles the original version. Afterwards a couple more months will have to be spend to fix each and every bug. All of that, while cool new functionality could have been written that partners and customers were actually waiting for. So: no JVM language, no cigar.

Static typing is essential to me, because I know I am only human (and my colleagues as well, as far as I know). I will make typos and I will screw up the variable types from time to time. When I do, I’d like to know as soon as possible and not when an application, running in production with 1000 concurrent users, stumbles upon an edge case for which no unit test was made. In addition, I love refactoring, but I always want to refactor safely. Of course some methods exist to make refactoring easier in dynamically typed languages, but you’re never really sure whether you did everything right until you run your program. I need to know this at compile time.

Pick me, pick me!

Let’s turn to our contestants now. Some languages out there have a lot of potential, but do not adhere to the stated requirements. A language as Smalltalk might have a lot going for it, the “roles” concept is quite elegant for instance, but it’s not a JVM language and not statically typed. Therefore, not feasible as the new Mendix Runtime language.

Looking at the JVM languages there are quite some interesting ones:

I probably forgot some, as a lot of new languages for the JVM are coming into bloom at the moment, but these seem to be creating the most buzz currently.

Of these languages JRuby and Jython, the JVM-variants of Ruby and Python, are not suitable as they are not statically typed. The same goes for Groovy and Clojure.

Groovy used to be quite popular, but does not seem to have so much momentum anymore. The creator, James Strachan, said the following a while ago: “I can honestly say if someone had shown me the “Programming in Scala” book back in 2003, I’d probably never created Groovy”.

Clojure does have some momentum right now. However, apart from being a dynamically typed language, which is a not a good fit for me, it also has strong roots in Lisp. I think this makes the leap from Java to Clojure a bit too big.

Kotlin and Ceylon are the new kids on the block. They are interesting languages, but do not seem mature enough yet. We need a language which is ready to run applications in production.

Fantom exists for quite a while now. Fantom seems like a solid language and would solve a lot of Java’s shortcomings. Although generally a statically typed language, Fantom also involves dynamic typing. The interoperability with Java is ok, but not seamless. Integration is accomplished using a Java foreign function interface (FFI).

Xtend is a language based on Xtext. Xtend generates Java instead of compiling to byte code. This language was not really on my radar when I was doing my investigation. The first version which made it more well known was the 2.1 release last November. Before that time, Xtend did not appear to have reached an adequate maturity level yet. In addition, I’m wondering if compiling to Java could lead to transformation issues. What happens if you first generate Java using Xtend and then change something in the Java code? Is this nicely reflected in Xtend?

Scala, the Scalable Wow?

Scala was a language I kept hearing good things about (the same goes for Clojure though). In 2001 Martin Odersky, who is known for his work on the Java compiler and Java Generics, started the design of Scala, which led to a first version in 2003. The current stable version is 2.9.2 which was released last April.

I will not go into the language details as there are numerous blog posts and books that cover this topic in a much more comprehensive way than I would be able to do here. I can recommend the “Scala for Java Refugees” blog posts by Daniel Spiewak and the “Programming in Scala” book (Odersky, 2010) to start with.

What made me choose Scala above the other mentioned languages? First of all, Scala fulfills my two main requirements: it’s a JVM language and statically typed. In addition, the interoperability with Java is very good. Scala can call Java classes, interfaces and methods and the other way around without needing an FFI.

The big advantage over Java is that Scala offers functional aspects, which makes the language much more powerful. This way the code is much less verbose and more readable. That solves my main frustration with Java, a lot of times I had to write too much code for something simple.

Looking back at the other contestants I think Fantom and Xtend would be the main rivals. I did not choose Fantom as the interoperability with Java seemed somewhat more complicated and the language is not fully statically typed. For the reasons I mentioned earlier I like static typing better. Xtend was not really in the picture at the time of my investigation, but in the end I think the fact Xtend generates to Java instead of compiling to byte code, would have made me choose Scala. Generating to a language which has some fundamental flaws does not sound like the way to go long term. The JVM also has some flaws, but when those would be fixed, you don’t have to potentially wait for Java fixing those as well.

Ultimately, I think, you also have to listen to your gut feeling, as comparing all languages in excruciating detail is not possible. My gut feeling placed Scala on top.

After my personal decision to go for Scala, I started playing around with the language. During that time, I discovered programming had become fun again! I’m not saying I was frustrated with Java all the time, but the times I was very happy with the code I had written had become quite sporadic. With Scala, however, every new language feature I discovered made me able to solve something a lot more elegantly. Also, the lines of code I needed were on average 50% less than when I would program in Java and the code became more readable.

Let’s have a look at some Scala code. Remember the Java example of filtering virtual attributes from the previous post?

In Scala this can be written as follows:

No more superfluous semi-colons and parentheses and far more to the point!

In the comments section of the previous post, similar code snippets were proposed as alternatives to the Java variant (although in other languages). I think they all have in common that the code is far more concise and also more readable. Although you may need some time to get familiar with the syntax when you are used to Java, I think the Scala code snippet is easier to read.
Eye-tracking experiments have shown that for program comprehension, idiomatic Scala constructions can be grasped 30% faster than their Java equivalents (Dubochet, 2009). When code takes less time to understand, it will also take less time to maintain.

Shifting into the next gear

Our search for a new language has resulted in the choice for Scala. Scala is a JVM language, statically typed and combines functional and object-oriented aspects. Its conciseness leads to increased productivity, better maintainability and more fun coding. For me, it definitely feels we’re shifting into the next gear by adopting a language with the expressiveness Scala has.

What lies ahead? We are gradually turning Java code in to Scala and will continue to do so. In a next post I plan to share my experiences so far.

With regard to the users of the Mendix platform, if enough interest exists, we could generate templates for Scala actions which can be called from microflows in your Mendix project. Another interesting direction is to facilitate the creation of third party modules in Scala. Making your own module is already possible in the Mendix Runtime, but I would like to take this to the next level.

Let me know what you think, what would you like to see?

That’s all for now, have fun with the new Mendix, hope you’ll enjoy it!

————————–

Dubochet, G. (2009). Proceedings of the 21st Working Conference on the Psychology of Programmers Interest Group, p. 174-187. Exton, Chris ; Buckley, Jim (eds.)
Limerick, Ireland: University of Limerick.

Odersky, M. Spoon, L., Venners, B. (2010). Programming in Scala Second Edition. Artima Press, Waltnut Creek, California.

Photo by John Liu

Subscribe to Our Blog

Receive Mendix platform tips, tricks, and other resources straight to your inbox every two weeks.

RSS Feed of the Mendix Blog

About Arjen Hoekstra

Software Developer at Mendix.

  • hjonck

    Scala is a fantastic choice.  The JVM+Static Typing+Functional Aspects+Cool stuff like AKKA+Good support all make Scala the best choice.  

    I am just concerned that the refactoring tools are not as good as for Java. Hopefully this is addressed in the Eclipse plugin – what is your experience here?

    Scala Actions in Mendix would be absolutely amazing.

    What would be even better – is to filter some concepts through to Microflows. The Filter/Map/Reduce pattern and Generics would make Microflows so much more concise without adding complexity for the Business Type Microflow coders.

    So, the typical GetCreate pattern in a Microflow would be implemented once and can be used  across all microflows irrespective of Object type.  Further, the typical pattern of DoThisOverThisCollection would be great with a Map type action instead of a Loop.

    Look forward to hearing your feedback on the real world results of working in Scala over the longer term and which features of Scala realize the less-code more readable results as opposed to the esoteric features which bends your own brain if you read your own code in three months time.

    Happy coding on cool new features.

  • Arjen Hoekstra

    Hi Hardy,
    Thank you for your comment.
    Yes, I really like working with Scala. Refactoring support exists for the Eclipse plugin, although renames are not yet propagated to Java source if you that in your project as well.

    Nice to hear you would like the idea of Scala Actions. 
    We actually added some concepts to microflows in Mendix 4. You can now use filter and find operations in the ‘list operations’ activity. We have ideas to extend this such that you can pass microflows as well for these operations, which would enable you to make higher order functions using microflows :-).
    I also like the ideas for generics and get-or-create and map patterns, the challenge for implementing this is to do this in a way it’s still understandable for everyone.

    I plan to do a blog covering my experiences with Scala so far, I will try to touch the topics you mentioned!

    Also, thank you guys very much for the amazing film introduction you made for Mendix 4 at http://mendixandbeyond.com/! We really loved it here at Mendix :-).

  • G’day Arjen,

    I’m not sure that “static typing” argument you put forward is entirely convincing, nor should the “so I can refactor” be in the enumeration of why you prefer Scala as the tooling is currently very poor — perhaps unless you want to use Ensime (but you don’t want to muddy your hands with a lisp).Your bottom-line on Scala, though, is on par with mine for the current work my team is doing: asynchronous, stateless, and highly-available distributed applications. That being said, I enjoy JRuby greatly for true productivity in today’s web app landscape, and feel Clojure is highly suited to some of our forthcoming initiatives.

    Thanks for the write up!

  • “. Its conciseness leads to increased productivity, better maintainability and more fun coding”

    That’s a pretty bold claim. Do you have any evidence at all to back up those words?

  • Meinte Boersma

    It’s a pity you didn’t really try Xtend, since it has matured _a lot_ since 2.1. The compilation to Java instead of bytecode is a fair point: I think originally this was done for lack of an Xtend debugger, so having no Java code would preclude any debugging whatsoever. Version 2.3 has a debugger and in any case: the compilation of Java -> bytecode is not really noticeable. I suspect the itemis guys might be adding a direct -> bytecode compilation at some point.

    Didn’t you run into the “typical Scala” problems yet (such as): slow compilation (even if it is directly to bytecode) and the need to grow a third brain half as soon as you start poking the type system beyond what you’re used to in the JVM?

    I’m using Xtend _everywhere_ these days and it really has the same advantages you mention for Scala.

  • Arjen Hoekstra

    Thank you for your comments!

    @twitter-14687218:disqus The static vs dynamic typing debate is a quite a religious one. I think both have their merits and it’s ultimately also a matter of taste. JRuby and Clojure are both interesting languages with which I would like to familiarize myself a bit more in the future. I do think Scala is closing the gap feature-wise if you look at what dynamic languages have to offer. They are still a bit more powerful in some areas, but the difference is not big anymore. Combining those features with type-safety as Scala does makes for a pretty versatile language.I agree with you that the IDE-support for Scala is not yet as mature as for Java. But for both Eclipse and IntelliJ a lot is going on. Typesafe is putting a lot of effort in the Eclipse-plugin at the moment which is definitely showing results. That being said, there is still quite some work to do, certainly with regard to mixed Java/Scala-projects.
    @twitter-14654522:disqus This claim is mostly based on my own experiences. With Scala I can express myself more powerfully, which makes it easier to write functionality faster. Understanding code is what takes the most time when maintaining code. The Dubochet research I referenced showed Scala code was easier to grasp, which means less time would be needed to understand code when maintaining.
    Whether the coding becomes more fun with Scala, is something everyone should judge for themselves, but for me it certainly does. It’s so much easier to write elegant code in a few lines, that gives me a good feeling.
    @google-52c58f066606bd7baf95874a974b4c8b:disqus 
    I can certainly see the advantages Xtend has to offer. Nice they have added a debugger as well now, that’s a big improvement over having to use the Java debugger. When they would support compiling directly to bytecode as well, they could become quite a rival for Scala if you look at people thinking about switching from Java to another language. I guess it would become mostly a matter of feature sets at that point.
    Compilation for Scala is not as fast a Java and it probably never will be. Scala has to do a lot more type inference which takes time. Besides that the IDE-plugins were not so efficient, certainly with regard to dependencies. This has improved a lot, although I think there could still be made some improvements, foremost, as I mentioned above, with regard to mixed Java/Scala projects.
    I did not run in to major type system problems so far. Although understanding some of the collections ScalaDoc might be somewhat challenging, using them did not give me troubles.