Mendix Improves Clustering to Provide More Resilient Applications
Mendix Improves Clustering to Provide More Resilient Applications by Andrej Koelewijn
Mendix 6 contains the start of some big improvements in the support for clustering and high availability for your Mendix applications. These improvements will enable you to run multiple Mendix runtime instances for a single application on top of a CloudFoundry PaaS, such as HP or Pivotal.
This will improve your users’ experience in several ways: when one runtime becomes unavailable, they will not notice any service disruption, as other runtime instances will seamlessly take over existing running sessions. Another benefit will be that you can easily add more runtime instances as the number of users grows.
Before Mendix 6, clustering relied on sticky session configuration: users were always sent to the same runtime in a cluster. Starting with Mendix 6, we are providing you with an easy way to flexibly distribute your users across multiple runtimes.
In this blog post, we’ll outline the improvements we’ve made to enable effortless high availability and elastic scalability in Mendix. As always, our focus is on providing these capabilities in a very easy to use manner, but to truly enable scalable applications, there will be some impact on your applications. We’ll also touch on these required project changes.
How is This Implemented in The Mendix Platform?
To provide clustering in a frictionless way, we’ve improved the Mendix platform in a number of ways.
The most obvious and perhaps biggest change is that runtime state is available to all runtimes, so we took it out of the individual runtime instance and stored it separately. We have support for different types of state storage, allowing you to choose between ease of installation and more performance oriented options.
Storing all runtime state externally impacts the way the runtime handles garbage collection. Mendix automatically removes unused objects from the runtime. Doing this for every request is expensive, as multiple runtimes and an external state store are involved.
To address this, we changed the default behavior for garbage collection: Non persistent entities (NPEs) are by default request scoped, which means that they will be garbage collected after every request. You can keep NPEs around for other requests and runtime instances by associating the objects with your user’s session.
The way sessions are administered has also been improved. When running clustered, sessions are always persistent sessions, meaning they are stored in the application database, and available across all runtime instances. This way we can ensure that all runtimes know a user has been authenticated or logged out.
Finally, we also need coordination across runtime instances to ensure database migrations are handled correctly, microflows that run when you start or stop the application are run correctly, and the result of long running asynchronous microflows is available on all runtimes.
What Does This Mean for You?
As I said before, we aim to make this as easy as possible to use for our users. You will, however, need to consider some points:
- You will need to setup a clustered environment. Our main focus currently is on running on Cloud Foundry, for example Pivotal Web Services or a private cloud based on a Cloud Foundry distribution from HP or Pivotal.
- You will need to configure an external state store, preferably Redis. On Pivotal Web Services, this is available as a service, which means minimal configuration effort is required. You can also configure Mendix to use a different Redis service, for example Elasticache on Amazon, or something in your own datacenter. If you do not have Redis available, for testing purposes you can configure your application database to also store runtime state, but this will have an impact on performance.
- Make sure your application only uses supported constructs. We currently do not support scheduled events, before startup or after startup type microflows.
- To get optimal results, minimize the use and lifetime of Non Persistent Entities. Associate all NPEs that need to be session scoped to the user session. Alternatively, you may consider making NPEs persisted entities.
One specific point to keep in mind is service calls. Calls to services, e.g., webservices or rest services, are often implemented using Non Persistent Entities. Switching to request scoped NPEs will have an impact on the availability of data returned by these calls across different pages and runtimes. We suggest you make sure your services return small amounts of data quickly, so calling these repeatedly is acceptable. Otherwise, consider storing the results of the webservice calls in entities in your application.
Mendix 6 Beta Scope
As mentioned before, frictionless clustering includes a number of changes to the runtime. Mendix 6 ships with a beta release of a subset of these improvements. They will enable you to start building and testing clustered Mendix applications, with some limitations.
The scope of Mendix 6 is as follows:
- Runtime state can be shared in an external state store, either Redis or application database.
- User sessions are synchronized between all the runtimes using persistent sessions.
- Changed garbage collection: default behavior for applications created with Mendix 6 is to garbage collect NPEs at the end of a request, unless associated with the user session. Session scoped objects can be deleted by explicitly deleting them, for example with a delete action in a microflow, or they will be garbage collected at the end of a user session.
- [Pending] Result of long running asynchronous microflows is available to all runtimes.
Please note that the garbage collection change for Non Persistent entities is not Beta: this will be changed for all new applications created with Mendix 6. We do provide you with a configuration option to switch back to the old behavior. For converted applications, the old behavior will be the default.
The following items be will addressed in future releases:
- Frictionless deployment – currently you need to scale down to 1 runtime before deploying a new version of your application to ensure correct database migration. This limitation will be removed in a future release.
- Support for application lifecycle microflows – the before startup, after startup, and before shutdown microflows are currently always run for every runtime. We will provide more fine-grained control in the future.
- Support for disallow concurrent execution for microflows.
More detailed information can be found here:
We are extremely excited with these improvement to the Mendix platform as they will make it a lot easier for our users to build web-scale applications. Your applications and your users will benefit from the frictionless support for scalability and high availability in the Mendix platform.