Optimizing SVN for the Mendix Team Server
Working with Mendix Team Server projects (showing the history and updating) has become significantly faster today. This tech tip describes how the search for a solution to a support ticket led to a speed up of all Team Server projects.
We recently received a support ticket from one of our partners (thanks!). The problem was that various Team Server operations executed from within the Modeler failed with the following error message:
Server sent unexpected return value (504 Gateway Time-out) in response to REPORT request for '/...PROJECT_ID.../trunk'.
We tried showing the history of the repository by using TortoiseSVN and this also failed. Together this told us that the problem occurred on the side of the Team Server which is implemented with Apache and Subversion.
The very slow response could not be explained by the fact that the server was busy because the CPU load of the server is extremely low all the time. The most likely reasons was that I/O was a problem.
We started Googling for answers and found a question at Stackoverflow (of course) that held the answer we were looking for:
(note that the problem applies to all Subversion versions, even though the question suggests it has to do with a specific version)
It was not the accepted answer that helped us but the second answer by Peter Parker. Please vote that answer up if you have enough reputation!
The Apache module that is used for subversion, mod_dav_svn has a curious default value that caused the performance problems. When retrieving history revisions, the server checks readability for each changed file in each revision. This is important if you have configured per-directory permissions by using a module such as mod_authz_svn.
In our case, each project is a separate repository and we do not need permissions at the level of directories. We can safely disable these checks by adding the following line to the httpd.conf configuration file:
We changed the configuration for the problematic project and the results where staggering. The time needed to show the history went from more than a minute (timeout) to one second. We also checked the ‘update’ operation of one of our own projects. In the first step an ‘update’ checks the latest revision and this step was the most expensive step in the process. With the fix in place the time for the full ‘update’ went from 12 seconds to 2 and the first step was too fast to register.
We have done some more benchmarks and found that history-related operations are up to one hundred times faster. We have rolled out this improvement to all Team Server projects already. But you might have already noticed that…
The R&D Subversion Server
We checked the Subversion server we use for development at R&D and found that it was also doing these expensive checks. We disabled them and measured the difference for a number of TortoiseSVN operations: checking out a huge project, showing its log and then pressing the ‘Show all’ button to show the complete history of the project. The results speak for themselves; showing all revisions took 0.4 seconds which is why it might be hard to spot the last bar in the chart.
If you are running an SVN server yourself and you do not need directory-level permissions, you might want to check whether you are still running with the default and slow setting. As you can see, there is a lot to gain!