In their 2018 report titled The Year of Enterprise DevOps, Forrester proclaimed: “With our data confirming that 50% of organizations are implementing DevOps, the questions have shifted from “What is DevOps?” to “How do I implement DevOps at scale?”
DevOps promotes small, more empowered and autonomous teams, along with automation and measurements. It has great potential to be a fantastic improvement in the way IT and business work together, and it is a welcome reversal of years of centralization and outsourcing.
But we also see a lot of organizations struggle with two areas:
- A proliferation of new “automation” tooling that rarely works together. It takes a very long time to set up as a pipeline for these tools, and they are not easy to change and manage for the DevOps teams.
- The mindset of architects is still where it was with a large IT organization, either microservices are not promoted at all, or the wrong approach to microservices is taken.
Without adopting the technology, design, and architecture to the new way of working, the DevOps exercise will be an unpleasant exercise without any clear benefits.
If CIOs introduce DevOps at a large scale across the IT organization, it is absolutely necessary to also roll out a new way of designing with Microservice Architecture (MSA) in mind. Otherwise, the teams will depend on each other and the result will be worse than before.
Using a high-productivity platform is highly recommended. These platforms already have a clear delivery pipeline within the package, which makes further automation, and adaptation of continuous integration, possible for the less technical people on the DevOps teams.
Figure 2: Architecture, Methodology, Organization, and Automation should all merge into a coherent journey for it to work.
Central IT Departments Have Central IT Components
It all goes back to Conway’s law from 1967: “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations“.
Since 2005, there was a strong movement in favor of centralizing the IT department, and within it, separating Development from Operations. The motivations for this movement were economies of scale, the concentration of expertise, synergies of IT between business departments, and the ability to negotiate large sourcing contracts with some off-shoring components.
Per Conway’s law, central IT departments often created IT components that resembled the organizational structure within that IT department. The department was often divided into areas of expertise (e.g. centralizing UX efforts, centralizing Business Process Management (BPM), the Integration in ESBs, the rules in a rules engine and so on). The SOA layered architecture was suitable for this, and many companies made it their reference architecture.
The central IT department was motivated by avoiding duplication of data and function. In a way it was understandable: Within a single application, there are good reasons to avoid duplication. It just doesn’t translate into an enterprise-wide strategy because dependencies grow out of control.
As we painfully learned in the past 10 years, the idea of better efficiency by larger scale did not pay off. Flexibility, specialization, business alignment and small-scale IT investment based on business-related profit-loss calculations all but disappeared.
It did not help that development, testing and operations were off-shored, maybe to different suppliers. Off-shoring allowed for doing the same thing at a lower cost, but it increased the distance between the people that should innovate and the business people that needed the innovation.
Cloud, Mode 2 and DevOps Stirring Things Up
Around 2010-2013, the Cloud offerings became more and more interesting. But companies found it hard to move to the cloud and take advantage of its benefits; They were constrained by rigid IT organizations and the fact that IT components were functionally highly dependent on each other.
At the same time, from the developer end of the spectrum, DevOps was taking form. Developers were saying: “There is no use to be agile in development if I cannot deploy the component afterward”. I.e. the entire organization needs to be agile if agile methodologies should work.
DevOps also held a large component of automation, where cloud and containerization allowed a whole new generation of continuous integration tooling. On-shore IT resources needed to compete with the lower hourly rates of off-shore resources, and the solution was increased innovation and automation.
Then, in 2014, Gartner proposed that enterprises need to work in two modes, one stable Mode 1, and a swifter, lighter and faster Mode 2, to take advantage of this new evolution. It allowed companies to start experimenting with smaller teams, new technologies and less red-tape in the areas that require a faster pace of innovation.
Two years later, the experience of speed and efficiency by enterprises moving to Cloud technologies, working in Mode 2 and using High Productivity platforms is so strong, that IT leaders are taking Mode 2 mainstream, thinking “why not be efficient for all areas of IT development?”
Are Digitization and DevOps Driving the Change?
The vehicle for this evolution going forward seems to be DevOps. As a reaction to the central IT department, with its strong separation between development and operations, DevOps goes all the way to the opposite end of the spectrum: Small business aligned DevOps teams that own their own components and manage their solutions also in production.
The first step was merging Ops and Dev, and then including the business to create (Biz)DevOps. This goes back to where things were before central IT, but now on the Cloud, using agile methods, and with much more automation. A small team can now build and maintain significant components.
The period with large, central IT departments may not be gone, but we expect that the time has passed for the IT department to be separated from the business. The merges between the business and IT also comes from the digitization of the business processes themselves.
IT is getting promoted from “something that should just run”, to the focus area where the wars of competitiveness are waged and won. Amazon, Airbnb, Spotify, Uber, Netflix, Facebook and others have shown that entire industries and business models can be disrupted by IT evolution. The CIO is now on the board of many companies, and the goal is to wake up the IT departments.
Microservices and Automation Will Follow
Per Conway’s Law: With small autonomous DevOps teams, the world will start producing small autonomous components, called microservices. Along with the team size, it also makes sense to produce deployable components that can fulfill a business function autonomously. This enables the ability to model the IT landscape closer to the way the business is run. The flexibility and time-to-market improve drastically.
We see speed improve five to ten times by moving to the cloud and using a high productivity platform such as Mendix. When Mendix components grow larger, we see up to five times improvement of speed in feature development by dividing the functional scope into microservices. DevOps and microservices are more or less inseparable, they can hardly exist in separation.
Steps for Getting DevOps to Work Successfully
This evolution is very exciting, but also requires our full attention.
We recommend any CIO, architect, manager, agile coach, designer, and developer that is involved in DevOps to gather at least a basic understanding of how microservices change the way we design things. Mendix plans to publish a set of blogs that describe microservices and our experiences in that area, so stay tuned for more content that can make or break your DevOps efforts.
We also recommend combining DevOps with a High Productivity Platform. A CIO with aggressive targets will benefit from skipping the lengthy phase of building automation from scratch and get straight to the business of delivering value to the business.
Keep an eye out for more blogs on how we see the DevOps and microservices style break through and evolve into the most important paradigm of the next 10 years.