Understand the Value of Migrating to Microservices

Consul service mesh provides a robust set of features that enable you to migrate from monolith to microservices in a safe, predictable and controlled manner.

This tutorial is focused on reviewing the motivations, benefits, and trade-offs related to adopting a microservice architecture, and is designed for practitioners who will be responsible for developing microservices that will be deployed to Consul service mesh running on Kubernetes. However, the concepts discussed should be useful for practitioners developing microservices that will be deployed to any environment.

This tutorial will:

• Provide an expandable starting definition for what a microservice is
• Discuss some common motivations for using microservices
• Identify some of the trade-offs when using microservices

If you are already familiar with the motivations, benefits, and trade-offs associated with the microservices architecture, feel free to skip ahead to the next tutorial in the collection where we discuss Consul from a design pattern perspective.

Source code

This tutorials in this collection reference an example application. You can review the source code for the different tiers of the application on github.

Definition of a microservice

Before beginning this collection of tutorials, or any development project, it is important to both agree upon a common vocabulary, and define scope. There is no canonical list of the defining characteristics of the microservices architecture. For this collection of tutorials, the characteristics of a microservice will be defined as:

• Small in size (lines of code)
• Focused on a specific area of business logic or application functionality
• Independently deployable
• Independently testable
• Well suited for deployment using an orchestrator such as Kubernetes or Nomad

For the purposes of limiting scope, the definition used in this collection will stop there. There are other attributes of a microservice that would make excellent additions to the list, but it is best to start small in scope and expand. If you are interested in reading more about the microservices architecture, this article by Martin Fowler is a great supplemental resource.

Motivations for using microservices

The motivations for migrating to microservices typically fall into two categories: organizational and technical. The following outline provides a framework for the discussion along two axes: stakeholder values and the weaknesses of the monolithic approach.

• Stakeholder values
  • Executive / Sales / Product: speed to market
  • Ops / Customer success: stability & customer satisfaction
  • Development: ability to innovate, agility, independence, maintainability
• Monolithic weaknesses
  • Slow to coordinate effort across teams
  • Difficult to maintain a separation of concerns
  • Difficult to innovate
  • Difficult to scale
  • Difficult to test

The microservice architecture attempts to address each of these values and weaknesses. We will consider the organizational motivations first because it is important that you validate that microservices are right for your scenario from an organizational perspective. As the technical stakeholder, you likely are already sold on the technical benefits, but in order to champion a migration to microservices, you need to be able to explain to the rest of the organization why such an expensive endeavor is in the communal best interest. In fact, you need to ensure that it actually is.

Organizational motivations

The executive, sales, and product teams of most organizations are typically focused on speed to market. The operations and customer success teams typically value stability and customer satisfaction. Development teams value agility, independence, maintainability, and the ability to innovate.

This section discusses some organizational motivations and benefits of the microservices architecture that are relevant to these stakeholder values with a focus on clarifying stakeholder expectations, explaining how microservices provide value, and what trade-offs are involved.

Faster iteration

Speed to market is often confused with faster iteration. If your organization is in a totally greenfield scenario, meaning you have not yet released any product to the market at all, even if the market is internal consumers, conventional wisdom would suggest that a monolithic approach is probably your best alternative in terms of satisfying stakeholders who primarily perceive value as "software in the customer's hands".

Eventually, however, this approach reaches a point where what previously enabled fast iteration now inhibits it. The tradeoff you make in going to market with a monolith, is the refactoring investment you will have to make when you hit critical mass, and the monolith becomes a barrier to speed. Monolithic solutions typically suffer from at least two primary deficiencies. The first is coordination of effort across teams, and the second is a lack of separation of concerns.

We'll cover these forces in more detail later, but for now let's reframe the discussion around speed to market. The speed to market benefit of microservices is typically more relevant when considering scenarios where your organization already has a solution, but your product or organization has reached a size where a single monolithic application has now become cumbersome rather that providing agility.

Microservices are a mechanism that allow you to extract sections of functionality from the monolith that can be deployed and managed independently. Because sections of functionality can be deployed and managed independently, you should be able to achieve a value chain built around deploying small units of functionality more frequently, and with a smaller blast radius in terms of deployment risk. This lower risk, constant value delivery approach speaks directly to the values of executive, sales, operations, customer success, and development teams.

Team independence

One of the key motivations for a shift to microservices is the desire to lessen the burden of coordinating efforts across teams. This doesn't mean that teams don't have to coordinate. They do, and they will. However, with microservices teams will coordinate in terms of interfaces rather than release schedules. The microservices architecture allows organizations to embrace independent team operations, sometimes called silos, which has traditionally been a friction point when teams are trying to integrate their work into a single common build.

On a practical level that plays out like this.

• The Security team produces the the Auth API, and documents the Auth API's HTTP interface
• The Orders team produces the Orders services that depends on the Auth API because users have to be logged in to place an order
• The Orders team should be free to manage their own release cycles without having to wait for the Security team to be ready to release whatever changes they are currently working on
• Coordination may need to take place if the Security team breaks interface
• The Security team should be very mindful of breaking interface, and is responsible for notifying all downstream consumers if they are going to make a breaking change and must be committed to making sure breaking interface is a rare occurrence
• With discrete boundaries like this, test automation should be able to more easily pick up breaking changes earlier in the dev cycle when it is less expensive
• If everyone puts forth a best effort to follow the rules, the Security and Orders teams are both free to iterate independently, and thoughtful guidelines around innovation have been established so that upstream teams can be better corporate partners to their downstream consumers

Enabling a culture of quality

Monolithic application architectures are notoriously difficult to test. Monolithic code is often described as brittle. This is because all logic exists in one place, and while it is completely possible to write well factored monolithic applications, it isn't easy. Over time, most monoliths start to experience a lack of separation of concerns, which is when functionality that should be isolated within a single component or layer bleeds over into another component or layer. The classic admonitions often cited are "don't put business logic in the UI" or "don't encode presentation logic in the DB".

When this happens, quality starts to diminish, and this can lead to cultures where there is little confidence that a "one line change" won't affect several other, seemingly unrelated parts of the system. Once the culture evolves in this manner, a "one line change" often triggers a full system regression test. In many organizations, theses tests are partially or entirely manual. It might also be true that regulatory compliance requirements force a full system regression test for any code deployment. Regardless of the why, full regression tests are expensive and time consuming.

By migrating to a microservices architecture, you are creating a workflow where code is deployed in smaller units, and typically with automated test and release gating. Because code is developed, tested, and deployed in smaller units, it becomes easier to reason through possible failure scenarios, and design more comprehensive tests. With a smaller deployed unit of functionality, a full regression test of deployed changes is often limited to a handful of function points.

Learning how adopt the microservice architecture comes with its own learning curve. No one learns new things without making mistakes. You will make mistakes at the beginning either in design, release automation, infrastructure automation, or some other area. In organizations where confidence levels are low, this might result in skeptical stakeholders calling to hit the eject button on the pilot project prematurely. It is important to manage stakeholder expectation realistically. The adage to under promise and over deliver was never more important to keep in mind than when planning a migration to microservices.

In any scenario, effecting a culture shift is unlikely to happen quickly. Stakeholders who have been burned by "one line changes" before, may be unwilling to forego the full regression test approach right away, or possibly ever. They may point to early failures as a reason to abandon the effort. Be prepared for these hurdles, and focus on constant learning. Over time, things should stabilize, and confidence levels should rise.

Technical motivations

As stated earlier, operations and customer success teams tend to value stability and development teams tend to value agility, independence, maintainability, and the ability to innovate. While this is an over generalization, it is a way to introduce the discussion around the technical motivations for microservices. In reality, operations and customer maintenance teams also care deeply about maintainability, especially if the product your organization produces has to be managed by your users.

This section covers some of the technical benefits of microservices that are relevant to these values with a focus on clarifying stakeholder expectations, explaining how microservices provide value, and listing some of the trade-offs that are involved.

Independent deployment

Independent deployment is a fantastic benefit, once you can achieve it, but the ease or difficulty in achieving this benefit is heavily intertwined with how well you monolith was written. Here's why.

If your product finds market fit, your team will likely expand, as will your features. As teams expand, two things happen. The cost of coordinating humans goes up, and familiarity with the code base per capita goes down. In fact, these two metrics are inevitable inverses of one another.

To illustrate this point, let's think through the impact growing a team relative to the code base. When more people are contributing to a code base, more code gets produced. As more code gets produced, each team member is less able to stay up with every single change. As team member awareness goes down, duplication of effort tends to go up, but even worse than that, coupling between components or modules almost inevitably rises.

In the best case scenario, this coupling is legitimate, in that, one component might really need to consume functionality from another. In a worse scenario, there is an actual lack of a separation of concerns, and functionality that should be isolated within a single component or layer has been implemented poorly and spans across two or more components, or even logical layers.

In an ideal situation, components are loosely coupled, meaning they have a well defined way of interacting, and the functionality of each component is discrete or self contained. Sometimes components or services are described as needing to have a well defined boundary or bounded context. Maintaining loose coupling is a very difficult cognitive exercise, and even accomplished developers find maintaining tight discipline in this regard challenging.

To achieve deployment independence for a service when migrating away from a monolith, you must first identify a component or service that you wish to extract, and then assess what changes will be required to do so. If you have a well developed, loosely coupled component model, the level of effort will be significantly reduced, but in no way insignificant. See the Extract a microservice tutorial for more details on the activities involved.

Once you've identified the service, and extracted it, you will start to see the benefits of independently deployability right away in the form of independent scalability & testability.

Independent scalability

Since microservices are deployed as independent services, they can be scaled independently of one another. There is a lot of hidden value implied in that statement, so let's unpack it a bit.

If you are still running a monolith you are in this situation: the cost of operating your least resource intensive functionality is equal to the cost of operating your most resource intensive functionality. If fact, it is worse than that. Since all of your code is in one place, the resource consumption of your most resource intensive functionality drives up the cost of your least resource intensive functionality because they are competing for resources.

Traditionally, on-prem practitioners might have chosen to scale this model vertically to meet the demands of the monolith. However, vertical scaling in the cloud is expensive. Also, your ability to scale dynamically based on load may be limited if caching large amounts of data is involved.

To illustrate this point, consider the following scenario. Your monolith has grown popular, and it serves data - a lot of data - from a relational database. In the beginning this was fine, because the monolith didn't see a lot of use. Over time the popularity of your product grew, and the excessive network calls to the database increased latency and bandwidth costs with the cloud provider. To mitigate this you added a caching layer inside the VMs that host your monolith, but in a cloud environment drives up your RAM price per monolithic VM host.

You built your monolith in a somewhat stateless manner, in that user sessions didn't necessarily have to always be served by the same host. So when you start to see performance degradation, you do, in theory, have the option of adding more monolith VM hosts, but they are very expensive. You don't really need that many copies of your data caches, but your architecture doesn't give you any options.

Here's what's worse. You are currently running 10 instances of your monolith, but you really only need that capacity during your peak hours. Unfortunately, bringing up a new instance takes quite a bit of time because you have to load so much data into cache. Ideally, you'd like to spin down to 3 instances after peak usage hours and then spin back up to 10 an hour or so before peak hours start, but you can't because that bogs down the relational database server, and also, you are paying for moving data across the wire at the cloud provider again and again each day.

The architecture of the monolith described here has a lot of weak points, and admittedly, things could have been done differently, but it illustrates clearly the challenges, inflexibility and costs that a monolithic architecture brings. Also, there are many applications in use today at very successful enterprise companies that have a very similar profile.

With a microservices architecture, you can scale different components of your application independently, and you can allocate to them only the resources they need when they need them. You may find that once your monolith isn't competing with itself for resources, you don't need 10 cache instances. How might things for your organization's margins improve if you found out you only needed 2-3 RAM heavy resources? What if you learned that one cache in non-peak hours was ok, because if it failed, database query times were acceptable under the smaller load? What if you learned that the database could handle spinning up 2 caches daily an hour before peak usage starts, and that the network transfer costs were lower than renting all the RAM you used to rent?

Independent testability

The value of being able test services independently cannot be overstated. If your organization has reached a point where services can be deployed and scaled independently, then almost by definition, services should be able to be tested independently. With a monolith, changing any part of your application, from a testing perspective, is effectively changing the whole application.

By breaking apart your monolith into discrete services you force a separation of concerns and establish obvious boundaries between subsystems. These boundaries become the surface area for test automation, and have the added benefit of helping developers with the cognitive load of maintaining loose coupling and tight cohesion between services. The service boundary becomes a first class citizen of the design and test planning processes.

Legacy system migration

One of the nicest thing about migrating to microservices, is that it can done thoughtfully over time. Big bang releases are risky and hard to sell within an organization.

By adopting a microservice architecture, you give yourself options when it comes to retiring your legacy monoliths. You can reason through which parts of your application can most easily be refactored and extract a little bit at a time. This approach is so common it has become part of the general software engineering pattern language, and is called the strangler pattern.

Trade-offs when using microservices

The reality is that while a migration to microservices can improve performance across a significant number of KPIs when employed by organizations that understand how a microservice architecture works, nothing comes for free. This final section outlines some additional trade-offs that have not yet been discussed when adopting this.

Simplicity

Distributed systems are complicated, and add a significant number of new operational concerns related solely to the complexities of operating a distributed system. Instead of having to manage the deployment of a single workstream, you have to manage N number of workstreams where N is the number of services deploy. Those services need to be secured, communications have to be configured, version control has to be setup, and on and on.

Monitoring

Another vector of consideration when discussing simplicity vs complexity is in the domain of monitoring. With microservices, the number of potential signals worth paying attention to increases considerably and somewhat unpredictably. Each new service has it's own deployment profile, and therefore its own monitoring profile. Also worth noting in this section is inter-service dependency. If one service goes down, and other services depend on it, you could create a cascading failure scenario. So not only does your external monitoring system need to monitor services, your downstreams need to monitor their ability to reach the upstreams they depend on and employ some sort of failure mitigation technique like circuit breaking.

Debugging

Another complexity that we will note in this section is debugging. Eventually, something will go wrong in your systems, and you will need to debug the problem. Debugging a distributed system can be more challenging than it needs to be, if you do not prepare for this eventuality. By design, a service mesh is secured from unauthorized observation at the network level. By virtue of running in VMs, Kubernetes, Nomad, or some other orchestrator, you are adding additional layers of obfuscation and security between your applications and would be attackers. That's fantastic except that you also now need to bypass those safeguards in order to debug. Having a solid telemetry strategy is not optional. Additionally, you will need to learn new debugging techniques specific to your tech stack. You may potentially even need to roll some homegrown tools to facilitate debugging when things go wrong, and as had been said many times before, you should plan for failure.

These examples and others, illustrate how migrating to microservices is an intentional trade-off of simplicity for resilience and flexibility. Pointing out these complexities may seem like an attempt to discourage from migrating to a microservices architecture. Nothing could be farther from the truth. Rather, this is an effort to empower you with information. By highlighting some of the considerations you will need to make when migrating your architecture, we hope that you can begin the undertaking well informed, and thus avoid some common mistakes others have already experienced.

Cloud Cost

One of the benefits outlined in the independent scalability section above was the ability to breaking apart the monolith so that different areas of functionality aren't competing for resources, and can be both understood and scaled based on their own performance profile. While that is a very desirable end state, there will be some amount of time where you have not yet reached that end state, and you are running more cloud resources in parallel to the monolith. This means that during the migration phase, and until you achieve your desired end state, you will actually be increasing your cloud spend.

It is also possible that by the end of your migration, you end up spending more to achieve the scalability, resilience, and speed your organization is now demanding. While it is true that vertical scaling in a cloud environment is very expensive, it is also true that horizontal sprawl has costs in terms of both the monthly cloud vendor invoice, as well as the human hours it takes to build and manage a complex cloud environment.

It is important to recognize and state these truths up front, so that stakeholders can know exactly what they are agreeing to. Everyone should be clear that the a migration to microservices is not a cost saving initiative. It increases complexity significantly. The migration to microservices should be motivated by your organization's growth, and the fact that a monolithic approach is no longer sustainable for your business reality.

Next steps

In this tutorial, we defined what a microservice is for the scope of this discussion of the topic. We also reviewed some of the motivations, benefits, and trade-offs associated with adopting a microservices architecture. The next tutorial in the collection will explain how Consul service mesh works, what design patterns it implements, and the value it provides on your journey from monolith to microservices.