A guide to open source for microservices

Microservices—applications broken down into smaller, composable pieces that work together—are getting as much attention as the hottest new restaurant in town. (If you're not yet familiar, dive into What Are Microservices before continuing here.)

However, if you have moved on from "Hello, World" and running a simple handful of microservices, and are building hundreds of microservices and running thousands of instances, you know there is nothing "micro" about them. You want your instances to increase when users increase and decrease when users decrease. You want to distribute requests effectively between instances. You want to build and run your services intelligently. You need a clear view of the service instances that are running or going down. How can you manage all of this complexity?

This article looks at some of the key terminologies in the microservices ecosystem and some of the open source software available to build out a microservices architecture. The focus is on building and managing high-scale microservices networks and solving the challenges of running services without fault and that scale correctly based on business demand.

Here is a wholesome, lavish spread of open source cuisine that is sure to be gastronomically, "microservically" appetizing. I'm sure I've overlooked some open source applications in this area; please let me know about them in the comments.

[Download the PDF version of this cheat sheet here]

Containers

The right way to deploy applications is in containers. Briefly, a container is a miniature virtual server packed with the software required to run an application. The container pack is small, smart, and easy to deploy and maintain. And deploying your application in a container is clever. You can deploy as many instances as you need and scale up or down as needed to meet the current load.

Open source containers

Container orchestrators

If you have hundreds or thousands of service instances deployed on containers, you need a good way to manage them. Container orchestration is the right solution for deploying and managing all of these containers. Orchestrators can move across; scale up, down, or out; manage higher or lower loads; regulate added, removed, and dead containers; and much more.

Open source container orchestrators

API gateways

An API gateway is a watchman that controls and monitors API calls to your application. An API gateway has three key roles:

1. API data and management: API listing, API subscription, API documentation, community support
2. API viewpoint and billing: Analytics, metrics, billing
3. API control and security: Subscription caller management, rate control, blocking, data conversion, production and sandbox support, key management

API gateways are usually multi-tenant solutions to deploy multiple applications on the same gateway.

Open source API gateways

Not all of the following API gateways support every function mentioned above, so pick and choose depending on your needs.

CI/CD

Continuous integration (CI) and continuous deployment (CD; it may also stand for continuous delivery) are the net sum of processes to build and run your processes. CI/CD is a philosophy that ensures your microservices are built and run correctly to meet users' expectations. Automation is the critical CI/CD factor that makes the build and run process easy and structured. CI's primary processes are build and test, and CD's are deploy and monitor.

All of the CI/CD tools and platforms listed below are open source. I don't include SaaS platforms that are free for hosting open source. GitHub also isn't on the list because it is not open source and does not have built-in CI/CD; it uses third-party CI/CD product integrations instead. GitLab is open source and has a built-in CI/CD service, so it is on this list.

Open source CI/CD tools

Load balancers

When your number of requests scale, you must deploy multiple instances of your application and share requests across those instances. The application that manages the requests between instances is called a load balancer. A load balancer can be configured to distribute requests based on round-robin scheduling, IP routing, or another algorithm. The load balancer automatically manages request distributions when new instances are added (to support higher load) or decommissioned (when load scales down). Session persistence is another load-balancing feature that redirects new requests to the previous instance when needed (for example, to maintain a session). There are hardware- and software-based load balancers.

Open source load balancers

Service registry and service discovery

When several hundreds or thousands of service instances are deployed and talking to each other, how do requester services know how to connect the right responder services, given that deployment points are dynamic as services are scaled in and out? A service registry and service discovery service solves this problem. These systems are essentially key-value stores that maintain configuration information and naming and provide distributed synchronization.

Open source service registry and discovery services

Monitoring

When your microservices and their instances cater to users' needs, you need to maintain a good view of their performance. Monitoring tools to the rescue!

Open source monitoring tools and software come in numerous flavors, some barely better than top. Other options include OS-specific; enterprise-grade; tool collections that provide complete integration; do-one-thing tools that merely monitor or report or visualize and integrate with third-party tools; and tools that monitor specific or multiple components such as networks, log files, web requests, and databases. Monitoring tools can be web-based or standalone tools, and notification options range from passive reporting to active alerting.

Choose one or more of these tools to enjoy a chewy crunch of your microservices network.

Open source monitoring software

The right ingredients

Pure open source solutions can offer the right ingredients for deploying and running microservices at high scale. I hope you find them to be relishing, gratifying, satiating, and most of all, microservicey!

Download the Microservices cheat sheet.