Why tools like Docker, Vagrant, and Ansible are hotter than ever

The complexity of application stacks keeps going up. Way, way up. Application stacks have always been complicated, but never like this. There are so many services, so many tools, so much more compute power available, so many new techniques to try, and always the desire, and the pressure, to solve problems in newer and cooler and more elegant ways. With so many toys to play with, and more coming every day, the toy chest struggles to contain them all.

If you're not familiar with stackshare.io, have a look at it. It's a great resource to see which pieces companies are using to build their applications. In addition to being useful, it also can be pretty entertaining.

Spend a few minutes browsing through some of the stacks out there and you'll see that some of the technology collections people have assembled are fascinating. Here's an example I particularly like: (deep breath) EC2 S3 Qubole MongoDB Memecached Redis Django Hadoop nginx Cassandra MySQL Google Analytics SendGrid Route53 Testdroid Varnish Zookeeper.

So that's web server, web application server, caching proxy server, discovery service, a few services-as-a-service, and six "databases" of various flavors and functions. (All of it either open source or proprietary service, of course. There tends to be very little in between anymore.)

It's highly unlikely that anyone ever stood in front of a whiteboard and wrote WE NEED SIX DATABASES!!! with a purple dry erase pen, but that's how things happen when your infrastructure expands rapidly to meet business demand. A developer decides that a new tool is best, rightly or wrongly, and that tool makes its way into production. At that moment, the cool new tool instantly becomes a legacy application, and you have to deal with it until you refactor it (ha!) or until you quit to go do something else and leave the next poor sucker to deal with it.

How to cope

So how can developers possibly cope with all of this complexity? Better than one might expect, as it turns out.

That awesome nextgen location-aware online combo gambling/dating/sharing economy platform is going to require a lot of different services and components. But every grand plan has a simple beginning, and every component of any ultrascalable mega-solution starts its life as a few chunks of code somewhere. For most teams, that somewhere is a few humble developer laptops, and a git repository to bind them.

We talk about the cloud revolution, but we tend to talk less about the laptop revolution. The developer laptop of today, combined with advances in virtualization and containerization, now allow complex multi-system environments to be fully modeled on a laptop. Multiple "machines" can now be a safe default, because these multiple, separate "machines" can all be trivially instantiated on a laptop.

The upshot: The development environment for a complex, multisystem application stack can now be reliably and repeatably installed on a single laptop, and changes to any of the environment, or all of the environment, can be easily shared among the whole team, so that everyone can rebuild identical environments quickly. For example, ceph-ansible is a tool to deploy and test a multi-node Ceph cluster on a laptop, using multiple VMs, built by Vagrant and orchestrated by Ansible, all with a single command: vagrant up. Ceph developers are using this tool right now.

This kind of complex multi-node deployment is already becoming commonplace, and it means that modeling the relationships between machines is now just as important as managing what's on those individual machines.

Docker and Vagrant are successful because they are two simple ways of saying, "This is what's on this machine, and here's how to start it." Ansible is successful with both because it's a simple way of saying, "This is how these machines interact, and here's how to start them." Together, they allow developers to build complex multi-machine environments, in a way that allows them to be described and rebuilt easily.

It's often said that DevOps, at its heart, is a conversation. This may be true, but it's a conversation that's most successful when everyone speaks the same language. Vagrant, Docker, and Ansible are seeing success because they allow people to speak the same languages of modeling and deployment.

This article is part of the Easy DevOps column coordinated by Greg Dekoenigsberg. Share your stories and advice that helps to make DevOps practical—along with the tools, processes, culture, successes and glorious/inglorious failures from your experience by contacting us at open@opensource.com.