Sometimes the title just gives away the answer. I’m a security guy, so this one is easy for me: the answer to "Where in the DevOps cycle do you do security?" is "everywhere". However, a couple of sentences doesn’t make a very compelling article, so I’ll go into a bit more detail.
What does the question really mean?
In the “good old days”1 of waterfall projects, there was a generally accepted practice of trying to lever security into the project just at the end of the process, right before deployment or shipping. Nobody thought this was actually good practice, but it’s been all too common—for those projects that bothered with security at all.
The problem is that there is no “right before deployment” when you’re doing DevOps. You’re always deploying, so at what point do you need to be doing security? The answer, as I suggested above, is “everywhere.” But we’ve just come full circle with very little additional information, so let’s take things apart.
When we think of DevOps, we think of it as a continuous process—whether the picture we use in our minds is a circle or an infinity symbol figure of eight—and I would argue that there’s not one part of that where we shouldn’t be “doing security”:
- At design time, security should be part of your requirements.
- At coding time, you should ensure the correct coding guidelines are followed.
- At testing time, there should be unit tests to cover security cases.
- At deploy time, you should ensure that the correct keys are deployed and adequately protected and that your workloads execute on the correct hosts.
- At run-time, you should monitor for compromises.
- And so on...
Notice in the list above that I chose examples from a variety of different levels of the DevOps “stack”. Coding guidelines aren’t about the product but about how you make it, unlike workload monitoring, for instance. Unit tests operate at a different architectural level than workload placement and scheduling. What this tells us is that in a DevOps world, security needs to be considered throughout the process, at multiple different levels.
A “where” question or a “who” question?
If the answer to our first question is “everywhere”, there may be an awful realisation dawning on your team: if security is everywhere, then it’s everybody’s responsibility. This is, of course, true, but it doesn’t mean what you might think it means: that everybody needs to suddenly become a security expert2. Rather, it means that it’s everybody’s responsibility to follow security processes. Who, then, puts these in place, and what do they look like?
Well, the first thing is to not pretend that there’s going to be one “security guru” who understands everything for all parts of the DevOps stack, for all projects and products that you’re running. There will be different people: one, for example, who decides what container images should be used, another who specifies workload placement and scheduling rules, and another who understands the arcane world of security monitoring.
The second thing we need to do is automate. Automation is, of course, key to DevOps, so if we don’t automate the security policies our experts have come up with as processes, they’re never going to be followed. Some will be more easily automatable than others, and some—hopefully fewer and fewer—will require some input from the other members of the DevOps team. But if we can bring the security folks fully into the team, then they should begin to understand the power of automation and it will become easier and easier to convince them to automate with the best of us.
Not just everywhere, but also every time
The last point comes back to the waterfall development model. In that model, you “do” security once4, but in the DevOps model, there is no “once”—everything goes round each time. Now, it may be tempting—very, very tempting—just to get the security folks in for the first iteration or two and then, well, let them head off and do their thing, wherever that is and whatever it is5.
This would be an enormous mistake. We all know that functional requirements change as we go through iterations—users interact with things in unexpected ways, customers change their mind, management becomes more or less clueless6, host configurations get updated. But so do non-functional requirements like security. If users are interacting with the product differently, do you need to change your authentication model? Have new regulatory requirements come in since your last review? Do the hosts you’re running your workloads on need to be patched to mitigate a recently discovered vulnerability?
The answer to all of these may be yes, so it’s vital to keep your security folks as part of your team throughout the process—and each time you iterate. Don’t think of this as a bad thing; their involvement is what’s stopping you having to redo the entire project when you discover some important security requirement you missed, or worse, needing to become security experts across the entire process.
1 This is irony.
2 Cue a collective sigh of relief from around 97% of developers, ops folks, testers, designers, and, well, everybody.3
3 Except us security folks; we love it.
4 If you’re lucky.
5 a) Probably down at the pub, and b) you don’t want to know.
6 It’s usually “more clueless”, but you knew that.
[See our related story, Security and privacy: Do you know what's lurking on your system?]
1 Comment