Automation is not automagic: Why better software automation matters

There's a lot of automation that can happen that isn't a replacement of humans but of mind-numbing behavior. —Stewart Butterfield

Many organizations are trying to create a culture of automation and struggling with what it will take to achieve it—or at least to begin to make a legitimate attempt at creating and maintaining it. This article is a different take on the topic for anyone interested in automation, including people struggling on a team that focuses on automation, a company that hasn't quite figured it out, or teams that aren't returning the expected ROI.

Why better software automation matters

Software development teams are most productive when they are happiest building software and bringing value to customers. This is supported in "Lessons from 7 highly successful software engineering cultures" by Mitch Pronschinske, evangelist from HashiCorp. The more they can do to build their software and watch it come to life, the happier and more productive they will be, which eventually leads to better business value delivered to the customer faster. The challenge that development teams have is software is growing in complexity, with the cloud, multi-cloud, private cloud, hybrid cloud, and now containers and container orchestration using OpenShift, Kubernetes, Docker, and XC/LXD. The amount of work required to ship software has also grown.

Because the most successful companies must be (or become) software companies, and technology and infrastructure are evolving and changing so fast, it is no longer sufficient to have processes and automation that are cobbled together. We must have well-designed, sophisticated automation that empowers teams at scale. The old way of doing things—writing scripts and sort of cobbling things together in an ad-hoc manner—does not scale. As many successful companies have proven, if you can iterate faster and ship sooner, you can capture the market faster. Companies that invest significantly and deliberately in automation will have the ability to move faster.

The sweet spot

How can companies create a culture where innovation is spurred by automation and where cross-cultural and collaborative contributions spark creativity, ingenuity, and new ideas? Rather than eliminating the need for humans, automation increases the demand for humans.

Automation might mean different things to different companies and even to different people; the common thread is the ability to scale at will and with speed, which creates freedom and flexibility that expands across organizations, teams, and departments.

In any case, the organization will have to start from the ground up and take a hard look at the automation wins and losses. Then they have to look at the people they have in place and the people they wish to hire. How deep is your bench? How proactive are you at creating a pipeline? Companies have to think creatively, and that creativity could come from unexpected places. Technology evolves at the speed of light, and companies need a mindset that allows solutions to come from all directions and automation specialists that are comfortable with disruption and even chaos.

Recruiting talent with automation skills

Companies should start by training their current folks, because that's easier than firing them, and then make sure they are hiring the right people with backgrounds in a variety of areas and/or sectors, including computer science, Linux, networking, systems administration or engineering, programming, software development, automation, and even non-traditional engineering fields like mechanical engineering. There is no hard and fast rule that an automation specialist must come from a traditional technical background. It's more about their collective experiences and how they think and solve problems. Are they coachable, amicable, and collaborative? Do they communicate effectively? Are they adverse to risk from an emotional intelligence perspective?

Referrals are a great way to find people with automation-related skills. Other options include taking on interns or hiring people with limited knowledge but a passion and desire to work in automation, then training them.

Automation isn't just about the tools that are used to achieve results; it is very much about the hands that perform the automation and, even more so, their minds and capacity to carry out the automation. The push will come from creative, resourceful, and time-sensitive practitioners. Those driven by the desire to automate all the things, whether they already have the skills or are taught, and who will deliberately put automation into practice. It's about creating, building, utilizing, and maintaining the things that will not only change the company for the better, but the things that will change the world.

Training and education for automation

Automation isn't just one thing or one person, one tool, or one technology. Is it a collective understanding or perfecting of the tools you build or adopt? Is it about applying critical thinking, dissecting problems, and—after understanding the problems and what tools fit best—not hesitating to automate? It's all of it. It's about having the ability to influence and persuade because change and technical implementations always require buy-in.

Organizations must invest in giving automation specialists the skills to keep up with the rate of change on the technological front. That's best done by investing in people—foremost in current staff members but also by supplementing your talent and creating a pipeline that brings in fresher eyes who haven't been subjected to the bad automation practices or the conflicts of the past.

As automation specialists change to keep pace with technology's evolution, the way they're trained on new skills has to change as well. I have sat in many flat technical trainings that end with me scratching my head and walking away with little real understanding of how to take the theory and actually turn it into practical application. 

So, what's the best solution?

First of all, there is no best solution, but for an organization or team to come close, they have to address the elephant in the room. What does this mean? It means that they have to truthfully answer the question, "what have been the failures that have kept the organization or team from getting to where we should be?" Do existing practices support the changes that true automation demands? Are there practices that can be tweaked? What practices need to be entirely eradicated?

Automation and the execution of best practices come after much agonizing over what was and what should and will be. Who are the best proponents of automation, and who can carry out the advanced parameters that are key to automation excellence? Automation must be continuously assessed and fine-tuned to get as close to being right with as little error as possible.

Ultimately, true automation success takes a collective of experts or people who have been automating for some time. In many cases, retraining, hiring, strategic planning, and thinking must happen almost simultaneously. In addition, improvements to existing practices and new applications (beyond just creating new tooling) will be needed. Thinking outside the box and reexamining past approaches will provide greater insight to sustain automation best practices with minimal erroneous results.

Skills and expertise to hire or develop

The skills and expertise that are most important for automation specialists will be ones that intersect one another. For instance, automation, development, and domain skills will be very important to ensuring the speed of delivery or the efficiency and effectiveness of automation. There is no one skill to look for; the right candidates have a combination of technical and soft skills and the innate ability to understand when and how to use them—and when not to.

Automation specialists have the ability to think broadly about how they apply automation and how automation will financially impact an organization. Some of the softer skills to look for include:

• Customer-first mindset
• Critical thinking
• Strategic thinking
• Troubleshooting
• Persistence
• Observability
• Enthusiasm
• Collaboration
• Leadership skills
• Creativity
• Real-time problem solving
• Communication
• Analytical skills
• Risk taking

Technical skills may include all or some of the following:

• Programming or scripting skills such as Bash scripting or Python
• Systems administration or engineering
• Application of CI/CD tools (e.g., Jenkins, VSTS, Octopus Deploy)
• Cloud (e.g., AWS, Azure, Google Cloud, OpenStack)
• Artificial intelligence, machine learning, blockchain, big data
• Repositories such as Git, Bitbucket, GitHub, GitLab
• Networking
• Performance/tuning
• Containers (e.g., Docker, OpenShift, LXD)
• Security (DevSecOps evolution)
• Linux
• Ability to pivot and learn new technical skills
• Infrastructure automation tools

Automation specialists should understand that technology evolves quickly, and specialists must to do the same. For instance, there was a time when infrastructure/automation engineers and system architects were in demand. While they are still being hired, things have evolved and now site reliability engineers, blockchain infrastructure engineers, and cloud architects and engineers are prevalent. Now chaos/intuition engineers, quantum computing experts, and quantum key distribution and cryptography skills are popular. The way technical roles are evolving begs for automation to do the same—and there is no doubt that it has.

Shifting is necessary to respond and adequately automate for efficiency and accuracy. It will require the intelligent processing of data and information that can provide the best guidance. Automation sometimes brings some trepidation, so being able to calmly and objectively look at the bigger picture—to know when to automate and when it is in the best interest of the company not to automate—are critical automation specialist tasks.

Processes and practices for building a culture of automation

Automation has to begin with the premise that without automation, companies will never be able to solve their business problems as effectively as their automating competitors. The process of constantly thinking about change and developing strategies for effectively executing automation will happen best in a culture that responds positively rather than negatively. Putting the best processes in place requires a top-down approach, where senior management is supportive and, by proxy, creates teams that share the same mindset around automation.

Having processes in place to address organizational strategy and flexibility, embrace failure, and expand skills and education is a good place to start building a culture of automation. Addressing weaknesses and creating a strategy for overcoming them is critical. Also, developing a process for addressing the inevitable resistance will be hugely beneficial, including supporting folks who want to automate but can't convince anyone else in their organization to take even the first steps. Some organizations have battle scars when it comes to automation and its own unique history.

Make sure to go slowly at first, which could be a challenge since, as automation specialists, we are used to going quickly and have an itch to automate. Build good, lasting engineering habits and set the right expectations from the beginning. It's important to remember why you're doing this—it's not because automation is cool; it's for the true, tangible, measurable benefits it can provide your business. Implementing automation best practices are like healthy weight loss—keeping weight loss off is more about the fundamentals—eating the right things in the right portions and exercising—than following a fad diet. Short-term gains are not nearly as important as building healthy habits over the long term. Automation is about long-term success with lasting benefits.

Automation practices are about improving operations across the entire organization more than the team dedicated to automation. A true culture of automation will require buy-in, even from those who don't totally understand it. Automation teams need specialists who are patient, who can operate within a team while being nimble, and are comfortable with constants and unknowns.

Maintaining constant, quality communication about automation with business leaders is an instrumental part of your purpose and strategy. Business leaders need to know that automation teams are focused on delivering value to the business, whether it's better uptime, stronger security, or just responding more quickly to changing business needs. Having a process to measure the success of automation at every turn is important for justifying the importance of having a culture of automation.

Finally, there must be a process for addressing failure when something goes wrong. This is another reason why it's important to have strong communication about automation with business leaders. When things go wrong (and they will), automation teams won't find themselves ripping out all those new tools and processes; they'll be working to fix them. Automation teams need to learn from their failures and strive to ensure they don't encounter the same problems twice. Each failure will be an opportunity to grow. Getting the business on board with this plan from the beginning will make sure that any failures are both learned from and planned for.

Any organization that adds automation, they have to manually create and standardize on one tool whether it be chef, ansible.  Also, it is harder to resurrect someone then to just get new blood in some cases. Specialists hav to be receptive to training and learning new technologies and that might only be achieved with new blood. Sometimes people may resist changes, but as organizations transform, people have to do the same.  Companies have a fiduciary duty to employees as they introduce automation, to make sure that specialists have the time and focus needed to level-up on the best practices or tools. Rome was not built in a day and automation won't be either.  

When centralizing around a tool, make sure that you emphasize the basics to meet the company’s  need and then go to the more fancy stuff. How much time are you letting your specialists actually spend on automation?  If they are only spending say 20% on automation and 80% on other operational things, then the time is not spent in the right place because if they can spend 80% automating, then they will save time and money in the long run.

The training should be more of a tiered approach.  It needs to be organized and it needs to be deliberate and intentional. Organizations have to make sure that they have the proper trainer or instructor to ensure knowledge transfer is achieved.  There is a science to teaching or training and getting someone who doesn’t really have this skill, means the likelihood that some automation specialists won’t possess the knowledge they are suppose to come away.  That automation specialists’ user experience is subpar to say the least and it is less likely that the specialist will effectively execute the requirements of the customer. What then? Well you will have wasted precious time and money and resources and you are back to square one and automation will suffer.     

Conclusion

Automation teams shouldn't try to define everything they will need to automate for decades. Just get started. Start small and automate the simple stuff, even if it's just writing a few scripts and running them with cron or focusing on automated troubleshooting.

Creating a culture of automation that stands the test of time will require an innate desire to effect change quickly and holistically. Automation teams may have to change often to meet the demands of innovation, so businesses should continually assess and regroup if called for. Creating a culture of automation must come from the inside out.