6 time-consuming tasks you can automate with code

No readers like this yet.
How has open source changed your life?

Opensource.com

Literacy used to be the domain of scribes and priests. Then the world became more complicated and demanded that everyone read and write. Computing is also a form of literacy, but having it only understood by a priesthood of programmers is not going to be enough for our complex, online world. "Learn to code" has become a mantra for education at all ages. But after clearing away the hype, why do people need to learn to code? What does it get us exactly?

Not everyone needs to become a software engineer, but almost every office worker uses a laptop as a daily tool. Computers are such a huge productivity booster because they support a large market of programs and apps designed for these workers. But commercial and open source software have a "last mile" problem: that they don't automate every conceivable task. There are still computing chores that require a lot of repetitive (and fairly mindless) typing and clicking. Even if you have an intern to push these tasks on, they're tasks that require a human because there's no software to automate it. These tasks are too small-scale or specific to your organization's workflow for it to be economical for a software company to create a custom solution.

This is where everyday programming can save the day. A little bit of coding knowledge can let anyone write small scripts to do these tasks and save them hours (or weeks or months) of effort. While Excel has programmable macros and Outlook has configurable email rules, coding in a real programming language like Python is vastly more adaptable and powerful. And despite its reputation for being akin to "rocket surgery," programming is just a skill like driving a car. You may have been nervous taking your license exam, but soon afterwards operating a motor vehicle becomes just a routine part of modern life.

My latest book, Automate the Boring Stuff with Python, skips the computer science and teaches just enough Python programming to get to a practical skill level. It's so important that everyone gain these skills that I've made the book freely available online at AutomatetheBoringStuff.com under a Creative Commons license, in addition to publishing a print edition through No Starch Press. Part one of the book is a general programming tutorial for beginners in the popular Python language. Part two covers the modules available for Python that help automate tasks.

But what are these tasks that people will be able to write? I've found they generally fall into six categories:

1. Gathering data, formatting data

Regular expressions (Chapter 7) allow you to program the computer to do text pattern matching. You do pattern matching yourself all the time: the text pattern is how you can tell 415-555-1234 is a phone number, but 4,155,551,234 is not. If you know the general pattern of the data you're looking for, your laptop can gather the exact information you need much faster than you can scroll through a document.

Say you have a large PDF of monthly sales data for the last five years. Or maybe you have this data as 60 separate Excel files. Or it's on a web site across several pages, requiring you to click a "Next" link after skimming each one. The data we need is often in an inconvenient format.

Python has several modules for reading data from PDFs, Excel spreadsheets, Word documents, web sites, CSV files, and other formats. Reading in data from thousands of files is no problem for your laptop. And once your program has loaded this information, it can then output it in any format your organization needs. It sure beats copying and pasting text by hand.

2. Batch of instructions

You may have learned a few keyboard shortcuts: It's much faster to press Ctrl+A, Ctrl+C than it is to move the mouse to the menu and click on Edit, Select All, then Edit, Copy. Imagine if you could create your own shortcuts. Writing a program that can launch other programs or execute existing commands (Chapter 15) lets you turn several steps into one step. You can write programs to control the keyboard and mouse for you (Chapter 18), filling out forms automatically and more. This is the sort of trick that can save you a lot of effort over time, especially for commands you do often.

3. Mass file operations

Copying a file to a flash drive is simple. But some file operations aren't so straightforward. What if you need to copy only the files that have a modification timestamp within the last week? Or if you need to rename hundreds of files to add a prefix to their name, based on the folder they're in? And you need to do these things to files spread across several different folders? Writing a program to copy, move, rename, delete, or compress according to your specific criteria (Chapter 9) can do this kind of filing for you quickly and precisely.

4. Automated checking

Checking for typos is a common task, and so it's found in software often. But what about checking for problems that are particular to you or your organization's needs? Say a coworker emails you daily spreadsheets with phone numbers, and you need to know if any of them are missing an area code. Or you have PDF reports that are sometimes missing numbers. Eyeballing this yourself is tedious and error prone. You can write scripts to check for missing data or data that doesn't fit the valid text pattern (Chapter 7) to free yourself from these chores.

5. Notification

Perhaps you've had the experience of clicking Refresh in your browser over and over, waiting for a web page to update with the information you need or a service you want to sign up for. This is a waiting-in-line kind of task that the computer can check for you, sending you an email or SMS text notification (Chapter 16) when it is ready. A long-running program can read a web page (Chapter 11) or other source daily, hourly, or even every few seconds (Chapter 15). It can check for updates whether you're using your laptop or not. This way you don't even have to be in front of your laptop to be notified. Your program can inform you by texting your smartphone.

6. Reporting

Compiling reports is a chore, but gathering and formatting information can be automated (described in the first category). Your report-generating program can fire off regular reports once a day or once a week (Chapter 15), and your code can email out these reports to the people who need them (Chapter 16). All of the skills you've learned to code the programs from the previous categories can be used to fully automate sending regular reports to coworkers and managers.

And if they've learned to program, your coworkers and managers could write code to automatically read the reports for any actionable information.

Your laptop is a powerful tool, and one that you've probably used in your professional or personal life for years already. You don't have to be limited by using only the software that others have created and sold to you. By learning to program, you can maximize its productive potential and your own.

To read examples of tasks people have automated to spur your own imagination, check out this conversation on Reddit.

User profile image.
Al Sweigart is a software developer and tech book author in San Francisco. He has written several Python books, which are available under a Creative Commons license at https://inventwithpython.com

6 Comments

Excellent article.

Getting started with these little coding projects is a great way to make one's work easier. It also conforms to the Linux Philosophy tenet, "Automate everything."

You are right throughout. But just look around you, how many persons still remember the little trick you showed them a few days later ? How many workers just consider the computer as another coffee machine ? How many secretaries do not know how to use a word processor ? Coding for the masses now ? IMHO, coding will still remain an elite priesthood's privilege .

True. To most people a computer is just another appliance. But I, personally, think it is worth it if even one of those people glimpses the true power behind the curtain.

In reply to by SET (not verified)

Assume I boost my productivity -- then what? More work will be assigned with no increase in pay, and quite possibly one of my fellow workers will be made redundant.

That really made me laugh. Thanks!

Unfortunately a true statement in any number of workplaces. The people who are more efficient in their jobs generally get higher work loads to compensate. Hopefully more organizations adopt merit based pay increases.

In reply to by George McDuffee (not verified)

As for getting more work because you automated some; Don't tell ANYONE that you automated the work. But have the documentation on HOW it is done so you can hand it off to someone should you find greener pastures. If the job should take an hour, but your script takes 5 minutes.......sounds like you have free time, about 55 minutes of it.

When your tedium is automated, just keep "working". No one needs to know that you are working smarter.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.