An inside look at open source at Twitter

In this exclusive interview with Chris Aniszczyk, the Head of Open Source at Twitter explains how company engineers are working on open source technologies internally and contributing to a range of community projects. Learn more about open source at Twitter in Chris's talk at All Things Open.

As Head of Open Source at Twitter, what exactly do you do?

In essence, I make the lives of Twitter engineers easier when open sourcing software, working with open source communities, or in places like standards bodies. Over four years ago, I had a fantastic opportunity to join Twitter to create an open source program (@TwitterOSS) from scratch. One of our main goals was to ensure that, as a company, we were good open source citizens—from the basics of respecting open source licenses to making it easier for engineers to open source code and ensuring we're giving back to the open source projects we depend on. Furthermore, we also wanted to be diligent in investing engineering time to open source projects and communities. (You can see this blog post for more information.)

In my opinion, it's a growing trend that companies like Twitter, Google, Facebook, Samsung, Walmart and even startups like Box are creating open source offices to maximize their value of open source participation. Heck, even Microsoft has an open source office these days.

What are open source developers currently working on at Twitter?

Twitter has about a couple thousand engineers across the company working on a variety of technologies, from as deep as the Linux kernel to front-end Javascript libraries. It's hard to pick anything in particular, but recently we have graduated Apache Parquet from the Apache incubator and are working on adding stateful service primitives to the Apache Mesos project so we can run MySQL in a Mesos cluster.

I can speak forever about this topic, but I highly recommend you view our open source site and check out our GitHub organization to see what we are working on.

How is open source part of the culture at Twitter?

Since Twitter's early days, open source has been a pervasive part of engineering culture. Every Tweet you send and receive touches a plethora of open source software on its journey from our Linux-based infrastructure to your web browser or mobile devices. Furthermore, we have engineers partake in open source training when they start their job to ensure they understand our processes and culture when it comes to interacting with open source communities.

Twitter was built on open source software, but what are examples of the open source solutions that Twitter has given back to the community?

Oh my, there are so many projects I'm proud of. First, I'm really proud of us sharing our expertise in scaling infrastructure by open sourcing technologies like Apache Mesos and Aurora. Over the past few years, Mesos has grown to a fully independent open source project at the Apache foundation and now runs the infrastructure for some of the largest scale companies like Twitter, Netflix, Airbnb, Cisco, Paypal, Uber, and even Apple. For example, every time you talk to Siri on your iPhone, you're speaking to an infrastructure backed by Mesos.

Second, Finagle is the JVM library we use to write highly concurrent distributed services that power Twitter timelines, direct messages, and more. By sharing the way we architect and run our services through Finagle, it warms our heart to see other companies like Foursquare, Nest, NYTimes, Pinterest, SoundCloud, and more adopt the lessons we have learned over the years.

Another project I'm proud of is twemproxy, which I think generally flies under the radar. In essence, twemproxy is a fast, lightweight proxy for memcached and redis clusters. It helps scale and power some of the largest cache clusters in the world, from Twitter to Wikimedia, Pinterest, Yahoo, and more.

Finally, I'm really proud of the popular Bootstrap front-end toolkit, which originated at Twitter. The Bootstrap project grew so much that we eventually set it free and made it a community-run project in its own GitHub organization as part of the v3 release.

In your talk, you'll discuss lessons learned from creating Twitter's open source office. Without giving away your whole talk, what are a couple of lessons learned?

1. Community doesn't come for free: There's a common belief that if you open source something, people will show up and contribute magical patches to your open source project. While I wish that were true, the mantra of "if you build it, they will come" doesn't really apply to most open source projects. You have to put in the hard work of community building to see growth over the long term.

   At Twitter, our open source program office has a team of developer advocates focused on growing open source ecosystems, which are important for us to ensure they thrive and evolve to our benefit. This includes making it easier to contribute and ensuring contributors feel welcome. In the end, happier contributors will contribute more.

2. Diversity is the spice of life; collaborate at the start: In general, projects that are more diverse in terms of committers have a better chance of long term success from the get-go. At Twitter, we prefer not to open source projects unless they have been running in production and have collaborators signed up from the start. What we mean by having collaborators signed up from the start is that we talk to our peers in industry before open sourcing any major project and do our best to get multiple parties on board in the beginning.

   For example, we started the Parquet project and collaborated with Cloudera before fully open sourcing the project. Launching with collaborators from the beginning can help build momentum. In a couple years, Parquet has 60+ contributors from 10+ organizations and has successfully graduated from the Apache incubator into a mature project.

Let's say you're organizing an "open source at Twitter" trivia contest. What are three Twitter trivia questions and answers you'd include?

Q: What Twitter open source project helps power Wikipedia's caching layer?

A: Twemproxy

Q: When you talk to Apple's Siri on an iPhone, what Twitter-originated open source project helps power its infrastructure?

A: Mesos

Q: What is the most popular open source project on GitHub in terms of combined stars and forks? (Hint: it's a front-end project that originated at Twitter.)

A: Bootstrap

If you read my interview, you should be able to answer all three questions.

This article is part of the All Things Open Speaker Interview series. All Things Open is a conference exploring open source, open tech, and the open web in the enterprise.