Brotli: A new compression algorithm for faster Internet

765 readers like this.
Open source network cable

Opensource.com

Brotli is a new open source compression algorithm designed to enable an Internet that's faster for users.

Modern web pages can often be made up of dozens of megabytes of HTML, CSS, and JavaScript, and that's before accounting for images, videos, or other large file content, which all makes for hefty downloads. Such loads are why pages are transferred in compressed formats; they significantly reduce the time required between a website visitor requesting a web page and that page appearing fully loaded on the screen and ready for use.

While the Brotli algorithm was announced by Google in September 2015, only recently have the majority of web browsers adopted it. The HTTP servers Apache and nginx now offer Brotli compression as an option. Besides Google, other commercial vendors (such as Cloudflare and DreamHost) have begun to deploy support for Brotli as well.

Compared to the good ol' gzip file compression and decompression algorithm, Brotli's open source algorithm shrinks data by an additional 20 to 25%. Brotli accomplishes this superior feat by making use of a dictionary that includes the most common words and syllables in English, Spanish, Chinese, Hindi, Russian, and Arabic, as well as common phrases used in machine readable languages (particularly HTML and JavaScript). This gives Brotli a clear advantage in compression and decompression over other algorithms, like gzip, bzip2, LZMA, etc.

To learn more, Cloudflare's engineers offer more details on how Brotli compares to these other algorithms. They provide an insightful exploration with a fascinating viewpoint on the potential impact that Brotli's improved compression may have on real-life applications. Another valuable take on the current state of Brotli (or one as of not too long ago, at least) is by Sam Saffron, co-founder of Discourse.org. He provides another set of benchmarking measures that demonstrate Brotli's size and speed savings, as well as a supportive opinion on why Brotli is mainly available for HTTPS.

With most browsers having added support for Brotli, here at DreamHost we followed suit and began deploying Brotli on our managed WordPress service, DreamPress, starting in December 2016. Since DreamPress uses nginx for TLS termination, we were able to use the Google-built ngx module in order to successfully build and deploy the Brotli library.

While browser support for Brotli is on a swift incline, there are notable holdouts as of this date. Apple Safari still has no plans for supporting Brotli, and Microsoft has plans only for its Edge browser, leaving legacy Internet Explorer users behind in the dust (that's kicked up by speedier alternatives). Microsoft's IIS server does have a free extension to add Brotli compression on the server side.

To check if a website is offering content compressed with Brotli, first make sure that your browser asks for the content-encoding extension br and the response in the header:

(As a sign-of-our-times side note, the initially proposed extension "bro" was rejected for having too much linguistic baggage attached to how the term is used in American English; in any case, naming it br rather than "bro" saves everyone a byte.)

Testing if a site supports Brotli using the tool provided by KeyCDN.com is arguably easier.

While in reality the advantage in speed that comes from using Brotli may not always be so readily apparent to users, it is for real. At the same time, seeing a new open source compression algorithm being developed at all is a rare thing, and that makes it worth appreciating in and of itself. Since we love open source at DreamHost, and all of the stars happened to align, we went ahead and started offering this option. And it works.

Tags
Stefano Maffulli smiling at the camera
Stefano is the executive director of the Open Source Initiative, an experienced leader of open source organizations, from non-profits advocacy groups and trade organizations to business ventures and community projects across countries. With a proven track record in community building, he’s also an active contributor to open source projects.

13 Comments

But what's its Weissman score?

Ah, but does it offer middle-out?
;-)

What about performance? If it attains 25%-30% more compression, but the cost of being twice as slow, it won't be much good.

what are the best level for brotli from 0 to 11 for best results if compared with gzip

The articles I linked from the post have benchmarks and more details. The gist of it is: the gains in speed and bandwidth depend a lot on how brotli is used.

In reply to by Krishna Pandey (not verified)

Why do all the links in this article have "?target=_blank" in them? What is that supposed to mean?

Hi QA,
target="_blank" instructs your web browser to open the link in a new tab or window.

In reply to by QA (not verified)

Glad to hear there already are plans to implement it widely. I cheered for the "bro" extension, though. :)

A quick test of:

https://www.google.com

yields:

"Negative! www.google.com does not support Brotli compression."

What ever happened to eating your own dog food?

Some of your links has an extra ?target=%22_blank%22:

Good catch! I just went in and took care of that.

Thanks for pointing it out!

In reply to by user00003 (not verified)

Irony. This article (well, opensource.com) uses the gzip compression algorithm. "Content-Encoding:gzip" :)

Talking about Brotli, we cannot forget about recent Facebook Zstandard which is usually much faster.
http://facebook.github.io/zstd/

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.