CloudFlare blows hole in laws of Web physics with Go and Railgun | Ars Technica

Today, a large collection of Web hosting and service companies announced that they will support Railgun, a compression protocol for dynamic Web content. The list includes the content delivery network and Web security provider CloudFlare, cloud providers Amazon Web Services and Rackspace, and thirty of the world’s biggest Web hosting companies.

Railgun is said to make it possible to double the performance of websites served up through Cloudflare’s global network of data centers. The technology was largely developed in the open-source Go programming language launched by Google; it could significantly change the economics of hosting high-volume websites on Amazon Web Services and other cloud platforms because of the bandwidth savings it provides. It has already cut the bandwidth used by 4Chan and Imgur by half. “We've seen a ~50% reduction in backend transfer for our HTML pages (transfer between our servers and CloudFlare's),” said 4Chan’s Chris Poole in an e-mail exchange with Ars. “And pages definitely load a fair bit snappier when Railgun is enabled, since the roundtrip time for CloudFlare to fetch the page is dramatically reduced. We serve over half a billion pages per month (and billions of API hits), so that all adds up fairly quickly.”

Rapid cache updates

Like most CDNs, CloudFlare uses caching of static content at its data centers to help overcome the speed of light. But prepositioning content on a forward server typically hasn’t helped performance much for dynamic webpages and Web traffic such as AJAX requests and mobile app API calls, which have relatively little in the way of what’s considered static content. That has created a problem for Internet services because of the rise in traffic for mobile devices and dynamic websites.

“The Web is changing,” said CloudFlare CEO Matthew Prince, “with more sophisticated websites that are API-driven like true applications. And more and more content is being viewed through mobile apps—it’s not a traditional webpage that people are developing now. And the last-generation CDN tricks become less effective for API-driven sites and mobile applications.”

Railgun attacks that problem by replacing the standard HTTP connection between CloudFlare’s customer websites and their data centers with a binary protocol. The protocol is used to communicate between a “listener” application that sits on the customer’s end and “sender” software that resides at whichever CloudFlare data center is closest to the end user that is the source of the API or page request.

When the CloudFlare software receives the request, it takes the last version of the page or API response it has kept in memory and creates a hash value from it. The hash value gets sent back to the Listener, which finds the version of the page or API response that matches the hash value. It then compares that version to the current page or API response, sending back binary instructions on what has changed. This lets the CloudFlare software construct the appropriate response and send it to the end user.

The hosting provider partners will offer the Listener capability to their customers with a single click, hosting the protocol for them without the need for any changes to servers; CloudFlare will provide software images for Amazon and RackSpace customers to install.

Over time, the two components build a dictionary representing chunks of content that “can get crazy compression ratios of 99 percent,” according to by John Graham-Cumming, the CloudFlare programmer who led the Railgun effort. “You can make references into the dictionary; if the page hasn’t changed at all, the server side could say, ‘Nothing’s changed, send the whole dictionary out.’“ Small changes in the page could be sent in messages as small as 5 bytes, he added. “That translates directly into bandwidth savings.”

Going with Go

The approach, which is similar in many respects to video compression techniques, was developed over the past year. Graham-Cumming said that the goal was to make sure that customers didn’t have to write any specially developed code for Railgun to work, as is needed to support Akamai’s Edge Side Includes (ESI) approach to doing dynamic content compression. He chose Go for the project because of its approach to handling concurrent sequential processing. “I was interested in it because we’re getting multicore machines everywhere and wanted to be able to exploit that parallelism. If it was written in C++, it would be threaded code.”

Graham-Cumming also found he was comfortable with Go in part because of one of the language’s sources of inspiration—Communicating Sequential Processes (CSP), the formal programming language devised by Sir Tony Hoare, who was Professor of Computing at Oxford when Graham-Cumming studied there.

“Go is very light,” he said, “and it has fundamental support for concurrent programming. And it’s surprisingly stable for a young language. The experience has been extremely good—there have been no problems with deadlocks or pointer exceptions.” But the code hit a bit of a performance bottleneck under CloudFlare’s heavy loads, particularly because of its cryptographic modules—all of Railgun’s traffic is encrypted from end to end. “We swapped some things out into C just from a performance perspective," Graham-Cumming said.

“We want Go to be as fast as C for these things,” he explained, and in the long term he believes Go’s cryptographic modules will mature and get better. But in the meantime, “we swapped out Go’s native crypto for OpenSSL,” he said, using assembly language versions of the C libraries.

“We think we must be one of the groups stressing Go the most,” Graham-Cumming said of the CloudFlare team. As a result of that work, they’ve submitted a number of fixes for the language’s core, including a fix for its logging functions and an external library for memcache.

To keep performance at its highest possible level, all of Railgun’s dictionary cache is kept in memory on the servers of CloudFlare cluster (for a full description of CloudFlare’s architecture, see our story on the company’s ten-data center rollout earlier this year). “We have hundreds of machines with cache on them for Railgun,” Graham-Cumming said.

Those bound to benefit most from Railgun are high-traffic sites running on Amazon Web Services and other cloud providers who charge based on bandwidth. One of those Amazon customers is Imgur, which uses AWS for much of its dynamic content. According to Prince, based on CloudFlare’s testing with hundreds of mobile applications and websites, Railgun has yielded an average 50 percent reduction in bandwidth usage and a 90 percent reduction in the “time to first byte” for Web and mobile clients. “If you’re on AWS, it’s a no brainer,” said Graham-Cumming. “It will pay for itself in no time.”

The hosting companies win too, as they’re able to reduce the load on their internal networks and the cost of their outbound bandwidth. Those signed on to support Railgun include 040hosting, A2 Hosting, Arvixe, Bluehost, ByetHost, CoreCommerce, DreamHost, ELServer, FastDomain, GreenGeeks, HostPapa, HostMonster, Just Host, InterServer, MapleTime, Media Temple, MDDHosting, NameCheap, PacificHost, PRO ISP, SiteGround, Sliqua Enterprise Hosting, Softcloud Hosting, SparkRed, VentraIP, VEXXHOST, WebHostingBuzz, WebHostingPad, x10Hosting, and Zuver.

Listing image by US Navy