divergent media, inc. - Forget Flash, Skip GIF, Control iOS

January 29, 2013

A few months ago, Apple announced the new iPhone5. To us though, the most interesting part wasn't the new hardware, but some subtle animation on the “Design” page. Obviously Apple wouldn’t use Flash to do animation on their site. And the HTML5 video tag doesn’t really allow video to be used as a design element the way it was used on these pages. The type of motion didn’t look like something that could be easily achieved with CSS.

So what was going on? Fortunately, we didn’t need to dive too deep, as @dbloom had already done some research.

Through a clever combination of still images, JavaScript and the canvas tag, Apple implemented a basic video decompression system, without needing browser support for video. We later found the makers of Sublime Text were doing something similar. We decided a tool that could generate this type of content without all the hassles might be useful for a web developers and designers at large.

There's an app for that (now)

A few months later, the fruits of that work are ready to show - we’re excited that Phosphor is now available on the Mac App Store, and we’re eager to see what folks do with it. In the spirit of Phosphor's beginings, here's a sample iPhone screencast:

You can see more demos on our phosphor product page and gallery, or visit the App Store.

How does Phosphor Work?

Phosphor takes each frame and breaks it up into macroblocks - 8x8 blocks of pixels - much like most video codecs. Then, it looks at each macroblock to see if it’s similar to any macroblocks it has previously encountered. For example, if your image mostly consists of a solid black background, Phosphor only needs to store a single 8x8 black macroblock, and then repeat that block as needed. When operated in a lossless manner, Phosphor only looks for blocks that are identical. In the default lossy mode of operation, Phosphor looks for blocks that are very similar, but not necessarily mathematically identical. For example, if the video you’re analyzing has been previously compressed, the codec may have introduced subtle noise (even though it may not be visually apparent). Lossy operation allows Phosphor to account for this.

After discarding all of the redundant macroblocks, Phosphor writes the remaining blocks into new images. These can be either JPEG, TIFF or PNG files. There’s nothing special about them - you can open them in any application that knows how to open images. Below is an example of the files Phosphor outputs for a composition.

You'll see a placeholder still of the first frame of video. This is displayed while your page is loaded. Then you'll see one or more "atlas" images. These are the simplified macroblocks written out in the file format chosen. It may look like a mess, however, Phosphor also saves out some instructions to tell the browser how to display these. These instructions are stored as the JSON and JSONP object files. You can open these files too - they’re just text. Inside, you’ll find an encoded set of instructions to tell the Phosphor framework where to place each macroblock, and when to display it. There are also some configuration values. The actual encoding standard is a bit too technical for this blog post, but visit our GitHub repository or check out our documentation for details.

Why not use ____?

There are a few advatages of this technique over other video and animation technologies. Compared to a format like HEVC or H.264, this type of compression is relatively primitive. But it’s far more efficient that transmitting entire frames, which was previously the only type of transmission available for this type of interactive, scriptable and standards-based animation.

HTML 5 video: iOS devices won't play video tags in the DOM, they instead open in a standardized player. Apple also stops them from auto-playing on iPhone. To support all the modern browsers, you'll need to deploy mutliple codecs. Phosphor will work out of the box on all modern browsers. Our compositions also support alpha, full layering, and custom frame metadata which video tags do not.

GIF: GIFs have less scriptablity. They play as soon as loaded. Colorspace is limited to 256 colors and alpha channel is binary only. For most content they are much larger files than Phosphor assets. For instance, the iPhone composition above is 7mb in GIF and less than 500kb in Phosphor.

Flash: Bwhahahahaa! Flash. Yes, use that.

Just the start

Releasing Phosphor is really just the first step - since it’s powered by web-standards technology and an open JavaScript framework, we know that developers will find uses for the application we haven’t even begun to imagine. We hope you’ll take a look, and show us your results!

Check out our phosphor product page and gallery, or visit the App Store. If you’re interested in extending our player framework, you can fork it on github.