The Supreme Court will hear broadcasters’ challenge to the legality of startup Aereo, in a case that may determine not only the future of digital streaming of station signals but of network television itself.

Without comment, the justices on Friday agreed to accept ABC Television Stations vs. Aereo, in which the television networks are seeking to halt the Barry Diller-backed venture, contending that its offering of streams of station signals in New York and other markets violates the public performance provisions of the Copyright Act. Justice Samuel Alito took no part in the consideration of the petition, the court said, without elaborating. Typically such recusals are for a potential conflict of interest, and Alito has previously said that his family owned stock in the Walt Disney Co.

“We are pleased the Court has agreed to hear this important case,” Fox, Univision, PBS and other plaintiffs said in a joint statement.  ”We are confident the court will recognize that this has never been about stifling new video distribution technologies, but has always been about stopping a copyright violator who redistributes television programming without permission or compensation.”

CBS, along among the plaintiffs, said that they “believe that Aereo’s business model, and similar offerings that operate on the same principle, are built on stealing the creative content of others. We are pleased that our case will be heard and we look forward to having our day in court.” ABC and NBC also released a joint statement, saying they were “gratified” that the high court is taking the case.

Chet Kanojia, founder and CEO of Aereo, said that “it was our hope that this case would be decided on the merits and not through a wasteful war of attrition. We look forward to presenting our case to the Supreme Court and we have every confidence that the court will validate and preserve a consumer’s right to access local over-the-air television with an individual antenna, make a personal recording with a DVR, and watch that recording on a device of their choice.”

Aereo argues that it is well within legal boundaries. It assigns each subscriber a dime-sized remote antenna, with individual copies of broadcast feeds recorded to a remote hard disk and, after a delay of at least six seconds, streamed to the viewer.

Aereo contends that its users, not the company, have “control” over the system, making it a private use akin to time-shifting on a VCR or DVR. It is relying in large part on a 2nd Circuit Court of Appeals decision in 2008 that determined that Cablevision was within the bounds of copyright law when it offered a remote DVR.

The next year, the Supreme Court declined to review the Cablevision case, letting the decision stand in what ultimately turned out to be an example of the unintended consequences of challenging emerging technology in the courts. Before declining media congloms’ petition to review, the justices asked then-Solicitor General Elena Kagan to weigh in, and she and other staff attorneys concluded that the Cablevision decision was “insufficiently important” to warrant it.

But Aereo was launched in March, 2012, the company mindful of the precedent of the Cablevision case. Several months later, U.S. District Judge Alison Nathan relied on the Cablevision decision in refusing to issue an injunction to halt Aereo, and a split panel of the 2nd Circuit affirmed her ruling. Another federal judge, in Boston, also has sided with Aereo.

Interestingly, as the TV networks sought review from the Supreme Court, Aereo also urged the justices to take the case, concluding that it would be better to have the issue resolved rather than to have a patchwork of decisions across the country as it launches in new markets. Broadcasters also are challenging Aereo in such places as Utah, and they have had greater success at putting a halt to one of Aereo’s rivals, FilmOn X. Federal judges in Los Angeles and Washington DC ruled against FilmOn X, and a decision is pending on that startup’s appeal in the 9th Circuit.

But the implications for broadcasters are greater than just whether the two companies continue to exist. They contend that Aereo threatens to undercut the growing stream of revenue coming from retransmission fees, estimated at more than $3 billion annually, money broadcasters say is more important in helping to finance original programming. If Aereo continues, broadcasters say, cable and satellite operators will see little need to pay those retransmission fees and can merely develop their own streaming services. Last year, News Corp. COO Chase Carey even threatened that they would consider putting its Fox network on a subscription platform if services like Aereo were not stopped.

The broadcast networks are confident that the justices will see the Cablevision decision as an outlier in an otherwise straightforward interpretation of the Copyright Act, particularly a provision that says a performance can be defined as ‘public’ whether viewers “receive it in the same place or in separate places and at the same time or at different times.” They argue that lawmakers who wrote the Copyright Act in 1976 intended it to restrict future new technology like Aereo from transmitting their signals without permission.

Aereo, meanwhile, argues that the Supreme Court’s opinion could affect the future growth of cloud computing, in that users are accessing a host of content stored remotely.

“The landmark Second Circuit decision in Cablevision provided much needed clarity for the cloud industry and as a result, helped foster massive investment, growth and innovation in the sector,” Kanojia said in his statement. “The challenges outlined in the broadcasters’ filing make clear that they are using Aereo as a proxy to attack Cablevision itself and thus, undermine a critical foundation of the cloud computing and storage industry.

He added, “The broadcasters are asking the Court to deny consumers the ability to use the cloud to access a more modern-day television antenna and DVR. If the broadcasters succeed, the consequences to consumers and the cloud industry are chilling.”

The prospect of courts shutting down Aereo has not stopped its expansion into other markets, although it is doubtful it would expand to Los Angeles any time soon, nor has it prevented it from raising additional money from investors kike Diller. Earlier this week, Aereo announced that it had raised another $34 million to help it grow.

For its part, Cablevision has said that it believes that Aereo infringes on copyright, but that the Cablevision decision was sound. “Cablevision remains confident that while the Aereo service violates copyright, the Supreme Court will find persuasive grounds for invalidating Aereo without relying on the broadcasters’ overreaching — and wrong — copyright arguments that challenge the legal underpinning of all cloud-based services,” Cablevision said in a statement on Friday.

Jonathan Steinsapir, partner at Kinsella, Weitzman, Iser, Kump & Aldisert, noted that Aereo represents the biggest copyright case involving new technology to reach the high court since MGM vs. Grokster, in which the justices decided that a file-sharing company could be held liable for distributing devices with the purpose of making infringing copies.

Studios, he said, are anxious to see the Cablevision decision overturned. “I think there is a feeling that it was too focused on the form [of the technology] rather than the substance,” he said. A Supreme Court ruling also could have an impact on another case that is in the 9th Circuit, the broadcast networks’ challenge to Dish Network’s introduction of its Hopper service, which includes an automatic ad-skipping feature. The networks are in the midst of appeals over that technology.

Broadcasters are taking the chance that they will get an adverse ruling from the Supreme Court, Stensapir said, but they have too much at stake not to seek a review. “It is a risk they have to take,” he said.

January 2014

I was accused recently of believing things I don't believe about women as programmers and startup founders. So I thought I'd explain what I actually do believe.

Some accused me of being sexist—of being biased against female founders. To anyone who knows Y Combinator that would seem a pretty implausible claim. It's hard to argue I'm biased against female founders when I have a female cofounder myself. And with 3 female partners out of 12, YC has slightly over 3x the venture industry average. While 3 out of 12 is not 50-50, it would be very hard to find another firm of our size in the venture business where women run the show to the degree they do at YC. I may be the public face of the company, but it's impossible to imagine YC doing something that Jessica, Kirsty, and Carolynn were against.

More thoughtful people were willing to concede YC wasn't biased against women, but thought we should be actively working to increase the number of female founders. As one put it, instead of being a gatekeeper, we should be a gateway.

But that is exactly what Y Combinator is. The people who caricature us as being only interested in funding young hotshots forget that when we started, in 2005, young founders were not a privileged group but a marginalized one. VCs didn't want to fund them, and when they did they often as not tried to replace them with "adult supervision." The fact that young founders seem a privileged group now is partly due to our efforts. We attacked the problem not by advocacy but by action—by funding more young founders than VCs would, and then helping them to overcome the bias against them that they'd encounter among other investors. It worked rapidly, because it had a double effect: if you support a young founder who otherwise would not have been able to find funding and they go on to succeed, you get not just one more young founder but also the additional ones they inspire by their example.

We're doing the same thing for female founders. We fund more female founders than VCs do, and we help them to overcome the bias they'll encounter among other investors. In the current YC batch, 16 out of 68 companies, or 24%, have female founders. That's almost twice the rate at which VCs fund such companies. [1] If these founders go on to succeed, they'll become what we know from experience will be the most powerful force for encouraging other female founders: examples of people like them who've done it.

The way we got so many female founders was by being less biased. It was the same with young founders. We didn't lower our standards for younger founders. We just tried to be better than other investors at seeing their true potential. That's why it worked.

Is there another organization that has done more to help female founders than Y Combinator, measured by number of female founders helped times how much we've helped each one? Possibly. But the amount of thought it takes to answer that question shows we are at least among the organizations that have done most. [2]

Eventually other investors will follow our lead. I can say more than that I hope they will, because as with young founders, they will make more money if they do, and investors tend to notice that fairly quickly. A lot of people outside the startup world seem to assume that investors have the same sort of naive bias ordinary people do when deciding who to invite to join a club—that they simply fund the people most like them. That is not true. [3] What drives most investors is money. As it should, because they have a fiduciary duty to their limited partners. So while they have biases, their biases are merely instances of stupidity, not principles they feel they have to uphold.[4] When they realize they've been overlooking some type of startup or founder, they stop, because it means they've been losing money. All it takes is another investor who's willing to fund the companies they're overlooking.

I saw this happen with Homejoy. When Adora Cheung first tried to raise a series A round, she couldn't find a VC willing to lead, despite their remarkable growth. Partly this was due to the nature of the business. Software-eating-the-world startups are still a novel idea to many investors. But I was convinced it was also partly because Adora was female. So I tweeted their revenue graph, saying that this was the fastest growing YC company I knew of, because I knew ultimately growth was what would convince investors. As in fact it did. If your numbers go up steeply enough for long enough, you could have eyeballs on stalks and investors will fund you.

I realize though that with female founders, efforts at our stage are not enough. We could in principle have fixed the problem for young founders by ourselves. If we funded enough young founders who went on to succeed, both investors and other would-be founders would learn from their example. But it would be naive to assume we could get the percentage of female startup founders to 50% so long as the percentage of female programmers is so much lower than 50%. Though this is less the case than it used to be, many startups still have a big technical component, and if you want to start that sort of startup your chances of succeeding are higher if you're a programmer. Adora Cheung is a programmer, for example. Software eating the world is still software.

So how would you cause there to be more female programmers? The meta-answer is: not just one thing. People's abilities and interests by the time they're old enough to start a startup are the product of their whole lives—indeed, of their ancestors' lives as well. Even if we limit ourselves to one lifetime we find a long list of factors that could influence the ratio of female programmers to male, from the first day of a girl's life when her parents treat her differently, right up to the point where a woman who has become a programmer leaves the field because it seems unwelcoming. And while the nature of this sort of funnel is that you can increase throughput by attacking bottlenecks at any point, if you want to eliminate the discrepancy between male and female programmers completely, you probably have to go back to the point where it starts to become significant.

It seems to be well underway by the time kids reach their teens. Which to me suggests the place to focus the most effort initially is in getting more girls interested in programming.

I'm not saying that's the only thing you could do, or trying to downplay other obstacles women and girls face in becoming programmers or founders. I'm just saying that seems to me where you'd get the most bang for the buck.

How would you get more girls interested in programming? I don't know much about girls specifically, but I have some ideas about how to get kids interested in programming. I think at a minimum you need two things: access and examples.

First of all, kids need to be able to program, in both senses of the word: they have to know how to write a program, and they need access to a computer they can write programs on, which nowadays probably includes Internet connectivity.[5]

But to turn kids into avid programmers—to get them to work on projects of their own in their spare time—you may need to do more than just expose them to programming. In my experience the best way to get people to work on ambitious projects is examples of other people who have. I'm pretty sure that's why so many more startups come out of some universities than others, for example. I don't think Yale students are inherently less able to start startups than Stanford students, or that Yale doesn't prepare them to. They just don't have examples of successful startups all around them like Stanford students do.

So if we want to get more girls to become programmers, we should give them more examples. Ideally in person, though examples also work through the media. Do the examples have to be female? They don't have to be to send the message that programming is one of the things people can work on, but female programmers might make more compelling role models.

Are there other things you could do to get girls interested in programming? Almost certainly. Are there other things you'd have to do? Quite possibly. But notice the language I'm using. In the course of writing this I've gone from certainty to uncertainty. That's because I've gone from talking about a topic I know more about than most people—what happens when women start startups—to a topic many people know more about than me.

I can say though that at our end of the funnel the trend for female founders is encouraging. Not just because 24% of the companies in the current YC batch have female founders. There is also starting to be a critical mass of successful female founders from previous batches, which means we can now run events like the one Jessica is organizing in which they share their experiences with other women interested in starting startups. That should further accelerate the growing number who do.

Notes

[1] In 2013, 13% of series A rounds went to companies with female founders. We did our own study to determine the number because the numbers we found online varied so much. We randomly selected 200 companies from Mattermark's list of those that raised series A rounds in 2013 and looked up all their founders. 26 of 200 had female founders at the time they got funded.

The 24% number is itself growing. Two years ago it was 9% (which was also more than VCs were funding then). Why the increase? Partly because more women are starting startups, partly because YC's applicant pool has been broadening beyond its initial core of programmers (a group in which women are underrepresented), and partly from efforts to recruit individual women.

[2] Which in turn shows how broken the public conversation is about this topic. There are a lot of people who are so agitated about it that they end up attacking those who are actually most sympathetic to their aims. If we really want to improve the situation it would help to stop blasting one another with friendly fire.

For example, if your goal is to get Y Combinator to accept more female founders, how does it help that goal to claim publicly that we're hostile to women? It just makes women less likely to apply.

[3] It's a useful technique for investors to fund founders who are like them in ways that help the startup, of course. So it's good if investors had been startup founders themselves, for example, because they can recognize their peers. Long ago a few people attacked me for saying we could recognize successful founders because they were "like us." It may have been the first time YC got trolled on this topic. But it was clear from the context that I meant we could recognize them because they were like us in being founders, not because they were demographically like us.

[4] Mediocre investors discriminate against women not because women are unlike them, but because they're unlike previous successful founders. Like mediocre people in any field, they're fighting the last war, and the last war was won by Mark Zuckerberg.

[5] Many kids now have computers with Internet access, but kids from poor families often don't. So to get them interested in programming you also have to solve the problem of hardware somehow. That is among the problems being attacked by one of the nonprofits in the current YC batch.

Thanks to Sam Altman, Alexandra Cavoulacos, Adora Cheung, Tracy Chou, John Collison, Patrick Collison, Danielle Fong, Kevin Hale, Aaron Harris, Elizabeth Iorns, Carolynn Levy, Jessica Livingston, Claire McDonnell, Kat Manalac, Kathryn Minshew, Kirsty Nathoo, Geoff Ralston, Garry Tan, and Olga Vidisheva for reading drafts of this.

All sarcasm aside, the above diagram has a kernel of truth. The important thing to note is that the intersection between "Proponents of dynamic typing" and "People familiar with type theory" is very small.

In an effort to increase the size of that intersection, I decided to familiarize myself with a little more type theory. I developed an implementation of Hindley-Milner which operates on a simple Lisp-like λ-calculus with let polymorphism. Everything you need for Hindley-Milner Algorithm W.

Background

Hindley-Milner is a type system that is used by ML and Haskell. Algorithm W is a fast algorithm for inferencing Hindley-Milner which is syntax-directed (meaning it is based on the type of expression) and recursively defined. In this way, it is similar to Lisp's eval.

Implementation

I based my implementation on a paper called Algorithm W Step by Step which implemented it in Haskell. My implementation started very similar to that implementation and diverged as I refactored it into a more Clojure-esque style.

This implementation uses no in-place mutation, instead using a "substitution style" which is slightly harder to read. It is, however, easier to write and prove correct.

In addition to the type inferencer, I wrote an interpreter for the same language, just because. My original intent was to expose (to myself) the similarities between syntax-driven type inference and eval. There might be some, but the full clarity I desire is yet many refactorings away. Note that the interpreter and type inferencer are completely independent, except that both apply to the same set of expressions.

I added a couple of minor luxuries to the language having to do with currying. Writing fully parentheisized function applications for curried functions is a pain, as is writing a hand-curried function with multiple arguments. I added two syntax transformations which transform them into a more Lispy style. For example:

(fn [a b c d e f] 1) => (fn a (fn b (fn c (fn d (fn e (fn f 1))))))

and

(+ 1 2) => ((+ 1) 2)

I'm pretty sure the syntactic transformation is completely safe. All of my tests still type check.

The final luxury is that it is a lazily-evaluated language. That's not strictly necessary, but it is strictly cool. It builds up thunks (Clojure delays and promises) and a trampoline is used to get the values out. This lets me define if as a function. The only special forms are let and fn.

Where to find it

You can find the code in the ericnormand/hindley-milner Github repo. I don't promise that it has no bugs. But it does have a small and growing test suite. Pull requests are welcome and I will continue to expand and refactor it.

What I learned

Type unification is why the error messages of most type inferencers are so bad. Unification by default only has local knowledge and is commutative (unify a b == unify b a). No preference is given to either argument. A lot of work must have gone into making the error messages of Haskell as good as they are.

Let polymorphism is useful and I'm glad that Haskell has it.

Hindley-Milner is powerful, but it does not by itself work magic on a languageq. A language still requires a lot of good design and a well-chosen set of types.

Your turn

I think you should implement Hindley-Milner in the language of your choice for a small toy λ-calculus. There is a lot to learn from it, even if you never program using a Hindley-Milner language. At the very least, you'll know what the fuss is about.

If you think it would be helpful, have a look at my implementation. Like I said, pull requests are welcome.

You may be interested in my LispCast Introduction to Clojure Videos.

Let’s get right to it.

SIMCITY OFFLINE IS COMING!

I’ve wanted to say those words for quite some time, so my apologies that I didn’t take the time to say Happy New Year first.

Yes, Offline is coming as a free download with Update 10 to all SimCity players. When we launch it, all of your previously downloaded content will be available to you anytime, anywhere, without the need for an internet connection. We are in the late phases of wrapping up its development and while we want to get it into your hands as soon as possible, our priority is to make sure that it’s as polished as possible before we release it. So, until then… testing, testing and more testing. As one of the final steps, we’re putting Offline into the hands of some of our most hardcore players, the DevTesters. This group of volunteers is going to put Offline through its paces before we release it.

In Update 10, you can still play solo in Regions on your own, or in Multiplayer with people from around the world. What’s new is the Single Player Mode, which allows you to play the game Offline by yourself. And because your saved games in this mode are stored locally, you can save and load to your heart’s content. Our team will be delivering a follow-up blog that will outline the full details in the near future so stay tuned.

So what does this mean for the Online game? All of the benefits of being connected will remain including access to Multiplayer, the Global Market and Leaderboards. And all of your pre-existing saved cities and regions will still be accessible should you log-in to the Online game.

Bringing the game Offline means big things for our wonderful community of Modders. They can now make modifications to the game and its components without compromising the integrity of the Online game. Modding is a big part of our studio’s legacy and we’re excited to see what you guys create. Check out this thread to learn about Oppie85’s Central Train Station, which you can put into your game right now, and if you’re interested in making your own content take a look at the Modding Policy. To get you started, we will be rolling out a series of tutorials from the studio that surfaces how we’ve created some of the content that you’ve seen so far in hopes of inspiring your creativity.

So that’s it from me for the time being. Keep an eye out for more information on Offline shortly. We’ll be releasing more information on how it all works. And as always, get in touch with me on Twitter @EAGamer, I’m always eager to read your feedback.

From me, and everyone at the studio, thank you again for staying with us. We look forward to getting Offline for SimCity into your hands as soon as possible.

Jelly, a highly-anticipated social networking app by Twitter co-founder Biz Stone, was released last week. One of Jelly’s underlying principles is “it’s not hard to imagine that the true promise of a connected society is people helping each other.” While using Jelly over the past few days, I’ve seen people ask whether or not to exercise, make unsubtle plugs for their product, sell a car Craigslist-style, and get amnesia.

Wait, what? How are these questions representative of “a connected society,” especially since I don’t know any of the question authors?

Jelly tells its users that they can “help friends, or friends-of-friends with their questions.” The vast majority of the questions I’ve seen are from Friends of Friends. I’m Facebook Friends with people who have thousands of friends; the probability that I have something in common with those second-degree friends is low.

Websites like Jelly and even Facebook are now utilizing the content made by second-degree “friends.” Why am I seeing pictures and statuses from friends of friends, who are essentially complete strangers?

A couple months ago, Facebook again tweaked the News Feed algorithm such that all new Posts on a friend’s Wall are shown on your News Feed, even if the person making the Post is a complete stranger to you.

Originally, Facebook showed posts in your News Feed that were posted on a Friend’s wall only if they were made by another Friend. If you are friends with both people, there’s likely a common link between all three of you, such as going to the same college or working at the same company. That can make the Wall post more relevant to your interests.

However, people have many social circles. What happens when a stranger not in any of my social circles posts on a Friend’s wall?

This post recently appeared in my News Feed:

I don’t know who the person with “1 mutual friend” is. (the “1 mutual friend” being the friend who received the post). I don’t live in Arizona. Why am I seeing this?

Although I may be Facebook Friends with someone, it doesn’t necessarily mean I should be Friends with their Friends. Facebook is showing me Posts that aren’t relevant to me at all.

This behavior also extends to your Friends’ birthday: all “Happy Birthday!” posts on your Friend’s Wall also display in your News Feed.

This entry was visible on my News Feed for an entire day:

Facebook really wants you to wish your Friends happy birthday.

I don’t need to see strangers with “1 mutual friend” wish my friends Happy Birthday. It’s just extra clutter on an already cluttered News Feed.

To be fair, other websites such as Twitter and LinkedIn push second-degree friends. Twitter’s new Discover feature highlights users who the people you Follow are themselves Following. LinkedIn shows your connections’ new connections so you can form connections with them too. But the difference between Twitter/LinkedIn and Facebook/Jelly is that both Facebook and Jelly push the power of intimate friendships, and imposing a sense of intimacy between strangers feels fake and intrudes on actual friendships.

An Uber driver and two technology company executives found themselves in the middle of a violent riot led by hundreds of Parisian taxi drivers near the city's Charles-de-Gaulle Airport Monday morning who were protesting against urban transportation services like Uber.

"Got attacked in an Uber by cab drivers on strike near Paris airport: smashed windows, flat tires, vandalized vehicle, and bleeding hands," Kat Borlongan, co-founder of open data firm Five by Five, tweeted.

Eventbrite co-founder and CTO Renaud Visage was also in the vehicle with Borlongan, French tech news blog Rude Baguette reported. No one was hurt and the car made it safely out of the area.

Uber rival, Allocar, also reported an attack in Paris on Monday. The company's marketing manager said via tweet that one of its drivers was assaulted and the car was damaged, CNET reported.

Cab drivers in France's capital city — who must pay a big fee to obtain a license — are riled up over urban transportation services, and are demanding a series of laws due to unfair competition. France has already instituted a rule that requires a minimum 15-minute wait before a service like Uber can actually pick up a customer.

Since July 2005, when Django was open-sourced, Jacob Kaplan-Moss and I have been the two Benevolent Dictators For Life (BDFLs) of the project. Today we're both retiring from our formal BDFL roles, given that (1) we don't have the time for it that we once had and (2) Django is in great shape with a vibrant community of contributors.

A BDFL, a term originally used by Python creator Guido van Rossum, is basically a leader of an open-source project who resolves disputes and has final say on big decisions.

In the early days, circa 2004-2008, Jacob and I had to make a fair amount of decisions, and we spent a ton of time promoting the framework, fixing bugs and adding features. Over the last few years, the codebase has stabilized tremendously and many fantastic developers from around the globe have joined the effort, contributing code, writing documentation, helping with process (bug triage, managing releases), publishing books/tutorials, holding conferences and organizing user groups. It's an incredibly healthy, friendly and diverse open-source project/community!

At the same time, I've gotten a development "life" outside of Django. In the old days, I'd spend basically all of my free time improving Django -- I considered it my baby, and it would be entirely fair to call it an obsession. Then, in 2007, I founded EveryBlock and found myself with a new baby, a young Internet company.

For the last year, I've been building Soundslice, a modern approach to sheet music and guitar tabs that I've wanted to exist for a long time. Soundslice has become the thing that I'm constantly thinking about and writing code for, from morning into late hours of the night. (And I'm very comfortable with that. I've always seen Django as a means to an end -- building great web products. I'm skeptical of anybody who builds frameworks for the sake of building frameworks.)

So, given that I can't give Django the time that it deserves, it wouldn't be fair to continue as BDFL. At best, it'd be phony for me to keep calling myself that; at worst, it'd be doing a disservice to the framework by slowing down our decisionmaking processes and setting a tone of complacency.

If you're a Django user, though, have no fear. Honestly, this title seems like a big change "on paper," but in reality it won't change much. I haven't been deeply involved in day-to-day development of Django in quite a while, and I think Jacob would say the same thing -- so, if anything, this change in titles just makes official what had already been happening. I suspect nothing major will change in the Django community, except maybe some committers will feel emboldened to build great new stuff. (But please, no more Django Pony. It's stupid.)

I'll still continue to contribute to Django as I find things that it doesn't do that I want it to do. And of course I continue to use it in building web apps. It's been a fun ride as co-BDFL, and I'm looking forward to Django's next chapter.

UPDATE:Here's Jacob's post about it.

British programmer John Graham-Cumming recently noticed something unusual as he watched the movie Elysium: a piece of code that he recognized, flashing across the monitors as the film’s characters interacted with a futuristic computer. “I tracked it down after taking a screenshot,” he told WIRED, “and found the code in an Intel manual.”

The programmer, who also petitioned the British government for an apology to mathematician Alan Turing, tweeted his findings earlier this month. It quickly received more than 500 retweets. The next day, Graham-Cumming created a Tumblr to explore the origin and meaning of the code in everything from major blockbusters to nerdy television shows to music videos. Since its inception a little over a week ago, Source Code in TV and Films has gained more than 12,000 followers and Graham-Cumming says he receives hundreds of notes and numerous submissions daily.

Turns out, much of the code featured in movies and television is copy-pasted from sources as inauthentic as that Intel manual code, and often originates from places as varied as Wikipedia, the Engadget home page, and a Canadian bank. Sometimes, it’s simply nonsense thrown up on screen, as in 1994′s Clear and Present Danger, because it looks legit. Other times the code contains programming jokes, like the code in 2011′s Charlie’s Angels (it’s Sudoku-solving software!).

“I suspect it comes down to the director and the time constraints on the set designers,” says Graham-Cumming. “I’ve actually had email from two set designers talking about the process, and it’s clear that sometimes they just don’t have time to work on the detail.” The weekly CSI: NY, for example, once used a random website’s basic HTML code.

But directors and set designers sometimes put real thought into it. Iron Man displayed valid, if botched, C source code, for example, and director David Fincher gets it right: When his characters — like Lisbeth Salander in The Girl with the Dragon Tattoo or Mark Zuckerberg in The Social Network– enter the digital realm, their coding language is consistent with what they’re trying to accomplish. Lisbeth at least knows SQL code, while Zuckerberg is using legitimate code that appears to have been created for the film.

“I think a big change happened some point in the early 1990s,” says Graham-Cumming. “As the Internet took off and open source became prevalent, it appears to have been easier for producers to find code examples [to copy and paste]. Prior to that, [the featured code] seem to have been written for the film or TV, or even typed in by hand from magazines.” Don’t believe it? Arnold Schwartzenegger’s POV shots in The Terminator include Apple-II code copied from Nibble Magazine.

There are many reasons accuracy varies so widely, according to graphics designers working in Hollywood.

“The ‘quality’ of the code really comes down to deadlines, budgetary constraints and when I get a script, and how soon those scenes shoot,” says Bob Ludemann, a motion graphic artist who has created code screens for films like The Wolf of Wall Street and television shows like Person of Interest, House of Cards, and The Following. “I think most people would be surprised if they realized that screens for most of these shows’ episodes are done in much less than an eight-hour day.”

The stuff that ends up looking the most legitimate, he says, is the stuff that needs to be the most legitimate.

“For Iron Man, that code was probably added later in post-production,” he explains. “Since they had the time and budget, somebody was most likely hired or had the time to do something more realistic. In most cases, producers don’t want to spend the money and writers are too lazy to write and research the actual code needed. Also, in a movie like Iron Man, they know people are going to pause and analyze everything that’s on screen.”

Ludemann says the decision to use garbage code can also stem from legal concerns, because while it may not be technically accurate – it sometimes lacks key elements, like semi-colons – it avoids the risk of copyright infringement. And then there’s the unpredictability of the set. Designers might set up computers with on-screen coding meant to be set-dressing or otherwise in the background, only to see the director changes the shot. This happened to Ludemann during the first season of White Collar.

“I was asked to do a generic screen of some FBI guy checking email — nothing hero, and they were shooting it from across the room and I had a half hour to do it,” says Ludemann. “I took an FBI website header that I built for the AMC show Rubicon and took a screenshot of my standard OSX Mac Mail program and inverted the colors so it looked dark and tech-y. I had it open to a piece of generic email I had in my inbox and sent it in….. White Collar ended up doing a full screen insert shot of this email opening — which had all of my personal email information, including purchases on Amazon, student loan statements, and requests to work in-house on Law and Order that week. This even made its way onto the DVDs.  To this day, I still have people emailing me a few times a year telling me that they, too, like Amazon and eBay and think student loans suck.”

If you’re interested in a little code policing of your own, Graham-Cumming recommends being at least somewhat familiar with multiple coding languages, as each has a distinct syntax. And looking for defining language characteristics is the key.

“I think it helps to be a programmer, because then you’re instantly familiar with the code and what language is used and what features of the language are going to be unique and Googleable,” he says, “But typically you are looking for things that are special to that code: comments, the names of the variables might be unique.”

Or, you could always just submit a screengrab and let him do it.

Schneier.com is a personal website. Opinions expressed are not necessarily those of Co3 Systems, Inc..

Today we are releasing Ghost 0.4 - the next minor release of the Ghost blogging platform. With a great deal of shiny new things.

This release represents a total of 374 commits across 178 issues from 88 incredible contributors all over the world. It's a cleaner, faster, more sophisticated Ghost all round. It enjoys Johnnie Walker Blue Label Scotch, Mozart, and the refined flavour of a premium cigar.

Aton - the Egyptian God of Sun - is the first release named for one of our 3 Kickstarter backers who pledged £1,000 to making Ghost a reality last year. Aton Energy develops smart, clean technologies to make your life easier and greener. They're pretty awesome.

What's in The Box

Ghost 0.4 is chock full of features, improvements and fixes. We really think you're going to love this update. Here are just a few of our favourite things in this release:

• Static pages. You can now toggle any post to be a "page" from within your post settings menu. This will remove it from your post feed. About / Contact / Terms galore!
• Unsaved changes notifications. We'll now give you a heads up when you're about to lose unsaved changes. So you can, you know, save.
• Featured posts. You can now mark posts as featured, and style them accordingly from within your theme.
• Sexy new loading bar. Always know when Ghost is doing something, a little blue bar crawls across the screen to let you know!
• Quick edit post urls. You can now slap /edit/ on the end of any post URL and, boom, you're editing it.
• Date based permalink support. If you like that kind of thing.
• SSL support and a more secure password reset process.
• Over 100 bugfixes and minor improvements!
• Much, much more

Ghost 0.4 also introduces an automatic update check service to let you know when a new version of Ghost is available (woo!). Ghost.org collects basic anonymous usage statistics from update check requests. For more more information, see the update-check.js file in Ghost core. This service can be disabled with a flag in config.js at any time, if required.

How to Upgrade

Users of our Hosted Platform on Ghost.org will be automatically updated over the course of the coming days. Have some Jonny Walker Blue Label and put your feet up. You're welcome.

For people running Ghost on their own servers, you will need to download Ghost 0.4 and then check out the upgrade documentation over on Github. If you're using the default theme without modifications, be sure to drop in a fresh copy of that - too!

Credits

This release was lovingly crafted by...

Hannah Wolfe, Fabian Becker, Sebastian Gierlinger, John O'Nolan, Harry Wolff, Jacob Gable, William Dibbern, Jakob Gillich, Matthew Harrison-Jones, Michael Bradshaw, Zach Schneider, cobbspur, jamesbloomer, Dane Springmeyer, Sebastian Gräßl, Zach Geis, buddhamagnet, Benjamin Chodoroff, Daniel Hanson, Gabor Javorszky, Mark Berger, Matt DuVall, Patrick Garman, Seb Gotvitch, Tim Griesser, Tony Gaskell, b1nd, germanrcuriel, remixz, sjama, Ben Gladwell, Declan cook, Derek Myers, Devin Doolin, Enrique Chavez, Harry Walter, Henning Sprang, Jacob Kaplan-Moss, Jacques Marneweck, Jeff Escalante, Jonathan Johnson, Jono Warren, Jorge Niedbalski, Karl Mikkelsen, Karolis Dzeja, Kumar Abhinav, Lev Gimelfarb, Lucas, Luke Arduini, Manuel Gellfart, Matheus Azzi, Matt Florence, Matt Hughes, Matthew DuVall, Michael Nason, Micheil Smith, Nick Donohue, Nick Pfisterer, Nick Schonning, Pascal Borreli, Paul, Paul Adam Davis, Peter deHaan, Ryan Powell, Ryan Seys, Sean Hellwig, Simone D'Amico, StevenMcD, Talon, Thomas Faurbye Nielsen, Tim Mansfield, Tom Gillett, Vineet Sinha, WangSai, Will Glynn, William Golden, Zlatan Vasović, abe33, ali, andy matthews, danschumann, enahs, jtw, moritz haarmann, nason, nicovalencia, omeid and rektide.

Thank you all!

Ghost 0.4 is available right now on Ghost.org. Users of the Ghost hosted platform will be automatically upgraded over the coming days.

Welcome back to the first meetup of 2014. We are very excited to announce we will have talks from Aral Balkan and Greg Brockman, CTO at Stripe. The event will be compered by Andrea Hubert, who was a huge hit at our October meetup. As always, we will have tons of free beer and pizza to keep you fed and watered.

This year we are also organising a one week ski trip to Chamonix from 9th to 16th March. We are staying at a kick-ass catered chalet with hottub and sauna. Places are still available if you want to join us. The chalet is £425, equipment hire, transport ski passes will cost about the same amount again. If you would like more info please email me at [masked]

If you are a front end developer you should check out JQuery UK conference in Oxford on 16th May in Oxford. Early bird tickets are still available at £130 + VAT at http://jqueryuk.com

If you are mobile developer, you may be interested in checking out my other meetup called Mobilise where we hold show 'n tells by popular mobile app developers. This month we'll have a demo of Citymapper by their lead developer. Sign up at http://www.meetup.com/Mobilise/events/159369282/

HNLondon is generously supported by

Stripe is the easiest way to accept card payments on your website or mobile app. Prior to Stripe, businesses had to spend days communicating with sales reps and filling out complicated forms to begin transacting online. With Stripe, you can create the payment experience you want in minutes, and Stripe will take care of all the heavy lifting, including security and daily transfers to your bank account. Stripe powers payments thousands of businesses of every size—from start-ups to blue chip companies. Get started at https://stripe.com/gb

Secret Sales: When you finish a feature, can you release it with confidence right away, or do you have to wait days or weeks to find out what users think? Secretsales uses continuous delivery to get super-rapid feedback on all the features we build for our busy, successful online shopping site. In addition, we are a team of generalising specialists, ready at the drop of a hat to refine an algorithm, write a tricky integration test, tune a SQL query, or discuss feature nuances with a product manager. We hold regular retrospectives to improve our working environment and lightning talks to share cool ideas whether work-related or not. Write to our CTO Squirrel at [masked] to find out more about jobs at Secretsales.

10gen is the company behind MongoDB, an open source, document-oriented database designed with both scalability and developer agility in mind. 10gen offers production support, training, and consulting for the open source database. The goal of MongoDB is to bridge the gap between key-value stores (which are fast and scalable) and relational databases (which have rich functionality).

The right to protest has always been powerfully upheld in France – whether it’s protesting an immigrant’s right to healthcare, the closing of a factory, or even Taxis, who have perfected the art of making their voice heard; however, how much is too much?

It seems that protest turned to guerrilla warfare this morning as one Uber driver, carrying Eventbrite CTO Renaud Visage & Kat Borlongan from the airport to Paris, was attacked by multiple assailants, who allegedly, after smashing one window and slashing two tires (as seen in the photo), as well as defacing one side of the car with glue, attempted to enter the vehicle. Borlongan says their Uber driver manoeuvered the two out of the situation before anything could happen, leaving the three stranded on the shoulder of the freeway.

There’s no confirmation that this attack came from the protesting taxi drivers; however, that certainly seems to line up with the sentiment felt by taxi drivers these days – unwarranted hate.

Update: Uber’s GM of France Pierre-Dimitry Gore-Coty has commented on the incident:

Taxi drivers today are protesting against the likes of Uber, and against what they consider to be a government that refuses to see things their way, according to an official press release. The only problem is that this protest comes less than a month after the French government caved in to the Taxi union demands to enforce the 15-minute law, not just for new users, as was originally intended, but for all users of Chauffeur apps.

It’s a question of credibility, according to Le Figaro’s Yann La Galès, who thinks the Taxis are looking to have their cake and protest the lack of the aforementioned cake, too.

Meanwhile, one Chauffeur App startup, Allocab, says it won’t take the protest, or the 15-minute law, lying down. According to Frenchweb, the startup is already pushing legal action through against the 15-minute law – we spoke last week about how the “law,” which actually is a Presidential decree, not voted on by the legistlative body, may be ruled invalid, due to the fact that decrees cannot take affect if they change the balance of competition – I think that’s an easy argument to make.

For now, what’s left of the fleeting reputation of Taxi drivers may have just been dwindled down to nothing but shattered glass and a flat tire.

Firebase

I wanted to build a web app that would allow data to easily be transmitted to other connected web clients. I had heard of Firebase before. So I started reading the Firebase documentation and playing around with the examples.There is a nice library that they created called AngularFire which provides some slick integration between Firebase and AngularJS. But for some reason, the provided chat example would sporadically not work. (As an aside, it seems that it would work most of the time in most of environments that I'd try, but for some reason, it rarely worked in one). So I needed to find a new solution.

LevelDB

According to the project page: "LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values." What's great about LevelDB is that it's fast and it doesn't have any external dependencies that users need to install before they install your app.

LevelUP is the Node.js bindings built on LevelDOWN which is the low-level Node.js bindings for LevelDB.

example:

varlevelup=require('levelup')vardb=levelup('./mydb.db',{valueEncoding:'json'})//default value encoding is 'utf8'db.put('somekey',{name:'some data'},function(err){db.get('somekey',function(err,data){console.dir(data)//{name: 'some data'}})})

As you can see, it has pretty intuitive API.

That's not the exciting part though. What's exciting is the ecosystem of modules and plugins built on LevelDB and LevelUP.

REST

So let's assume that you want your database to be accessible by others. You can create a simple REST API over your LevelDB database by using multilevel-http. multilevel-http just wraps Express and adds REST routes.

npm install --save multilevel-http

example (server.js):

varlevelup=require('levelup'),multilevelHttp=require('multilevel-http'),http=require('http')vardb=levelup('./mydb.db',{valueEncoding:'json'})varapp=multilevelHttp.server(db)varserver=http.createServer(app)server.listen(3000,function(){console.log('listening on port %d...',3000)})

you can now run:

node server.js

and access the REST interface via your browser or curl:

curl -X POST -d '{"name":"data from curl"}' -H "Content-Type:application/json" http://localhost:3000/data/somekey

retrieve the key somekey:

curl -x GET http://localhost:3000/data/somekey

serving HTML

create the following file (index.html):

<!--  watch this: http://www.youtube.com/watch?v=WxmcDoAxdoY --><!doctype html><metacharset="utf-8"><title>LevelDB Rules the World</title><h1>hi</h1>

As a quick aside, you don't need the html, body, and head tags in HTML5. Watch this talk by Paul Irish explaining why.

let's modify (server.js):

/* ... */varapp=multilevelHttp.server(db)app.get('/',function(req,res){res.sendfile('./index.html')})varserver=http.createServer(app)/* ... */

now, rerun:

node server.js

notice now that you're redirected to /meta? This is because multilevel-http has setup this redirect. Here's how you can fix it:

/* ... */functionremoveRoute(app,method,routeMatcher){varroutes=app.routes[method]for(vari=0;i<routes.length;++i){varroute=routes[i]if(route.path===routeMatcher)break;}routes.splice(i,1)}varapp=multilevelHttp.server(db)removeRoute(app,'get','/')app.get('/',function(req,res){res.sendfile('./index.html')})varserver=http.createServer(app)/* ... */

now run:

node server.js

notice now your index.html page is being served up correctly.

WebSockets

A REST API is nice, but there is still more that you'd need to do to get it working with your client-side JavaScript. Yes, you can easily interface with a REST API via AJAX calls, but let's make things even easier and use RPC over WebSockets.

install multilevel and shoe:

npm install --save multilevel shoe

shoe requires browserify for it to run client-side:

npm install -g browserify

browserify is an awesome solution for client-side package management. Probably the best at the moment.

multilevel isn't the same as the package above multilevel-http. This is its sexier sister. shoe is a wrapper for sockjs. It makes dealing with WebSockets more like Node.js streams.

server.js:

/***  other requires***/varmultilevel=require('multilevel')varshoe=require('shoe')/*** other code ***/varwsdb=shoe(function(stream){stream.pipe(multilevel.db(db)).pipe(stream)})wsdb.install(server,'/wsdb')

client.js:

varmultilevel=require('multilevel')varshoe=require('shoe')vardb=multilevel.client()varstream=shoe('/wsdb')stream.pipe(db.createRpcStream()).pipe(stream)/****later in the script, you use the leveldb apie.g.: db.get, db.put, etc*****/

(reference app.js in index.html)

browserify:

browserify client.js > app.js

run it:

node server.js

That's it. Now client-side/browser scripts can use the levelup API.

Live Changes

Part of the utility of Firebase is that changes propagate to other connected clients. Fortunately, you can do the same with LevelDB. We'll use another WebSocket to broadcast the changes.

install deps:

install --save event-stream level-live-stream

server.js:

/***** other requires******/varleveLiveStream=require('level-live-stream')vares=require('event-stream')/* ... */varliveDBStream=levelLiveStream(db)varchangesSocket=shoe(function(stream){es.pipeline(liveDbStream,es.map(function(data,next){next(null,JSON.stringify(data))}),stream)})changesSocket.install(server,'/wschanges')/* ... */

client.js:

varchangesSocket=shoe('/wschanges')changesSocket.on('data',function(data){console.dir(JSON.parse(data))})

Chat Example

Let's put together what we learned to create a chat example. Similar to the one found on http://angularfire.com.

Install deps:

npm init
npm install --save levelup leveldown multilevel event-stream shoe level-live-stream browserify

create server.js:

varlevelup=require('levelup'),multilevel=require('multilevel'),levelLiveStream=require('level-live-stream'),http=require('http'),shoe=require('shoe'),fs=require('fs'),browserify=require('browserify'),es=require('event-stream')vardb=levelup('./chat.db',{valueEncoding:'json'})varliveDbStream=levelLiveStream(db)varmessages={}//load initial messagesdb.get('messages',function(err,data){if(err)returnmessages=data})liveDbStream.on('data',function(data){if(data.type==='del'&&data.key==='messages'){//'clear' pressed, doesn't actually remove all of the keys, although you easily couldmessages={}}if(data.key.indexOf('message:')>=0){varidx=data.key.split(':')[1]messages[idx]=''//not sophisticated enough to handle messages generated at exact same timedb.put('messages',messages)}})varserver=http.createServer(function(req,res){switch(req.url){case'/':fs.createReadStream('./index.html').pipe(res)break;case'/client.js':res.writeHead(200,{'Content-Type':'application/javascript'})browserify('./client.js').bundle({debug:true}).pipe(res)break;default:res.writeHead(200,{'Content-Type':'text/plain'})res.end(res.url+' not found')}})vardbSocket=shoe(function(stream){stream.pipe(multilevel.server(db)).pipe(stream)})dbSocket.install(server,'/wsdb')varchangesSocket=shoe(function(stream){es.pipeline(liveDbStream,es.map(function(data,next){next(null,JSON.stringify(data))}),stream)})changesSocket.install(server,'/wschanges')server.listen(8000,function(){console.log('listening...')})

create index.html:

<!DOCTYPE html><metacharset=utf-8><title>chat example</title><script src="client.js"></script><form><inputtype="text"id="name"value="guest"style="width: 75px;"><inputtype="text"id="message"placeholder="type message here..."style="width: 300px;"><inputtype="submit"onclick="send(); return false;"value="send"><buttononclick="clearMessages(); return false;">clear</button></form><hr><divid="messages"></div>

create client.js:

varmultilevel=require('multilevel'),shoe=require('shoe')vardb=multilevel.client()vardbSocket=shoe('/wsdb')varchangesSocket=shoe('/wschanges')dbSocket.pipe(db.createRpcStream()).pipe(dbSocket)changesSocket.on('data',function(updateData){varupdateData=JSON.parse(updateData)if(updateData.type==='del'&&updateData.key==='messages'){document.getElementById('messages').innerHTML=''return}if(updateData.key.indexOf('message:')>=0){appendMessage(updateData.value)}})functionappendMessage(msg){varp=document.createElement('p')vartext=document.createTextNode(msg.name+': '+msg.message)p.appendChild(text)document.getElementById('messages').appendChild(p)}window.send=function(){varnameEl=document.getElementById('name')varmsgEl=document.getElementById('message')varobj={name:nameEl.value,message:msgEl.value}msgEl.value=''db.put('message:'+Date.now(),obj)}window.onload=function(){varnameEl=document.getElementById('name')varid=Math.random().toString().substr(2,3)nameEl.value+=id//get initial chat statedb.get('messages',function(err,messages){if(messages==null)returnvarids=Object.keys(messages).slice(-15)//take last 15ids.forEach(function(id){db.get('message:'+id,function(err,data){appendMessage(data)})})})}window.clearMessages=function(){db.del('messages',function(err){if(err)alert(err.message)})}

now run:

node server.js

Boom! Now you have a hacky chat server ready to rock!

If you made it this far, you should follow me on Twitter.

-JP

I have been playing with Docker1 for a while. Initially, I had been using it like a hypervisor - spin up containers, play around with them, and shut them down. But in the past few days, I have recognized two very useful utilities of docker, that make life easier.

This past week, I have been trying to deploy some of the projects I have worked on over the last two years. Each of these projects was different from the others: One was in Node.js, another was in Django, yet another in plain Python. I wanted them all to be running forever, didn’t want one to affect the others, and wanted them to be resistant to failure on account of changes to the environment in the server box. A perfect use case for docker.

Using docker to create a deployment environment involves writing a Dockerfile. Dockerfiles use a simple DSL which allows you to automate the steps you would normally manually take to create an image. Think of it as a make utility for deployment. The advantage of using docker is that it is portable; ie, the same container that you build and test on your laptop can run at scale, anywhere; in production, on VMs, bare metal, OpenStack clusters, public clouds etc.

You will find documentation and examples for Dockerfile here

Docker as an application sandbox

This is something that isn’t obvious on first thought. I remember that when I first heard about CoreOS, I had no idea how it would work. CoreOS is a minimal distro that contains just a linux kernel and systemd and the root partition is completely read-only. It uses docker as a package manager to run applications on top of it. That means, every user application will run in its own linux container.2

I have seen a lot of projects in Github that I found really interesting, but kept away from trying out because of the complexity of their setup. First you have to do this, then that … If there were a simple Dockerfile, that built the complete setup, then a single command would suffice. A great example for this is StackEdit, the markdown editor used to write this document. I wanted to have a local copy as I wanted to use it even I was offline. I found their Dockerfile in Github 3 , so all I needed to do was build a container, and run it as a daemon. I don’t need to worry about the version of node, nor of the dependencies nor anything else. If it’s worked for them, it’ll work for me. As simple as that!

Another container I’m running is that of octopress. All my attempts to setup octopress in my local system were failing for some unknown reason, so I wrote a simple Dockerfile4 to set it up. I now have a completely sandboxed octopress setup, that I can use for any number of octopress blogs without needing to install everything again. The instructions to do the same will be provided in the next blog entry.

Have you found additional usecases for docker? Is there anything wrong about all that I’ve said above? If so, let’s discuss it in Hacker news

On Friday evening our service went down during scheduled maintenance. The service was back up and running about three hours later, with core service fully restored by 4:40 PM PT on Sunday.

For the past couple of days, we’ve been working around the clock to restore full access as soon as possible. Though we’ve shared some brief updates along the way, we owe you a detailed explanation of what happened and what we’ve learned.

What happened?

We use thousands of databases to run Dropbox. Each database has one master and two slave machines for redundancy. In addition, we perform full and incremental data backups and store them in a separate environment.

On Friday at 5:30 PM PT, we had a planned maintenance scheduled to upgrade the OS on some of our machines. During this process, the upgrade script checks to make sure there is no active data on the machine before installing the new OS.

A subtle bug in the script caused the command to reinstall a small number of active machines. Unfortunately, some master-slave pairs were impacted which resulted in the site going down.

Your files were never at risk during the outage. These databases do not contain file data. We use them to provide some of our features (for example, photo album sharing, camera uploads, and some API features).

To restore service as fast as possible, we performed the recovery from our backups. We were able to restore most functionality within 3 hours, but the large size of some of our databases slowed recovery, and it took until 4:40 PM PT today for core service to fully return.

What did we learn?

Distributed state verification

Over the past few years our infrastructure has grown rapidly to support hundreds of millions of users. We routinely upgrade and repurpose our machines. When doing so, we run scripts that remotely verify the production state of each machine. In this case, a bug in the script caused the upgrade to run on a handful of machines serving production traffic.

We’ve since added an additional layer of checks that require machines to locally verify their state before executing incoming commands. This enables machines that self-identify as running critical processes to refuse potentially destructive operations.

Faster disaster recovery

When running infrastructure at large scale, the standard practice of running multiple slaves provides redundancy. However, should those slaves fail, the only option is to restore from backup. The standard tool used to recover MySQL data from backups is slow when dealing with large data sets.

To speed up our recovery, we developed a tool that parallelizes the replay of binary logs. This enables much faster recovery from large MySQL backups. We plan to open source this tool so others can benefit from what we’ve learned.

We know you rely on Dropbox to get things done, and we’re very sorry for the disruption. We wanted to share these technical details to shed some light on what we’re doing in response. Thanks for your patience and support.

Akhil
Head of Infrastructure