Sorry, Geeks, Microsoft Excel is Everywhere

In 2012, Bruno Michel Iksil, the “London Whale”, was responsible for six billion dollars in losses at JP Morgan. The handling of this situation raised as many eyebrows as the loss itself, with many wondering why JP Morgan relied on internal investigators instead of someone more objective. But this week the tech world is fixated on one line buried in the appendix of the investigators’ report: the bit about Microsoft Excel.

“During the review process, additional operational issues became apparent. For example, the model operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another.” - Report of JPMorgan Chase & Co. Management Task Force Regarding 2012 CIO Losses, page 127

Software folks are shocked that so much wealth was entrusted to a simulation written by an amateur with a spreadsheet program. But they shouldn’t be surprised, because that’s how it’s been for years.

Birth of the spreadsheet

It’s been said that the first spreadsheet program, VisiCalc, sold more Apple II computers than Apple’s own marketing department. Businesspeople begged their bosses for an Apple to use at work, and those that were turned down sometimes borrowed their kids’ computer while the little scamps were at school. Before the dawn of VisiCalc, a spreadsheet was a literal sheet of paper with a grid drawn on it, and every cell needed to be figured using an adding machine or calculator. Testing a simple change to a model required minutes or even hours of work. VisiCalc took seconds.

VisiCalc inspired Lotus 123, a similar program for IBM PCs. Bosses were much more willing to order a PC for their staff than something built by California hippies. As they used to say, you’ll never get fired for buying IBM.

VisiCalc and Lotus 123 transformed business. They put control over modeling and predictions under the purview of the decision makers, not the number crunchers. Electronic spreadsheets were an empowering tool, championed by those who dreamed of a personal computing revolution.

Today, Excel is one of the most popular programs on the planet. It’s installed on practically every corporate PC. That ubiquity is one of its strongest features.

The business world runs on Excel

We live in a wonderful time when programmers can be so naive about the workings of Big Business. I’m not being the least bit snarky there, either. I see so many young developers working in startups and progressive agencies, blissfully unaware of just how poisonous corporate culture can be sometimes. But they do tend to live in a tech-savvy bubble, and sometimes it’s helpful to see how “real people” use the tools our industry gives them.

I once worked for a company run off a couple Excel spreadsheets, just like JP Morgan. Cron jobs in the IT department would spit out sales reports every hour, and the numbers from our bottom line got fed into Excel to make the “real” report sent to the top. This let the decision makers play “what if?” to their hearts’ content. It was also a way to filter out the details – we needed to show our work, sure, but they just needed an executive summary.

Excel’s biggest advantage, as Hacker News user breckinloggins points out, is that it lets people “circumvent the official software and just write something that works”. breckinloggins goes on to describe a typical corporate software project full of metrics, scope creep, and bikeshedding. What finally gets delivered is an over-engineered wreck that’s not really suitable for its original task. But, it doesn’t take a committee to build a spreadsheet, and Excel “developers” don’t need to fret about best practices.

Programmers crow about test coverage and continuous integration. Spreadsheet users try a couple different things and see if the numbers are close enough. Programmers track changes in version control. Spreadsheet users save a new version with the date tacked on the end of the filename. Programmers store data in redundant systems audited for security and bound by access rules. Spreadsheet users just copy their file onto a thumb drive and throw it in their pocket.

It’s not just a spreadsheet

I’ve seen Excel used in amazing ways.

Yes, I’ve seen it used as a spreadsheet, to crunch numbers. I’ve also seen people sit down with a calculator and compute cell values themselves, blissfully unaware that the software could do it for them. Excel is a tool for lining up numbers.

We’ve all seen Excel CSV files used for reporting and data interchange because they’re easy for programmers to generate. I’ve seen an application’s configuration managed by tweaking a CSV file in Excel and uploading it to a server. Excel is a tool for separating labels from values.

I’ve had screenshots pasted into Excel and attached to an email. Excel is an ubiquitous file format.

I’ve received an entire application wireframed by the accounting department, with each screen drawn with filled-in ASCII-art columns on its own worksheet and every button hyperlinked to a worksheet showing the next application state.

That really threw me off when I saw it. But when I look back, I realize just how clever it was. Front-end developers are always praising grid-based design, aren’t they? That’s exactly what what Excel was being used for. Excel is the first layout grid many people encounter.

What can we do about it?

Excel’s popularity and the crazy things it’s being used for are proof that business people need flexible, ubiquitous, approachable platforms that empower them to build their own tools. A proper programming language is still too intimidating to average folks, and that’s ok. Their output doesn’t need to scale to millions of users, or pass a PCI audit, or even come with a disaster recovery plan. But it needs to work reliably.

Apple’s iPad line has been successful in the business world, but Microsoft still doesn’t make a version of Office for the portable phenoms. Instead, iPad-savvy suits are turning to whatever’s in the App Store, and IT can steer them toward apps that strike a balance between the business’s security needs and the flexibility of tools like Excel.

Intranets can address the need for ubiquity, and offer a secure, backed-up place to collaborate. IT departments should be looking at what Excel is being used for in-house and offer web equivalents, perhaps leveraging HTML5′s offline capabilities for those who like to work from home after the kids go to bed.

And, lastly, we mostly just need to take a deep breath. Personal computers might never have taken off the way they did without Excel’s ancestors. We should be happy that VisiCalc and 123 empowered the common suit to do their job more effectively, that they found value in platforms that enabled this, and that tons of money flowed into personal computer companies in the 1980s. Isn’t that what geeks like Woz wanted? Regular people using computers? They are! And they’re sorta-kinda programming! Voluntarily! We just need to steer them toward better tools, and take advantage of that passion.

Don’t hate the spreadsheet, hate the game.

Excel wasn’t the problem at JP Morgan. There was a reckless culture that thumbed its nose at rules, ignored the guidance of review committees, and tried to sweep things under the rug when they got caught. That would have happened whether the models were written in Excel or Ruby.

Likewise, spreadsheets drive decisions at a number of the biggest companies in the world, yet they manage to avoid losing billions of dollars that easily. The top people at those companies can be more agile, responding to changes quickly by asking endless “what if?” questions and getting immediate results. This is the reason we have spreadsheets. It’s a big part of why you have a computer on your desk, on your table at the coffee shop, or in your hand to read this. So don’t blame Excel for what is ultimately an issue of corporate oversight and our flawed human nature.