You aren't getting any better | Robert Heaton

My limited and very biased samples suggest that hackers have a much greater intrinsic desire to improve their skills than those in other professions. We want to be ninjas not just because it will mean better-paying and more exciting jobs, but because there is something innately cool about knowing how to properly wield Javascript, Erlang or Clojure. So we spend an inordinate amount of time in after-hours experimentation, blogging and productivity optimisation, with the unspoken assumption that the weight of this combined experience will necessarily and always lead to a sustained, continuous improvement in knowledge and chops. But it won't.

Many studies, collected most notably in Geoff Colvin's "Talent is Overrated", find that in a wide range of disciplines experience has absolutely no correlation with efficacy [1]. Lifelong stockbrokers perform at exactly the same level as those in their 20s; or, rephrasing this slightly, lifelong stockbrokers perform at exactly the same level as when THEY were in their 20s. Despite having overseen numerous trades, been to countless conferences and devoted thousands upon thousands of hours to the craft of stockbroking, they are not even slightly more skilled that they were 40 years before. As a slightly arrogant and somewhat ambitious software guy, I find this possibility terrifying.

You can't just show up and expect to get better. You can't even just show up, work hard and expect to get better. You have to show up, work hard and deliberately focus on the specific things that will make you get better. Sometimes they will be the same as the cool things that you instinctively want to work on, but there's no guarantee of this. Working through problems you've already solved won't help much. Working on easy problems won't help much. Working without feedback won't help much. Working with only a superficial understanding of the technologies and frameworks you are using won't help much. You'll be OK for a while, but before long you might just be spinning your wheels in the mud, and if you're unlucky you might not even notice.

Instead you must obsess over actual progress. Work out why things are the way they are and not the way they aren't. Grok the fundamental principles that drive the guys who are the best in your game. Think about the time in your career when you were improving the fastest. When things were just snapping into place left, right and centre. Your career might be 40 years long. Are you improving that fast now? Why not? How ridiculously skilled could you be after even just 10 years improving at that rate? And how ridiculously rubbish would it be to wake up in 2053 and not have any skills to show for the last 40 years?

You might just want to have fun. You might just want to mess around with some cool APIs or quickly throw together some sweet but kind of janky object model to impress the laydeez. That's cool, but you shouldn't kid yourself that it's anything more than a bit of fun. And there will be times in your professional life when you have to optimise for pure output and forget about improving. That's also cool, but make sure it doesn't go on for too long, and make sure that you aren't just rationalising being stuck in a crappy situation that is never going to get better.

No one would argue that some ways of learning are more effective than others, but it is important to realise that the ineffective ones are not only sub-optimal, but completely useless. Whilst you are unavoidably picking up points on your CV with every project you complete and every line of code you write, you may not, and indeed probably are not actually getting any better. If that's what you want, you have to make a conscious effort and chart a deliberate course. Following your nose will lead you to some interesting places, but it will not make you exceptional.

[1] "Talent is Overrated" by Geoff Colvin, p7

Discussion on Hacker News