I don't believe that most professors (even tenured ones) can genuinely work on "anything they want." In practice, as a professor you are constrained by at least four things:
- What you can get funding to do;
- What you can publish (good) papers about;
- What you can get students to help you with;
- What you can do better than anyone else in the field.
These are important limitations to consider, and I want to take them one by one.
Funding doesn't come easy. When I was a PhD student at Berkeley, I was fortunate to be a student of
David Culler's, who had what seemed like an endless supply of funding from big DARPA and NSF grants, among others. When I went to start my faculty career, he (and many others) told me I would have "no problem" getting plenty of funding. This turned out not to be true. Shortly after I started my faculty job, DARPA all but shut down their programs in computer science, and NSF grants became heavily constrained (and much more competitive). Being a freshly-minted faculty member meant I was essentially a nobody, but that didn't mean that NSF review panels took pity on me -- apart from special programs like the CAREER award, you're competing with the most senior, established people in your field for every grant. To make matters worse, I didn't have a lot of senior colleagues in my area at Harvard to write proposals with, so I mostly had to go it alone.
Now, I will readily admit that I suck at writing grants, although according to my colleagues my hit rate for funding was about on par with other profs in my area. However, there were several projects that I simply
could not do because I couldn't get funding for them. I tried for
four years to get an NSF grant for our work on
monitoring volcanoes with sensor networks -- which was arguably the thing I was most famous for as a professor. I failed. As a result we never did the large-scale, 100-node, multi-month study that we had hoped to do. It was a huge disappointment and taught me a valuable lesson that you can't work on something that you can't get funding for.
Who decides which problems are sexy (and therefore publishable)? I'll tell you: it's the 30-some-odd people who serve on the program committees of the top conferences in your area year after year. It is very rare for a faculty member to buck the trend of which topics are "hot" in their area, since they would run a significant risk of not being able to publish in the top venues. This can be absolutely disastrous for junior faculty who need a strong publication record to get tenure. I know of several faculty who were denied tenure specifically because they chose to work on problems outside of the mainstream, and were not able to publish enough top papers as a result. So, sure, they could work on "anything they wanted," but that ended up getting them fired.
Now, there are some folks (David Culler being one of them) who are able to essentially start new fields and get the community to go along with them. I argue that most professors are not able to do this, even tenured ones. Most people have to go where the funding and the publication venues are.
What can you get students to work on? I don't mean this in a kind of grad-students-won't-write-unit-tests kind of way (although that is also true). What I mean is how likely is it that you will find grad students in your field who have the requisite skills to undertake a particular research agenda? In my case, I would have killed for some students who really knew how to design circuit boards. Or students who had some deep understanding of compiler optimization -- but still wanted to work on (and publish) in the area of operating systems. A bunch of times I felt that the problems I could tackle were circumscribed by my students' (and my own) technical skills. This has nothing to do with the "quality" of the students; it's just the fact that PhD students (by definition) have to be hyper-specialized. This means that grad students in a given area tend to have a fairly narrow set of skills, which can be a limitation at times.
Can you differentiate your research? The final (and arguably most important) aspect of being successful as a faculty member is being able to solve new problems better than anyone else in your area. It is not usually enough to simply do a better job solving the same problem as someone else -- you need to have a new idea, a new spin, a new approach -- or work on a different problem. Hot areas tend to get overcrowded, making it difficult for individual faculty to differentiate themselves. For a while it felt like everyone was working on peer-to-peer networking. A bunch of "me too" research projects started up, most of which were forgettable. Being one of those "me too" researchers in a crowded area would be a very bad idea for a pre-tenure faculty member.
Do things get better after tenure? I didn't stick around long enough to find out, so I don't know. I definitely know some tenured faculty who are coasting and care a lot less about where and how much they publish, or who tend to dabble rather than take a more focused research agenda post-tenure. Certainly you cannot get fired if you are not publishing or bringing in the research dollars anymore, but to me this sounds like an unsatisfying career. Others -- like David Culler -- are able to embark on ambitious, paradigm-shifting projects (like
NOW and
TinyOS) without much regard to which way the winds are blowing. I think most tenured faculty would agree that they are subject to the same sets of pressures to work on fundable, publishable research as pre-tenure faculty, if they care about having impact.
Okay, but how much freedom do you have in industry? This is worth a separate post on its own, which I will write sometime soon. The short version is that it depends a lot on the kind of job you have and what kind of company you work for. My team at Google has a
pretty broad mandate which gives us a fair bit of freedom. But unlike academia, we aren't limited by funding (apart from headcount, which is substantial); technical skills (we can hire people with the skills we need); or the somewhat unpredictable whims of a research community or NSF panel. So, yes, there are limitations, but I think they are no more severe, and a lot more rational, than what you often experience as an academic.