Interviewing in Silicon Valley « Symbo1ics Ideas

Over the last year, I’ve done enough job interviews (all for software engineering positions) that I’ve lost count. I’ve done interviews for both big and small companies, young and old. The types of interviews have ranged from chats over email, to phone calls, to laborious hours on-site with a half-dozen interviewers.

Almost all of the interviews follow the same cookie-cutter format, probably inspired by Google or the likes:

Do a general screening over the phone. Do a couple technical interviews over the phone (these can range from simple discussions to solving puzzles). Do an on-site interview (this can range from an hour to six hours) where you demonstrate your coding/problem solving skills. Discuss with a recruiter/HR person about compensation, etc.

Between each of these steps spans a time ranging from a day to a week. After each of these steps, usually within a day or so, you’re let known whether or not you pass to the next round, and when you’d be available for the next round.

Usually the fourth step above implies you will be receiving an offer, of course unless there isn’t any agreement. But it can be safely assumed—in my experience—that at step 4, you’ve essentially locked in your position.

In the following, I’d like to chronicle some of my experiences, and perhaps suggest some improvements. Programming interviews are a topic of hot debate, and the style of interviewing is one thing that can set a company apart from the others. What I will not do, however, is suggest exactly how interviews should be done, precisely because of the aforementioned reason.

Vanishing communication

Bar none, the most annoying thing about interviewing with some companies is the horrendous communication. Almost always, communication is done over email with a recruiter, and the average response time is several hours to a day. Response time isn’t really an issue (provided it isn’t too long), but the willingness to respond is.

So far, I can think of over five companies I’ve applied and interviewed with (at various stages, see above), who simply cut communication. Let me give a few examples.

• I did a technical interview for a software engineering position focused on performance for a very large and successful non-profit. I thought the interview went well. We discussed topics ranging from sampling profilers to statistical significance testing. We also discussed past things I worked on, and how I might bring some impact to the company with my own experiences and expertise. Up until that point, there was a steady flow of communication with the recruiter on-site. After the interview, everything was dead. I emailed them back a few weeks later to try to reignite, but it has been about a month now, and there has been no response or indication of the status of things. As the interviewer, I am compelled to think they are not interested, and to not focus my efforts on them any longer. It was also a company which I thought was on rather high moral ground (being a proponent of open source development), but I find their actions rather rude.
• I did a couple phone interviews and a semi-technical interview (no puzzles, but discussions about engineering, etc.) with the VP of Engineering, and with the CTO of a company. Again, I felt a good vibe personally. My interests aligned with theirs, and it was a very exciting career avenue. They then emailed me asking if I’d like to stop by their office for a few hours for an on-site interview. I responded after a couple days when my schedule wasn’t so clouded, asking what time would work for them, giving them a range of days that work for me. I had not heard from them after a few weeks, so I email asking if there was still interest. Around a month went by, and no response.
• A startup contacted me and was interested in me interviewing. The recruiter was very nice, and asked if I could do an on-site interview immediately after a short phone call. In other words, they wanted an on-site interview the same day they contacted me. My schedule was open, so I was perfectly okay with this. I headed over there, five minutes early, and was greeted and escorted to an interview room. I was interviewed by an engineer and a manager. I was asked technical questions, ranging from how I would set up cache hierarchies for servers, to how I would do certain chess computations efficiently. They used Scala, so I actually gave my solutions, along with good time complexities, in purely functional code. They asked questions about past projects I’ve worked on, and was more than happy to discuss those. I felt pretty good after that one. A month passes by without a response. Like the previous two, I mail them back. To my surprise, I do get a response:
  It was great meeting you last month but unfortunately we did not feel it was a fit.

  I wondered, in my mind, why they couldn’t have told me that as soon as they knew, as opposed to me waiting a month for a follow-up. I emailed them back asking if they had any feedback from my interviews, but I’ve failed to receive a response.

• I did a short phone screen with a company in “stealth mode”. Originally they were interested in me doing contract work, but during a full-on on-site interview, they actually asked if I could be a full-time employee instead, to which I said yes. The interview was very good. It was very conversational, occurred in the office, and over the course of a nice meal at a restaurant. I actually think that I was probably the perfect fit with this company of all of the others I’ve interviewed with. My interests and the founder’s interests were very in-tune (which is rare, my interests are in computer algebra, Lisp history, etc.), my skill set very closely matched with the projects needing work, and so on. The interview ended with a brief discussion about compensation. I only suggested what I previously received as a base salary, and said I was open to negotiation. I thought I had it in the basket with this one. A little later, I received an email with a very long to-do list, asking for my comments. I gave very general comments about each item in the to-do list, along with my enthusiasm and experience with the subject. (Some of the tasks were not so interesting, which I said, and some were incredibly interesting.) A couple weeks pass, and I send an email asking for a follow-up, and the response is
  Thanks for checking in. We are still interviewing people and are not yet ready to make a decision yet. Sorry. I will let you know when we do.

  I completely understand the desire to find the right candidate, but that follow-up was nearly a month ago. The reality might be that they were not so compelled to make an offer, and only using me as a last resort. That’s fine, but better communication would be highly appreciated.

A friend told me that I should take the strategy of sending email after email until they give me some sort of response. (“Stop sending us damn emails. We don’t want you.”) I’ve not really done this, but I’ve done more than what I think I am responsible for doing, which is sending requests for a follow up asking, “hey, what’s the status?”

The common pattern in all of the above companies is that they are smaller companies. The largest one—the non-profit—is still somewhat small relatively speaking.

It has been my experience that other larger companies, such as Facebook, Amazon, or Apple, are very expedient with scheduling and setting things up, and keeping the loop. This is extremely to their advantage. When you’re doing several interviews concurrently (and provided you’re a good candidate), it’s to their advantage to hustle.

Puzzle-oriented interviews

This is a subject open for intense debate. Almost all of the software engineering roles I’ve had have included some sort of coding puzzle. They are definitely beyond that of Fizz Buzz. Some examples:

• Given an array of integers, find all combinations of $n$ values in the array such that their sum is $0$.
• Write routines to serialize and deserialize a binary tree.
• Write a regular expression matcher for regular expressions containing alphanumeric characters, a wildcard “.”, a Kleene star “*”, and grouping “()”.
• Given this bit of code, if it were run concurrently with $N$ threads, what are the possible values of the variable $x$? Give a sketch of a formal proof of correctness of your answer.
• If you are downloading data from a CDN, what is an efficient way to cache data? Explain design decisions and trade-offs of different techniques.

I am definitely not an expert puzzle solver. Almost all interview questions I take a long time with. If I had to guess, I probably have a 85–95% rate of solving puzzles—using almost all of the allotted time—on my own, and the rest I’ve needed a very small hint to get me in the right direction to begin with. I can’t think of an instance where I’ve been given the answer because of being unable to solve the problem.

While usually I am able to solve the puzzle after lots of thinking, and also be able to effectively communicate my thoughts while puzzle solving, I think that taking long breaks to just think, and not being able to write a solution down in the first ten minutes has usually not been a good sign for them. I recently did an on-site interview, which was composed of four sub-interviews, each of which was some kind of puzzle. In each of them, there was a time I balked. For example, when deserializing a binary tree, I hesitated when thinking about an effective way to do a recursive solution without mutating state. (I was very keen to not make a solution which mutates state, because the correctness of the recursion becomes very fuzzy.) When asked about threads, I got the general idea correct from the get-go, but I had a hard time keeping a lot of bookkeeping correct when I was explaining a proof. By trouble, I mean I’d begin to confuse myself. I notice this happens a lot when I am put on the spot, and when I am expected to both think and talk at the same time, which is harder than it seems.

Anyway, after that interview, I was told one of three things would happen: I would get rejected, I would get accepted, or if the signal wasn’t strong enough, I’d be asked for another interview. Indeed, I was asked for another interview, but I’m not quite sure why the signal wasn’t strong enough. (Was it because I took too long, but did manage to solve things?) This will be my fourth interview with the company. Is a total of eight cumulative hours of interviewing with around seven different people not enough to get a good signal? What does this say about the interviewing process?

I’ve debated with myself what puzzle solving is measuring. I am especially bewildered by the fact hard CS questions are given to prospective employees who will only be doing boilerplate PHP development for the actual job. However, there is lots of good discussion elsewhere about the effectiveness of puzzles, and I won’t delve into the subject here.

I will note, however, that I think interviewers should remain humble. I propose a simple test in order to keep interviewers on even ground: let the interviewee propose a question for the interviewer. This allows for a couple things to happen:

Secretly, the interviewer is actually seeing if the interviewee knows or has any interesting questions or puzzles him/herself. It gives the interviewer a chance to reflect on his or her difficulty with a problem, which the interviewee also might have had during the interview. (In other words, it is possible the interviewee had trouble with a question, which initially might be looked down upon, but if the interviewer also has trouble with a different puzzle, then the interviewee’s trouble might be sympathized with.) If the interviewer doesn’t know very well how to answer the question, then it gives a chance for the interviewee to be the one in the position of teaching and instruction, which can be very informative.

I suspect this type of interview style won’t take off, because it means that the company interviewing is closer to a position of vulnerability: the company now is also being tested, which can tear down the façade that they are truly filled with the best of the best of problem solvers. (This of course assumes that the company feels it is in their best interest to make everyone think they have an intellectually impenetrable staff of engineers.)

Aside from the ability to ask simple questions about the company as an interviewee, there has always been a sense that the interviewers are in the superior position.

Where are the technical challenges?

So, all of these companies seem to want rockstar problem solvers for their “full-stack engineer” positions. They want people who are computer science wizards and have experience in 20 different technologies for their social web app. Every company I’ve interviewed for has told me that they are solving “very difficult problems”, but when asking what problems they are solving that perhaps might require some sort of research, they just respond talking about their troubles with integration or scaling. I am not saying integration and scaling aren’t legitimate problems, but that’s what everyone is doing. I think I interviewed with two companies who actually were working on interesting problems. One was a very efficient, compiled pattern matching system for NLP, and the other is a position for representing and storing very unstructured textual data in a uniform, tree-like format, and then doing interesting operations on it such as diffing.

I understand that perhaps the more interesting problems are being solved by research firms, think tanks, academic institutions, etc. (each of which almost always requires a masters’ or PhD to even be considered), but I am baffled that a lot of companies have a hard time telling me about technical challenges—whose solution isn’t in a Stack Overflow post or in some hip technology such as Hadoop or Redis—they are having.

Is Silicon Valley stuck in a period where all that’s wanted are web engineers? That’s what most job listings indicate.

Don’t sell me your product, sell me your challenges

In most interviews I’ve had, engineers and recruiters I speak to spend a great deal of time gloating about their product, almost as if they are trying to sell me their product. The fact of the matter is, I am not interested in being spoken to as if I would be a potential consumer of your product. I want to be spoken to as if I would be a potential employee working on your product. As an engineer, that means I want to hear about the things I discussed in the previous section: what makes the work interesting.

At the last company I worked, there was a systemic problem (in my opinion) about where the motivation to work comes from. Most people were excited to work at this big name company simply because it was Big Name Company, and not because of the technical challenges presented. I see this as a problem because then, when you actually get down to working, motivation isn’t actually coming from the underlying work, which means it will probably be of poorer quality. If you’re excited about a subject, then there will be a tendency to perform tasks in that subject area better.

As I said, every company just told me how great their product is. Many of them, when I asked about the actual underlying engineering, were somewhat complacent. “Well, it’s just $X$ and $Y$ on top of $Z$. But our customers are telling us they are saving…”

When I talk about some of the software projects I’ve worked on, I am incredibly enthusiastic about the engineering, not the final product. Of course, having enthusiasm in the final product is important; it is nice to have an end goal you can be proud of. I do think that it’s important for the engineers to be excited about the engineering, just as salespeople are excited about selling the product. I think that it is perhaps detrimental to the rate of turnover if there isn’t a focus on making the actual work interesting.

Final remarks

So far, the many companies I’ve interviewed with have provided an underwhelming experience. This has either been a result of poor communication, a pinpointed focus on solving computer science puzzles, or a lack of genuine interest in the actual engineering work. It is my hope that over time, engineers in Silicon Valley will again become engineers, and shy away from this trend of Full Stack Web Developer. Some interviews have showed a glimmer of hope, but they are the exceptions and not the norm.

If you do have interesting work, don’t hesitate to contact me. I would love to discuss what you are working on.