Interviewing Programmers

As anyone who reads this blog knows, a few months ago I went through the process of finding a new job.  That of course included a number of interviews.  Which got ‘interviewing’ on my mind and encouraged me to write this article.  Now, months later, I finally got around to it.

I’ve interviewed my fair share of programmers over the years, and I find personally that how you interview a ‘junior’ versus ‘intermediate’ versus a ‘senior’ programmer needs to change.  Specifically in regards to one topic.

What’s that topic?   It’s giving a coding test (as well as just whether you ask coding questions in the interview)

I personally find that asking a coding test of junior programmers is a GREAT tool for helping to evaluate them.  Typically just a very simple test, a basic CRUD application.   Looking at what they create not only gives you insight into how they think and code (did they do MVC?  a simple single .php file?  handle web security? etc).  It also gives you a great starting point to begin discussions with them in person, asking them about why they did (or didn’t) do certain things.

It’s also worthwhile to go into great depth with junior folks just to make sure that they understand some basics of programming.  Recursion?  Object Oriented? etc.

That starts changing somewhat when you start interviewing folks who are more at an intermediate level.  At the that level, asking for the coding test is fine, but I like to offer for them to instead just provide some code they’ve written before if they wish.  From looking at their code, you may come up with a few directions that you want to guide the discussion with them in an interview to understand certain parts of their knowledge.

This completely switches over in my mind when it comes to interviewing a senior person.  Someone who is ‘senior enough’ (use your own definition) should have their experience speak for them enough that there is no reason to ask for a code sample, let alone giving them a coding test.  Typically looking at their experience and asking them a number of probing questions about their experience should rapidly inform you if they know what they are talking about.

Now that’s not to say, that if after doing an initial interview, if something seems fishy, or if you are concerned about their experience, then go ahead and ask for a sample/test after the fact to dig deeper.

But my point is this, when interviewing for senior candidates, you should be spending time researching their experience, their history, what they’ve publicly done, presentations/articles/etc they’ve done and more.   You should take the effort to learn enough about the candidate that it should make most questions about their ‘coding skills’ a moot point, and leave you instead asking more about bigger picture questions, and to see how well they fit within your team.

Also I highly suggest that any coding test you give be a ‘take home’ test.   Let them take their time to do the problem in their own environment.   Asking someone to do a live coding test in front of you is asking for failure.  Are you going to be standing over them watching their every keystroke when they are working for you?  no?  Then why would you test them in that environment?  It doesn’t give a good indication at all as to how well they code (and the style in which they code) when given the freedom to do so.

16 Responses to Interviewing Programmers

  1. […] a new post to his blog Eli White looks at something that can be a difficult task at times – interview programmers to find […]

  2. […] a new post to his blog Eli White looks at something that can be a difficult task at times – interview programmers to find […]

  3. alantmiller says:

    Being on my own for a while I recently set out on a number of interviews and noticed a shift when it came to hiring developers which you touch on in this article. That shift is this focus on presentations/articles/etc that a developer has done. My question is, does it really matter? I mean is this the benchmark that defines one programmer’s seniority over another’s? I would think that when you are hiring a senior level programmer you would not be as concerned with their efforts to self promote themselves as you would be concerned with their actual work. I mean if you were hiring someone to go out and write presentations, articles, etc, I can understand why this would be important, but it seems to me in my experience that companies looking to hire senior PHP developers would be better served hiring people that had focussed more on their work and were better programers?

    You really have to ask yourself, what are the requirements of the role I am trying to fill, one of a good presenter, and blogger, or someone who can do the job better than anyone else?

    • Pogo says:

      I have long respected Joel Spolsky, though I sometimes disagree with him. This is one time. Having programmers code solutions to problems at a white board while his inquisitors look on is counter productive. It is not the way a competent programmer writes production-quality code. It IS, however, an excellent way of testing whether the programmer has potential to be a “competition programmer,” which is mostly of advantage to programmers in the better grad programs.

      I have developed a number of “take home” programming challenges. I give each candidate four and ask the candidate to do at least two and email them back the next day with notes and/or intermediate solutions to show how the coder worked from the original brute force method that almost invariably comes to mind first and the final, refined answers. That lets coders analyze the problems in depth, define them the most clearly, select the best data structures and algorithms and use whatever language you use as your problem-solving toolbox to implement the best, most elegant solution they can come up with. Unlike coding exercises during an interview, which are NOT the way a good programmer works in real life, this gives me a good read on how the programmer goes about solving a problem.

      All of the problems are ones that would (to my knowledge) never come up in a real situation, so googling is no help.

      Top programmers I never give tests to, but will engage them in discussion in depth about more abstruse elements. The best interview I ever had to go through involved two questions. First the interviewer asked me how I would set up a symbol table. We talked about it for an hour. First, I had to elicit requirements on how the table was to be used. Did the table need to support quick insertion, quick key-based retrievals, inorder traversals, duplicate entries, etc. Then we discussed the merits & disadvantages of arrays, linked lists, b-trees, self-balancing trees, hash tables (open & closed), circular or doubly linked lists implenting stacks or queues, and on and on. At the end of the hour, I was pretty much drained. Then he asked me to talk about parsing (lexing). We discussed top-down vs bottom-up, recursive-descent with & without backtracking, LALR(1) and its variations, finite state automata.

      At the end of 2 hours, he had teched me better than just about any other interviewer I ever faced (I got the job, but backed out when he mentioned a rate half of what I usually charged).

      At the other end, after enduring a 4-hour, 4-programmer panel technical grilling, where it seemed that each programmer was trying to gotcha me to show the manager he was better than I was, the company’s most senior programmer walked into the room and said, “I have just one question. Can object-oriented code be written in assembly language. I didn’t hesitate, answering “Of Course. But, why would anybody want to?” adding that all code ends up eventually as machine language before running on the CPU(s). He told me that he had been asking that question for 5 years and I was the first one who got it right – and offered me the job on the spot.

      Go figure…

  4. caseysoftware says:

    @alantmiller

    While I would agree that “self promotion” has little to do with a candidate’s suitability for a senior position, I think you’re missing some of the point. A senior developer – especially one who cares about his/her craft – will write, blog, present, teach, etc to sharpen their own skills *and* to sharpen the skills of those around them. Some of the hidden parts about being a Senior [anything] and the leadership and communication components.

    If someone can a) come up with the “best” answer*, b) express the idea successfully, and c) convince people that it is the “best” answer*, your team wins.

    * “Best” obviously varies depending on the requirements and circumstances. 😉

  5. […] application. This is absurd, for several reasons. Eli White spends a good deal of time arguing why coding tests are bad. I won’t rehash that […]

  6. robakehoe says:

    I agree with your post. I am fairly senior (with 15 years experience). Recently I had an interview where they gave me a programming exam to complete while the interviewer sat in front of me. Needless to say I failed. Not only did I think this was harsh, but it also told me that the interviewer was not technical, and did not know how to recruit a programmer.

  7. Piotr Kundu says:

    I have to admit, I suck at doing programming tests myself. I bet I would fail even if my life depended on it. I’m I a bad developer, I wish to think I’m not.

    Recently tables have turned and I’m doing the interviewing and as a “”recruiter”” I’m by default “not technical”. I’ve been programming for a living for the last 18 years, enough for most to considered “senior”. Why is there an assumption that all recruiters are idiots?

  8. Piotr Kundu says:

    Most senior developers who suck at their job, are actually great on cheating interviews, they have a great CV, 20+ experience and they can talk themselves out of any situation. I had a bunch of those last year and I asked pretty basic probing questions, like “whats the difference between a stack and a queue” (answer I got – “there is no difference”). So I gave them the test and they failed miserably upon which they crumbled and admitted that programming was never a passion it’s a mean to put food on the table.

    Were few people score high on the tests and flunk out in real life in my opinion. I rather loose a bunch good programmers than hire fake ones. I do understand the point in respecting somebody’s time – that is the backside of tests and programming assignments.

    These test are meant to become a way to start a dialog, so you can discuss the code and the solution – get a peek inside the developers head. Being the “dumb recruiter” I want to see how they react to “dumb colleagues”. Do they get mad or do they coach and explain?

  9. Piotr Kundu says:

    Since I’m a developer I want to SEE code, preferably something in production and to be honest do you really think my way of having fun is to “compile” every single developers code in my head? If you look for jobs seriously you would need to spend 2 hours per application for all the research, writing the cover letter and cleaning up your CV. Thats 4 jobs a day but as a recruiter you might end up with 30-40 applicants a day, all very cool programmers that you would love to code with yourself.

    Give 40 guys a programming test will most probably render 41 different solutions that the recruiter needs to have a look at- so you do that if you really need to. If the recruiter is non technical – he will have hard time to deal with the results. Actually I don’t mind paying a full days salary just to see the developer code for a day.

  10. IP Phone System Dubai

    Interviewing Programmers | Eli's Ramblings

  11. totesmagotes83 says:

    As a junior/intermediate developer, I hate it. They always say “It should only take a couple of hours”… so you expect me to work for 2-4 hours for free, so that MAYBE you’ll consider hiring me!? What happens when I get 3 different companies asking for that in the same week? Just because I’m unemployed doesn’t mean I don’t have other things to do! Like go to interviews, send resumes, etc.. Hell, sometimes people are looking for work while employed, so now you’re asking them to spend all weekend working on your test. (Let’s be honest, it’s hardly ever “just a couple of hours”).

  12. http://Www.caxa.com

    Interviewing Programmers | Eli's Ramblings

  13. Regarding these tests, there is another problem that has not been addressed. These tests suppose that the people that pass are better in the job. But that assumption need also to be verified. Have the companies/tests developers checked the performances of people that fail/pass the tests? Until then these tests are just pseudoscience.

  14. […] application. This is absurd, for several reasons.Eli White spends a good deal of time arguing why coding tests are bad. I wont rehash that […]

Leave a comment