Setting out a new path in your career is an exciting venture. Bookmarking promising positions and getting those next-step emails from a potential employer can feel like the beginning of something big. And then comes the tedious test — a time-consuming coding task that grinds down your bandwidth so much, that you’re already eyeing positions with other companies just to avoid it.
After experiencing this interview cycle firsthand for long enough, we knew it was high time to switch up our approach. Our software engineering Circle lead Dominik Mostek and Python Circle lead Stanislav Prokop had the task of improving this framework, and we’re already noticing positive results!
See for yourself how we’re getting to the core of what’s really important to us during our interviews — not to mention saving hours in the process!
What are some of the pain points that come up in typical interviews?
DM:
Most engineers don’t want to spend a lot of time on interviews. As interviewers we’d like to spend time with them, to get to know them better and their opinions and experiences. But they would drop off immediately if we told them they needed to spend a lot of time on it. So one of our goals was to make the interviews as quickly as possible while preserving the amount of knowledge we get about the candidate.
What is the standard process like? Why doesn’t it work?
DM:
A standard way to assess developers in many software companies is to give them some coding tasks to do on their own, and they present it. We don’t like it because if you want to make your code as good as possible, you have to spend at least one or two days on even a simple task. I remember recruiters mentioning long homework, and I thought, Okay, I’m not doing it. If I wanted to go to five interviews, and each has homework that will take at least two days to complete, that’s 10 days of work I won’t see again.
We also wanted to avoid live, whiteboard coding because it puts a lot of pressure on the candidate. And you have to think of code on a whiteboard, and that’s not what you will be doing at your job. We also wanted to avoid small code examples, which is another standard way of interviewing. You present a few lines of code which proves that you know how HashMap works and it’s a nice academic discussion but says nothing about your abilities as a software engineer.
SP:
When I did an interview for a senior position at Microsoft in the past, an interviewer asked me how HashMap is implemented, including details like how hash function collisions are dealt with. That’s university level knowledge you never ever implement on your own. To me it feels like laziness to evaluate candidates based on such questions.
How is the hiring framework different at Ataccama now?
SP:
First, there’s a screening round with our recruiters to learn about the candidate’s skills and if they might be a cultural fit for the company. Then the recruiters send the candidate a code review task. In this task, we prepared some code that has some fundamental and more complicated issues. They are supposed to take a look at the code and provide feedback about how to improve it. And this is essentially the difference, that we ask them to do what they are supposed to do in their job, using tools which they would use as well.
DM:
There is a statistic that you spend 90% of your time as a developer reading someone else’s code, and only 5% writing code and maybe 5% debugging. So at the interview why are we asking them to write code, which is only 5% of their time?
How do the interviews continue after the code review?
DM:
After the technical rounds, there’s a Spaceport round where we try to evaluate in which of the seven Spaceports the candidate would most belong based on their past knowledge and experiences. Then, representatives of those Spaceports meet them in the last round.
How does this help you save time on interviews?
DM:
I’d say it saves a lot of time, because we use the same code for every candidate and we know where the problems and possible improvements are. So it saves the candidate’s time because the code is ready, and they only need to review it and focus on some parts. If they were doing a coding task, we’d have to review the code for at least an hour or two. Now it only takes us about 10 minutes to prepare for an interview.
SP:
Sometimes I don’t even spend 10 minutes. Usually, there are two interviewers in the interview. Often one is going to really take a look, and the other is just following. And it’s easy to take a look at the code because you know the code and the candidates are familiar with it as well. And often we don’t go by code commands, we just focus on certain areas where we know there is a lot of room for improvement. We also try to find something the candidate missed, and try to understand their reaction. If they claim to be a subject matter expert, but don’t notice an obvious violation of their particular subject and then try to neglect that…it’s just a bad sign.
But maybe some are more like, “Oh, I missed that!” That’s a more positive, human reaction, and you know they can openly acknowledge mistakes. That’s more human, you know they’re not lying. It really sparks the discussion around written and missing commands.
How was the Dreyfus model of skill acquisition involved in creating this framework?
DM:
It basically says that your skill acquisition path isn’t linear. You have to go one way, then go back, try another way, and that’s how we improve. And we took that and reflected it in our candidates’ evaluation. So now during interviews, I challenge candidates when they say that they’d do something different in the code. I ask why, and the usual answer is because they’re used to doing it that way. If this is the answer, you know they’re more of a junior software developer with less experience. For a junior developer, this is actually the best answer they can really give.
But if this is coming from a senior developer, after 12 or more years of experience, they should have more of an opinion about it. And that’s what the Dreyfus model actually tells you. If you’re a junior, you follow blindly. As you discover more you reach the next stage, but you might not know how to choose the best option. Then if you are senior, or what the Dreyfus model calls competent, then you not only know the paths, you know how to choose between them. And it goes further, into proficiency, where you can even invent different paths.
SP:
To understand if the candidate is junior or more senior, we try to challenge their points. They tell us to change something in the code, and we pretend to be curious interns — what’s the point? Could you help me to understand why it’s a good practice? How does it relate to your experience? So it’s an interesting discussion, which also shows their experience. It’s way more fun than the interviews I was doing in the past.
Why did you want interviewees to perform a code review, rather than a typical coding task?
SP:
Before joining Atacama, I was part of Microsoft and GitHub, working on a Microsoft product. The interview process was quite bad. It was the way we discussed, with a lot of rounds, and asking for a HashMap. Before you could participate in interviews you had to go through a tool similar to HackerRank, and the coding was online as well, in a browser, which you don’t use to code. It’s bad.
We also noticed that when senior candidates were supposed to do a code task, they just didn’t follow up. Basically, they started ghosting us. We thought about how to improve it, and we couldn’t improve it due to some constraints. It’s not that easy to just make changes within a corporation. And the code review sounded like a really promising idea. And there are many other companies also using it.
What are the five main topics covered in the interviews?
DM:
One of them is knowledge of the programming language. Then there’s knowledge about frameworks and libraries and ready-made tools. Then it’s architecture, clean and maintainable code, and distributed systems. So it focuses on basic coding, knowledge about the environment, the ability to create a single, let’s say, service or module. Clean coding is about writing code that will survive for many years. Distributed systems is a very senior concept, having many services and the coordination and communication between them. This should cover most of what you do as a developer.
One developer asked why we don’t test databases during the interviews, since we’re a company that deals with data. It’s actually tested in the distributed systems part of the interview. But most importantly, databases are more of a detail. If you know how to write maintainable code and design their communication, then databases are only an implementation detail. I think most other things are covered in the same manner.
What kind of feedback have you gotten?
DM:
The feedback is starting to come in now and it’s overall very good. We have a lot of feedback from candidates who accepted the offer, but that is not that relevant because it’s from people who accepted the offer, so they probably liked it. It’s kind of biased, and we should be focusing more on the people who we didn’t hire.
We weren’t doing this before but we’re starting to focus on getting feedback from the people who didn’t want to continue the process. After candidates are done with the technical round, they will now get an email with a feedback form. So now we even see feedback from all the candidates, not only from the ones we hire.
We also asked at the end of interviews. I don’t remember anyone who didn’t feel positive about the process. And the majority of the feedback is that it’s really interesting, unusual, and refreshing. We actually had one candidate who had an offer for a different company already, who still agreed to do the review task just because he liked the idea of code review.
Want to challenge yourself with our code review task? We’d love to hear from you! See our open positions over on our job portal and find the role that best fits you — or would get you out of your comfort zone!
Can’t wait to meet you!