Engineering persona

I need to admit I always enjoy interviewing candidates for software engineering positions: there is a unique power that comes with choosing the right people, bringing them into a team and observing how the best individuals can really make the difference.

While I have been relatively lucky in my hiring endeavour, it always somehow bothered me that I didn’t know why it worked the way it did.

Too many of our gut feelings and instincts are just plain wrong so when the data actually says otherwise it’s intriguing to try and understand exactly why.

It was only 3 years ago - when I was tasked with hiring really good engineers at scale - that I was forced to really analyse and systematize my approach, to make it scalable and improve it where possible.

I always thought - and still do - that most recruitment processes are inherently flawed and that most job interviews simply have the wrong format and sadly ultimately fail to achieve what they are designed to do way too often.

They either:

• don’t find the right people for the job - leading to random hires - or
• they are wasteful and with a high rate of false negatives - like in the Silicon Valley style 6/7 interviews and 0.2% success rate.

I believe the culprit is what is commonly called a dumbed down implementation of wisdom of the crowds approach.

This technique is ubiquitously used across most industries in the hiring process of knowledge workers.

Even if you have never heard of this term you may already be familiar with how it works: let multiple people take a guess at evaluating a potential hire - through multiple interviews - and proceed with a hire when there is agreement among all the interviewers.

Laszlo Bock - the former head of People Operations at Google - describes the wisdom of the crowds approach in hiring as a foundational choice that shaped the company. In his very good book - Work Rules! he describes how:

The founders realized it was important to hire by committee, often interviewing candidates together ….They intuited that no individual interviewer will get it right every time, an instinct that would later be formalized in our “wisdom of the crowds” study in 2007

Wisdom of the crowds - corrected

In fact it’s been known for years that structured interviews and standardized tests are significantly better in hiring the best candidates than just “hiring by committee” and to be completely honest Google has been continuously evolving their hiring practices to incorporate these findings.

In reality while most top tier US West Coast IT companies claim to follow a structured and standardized approach, it’s actually more of a hybrid, with the constant tendency of falling back into the unstructured.

Multiple independent studies on this subject have shown that the wisdom of the crowds only works if:

• Interviewers are from totally different backgrounds,
• Assessments are totally independent,
• Outputs and evaluations are *numerical.

In practice these requirements are never really met as

• interviewers come from very aligned backgrounds, at most different teams within the same department and are actively encouraged to look for sometimes vaguely defined cultural fitness and this becomes an easy way for unconscious biases to sneak in,
• assessments are not independent: in most companies - but not at Google for example - the hiring manager has the power of veto or skew conversation around the final decision
• but above all, the evaluation of the candidate cannot be expressed numerically.

It’s been proven that an interesting side effect of a properly structured hiring approach is that it leads to more inclusive and diverse hiring decisions. Unfortunately - as the literally hundreds of articles and studies showing the need for more diversity in Software Development demonstrate - we are simply far from it.

Of the main 3 issues described, I believe that the last is the single most important and - as it turns out - the easiest and yet the hardest to fix.

Engineering Persona

How many of us have actually thought about what the best person for a specific job would look like? I am not just talking about the description of a hypothetical good candidate but rather the list of traits and characteristics that really make the difference in a role?

Not many I could guess: there is a lot of talk in the industry about 10x developers but very few people have really spent any time trying to describe what makes these people’s performance 10 times better than the average.

Yet it is self evident that if you don’t know what good looks like, you won’t be able to find it. It becomes practically impossible to evaluate a candidate without falling into the gut feelings approach - which leads to bias, lack of diversity but - above all - random results.

Our actions are the result of our mental maps, the approaches that we take in making sense of and dealing with the situations that involve us.

The NLP Modelling approach - not scientifically proven but widely popular in the field business - suggests that not all maps have the same value and that specific behaviours and approaches lead to significantly better outcomes.

In other words: success leaves clues.

And this is where the concept of engineering persona - a blueprint for what an ideal candidate thinks like - kicks in.

My engineering persona

This concept is easier to describe through an example rather than a theoretical explanation so let me start by describing the engineering persona that I always use during my hiring.

I am always looking for 5 distinct traits:

1. Technology as a means to an end: we all have different triggers to our motivations and understanding if a candidate has the right ones is crucial. The industry is plagued by technologists by technology sake - people that look at tools and approaches without any interest in the bigger picture. These are exactly the people that I tend to screen out.

2. Curiosity: Demonstrated interest in multiple technologies. Approaching technologies with a critical thinking and a PROs and CONs approach. Existence of any side project. One could say that curiosity is a must for all high performance individuals.

3. Entrepreneurship: interest in going beyond technology and understanding how every field of business that technology touches on - marketing / sales / product management / delivery - ideally works so that different and competing requirements can be foreseen, understood and balanced appropriately.

4. Technical Knowledge: the result of a tech assessment of some sort. The tech test has to have a high relevance to the job you are hiring for. Multiple options are available but this step simply cannot be skipped.

5. Feedback: How does the candidate receive and provide feedback. Using the PROs and CONs detached approach to resolving conflict.

As you can see:

• this approach is focused on problem-solving and future-oriented,
• every item on the list describes a specific attitude that in my experience has been associated with success in software development.
• there is no mention of work experience, formal or informal education, CV and/or personal references which are all interesting but - in my experience - almost irrelevant.

The power of the engineering persona lies in the fact that it can be used as the foundation to support the objective evaluation of a potential candidate across these 5 dimensions.

Each interviewer can be instructed to look for specific and measurable traits and provide a numeric score.

Questions and answers

In practice I have found it very useful to prepare upfront a set of predefined and relevant questions that can guide the evaluation.

Good examples of questions that would match the engineering persona described above are:

• What was your best day in your current or past role and what made it so?
• How do you go about evaluating a new technology? What attracts you to it and why?
• What has been your greatest learning working with product, marketing or sales in your career?
• What has been so far your relationship with product / delivery / marketing / sales? What can you do to help people in this area do a better job?
• How do you handle conflict in your day to day?

As you can see these questions are open ended yet designed to spark a conversation. There are no right or wrong answers because all that matters is understanding the thinking behind the answers provided.

As the process matures use the learnings accrued to evolve them - to prevent copycat answers - but also to continuously keep them relevant and up to date.

Conversational engineering persona

I have argued before at length that the market for software engineers is very tight - despite Covid19 - and the overall experience of the hiring process is your biggest asset in convincing the right candidates to accept your offer.

To this end it’s crucial not to fall into the trap of using the who wants to be a millionaire approach - asking questions from a teleprompt and noting down answers without betraying emotions. On the contrary, try and create a pleasant conversation that is carefully crafted to lead you to the answers you are looking for while still providing a feeling of a friendly chat.

Here are my top suggestions to achieve this:

• Start with events taken from the CV and ask generic questions on how a common situation/issue relevant to the role would be handled.
• Focus on the future (how do you suggest we handle …) rather than past experiences (tell me about a time when you…) as the past can be easily scripted but the future requires ideas and creative thinking.
• Once that candidate has exposed an idea or approach, feel free to express your take on it and look for ways to spark a debate. Your job is to understand the true self behind the words and get rid of canned answers so you need to challenge the candidate.
• People will have different opinions than yours but opinions are not what you want to evaluate people on. What you really care about is how solid the interviewee’s thinking is and how easy or difficult it is to poke holes into the thinking that underpins the opinion.
• A great indicator of maturity and seniority are signs that the candidate is listening to your ideas by mentioning them back to you.
• You want to have a conversation but all areas need to be evaluated so write down notes and continuously go through them during the interview to understand what has already been covered and what’s missing.

Summing it all up

Hiring the right people is crucial, in Software engineering like with all modern knowledge workers. Unfortunately most recruitment processes are oversimplified and end up making decisions based on gut instincts whereas structured interviews have been proven for many years now to provide much more reliable results. The simple most impactful element in creating a structured process is defining what we are looking for - I call this the engineering persona.

Use it as a blueprint to guide your conversations while breaking free from scripted answers and boring Q&As: that’s when you can really harness the true wisdom of the crowds.