I recently interviewed people for a couple positions where I work. We had to ask questions that got at a bunch of different skills, and at the end we were supposed to score their responses in a spreadsheet.
It quickly became obvious to me that there was a flaw in the method: If the candidate didn't get a chance to answer all the questions, their score would be really low. This is through no fault of their own: they don't know they're on question 4 of 12, and there's only 15 minutes left in the interview. They think they're just answering the question thoroughly and thoughtfully. So, it left me in the position of having to do a "lightning round" style assault at the end of the interview, to make sure they at least got a chance to answer everything, and not be torpedoed by the scoring system.
The solution to this is likely one of:
* ask everyone different questions, or different numbers of questions
* ask fewer questions to everyone, and potentially not cover some areas.
* timebox each response, and cut them off to say "time's up, let's move on".
* ignore questions they didn't answer, which means deciding on a candidate you know less about.
* increase the length of the interview.
I think I'd be happy asking everyone a different set of questions, but companies are apparently scared about the bias that could introduce (at any rate, we were instructed not to do that). Of the remaining options, the best one seems to be to increase the length of the meeting.
I’ve found standardized questions to be useless. I’ve found the best people by reading their cvs carefully. Then reading whatever published work they have. Then asking them to talk about that. If that goes well then I describe the work the team is doing and ask them their thoughts. If I really need validation I have a data problem I send them and ask them to send it back. (I don’t ask people to code in front of me - that’s ridiculous and tells me nothing) The standard problem Id ask was to figure out how to calculate something I describe only conceptually, and then give them a data set to do it on. Most mediocre so called data scientists will throw a bunch of Python libraries at the data and not even think about what was asked. This, sadly, is the state of my profession after 10 years of “boot camps “ and incursions from other disciplines.
It quickly became obvious to me that there was a flaw in the method: If the candidate didn't get a chance to answer all the questions, their score would be really low. This is through no fault of their own: they don't know they're on question 4 of 12, and there's only 15 minutes left in the interview. They think they're just answering the question thoroughly and thoughtfully. So, it left me in the position of having to do a "lightning round" style assault at the end of the interview, to make sure they at least got a chance to answer everything, and not be torpedoed by the scoring system.
The solution to this is likely one of:
* ask everyone different questions, or different numbers of questions
* ask fewer questions to everyone, and potentially not cover some areas.
* timebox each response, and cut them off to say "time's up, let's move on".
* ignore questions they didn't answer, which means deciding on a candidate you know less about.
* increase the length of the interview.
I think I'd be happy asking everyone a different set of questions, but companies are apparently scared about the bias that could introduce (at any rate, we were instructed not to do that). Of the remaining options, the best one seems to be to increase the length of the meeting.