I tested three AI interviewers to see whether we're out of the uncanny valley
The shorter answer is no. The voices are getting better but being interviewed by AI is like being on a bad date where one side has no conversation skills.
AI summary
- I sat through AI interviews at Mercor, Crossover, and Ribbon as a candidate. The voices have improved a lot. The conversations have not.
- All three had the same problem: generic questions, paraphrasing my answer back to me, and no reaction when I said something a human would have questioned.
- Underneath the voice, an AI interviewer is doing what a one-way interview already does. It collects structured answers and scores them against a rubric. The voice is an expensive layer on top.
One debate we consistently have at Truffle is whether we want to add the option to add AI interviews to our AI screening toolkit.
The initial argument against them was that they’re too uncanny valley. The voices sounded robotic, they failed in mysterious ways, and candidates wouldn’t complete them.
We’ve come a long way since then, and with the launch of Astra, and Fable continuing to blow our minds, we figured it was time to revisit the state of AI interviewers.
What follows is my experience with three well-known AI interview platforms and my honest take on the experience with each one.
Mercor
Mercor’s website reads: “Mercor is organizing human intelligence to power the AI economy.”
(I suspect “Mercor is using you to train your AI replacement” was tested at some point, but didn’t perform quite as well.)
What started out as a straight AI interviewer morphed into finding humans to train AI models, which I guess proved to be a lot more lucrative. They’ve raised north of $450 million and are valued at $10 billion so if there is a company with the resources to create world-beating AI interviewers it’s Mercor.
The company works a bit like an Upwork where you sign up and create a profile. This only took a few minutes and involved my email address and LinkedIn profile.
I applied for a Sales and Marketing Expert role to access the interview experience with Hemingway’s face disapprovingly watching over me.
There is no human-like avatar which in my humble opinion is a better experience than having something from Zuckerberg’s metaverse staring at you like an alien lifeform.
The AI voice proceeds to provide some instructions and guidance. (I appreciate how much better these AI voices have gotten.)
Just as we were getting to the end of the instructions it failed to adjust a personalized field that was empty because I had not yet uploaded my resume. This made the experience feel a bit janky out of the box.
I found the experience to be painful.
When I said my expertise was marketing it proceeded to ask me a very generic question: “What is a marketing challenge you’ve overcome recently?” Most of the questions that followed were generic and open-ended, and therefore difficult to answer.
As I was speaking it interjected at unnatural points and then we got into the most hilarious exchange around CAC (customer acquisition cost). I’ll just leave this screenshot here.
Overall, I wasn’t impressed by the experience. The voice and the interface are slick, but the questions and the conversational flow were not smooth.
Crossover
Crossover recruits full-time contractors worldwide for a group of US software and edtech companies that share one owner, ESW Capital (Trilogy, IgniteTech, GFI, DevFactory, 2 Hour Learning, Alpha School, Prequel, GT School).
Truffle has also worked with PE firms looking to roll out AI screening across their portfolios so it makes sense they built their own platform, though they could have just called us. ;)
Every role runs the same six-step gauntlet: a 15 to 20 minute AI chat-style screening interview, a cognitive aptitude test (CCAT), a real-work skills assessment, a hiring-manager interview, a proctored test, then an offer.
The AI interview is text-chat rather than voice or video, which makes it a good contrast to the rest of the vendors on the list.
The first step is a few qualification or knockout questions (we offer a similar feature in Truffle.)
The challenge with the chat experience is that it’s incredibly tedious for candidates. Unless you’re the kind of person who enjoys customer service chats where a rep asks one question, then follows it with several clarifying questions that could have been asked upfront, you’re probably not going to enjoy this.
While I am all for journalistic integrity, I soon abandoned the chat experience when the conversation became a little circular. I can’t imagine what fresh hell it would be to spend 30 minutes doing this.
Ribbon
Ribbon is a Toronto startup that raised $8.2 million from Radical Ventures last year and claims to have run over a million interviews.
Of all the roles I mock interviewed for, founding marketing lead was probably the one I was the most “qualified” for so I was determined to give this one a proper shot.
The setup is the best of the three. A sound check, a “camera is recording” warning, a promise it would take under 15 minutes, and a question about whether I had any questions.
That said, there are a growing number of states and countries that require notice before the interview that AI may analyze it, an explanation of how it works and what characteristics it evaluates, and consent to that specifically. I did not see this, and it seems like a miss on Ribbon’s part.
Every exchange followed the same three beats. I’d answer, it would paraphrase my answer back to me, tell me it was a great answer, and read the next question off the list.
“It sounds like you have a keen understanding of the current hiring landscape. Now, where are you currently located?” I told them I was in Belgium. It said “That’s interesting, being based in Belgium with dual citizenship?” and moved on.
The job is hybrid in Toronto so it seemed like a key piece of information that would disqualify me.
I decided to test how much it was actually listening. When it asked about successful marketing campaigns I’d run, I told it I don’t believe in marketing campaigns and that the whole concept is a bit silly.
A human interviewer would have at least raised an eyebrow. Ribbon congratulated me on “creating sustainable marketing systems rather than short-term campaigns.”
I’m not quite sure how this experience is better than a standard one-way video interview. It ultimately is collecting the transcript to analyze, and when you try to engage with it, you get something like:
“My role is primarily to facilitate this conversation, ensuring that I capture your experiences and how they align with Ribbon’s needs, rather than evaluating or debating points.”
And it goes back to asking the questions it’s programmed to ask. Even if you like the fact that AI repeats a candidate’s answer, Ribbon’s published entry plan starts at $499 a month, roughly ten times Truffle’s Starter tier of $49.
The uncanny valley has taken a new shape
I’m still not convinced the juice is worth the squeeze with AI interviews, despite how much better the voices and responsiveness have become.
A candidate answering five written questions knows exactly what’s happening. An AI interviewer presents itself as a conversation, so candidates judge it like one, and every odd interaction stands out.
That leaves it in an awkward middle ground: slower and more tiring than a form, but less revealing than a real call. Underneath, it’s still doing what asynchronous interviews already do: collecting structured answers and scoring them against a rubric.
The voice is an expensive layer on top, and it’s not clear what you lose by removing it.
.png)