Eight questions that separate the vendors
A demo tells you how a product performs under conditions the vendor chose. These questions tell you more, and they work on us as well as on everyone else.
The short version
- Ask for the overage rate and the billable-unit definition in writing. Four vendors we track publish neither.
- Ask what integrates with means, specifically: does it write, and what does it write?
- Distrust any resolution-rate figure that arrives without a method and a sample size.
- Test with a real call from a noisy place, not from a quiet office.
The money questions
What is the overage rate at the tier I would buy, in writing? Rosie, Ruby, Slang.ai and AnswerConnect do not publish one. That is four of the eight vendors we track, and it means a busy month cannot be estimated from public information.
What counts as a billable unit? Does a hang-up during the greeting count? A wrong number? A call transferred straight to my team? On a per-call meter those add up, and on a per-minute meter that rounds up, a twelve-second robocall costs a full minute.
Is there a setup fee, and what is the contract term? We charge no setup fee. AnswerConnect is reported to charge around $49.99, which is third-party information since they gate pricing.
The capability questions
Does it write into my calendar, or only read it? A tool that can see availability but not create an event is not going to book anything, and a logo grid will not tell you which you are getting.
What exactly lands in the calendar entry? A blank slot at the right time is a much weaker product than one carrying the caller's name, number and reason.
What happens when the assistant cannot help? Ask for the mechanism rather than a percentage. Who does the transfer reach, and what does the caller hear if that rings out? That last one is the failure mode that actually occurs.
Which languages are verified, not planned? We handle English at launch and say so. A vendor listing languages should be able to say how each was tested.
The claims to push back on
Any resolution rate. Figures about the share of calls handled without a human circulate constantly in this category, and we could not find a primary source for the ones quoted most. Ask for the method and the sample before treating one as information.
Any uptime figure. Nobody in this category publishes measurements you can verify, so an uptime number on a marketing page is decoration. We publish none for the same reason.
HIPAA compliance, if you are a practice. It is contractual rather than a certification. Ask which layers they hold agreements with: carrier, speech-to-text, language model, voice synthesis.
And indistinguishable from a human. Beyond being hard to verify, that is close to the condition Maine's disclosure law makes actionable. A vendor selling on it while also selling to licensed professions has a tension to resolve.
The test to run before you sign
Ring the demo number from the environment your real callers use. A busy dining room, a van with the engine running, a street. Vendors demo from quiet offices because everyone does, and background noise is where these systems differ most.
Interrupt it. Change your mind halfway through. Give a date and then correct it. Ask for a person. Those four moves reveal more about a product than ten minutes of a scripted booking.
Then ask the same vendor what their product does badly. An answer that is a genuine limitation rather than a humblebrag tells you something about how the rest of their claims were written.
Hear it handle one of your own calls
Your scenario, your greeting, a couple of minutes.