Deepgram benchmarks its speech recognition on aixplain, so builders can choose it on evidence

aixplain is an ecosystem that fuels its operating system. Model and tool providers bring their assets, and the people building agents choose among them. A provider needs to show where its model is the right choice. A builder needs to know which model fits their language, their data and their budget. Independent benchmarks answer both. Deepgram, a speech recognition provider, used aixplain to benchmark its models in Hindi and Indian English.

The challenge: every provider has its own numbers

Speech recognition is a crowded market. Each provider publishes results on its own data, so buyers can't compare them, and providers can't prove where they lead.

A fair benchmark is slow and costly to build in-house: recordings in each language, the same conditions for every system, metrics everyone accepts, and a report others will believe.

Agents raise the bar. A voice agent needs a model that is accurate in the caller's language, fast enough to answer in time, and cheap enough to run on every call. A provider that wants to be picked for agents has to show all three.

Why aixplain: neutral ground with the ecosystem on it

• Independent. aixplain runs the benchmark, so the provider isn't grading its own work.
• Same harness for everyone. Every system runs on the same recordings, through the same interface, under the same conditions.
• Metrics agents care about. Accuracy, cost and speed side by side, plus how often calls fail.
• Data from the ecosystem. Data partners such as VoiceLoft supply recorded speech in the languages providers need.

The solution: an independent evaluation in Hindi and Indian English

The data. Scripted text read aloud by many speakers, in Hindi and in Indian English, supplied by VoiceLoft. The text comes from public sources such as Wikipedia, question-and-answer sites, reviews and social posts.

The systems. Deepgram's speech recognition next to other leading cloud speech services, all run through aixplain.

The metrics. Word error rate and BLEU against the reference transcripts, each under several text normalization settings, plus cost per minute, latency per segment and the share of failed calls.

The reports. One report per language, with charts that weigh accuracy against cost and against speed.

What it showed: no single best system

In both languages, Deepgram was the fastest system and among the least expensive, which is what a voice agent answering in real time needs. On accuracy the picture split. In Indian English, Deepgram had the lowest word error rate in most settings. In Hindi, other services were more accurate.

That is the point of an independent benchmark. Rankings changed between languages and between normalization settings, because a model does best on data like the data it was trained on. A provider learns where to sell and where to improve. A builder learns to test on data like their own.

What's next: agent-ready, on evidence

Providers in the ecosystem can benchmark their models on aixplain the same way, on accuracy, cost and speed, so builders can choose them for agents on evidence, tested on data like their own.

See aixplain at work

aixplain is the operating system for accountable AI agents you own. Organizations run more of their operations and services on agents, on any model and any server. Every run is checked against policy, and every agent answers to an owner.

Book a demo to see how your model, or the one you're choosing, performs on data like yours.