Nvidia Releases Open-Source Voice Model Supporting 40 Languages

Nvidia has entered the speech recognition space with an open-weight model. Nemotron 3.5 Transcribe, released by the company, combines speech-to-text transcription, speaker diarization, and translation into another language in a single model. Until now, these tasks were usually handled by several separate tools run side by side.

The model is designed not as a classic transcription tool, but as a command-guided speech-augmented language model (SALM). In other words, you don’t need retraining or a separate script to change the output format; an instruction in natural language is enough.

34 billion parameters, 4.3 billion per token

The model’s backbone is based on the Nemotron 3.5 Lightning architecture. The mixture-of-experts (MoE) structure, made up of hybrid Mamba-2 and Transformer layers, carries 34 billion parameters in total, but only about 4.3 billion of them are activated for each token. Nvidia explains this choice as combining the language ability of a 34B-class model with the inference speed of a 4B-class model.

The linear memory scaling of the Mamba-2 layers also reduces the memory bottlenecks seen in long recordings. Thanks to the 65,536-token context window, a meeting or lecture lasting up to 60 minutes can be processed in a single pass without being split into chunks.

Forty languages, eight speakers, two formats

The model performs automatic speech recognition in 40 languages; Turkish is also on the list. Direct speech translation between 31 languages and English is supported, so there is no need to insert a separate text translation model in between. Up to 8 different speakers can be labeled in the same recording; even overlapping speech is separated.

The output format is also selected by command: a verbatim transcript (including filler words and stutters) or clean, minutes-style text. You can give the model a list of custom terms, drug names, or code libraries; Nvidia says this reduces misspellings of proper nouns by up to 70 percent. A CTC alignment adapter added to the output provides the start and end time of each word at millisecond precision.

Running it on your own server

The weights were released on Hugging Face in BF16, along with FP8 versions that halve memory and NVFP4 versions specific to the Blackwell architecture. The model works out of the box with vLLM, TensorRT-LLM, and Nvidia Riva. According to the source, hundreds of concurrent audio streams can be processed at full precision on a single 80 GB H100 or A100; quantized versions also run on machines with lower VRAM. Nvidia also open-sourced the Granary v2 data pipeline under the NeMo umbrella; the community can train a speech model tailored to their own language or industry through this pipeline.

Why it matters

As voice interfaces become more widespread, organizations’ biggest concern was sending call center recordings and closed-door meetings to cloud APIs. An open-weight model that can run on your own hardware makes it possible to process these recordings without sending them outside. Turkish support also makes the model directly testable in local projects.

Source: WebTuring