August 17, 2025
TL;DR
SDBench is our open-source and reproducible benchmark suite for popular speaker diarization systems, commercial and academic. It includes 13 datasets covering various languages and use cases, standardized test splits, and evaluation metric configurations. Here is the link to the PDF.
We recently renamed SDBench to OpenBench to reflect the expanded benchmarking scope. OpenBench now benchmarks real-time transcription and keyword boosting as well. The link to the GitHub repository is here.
The number of people inquiring about Frontier Models On Device (FMOD) far exceeded our expectations! We also welcomed friends of Argmax like Hugging Face, Nvidia, pyannoteAI and aicoustics as well as many first-timers who were excited to see frontier models like Nvidia Parakeet v3 (released 1 day before the conference) running on-device in real-time.