Customize Speech-to-Text for Healthcare

Transcribing medical conversations requires systems that continually adapt to the newly developed medications, tests, and procedures. Furthermore, there are more than 135 medical specialties, each bringing its unique vocabulary to learn. General-purpose systems are not suitable in these settings.

Argmax enables developers to customize speech-to-text in real-time in a self-serve fashion with the Custom Vocabulary feature:

• Updating the system vocabulary is a configuration change, not a model or code change
• Each medical specialty can easily configure its unique vocabulary to customize behavior in a fine-grained fashion

‍

For companies that already fine-tune in-house

Companies that have already productionized fine-tuning workflows to update their speech-to-text systems can leverage Argmax Custom Vocabulary for additional targeted accuracy improvements.

Fine-tuning improves general accuracy in the healthcare domain. However, for any given doctor-patient encounter, the following specific context can be further leveraged for personalized and specialty-specific accuracy improvements.

Patient Specific:

• Patient name
• Clinical terms from patient pre-chart

Doctor Specific:

• Doctor name
• Names of the medical staff
• Clinical terms for doctor's specialty

‍

How do I test this on my own in 5 minutes?

‍

You can follow these steps to reproduce the concrete example below on your iPhone within the next 5 minutes! This example includes 5 audio files that vocalize all 238 medications approved by the FDA since 2021.

The exact match transcript results from Argmax, with and without Custom Vocabulary, are plotted below. For comparison, we also tested top cloud APIs that offer general-domain (Nova 3) as well as medical-domain fine-tuned (Nova Medical) systems. Note that these cloud APIs offer a Custom Vocabulary-like feature, but limit the keyword count to 100, so they could not be tested with 238 keywords.

‍

‍

‍

Step 1

‍Install the free Argmax Playground app

‍

Step 2

‍Launch the app, select parakeet-v2_476MB, toggle Enable Custom Vocabulary, and click Load Model  as shown on the left. The screen will look like the one on the right once the model is fully set up within 2 minutes. Note that this step requires an internet connection for downloading the speech-to-text system.

‍

‍

Step 3

Download any of the audio files below or choose from your own local audio files. The files below vocalize each medication approved by the FDA since 2021 based on the FDA's Novel Drug Approvals webpage.

‍

‍

‍

Step 4

Register all 238 of the medications that appear on the FDA's website (or your own desired list of clinical terms) by clicking the book icon with the letter A on it as shown below on the left and then copy pasting your comma or space delimited list of special terms as shown below on the right.

‍

‍

Here is a convenient concatenated list:

Nereus, Yartemlea, Myqorzo, Exdensur, Cardamyst, Nuzolvence, Lerochol, Voyxact, Hyrnuo, Redemplo, Komzifti, Kygevvi, Lynkuet, Jascayd, Rhapsido, Palsonify, Inluriyo, Forzinity, Keytruda Qlex, Wayrilz, Dawnzera, Brinsupri, Hernexeos, Modeyso, Vizz, Sephience, Anzupgo, Ekterly, Zegfrovy, Lynozyfic, Andembry, Ibtrozi, Enflonsia, Tryptyr, Emrelis, Avmapki Fakzynja Co-Pack, Imaavy, penpulimab-kcqx, Vanrafia, Qfitlia, Blujepa, Romvimza, Gomekli, Journavx, Grafapex, Datroway, Alhemo, Alyftrek, Tryngolza, Ensacove, Crenessity, Unloxcyt, Bizengri, Iomervu, Rapiblyk, Attruby, Ziihera, Revuforj, Orlynvah, Vyloy, Hympavzi, Itovebi, Flyrcado, Cobenfy, Aqneursa, Miplyffa, Ebglyss, Lazcluze, Niktimvo, Livdelzi, Nemluvio, Yorvipath, Voranigo, Leqselvi, Kisunla, Ohtuvayre, Piasky, Sofdra, Iqirvo, Rytelo, Imdelltra, Xolremdi, Ojemda, Anktiva, Lumisight, Zevtera, Voydeya, Vafseo, Winrevair, Duvyzat, Tryvio, Rezdiffra, Tevimbra, Letybo, Exblifep, Zelsuvmi, Leqembi, Brenzavvy, Jaypirca, Orserdu, Jesduvroq, Lamzede, Filspari, Skyclarys, Zavzpret, Daybue, Zynyz, Rezzayo, Joenja, Qalsody, Elfabrio, Veozah, Miebo, Epkinly, Xacduro, Paxlovid, Posluma, Inpefa, Columvi, Litfulo, Rystiggo, Ngenla, Beyfortus, Vanflyta, Xdemvy, Zurzuvae, Izervay, Talvey, Elrexfio, Sohonos, Veopoz, Aphexda, Ojjaara, Exxua, Pombiliti, Rivfloza, Velsipity, Zilbrysq, Bimzelx, Agamree, Omvoh, Loqtorzi, Fruzaqla, Defencath, Augtyro, Ryzneuta, Truqap, Ogsiveo, Fabhalta, Filsuvez, Wainua, NexoBrid, Briumvi, Xenoview, Lunsumio, Sunlenca, Krazati, Rezlidhia, Tzield, Elahere, Tecvayli, Imjudo, Lytgobi, Relyvrio, Omlonti, Elucirem, Terlivaz, Rolvedon, Sotyktu, Daxxify, Spevigo, Xenpozyme, Amvuttra, Vtama, Mounjaro, Voquezna, Camzyos, Vivjoa, Pluvicto, Opdualag, Ztalmy, Vonjo, Pyrukynd, Enjaymo, Vabysmo, Kimmtrak, Cibinqo, Quviviq, Adbry, Leqvio, Vyvgart, Tezspire, Cytalux, Livtencity, Voxzogo, Besremi, Scemblix, Tavneos, Livmarli, Qulipta, Tivdak, Exkivity, Skytrofa, Korsuva, Welireg, Nexviazyme, Saphnelo, Bylvay, Rezurock, Fexinidazole, Kerendia, Rylaze, Aduhelm, Brexafemme, Lybalvi, Truseltiq, Lumakras, Pylarify, Rybrevant, Empaveli, Zynlonta, Jemperli, Nextstellis, Qelbree, Zegalogue, Ponvory, Fotivda, Azstarys, Pepaxto, Nulibry, Amondys 45, Cosela, Evkeeza, Ukoniq, Tepmetko, Lupkynis, Cabenuva, Verquvo

‍

Step 5

‍Finally, go to Transcribe > From File in Argmax Playground and transcribe these files and enjoy the frontier accuracy that you just crafted in real-time simply by copying and pasting a list of words!

‍

‍

What is the secret sauce?

We will publish the technical report in February.

‍

How do I get this tool for my app?

If you are deploying medical speech-to-text in your app and want this real-time customization capability, start your 14-day Argmax Pro SDK trial today!