Robotic Automations

Paris-based startup Nabla just announced that it has raised a $24 million Series B funding round led by Cathay Innovation, with participation from ZEBOX Ventures — the corporate VC fund of CMA CGM. This funding round comes just a few months after Nabla signed a large-scale partnership with Permanente Medical Group, a division of U.S. healthcare giant Kaiser Permanente.

Nabla has been working on an AI copilot for doctors and other medical staff. The best way to describe it is that it’s a silent work partner that sits in the corner of the room, takes notes and writes medical reports for you.

The startup was originally founded by Alexandre Lebrun, Delphine Groll and Martin Raison. Lebrun, Nabla’s CEO, was the CEO of Wit.ai, an AI assistant startup that was acquired by Facebook. He then became the head of engineering of Facebook’s AI research lab FAIR.

A few weeks ago, I saw a live demo of Nabla with a real doctor and a fake patient pretending that they had back pains. When a physician starts a consultation, they hit the start button in Nabla’s interface and forget about their computer.

In addition to the physical examination part, a consultation also includes a long discussion with a bunch of questions about what brings you here and your medical history. At the end of the consultation, there might also be recommendations and prescriptions.

Nabla uses speech-to-text technology to turn the conversation into a written transcript. It works with both in-person consultations and telehealth appointments.

After the patient has left, the doctor hits the stop button. Nabla then uses a large language model refined with medical data and health-related conversations to identify the important data points in the consultation — medical vitals, drug names, pathologies, etc.

Nabla generates a thorough medical report in a minute or two with a summary of the consultation, prescriptions and follow-up appointment letters.

These reports can be customized to the doctor’s needs with a personalized format for your notes. For instance, you can add instructions to make the note more concise or more verbose. Or you can ask to generate notes that follow the Subjective, Objective, Assessment and Plan (SOAP) note pattern that is widely used in the U.S.

During the demo that I saw, I was extremely surprised by the effectiveness of Nabla in general. Even though we were in a crowded room and Nabla was running on a laptop a couple of meters away from the demo presenters, the tool was able to generate an accurate transcript and a useful report.

With Nabla Copilot, as the name suggests, the startup isn’t trying to take the human out of the medical loop. Physicians still have a final say as they can edit reports before they are filed in their electronic health record system (EHR).

Instead, the company thinks it can help doctors save time on admin work so that they can spend more time focusing on patients.

“What we know is the near future is we don’t want to try to replace doctors. You’ve seen companies — like Babylon in the U.K. — burning $1 billion trying to do chatbots and trying to automate things right away and remove doctors from the loop. And we’ve decided a long time ago with Nabla Copilot that [doctors] are the pilots and we work by their side,” Nabla co-founder and CEO Alexandre Lebrun said.

“It’s a little bit like automation for autonomous vehicles. We are still at level two today. We will start level three very soon with clinical assurance support. Then level four is clinical decision support, but with FDA approval, because you make decisions that you cannot really explain,” he added.

At some point, you could even imagine a level five of autonomous healthcare, which would mean removing physicians from the room. But Lebrun is still very cautious on this front.

“For some situations in some markets, like in some countries where they don’t have any access to healthcare, it would be a relevant thing,” he said. Over the long term, he sees the diagnostic process as a “pattern matching problem” that could be solved with AI. Doctors would focus on empathy, surgery procedures and critical decisions.

While Nabla is based in France, most of the company’s customers are in the U.S. following a rollout across Permanente Medical Group. Nabla isn’t just a work in progress, it is actively used every day by thousands of doctors.

Nabla’s privacy model

Nabla is currently available as a web app or a Google Chrome extension. The company is well aware that it is handling sensitive data. That’s why it doesn’t store audio or medical notes on its servers, unless both the doctor and the patient give their consent.

Nabla focuses on data processing instead of data storing. After a consultation, the audio file is discarded and the transcript is stored in the EHR that doctors are already using for their patient files.

In more technical terms, when a physician starts a recording, the audio is transcribed in real-time using a fine-tuned speech-to-text API. The company uses a combination of an off-the-shelf speech-to-text API from Microsoft Azure and its own speech-to-text model (a refined model based on the open-source Whisper model).

“When you have just a normal speech-to-text algorithm, they may or may not be good on medical data. But we have a fine-tuned one. And, as you probably have seen, the text is very light at first, and then it becomes dark. And when it becomes dark, it means that we verified it with our own model and we corrected it with medication names or medical conditions,” Nabla ML engineer Grégoire Retourné said during the demo that I saw.

The transcript is first pseudonymized, meaning that personally identifiable information are replaced with variables. Pseudonymized transcripts are processed by a large language model. Historically, Nabla has been using GPT-3 and then GPT-4 as its main large language model. As an enterprise customer, Nabla can tell OpenAI that it can’t store its data and train its large language model on those consultations.

But Nabla has also been playing with a fine-tuned version of Llama 2. “In the future, we envision using more and more narrow models as opposed to general models,” Lebrun said.

Once the LLM has processed the transcript, Nabla de-pseudonymizes the output. Doctors can see the note, which is stored on the computer in the local web browser storage file. Notes can be exported to EHRs.

However, doctors can give their approval and ask for the patient consent to share medical notes with Nabla so that they can be used to correct transcription errors. And given that Nabla is on track to process more than 3 million consultations per year in three languages, chances are Nabla will improve really quickly thanks to real world data.

Image Credits: Romain Dillet / TechCrunch