Musings on healthcare IT, FHIR, EHR systems, and digital health innovation

🔥 FHIR in the hole!
Back to posts

Fun demo for Friday afternoon: voice agents! Check out gpt-live-1 in a…

Original post on LinkedIn

Fun demo for Friday afternoon: voice agents! Check out gpt-live-1 in a medication adherence check-in scenario. This should give you a sense of how the “live” model coordinates with an underlying reasoning model (through explicit delegation that’s… not quite tool calling) and how thoughts/suggestions can be injected into live conversation. It’s far from flawless but it still beats the pants off anything in my “real life consumer” experience.

Video
Watch on YouTube

Conversation on LinkedIn

15 comments · 56 reactionsReply on LinkedIn
  • William LaolagiSep 18, 2026

    Interoperability specialist serving the state of California in Riverside and San Bernardino counties

    this is really cool Josh!

  • Paulius Mui, MDSep 18, 2026

    Care quality and safety

    Love the visual of the handoffs between tools 💪

    1 reaction

  • CEO, Managing Director, Board Member, Expert in bringing transformational products to market

    Sounds good, how well does it handle api calls for it ms like ins verification, npi look up or pharmacy? Would love to talk through it some time

    • Josh Mandel, MDAuthorSep 19, 2026

      This is all done through "delegation" -- so gpt-live-1 basically mutters "help me out", and you have some underlying model watching for these delegation requests, turning them into tool calls / info retrieval / processing, and whispering back in gpt-live-1's ear to guide the conversation with resulst.

  • Shannon ManziSep 19, 2026

    PharmD, BCPPS, CPHRM, FPPA Assistant Professor, Pediatrics at Harvard; Director, Safety & Quality, Dept of Pharmacy Boston Children’s HospItal

    Hello my friend! A+ for pronunciation of amlodipine, C- for not warning to keep it out of reach if there are children in the home. Interesting flow, unsure behavior would change but certainly worth study!

    • Josh Mandel, MDAuthorSep 19, 2026 · edited

      Hi Shannon!

      Super good points.

      Re: behavior, I don't expect the sort of *in-band counseling* I show here is super effective, but the *information gathering* could be a useful input to new processes (i.e. knowing the situation could help providers decide when to step in an assist).

      On pronunciation + safety precautions : well, my prompt included a pronunciation guide and lacked safety precautions, so I can credit or blame the models on those :)

      1 reaction

  • David K.Sep 19, 2026

    Toddler dad working on better work/life balance at early-stage healthcare companies. /s

    Definitely a lot to be excited about here. Thanks for sharing!

  • Co-Founder at HeraIQ

    Dr Mandel, decoupling low-latency conversational audio from an underlying reasoning model via trusted sideband delegation is a brilliant pattern, because it allows clinical logic and stop gates to intervene dynamically without breaking the live patient flow.

  • Dr. Nadeem AhmedSep 19, 2026

    Forbes 30u30 | Ex-McKinsey | Physician building the AI evidence layer for Longevity

    Building the thing rather than describing it is why this is worth reading. Also a fair point that the bar set by actual adherence outreach calls is not a high one.

  • Doug RogersSep 19, 2026

    TOM Evangelist | Digital Health, Strategy & GTM Leadership | Founder & Host of “The Counterweight” Podcast | Board Member

    This is pretty cool. The tone and cadence seemed pretty natural throughout. Curious what would be the trigger for this Agent to do the call? Someone in the practice asking if or keying off something in the chart?

    • Josh Mandel, MDAuthorSep 19, 2026 · edited

      Many possible triggers (e.g., routine outreach, or follow-up triggered by a concerning pattern of adherence from pharmacy, pickup data, clinical measurements,  etc)

      1 reaction

  • Founder & CEO @ iTech Clinical Intelligence | Radiology | Medicine | Computer Science | Data Science | AI

    Josh, interesting, and thank you. I wasn’t initially a fan of GPT-Live 1 relative to GPT-Realtime, but based on your demonstration, I’ll definitely reconsider it.

    We experiment with these models while developing conversational workflows for patient appointments, follow-ups, and related use cases. We’ve found that comfort with talking to computers varies by generation, particularly when the voice is perceived as artificial. Prosody and conversational timing matter.

    I like the direction OpenAI is taking, and I still consider its integration of NLP and signal processing the strongest on the market. Of course, that can always change, so it’s important to remain flexible.

  • Benjamin KrasinSep 20, 2026

    Founder, Krasyn. Building AI for clinicians and learners, and testing it in public.

    In a medication check-in the quiet failure sits upstream of the reasoning. Benchmarking 3 speech services on our own clinical audio last week, the one we had in production dropped the "no more than" out of an acetaminophen dose limit, and another truncated a 30-second chunk of continuous speech down to 89 characters. A live model can then coordinate flawlessly over a sentence the patient never said.

    • Josh Mandel, MDAuthorSep 21, 2026

      What models?

    • Benjamin KrasinSep 23, 2026

      Founder, Krasyn. Building AI for clinicians and learners, and testing it in public.

      Josh Mandel, MD Whisper on Azure OpenAI was the one in production and dropped the "no more than". gpt-4o-transcribe did the truncating, and giving it the previous chunk as a prompt made recall worse, not better. Azure AI Speech fast transcription kept all 72 negated statements across 13 synthetic visits, all fed as 30-second chunks, which is not how a live voice agent hears audio.