Google’s AMIE Matches Doctors in Video Consultations (In a Lab)

Google has published research showing its AMIE medical AI system conducted real-time video consultations with professional patient actors and received ratings comparable to primary care physicians on history-taking, diagnostic accuracy, management quality, and communication. The study used a multi-arm randomised format: AMIE (Video), a text-only AMIE version, and ten board-certified physicians all completed consultations, which were then reviewed by a panel of 20 independent physicians. Google is explicit that real-patient trials are required before any clinical conclusions can be drawn.

What happened

Detail	Value
System tested	AMIE (Video), a multi-agent medical AI
Patient actors	15 trained professionals
Body systems covered	Cardiopulmonary, abdominal, HEENT, neurological/psychiatric, musculoskeletal
Physician comparators	10 board-certified primary care physicians (same video interface)
Independent reviewers	20 experienced primary care physicians
Evaluation format	Multi-arm randomised study with standardised scenarios

Google’s AMIE research system has been extended from text-based consultations to live video, using an architecture that splits work across three separate agents rather than asking one model to do everything at once. According to Google, a single model cannot sustain natural conversational response times while simultaneously reasoning clinically and processing audio and video input.

The three agents each carry a distinct role. The talker agent manages the spoken conversation with the patient, aiming to keep the dialogue flowing. The planner agent runs in the background, updating differential diagnoses and treatment plans as the conversation develops, flagging missing information and adjusting clinical priorities. The perception agent continuously monitors the video and audio streams for non-verbal cues, physical findings, and auditory signals, then feeds those observations into the clinical context.

This separation is largely a latency fix. Deep reasoning takes time, and pauses during a medical consultation can damage the patient-clinician relationship. By letting the talker respond independently of the slower background processes, Google reduces the lag a patient would notice.

How the study was structured

The human evaluation used a multi-arm randomised design. AMIE completed synchronous video consultations. A text-only version of AMIE served as a modality baseline. Ten board-certified primary care physicians used the same video interface. An independent panel of 20 experienced physicians then reviewed every consultation using established clinical rubrics, first assessing general clinical competence, then applying scenario-specific criteria for each case.

According to Google, the independent panel rated AMIE (Video) on par with the physician group for:

• History-taking thoroughness
• Diagnostic accuracy
• Management appropriateness
• Communication quality

AMIE (Video) matched or exceeded AMIE (Text) across all four of those measures. The video system also scored higher than both the physician group and the text-only system on eliciting physical signs and proactively guiding actors through virtual examination steps. Patient actors preferred the video interface to text chat and rated AMIE favourably on empathy, rapport, and confidence in care.

Before the actor-based study, Google ran an automated evaluation suite to develop the system. This covered defined perceptual tasks such as identifying anatomical laterality and signs of respiratory distress, as well as multi-turn simulated audio consultations. In some scenarios, visual information was fed in as text descriptions rather than live video, which Google acknowledges does not replicate a real end-to-end video feed.

What are the real limits of this study?

Google lists several openly. Professional actors cannot reproduce the full variability of real patient encounters. The scenarios excluded presentations that actors could not portray authentically, which may happen to be cases where audio-visual perception matters most diagnostically. Automated evaluations found occasional perception and reasoning errors. Technical issues intermittently disrupted conversational naturalness.

Most importantly: no real patients were involved. Google states that studies with actual patients and their own health conditions must happen before clinical use conclusions are warranted. Some related work is already underway with the text-based AMIE in clinical settings, including a feasibility study with Beth Israel Deaconess Medical Center.

The split between automated testing and actor-based consultations is worth keeping in mind for anyone thinking about AI in healthcare procurement or governance. Automated assessments can test defined tasks at scale. Simulated video consultations can assess interaction quality under controlled conditions. Neither tells you how the system performs when a patient has an unusual history, poor connectivity, or symptoms outside a prepared script.

Why it matters

This is one of the more rigorous published evaluations of a medical AI conducting real-time video consultations. The multi-arm randomised design with independent physician reviewers is a step above most self-reported AI benchmarks, and the three-agent architecture addresses a genuine engineering constraint that anyone building AI into live communication workflows will recognise.

For businesses, the relevant signal is not “AI is replacing doctors.” It is that AI systems are being seriously tested for synchronous, multi-modal (audio plus video) interactions, and the evaluation frameworks being built here will shape how clinical AI gets regulated and procured over the next few years. For teams building AI-driven support tools or automated workflows, the latency-splitting architecture Google describes is a pattern worth understanding. We have written about similar agentic approaches in the context of AI-driven software development workflows, where splitting reasoning and output tasks is becoming a standard design move.

If you are working on AI integration for client-facing applications, the core lesson here is that conversational naturalness and deep reasoning are currently in tension, and architecture choices matter. Our AI integration work regularly runs into this trade-off.

Our take

The headline finding is genuine: an AI system rated alongside physicians by an independent panel of 20 doctors is not nothing. But “on par in a controlled study with actors” is a long way from “ready for clinical use,” and Google says so directly.

The more durable contribution here is probably the evaluation framework itself. A standardised rubric covering telehealth competencies, physical sign elicitation, and latency is the kind of infrastructure that makes future comparisons meaningful. That matters more than the performance numbers from a single study.

The three-agent architecture is also the honest engineering answer to a real problem. Rather than claiming one model can do everything, Google has built a system that acknowledges trade-offs and routes work accordingly. That is a more credible foundation than a single monolithic model trying to be fluent and deeply analytical at the same time.

Watch for the real-patient feasibility results from Beth Israel Deaconess. That data will be far more informative than anything produced with actors and prepared scenarios.

Source: AI News (artificialintelligence-news.com)