What does Google's AMIE actually do during a video consultation?

The system that sees the patient

AMIE stands for Articulate Medical Intelligence Explorer. It is a research system, not a product you can download. Google Research and Google DeepMind announced the latest version on August 11, 2026. The announcement describes a first-of-its-kind demonstration of expert-level AI capabilities in real-time clinical video consultations.

A consultation extends far beyond words. A physician notices a cough, observes gait, or registers visible signs of discomfort. AMIE now interprets visual and auditory cues, guides virtual physical exams, and reasons diagnostically in real time. The system runs on Gemini and Project Astra using a multi-agent architecture.

How the multi-agent architecture works

Gemini provides the large language model backbone. Project Astra contributes real-time multimodal understanding. Together they let AMIE process video and audio streams simultaneously. The system does not simply transcribe speech. It tracks facial expression, posture, and movement. It can ask a patient to raise an arm or turn their head. It integrates those observations into a running diagnostic hypothesis.

The architecture uses multiple specialized agents. One agent handles history-taking. Another manages the visual examination. A third synthesizes findings into a differential diagnosis. A fourth formulates the management plan. They communicate with each other during the consultation. The announcement does not specify how many agents or their exact boundaries.

What the randomized study measured

Researchers ran a randomized study using simulated consultations. Patient actors portrayed medical cases. A group of primary care physicians conducted the same consultations for comparison. Clinical evaluators assessed both groups across core clinical competencies.

Evaluators rated history-taking thoroughness. They rated diagnostic accuracy. They rated management appropriateness. They rated communication quality. AMIE was assessed favorably across all four domains. Patient actors also preferred the video experience over text chat. The announcement does not give numerical scores, confidence intervals, or the number of cases tested.

Where research stops and clinical deployment begins

The system remains a research system. More research is needed before responsible real-world clinical deployment. The announcement does not define what that research entails. It does not name a timeline. It does not mention regulatory pathways. It does not describe safety monitoring for live patients.

The study used simulated consultations with actors. Real patients present messier histories, poorer lighting, and unstable connections. The announcement does not address how AMIE handles those variables. It does not discuss privacy architecture for video streams. It does not explain how a clinician would override a wrong suggestion in real time.

The glimpse is exciting. The gap between simulation and clinic remains wide.