How Well Do FDA-Cleared AI Tools Actually Detect Pulmonary Embolism in Real Hospitals?

The platform and the two algorithms

Aidoc Medical BriefCase supplies both algorithms.

One triages dedicated CT pulmonary angiography studies. The other scans routine contrast-enhanced CTs for incidental pulmonary embolism. Researchers pulled 30,678 CTPA exams and 37,191 routine contrast-enhanced CTs from the 17-facility network. Reference labels came from radiology reports parsed by a validated LLM pipeline. That pipeline reached 97% accuracy with a kappa of 0.94.

PE triage model results

The PE model posted 86.8% sensitivity and 99.1% specificity.

Sensitivity dropped sharply as clots moved outward. Saddle emboli hit 99.3%. Subsegmental PE fell to 72.9%. Acuity mattered too. Acute PE reached 89.7%. Non-acute PE landed at 65.3%. Both declines mirror known human reader limits. The paper notes the model still exceeded its cleared specificity threshold. Sensitivity came in below the FDA-clearance benchmark.

Incidental PE model results

The iPE model returned 73.5% sensitivity and 99.8% specificity. Again specificity surpassed the cleared mark. Again sensitivity lagged behind the clearance benchmark. The study does not break down iPE performance by clot location or acuity.

What the gap means for post-market surveillance

Both algorithms underperformed their clearance sensitivity targets while beating specificity targets. The pattern repeats for peripheral and non-acute emboli. That pattern matches human limitations. The authors argue this underscores the need for standardized post-market surveillance of AI-enabled medical devices. No regulatory framework for such surveillance exists today. The study offers no cost data, no workflow integration metrics, and no patient outcome follow-up.