Stanford University: ‘Black boxes’ in Stanford Hospital operating rooms aid training and safety

For decades, the airline industry has benefited from so-called black boxes, which record flight data and cockpit conversations. The collected information allows airlines to boost safety by predicting and mitigating risks.

Now, similar technology is moving into another high-stakes arena: the operating room. The new technology, called OR Black Box, has been installed in four operating rooms at Stanford Hospital — the first on the West Coast — with plans to expand to several more rooms.

These sophisticated monitoring systems capture everything that goes on in the operating room, from human performance to distractions, patient vital signs to equipment malfunctions — all factors that might affect the outcome of a procedure.

“I was inspired by aviation and what made aviation the safest form of transportation,” said Teodor Grantcharov, MD, PhD, the inventor of the OR Black Box, who recently joined Stanford Medicine as a professor of surgery.

“Our patients are not aircraft, but a lot of the methods — the culture, the approach to safety, the never-ending pursuit of ‘safer’ — is transferable,” he said.

The information collected by the monitoring system becomes a rich resource from which new protocol solutions can emerge. When quality, safety and operations teams at Stanford Hospital notice potential problems in efficiency, collaboration or safety, they can turn to the data to pinpoint factors that can be improved.

For instance, nurses might note an inefficiency when some of the surgical instruments they are asked to bring into the operating room are rarely used. A review of the data could reveal how often each tool was used in the previous 12 months and help reduce the number of instruments unnecessarily transported and sterilized.

“We can’t get to precise quality improvements without high-quality information,” Grantcharov said, explaining that the new system generates that high-quality information.

Differences in performance
In the 1990s, when Grantcharov was a surgical resident in Copenhagen, he noted a variability in skill levels and surgical outcomes among his surgical colleagues. So he developed a system that could evaluate surgeons as they performed virtual reality operations, and confirmed that wide disparities existed.

He hoped then to develop a way to put the system to work in the real world, but it would be years before technology made that possible.

In 2006, Grantcharov moved to the University of Toronto and began working with a group of engineers, designers and researchers on a black box for the OR. Within a few years, they installed their first prototype in a hospital in downtown Toronto.

“We started seeing things that we’d never seen before,” he said. “Things that mattered to our performance, like how we communicate, how we treat each other in the operating room, what distracts us, what kind of technology we use.”

The more data they collected, the more opportunities they found for improvement. Unlike an airplane black box, the purpose of this technology is not so much to replay adverse events as it is to understand all the factors that could lead to harm and to minimize those risks, Grantcharov said.

One of the surprising initial findings from the system, now installed in about a dozen hospitals around the country, is how much distraction there is in a modern operating room, Grantcharov said — constant opening and closing of doors, machine alarms, questions and side conversations. But once noted, they can be mitigated.

Objective metrics
The black box isn’t a physical box, but a system of sensors and data recorders that blend into the operating room. “You don’t see it,” he said. “We intentionally designed it to not be distracting nor require human input. It’s an autonomous system in the background that’s running all the time.”

The multiple streams of information collected by the system — including audiovisual data about the surgery and the environment, physiological signs from the patient and equipment performance — are synchronized and analyzed by artificial intelligence.

A surgeon’s execution of a particular surgical technique, for example, can be assessed by algorithms trained on years of real data, said Mary Hawn, MD, chair of the Department of Surgery and the Emile Holman Professor in Surgery.

Communication among the surgical team, which is often overlooked in quality improvement, can also be analyzed using clinically trained AI. “Is the anesthesiologist alerting the surgeon that the patient’s vital signs are changing? Is the surgeon alerting the anesthesiologist that they’re having more bleeding than they would typically expect in this case?” Hawn said.

Other data can reveal ways to increase efficiency, reduce environmental impact, increase compliance, and improve teaching and mentoring in the operating room.

For example, the technology could flag an unusually long turnover time between surgical procedures and help identify reasons for the delay: perhaps cleaning the room or bringing in new equipment is taking too long, or there is poor communication about when the room is ready.

“These are data we just never have had before to actually understand what’s going on,” Hawn said.

The system can also monitor and improve compliance to safety protocols. Use of the now widely adopted surgical safety checklist, promoted by the World Health Organization, has helped reduce errors and adverse events in the operating room, but studies using black box data reveal there is room for improvement: Clinicians often overestimate their own compliance, Grantcharov said.

Culture of safety
The purpose of the technology is to promote a culture of safety and constant improvement, not to cast blame, Hawn emphasized. Faces are blurred and voices are distorted in videos to ensure privacy and confidentiality of both staff and patients, and the recordings are deleted after 30 days.

Stanford Medicine is customizing the feedback its system is generating, said Sam Wald, MD, vice president of perioperative services at Stanford Health Care, who is leading the rollout. “The feedback will be presented as a theme focusing on the system rather than the individual,” he said. A black box working group meets weekly to review reports and identify opportunities for improvement.

Stanford Hospital has also joined the Surgical Safety Network, a consortium of hospitals that use OR Black Box data to benchmark and share best practices.

“We know that comparing progress internally can take us only so far as an organization,” Wald said. “Gaining insights from others and using shared data for quality and performance improvement will take us from a linear path to an exponential one.”

When Grantcharov tells patients about the technology, they’re usually surprised that it’s not already being used.

“Stanford Medicine is already a data-driven, well-functioning machine, so we’re just adding another layer and another opportunity to keep improving,” Grantcharov said.

In five or 10 years, he said, “Nobody will imagine a modern operating room without a black box.”