Medical scribes track doctors’ examinations from thousands of miles away

PALO ALTO, Calif. – Jim Andrews is in a medical office wearing just a hospital gown, staring at his doctor of 11 years, who is staring back at him through the sleek, metallic lens of Google Glass.

As the doctor examines Andrews, a new kind of medical scribe is watching the examination, transcribing everything he sees. The scribe, named Rahul, is thousands of miles away in India, and he is viewing the office visit live through the tiny, Wi-Fi-connected camera attached to the doctor’s glasses.

“When was his last physical?” the doctor, Albert Chan, asks as he listens to Andrews’ breathing and checks his reflexes. Rahul’s nearly immediate answer pops up in a text bubble display in the right corner of the doctor’s field of vision. “June 3, 2014!”

The entrepreneurs behind the technology – which could one day morph into something as tiny as a contact lens – say it is more than just a transcription tool: It’s the first step to supercharging doctors with instant information that transforms how medical decisions are made. Instead of human scribes taking down notes, they envision a world in which artificial intelligence software could transcribe the office visit in real-time, while immediately comparing the patient’s medical issues with those of millions of others, then making predictions about what treatments would work best. Moreover, they say the technology is bringing health care professionals back into the moment with their patients – returning a sense of humanity that has been lost as computers have become a fixture in the doctor’s office.

Yet, like much new digital technology in health care, Google Glass has the potential to create more problems than it solves. Medical scribing is a tough job that requires a high level of accuracy – note takers in far-off lands need to know terms such as “eosinophilic esophagitis” – and getting a word wrong could have grave consequences.

Health systems are also far more vulnerable to cyberattacks than other industries, such as banking, a gap hackers have aggressively exploited in recent years, said Avi Rubin, director of the Health and Medical Security Lab at Johns Hopkins University. In recent years, hundreds of hospitals have been victims of so-called ransomware attacks, in which outsiders block the hospital from accessing its own data until the hospital pays them money.

Rubin called hospitals prime targets because of a “perfect storm” of weak security systems, the increasing digitization of data, and the fact that medical records can be so time-sensitive and critical to making life-or-death decisions. A live-streaming video of a patient exam with a doctor could be most sensitive of all.

“You’re taking something that was in the doctor’s office and now you’re streaming it across the world,” Rubin said. “The whole health-care industry has lax security, and if you combine that with how private the information is and how it needs to be immediately available, that’s where I have concerns.”

Google Glass is partly a response to the last major technology introduced into the doctor’s office: electronic health records have received billions in federal funding and been hailed as a major step forward for the medical field. Doctors now say they are turning to Google Glass so they can escape the burden of spending an entire patient visit - and several hours later that night - typing information into forms.

“I can look my patients in the eye again,” said Teresa Nauenberg, a primary care physician who works with Chan at Palo Alto Medical Foundation and has been using the technology for two years.

Today, roughly 500 doctors in 27 states put on Google Glass. They wear them through the day, live-streaming their office visits to the virtual scribes, who look on and take detailed notes.

Ian Shakil, the 32-year-old chief executive of Augmedix, the San Francisco startup that has partnered with search-engine giant Google to distribute the technology, hopes that in the near future, seeing a doctor wear Google Glass will feel as normal as seeing one wear a stethoscope. Augmedix has raised more than $36 million from venture capitalists and health systems, including Dignity and Sutter Health, which owns Palo Alto Medical Foundation. The start-up charges doctors between $1,500 and $4,000 per month for the service, which Sutter says saves doctors up to two hours a day.

Scribes receive about three months of training from Augmedix before they can do the work, which is performed in sealed-off rooms. The company’s 200 scribes, divided between San Francisco and Bangalore, must leave pens, paper, smartphones, and all other belongings at the door.

For Shakil, accuracy is critical. Beyond extensive training of scribes, he says that doctors must ultimately review the notes to make sure all details were recorded accurately.

If Augmedix is successful, it would also be a huge coup for Google, which has had a troubled history with Glass and health data.

Glass emerged in 2012 as a pet project of Google co-founder Sergey Brin. Brin wanted to sell the $1,500-a-pair glasses as a high-end fashion item, and went so far as to partner with designer Diane Von Furstenberg. But the product fell flat.

Even in a society that is always recording, the glasses felt too much like a privacy violation – or just a gimmicky party trick. Google discontinued its limited consumer sales last year and retooled the devices. (Snap, the company behind Snapchat, last week launched sunglasses with an embedded camera that can create 10-second videos.)

Augmedix is now one of nine Google partners, which include startups using Glass to enhance museum tours, record surgeries, and train workers doing equipment repair.

In 2013, Google shuttered its personal health record platform, Google Health, because so few people were using it. Today, the search giant is competing with Amazon (whose chief executive, Jeffrey Bezos, owns the Washington Post), Microsoft and others for hospital customers to store data on its cloud servers. Many hospitals are wary of tech giants storing their data. Augmedix does not share any data with Google.

Google declined to comment.

Like many entrepreneurs, Augmedix’s founders say they believe in the Silicon Valley vision of better technology liberating people from older technology’s burdens. Yet in health, the record has not been so clear.

Electronic medical records were supposed to make the medical system more efficient – reducing errors and enabling doctors to track a patient’s history. But their implementation has been rife with unintended consequences, said Robert Wachter, interim chairman of the Department of Medicine at the University of California, San Francisco, and the author of a 2015 book on the subject.

A 2014 study by the American Medical Association and RAND Corp. found that digital records were a major contributor to physician dissatisfaction. Another article, in the journal Health Affairs, found that two-thirds of a primary care physician’s day was spent on clerical work. There’s also some evidence that electronic health records have increased the practice of overbilling Medicare by stuffing digital data fields with unnecessary procedural codes.

“You see rates of physician burnout skyrocketing, and many blame it on the electronic record,” said Wachter.

Augmedix’s solution could usher in unintended consequences of its own.

The startup takes extensive precautions in protecting data. The video, which is scrambled, is sent directly from Google Glass to the scribes. It doesn’t pass through hospital servers, which are often the vulnerable point that hackers exploit. The video is deleted after each medical visit. But the company is exploring ways to expand its business to make the video available to doctors who want to review it later.

The latter is a riskier proposition, Rubin said. “Hackers look for the weakest link in the system.”

Few patients seem concerned. “It’s a conflict, there’s no doubt about it,” Chan’s patient, Andrews, said of the privacy issues as he got ready to head back to work from his appointment. “How I’ve rationalized it is, if this can help my doctors provide me with better medical care at this stage of my life, I’m fine with it. If someone finds out I have an artificial knee, then so be it.”

Andrews, who is 61, noted that Chan seemed “more relaxed” ever since he started wearing the device.

Chan says that in his two years wearing Google Glass, only a fraction of patients – just 4 percent – have asked him to remove it. Many forget it’s even there, he said.

Augmedix says under 2 percent of patients ask their doctor to take off the lenses. Other doctors said they take their own measures to protect privacy – flipping the glasses over their foreheads, for example, when they are examining private parts.

Stuart Peterson, senior partner with the venture capital firm Artis Ventures, said he felt uncomfortable after his visit.

“You don’t know who is on the other side of it. It’s a scary thought,” he said.

Augmedix’s capacity to become a wider solution to entrenched problems in the health care system will depend, for now, on the startup’s ability to find and train low-cost scribes.

For decades, physicians have dictated their notes into tape recorders; paid transcriptionists converted them into written reports. Today, the growing use of medical records has created a demand for human scribes, who largely tag along full-time with doctors. Shakil has spent the better part of the past several years refining the training process; Augmedix’s San Francisco scribes – mostly 20-somethings looking to get experience before medical school – are now trainers for people like Rahul.

Like many startup founders, Shakil is planning for a future in which the work done by humans is outsourced to machine. In recent years, artificial intelligence software such as Apple’s Siri have made big strides in understanding human speech. Medical transcription, however, presents far bigger challenges than asking Siri what the weather is. Still, he’s confident it will happen.

“There’s not enough scribes in all of South Asia to serve the nearly million doctors here,” Shakil says. “Uber has driverless cars. One day, we’ll have scribe-less scribing.”