The following is an excerpt from The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI by Dr. Fei-Fei Li.

When you purchase products through the Bookshop.org link on this page, Science Friday earns a small commission which helps support our journalism.

Buy The Book

The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI

Buy

“So, to sum up,” I said as I clicked into the final slide of the presentation. “Occasional lapses in hand hygiene might not seem like a big deal, but they account for tens of thousands of hospital infections a year, alone, and we’re looking to help clinicians spot them as they happen with computer vision algorithms. We call it Ambient Intelligence for healthcare delivery,” I concluded. “Any questions?”

My audience of one, seated on the red couch in my office, was an especially bright double major who divided his time between computer science and statistics. He was starting his second year as a PhD candidate, and had been looking for a more permanent place to finish the remainder of his research. All three of our previous interviewees had decided not to join our team, making him our fourth attempt. I did my best to conceal the fact that our morale was getting low.

“I mean, it sounds super interesting,” he replied, his tone sincere enough. I chose to ignore the fact that he was the fourth candidate in a row to call us “super interesting.”

“What I’m wondering, though, is whether I’ll still get to publish in the usual venues. You know, NeurIPS and CVPR and stuff.”

“Absolutely,” I said with a smile. “We’re exploring a lot of unsolved problems.”

That much was true. It was 2015, and as unorthodox as the hospital setting was, the computer vision under the hood would be absolutely state of the art. We were advancing the frontier of identifying human activities, rather than static objects—already a delicate, experimental technique—and our algorithms would face the extra pressure of recognizing unusually subtle movements with high demands for accuracy. At the same time, we were taking object recognition to the next level, as our classifiers would have to contend with dense layers of movement, clutter, and ambiguity. It would be exceptionally hard work, offering plenty of opportunities to build a reputation.

“Frankly, though, we’re looking to make a real, clinical impact here. That means partnering with clinical collaborators and submitting to clinical journals, too—not just computer science.”

The student took a moment to consider it. “Okay, but, like, what’s the timeline for journals like that?”

Given how much an academic career depends on publication, especially in the early years, it was a good question. He saw the glacial pace of medical journals as an anchor, weighing him down when he needed to sprint, and he wasn’t wrong to worry. He’d be lucky if he published half as frequently as his peers. I winced inwardly as I answered.

“Honestly, I haven’t done it myself. But my partner, Dr. Arnie Milstein, says they tend to take a year or two.”

Wide eyes. Another pause.

“Wow. That’s . . . a lot longer than I expected. I mean, computer science papers normally take like a few months.”

He was stating the obvious, but he was right. There wasn’t much I could add.

“Uh, Professor Li, one last question,” he began as he folded his arms. “I know how long you spent building ImageNet, and how important it was for computer vision. Will we have access to a similar data set for this, uh, ambient intelligence idea?”

I sighed, probably too loudly.

The answer was no. Yet another “no” among so many others. No existing data sets. No known literature to base our ideas on. No labs working on similar problems to collaborate with. Although issued politely, his answer was “no” as well.

As the months wore on, our struggle to recruit even a single collaborator began to keep me up at night. I was on the precipice of what promised to be the most meaningful chapter of my career, but we wouldn’t get anywhere without help. I thought about the lonely, early days of ImageNet. They felt tame by comparison.

Today, however, I’d have the luxury of distraction. Perhaps noticing I needed a push to keep my head in the game, Arnie had sent me on a field trip.

“Are you sure this is gonna be okay?” I asked as I adjusted my mask. I’d spent so much of my life surrounded by people in scrubs, but it was the first time I’d worn them myself.

“Absolutely. We do it all the time. Nurses, med school students, graduates doing their residency, you name it. Don’t worry. You’ll blend right in.”

Arnie had arranged for me to shadow Dr. Terry Platchek, a pediatrician at Lucile Packard Children’s Hospital, so I could observe the realities of maintaining hand hygiene throughout a shift. But I wanted to see everything: the patients, the nurses, all of it. The full gamut of their experience. I knew their world was chaos, and I wanted to see it the way they do.

I had no idea what I was in for.

Christmas season had come to the general ward, and I couldn’t believe how many children were there. Each had a story, and each was heartbreaking. Some encounters were good news, some were bad, and most were just another step in a long, often numbing journey. Some parents asked who I was and why I was there. Most didn’t even seem to think twice, so used to a revolving door of faces as they attempted to understand what their loved one was going through. 

I was supposed to be keeping track of something mechanical and easily quantified—hand hygiene routines—but I couldn’t take my eyes off what I quickly understood to be the real demonstration: the human act of delivering care. A good doctor is a font of information, a source of strength, and sometimes the bedrock for patients and their families in moments of distress. Years of caring for my mother had made me confident I knew the health care space intimately, but what I saw completely upended my understanding. I was certain that no technology, no matter how advanced, could replace what I witnessed that day.

Nevertheless, I learned that in certain, decisive moments, new tools were sorely needed. I met a veteran nurse who recently had a patient fall, the first of her career, and I was surprised by how deeply it affected her. It was a statistical inevitability that someone would get hurt on her watch at some point—she’d been a nurse for decades, after all—but when the moment finally came, her lifetime of distinguished performance didn’t seem to make a difference to how she felt about it. She was as emotionally devastated as she would have been had it happened on her first day. If AI could help avoid this—two people profoundly wounded—it seemed more than worth the effort.

As physically demanding as the day was, my emotional exhaustion overshadowed whatever fatigue my body felt at the end of the shift. It was as if I’d watched every moment I’d faced with my mother—but playing on a loop, hour after hour. Dazed, I shook hands with my host and exchanged pleasantries as I prepared to leave. But something occurred to me on the way out.

“Terry, I’m curious about something. What made you so willing to let me into your world? I mean, let’s be honest—I’m a bit of an outsider.”

He thought for a moment before responding.

“You know, there’s a lot of talk in the news lately about AI, and frankly, I don’t like most of it.”

I smiled, perhaps cynically. I knew where this was going.

“Sure, it’d be great to automate more of my day. Whatever. I get it,” he continued. “But I’m a little tired of tech executives talking about putting people like me out of a job. You and Arnie are the only ones who actually seem to want to help me, rather than replace me.”

I considered my own response for a moment as well.

“I know we’ve talked a bit about my mother, and how my experience with her health over the years has influenced me,” I said. “But there’s another side to that story. In all the time I’ve spent in rooms like this, there’s been one silver lining.”

“What’s that?”

“There’s something special about . . . I dunno, I guess you’d call it the act of delivering care, whether it’s a nurse helping my mother sit up or a specialist outlining a treatment strategy. It’s just so human—maybe the most human thing we’re capable of, you know? It’s not just that I can’t imagine AI ever replacing that—I wouldn’t even want it to. I appreciate the role technology is playing in keeping us all alive these days, but it’s not an exaggeration to say that the real reason my mother and I have made it through it all is people. People like you.”

The sun had set during our shift, and I emerged from the hospital to the brisk air of an evening well under way. The relative quiet was all the license my thoughts needed to unravel, the day’s memories replaying with a dull tingle. But as harrowing as it’d been, Arnie was right—this was exactly what I needed. It was an education that no degree in computer science could offer: the bustle of the ward, the pleading looks of uncertainty, the desperation for comfort in any form. Sore feet and worn tennis shoes. Cold pizza in the breakroom. Hour after grinding hour. Arnie knew that for all my years of experience at my mother’s side, I still had no idea what it was like to be a clinician. So he’d invited me to see it for myself.

A strange thought occurred to me as I made my way home, the sky now nearly black: I was glad we hadn’t recruited any students yet. I’d have inundated them with a computer scientist’s reading list, conditioning them to think in terms of data, neural networks, and the latest architectural advances. That was important, of course—there was no dodging the science on a project like this. But now I knew that wasn’t the right place to start. If AI was going to help people, our thinking had to begin with the people themselves.

A decision was made, instantly. From that day forward, no prospective member of our team would write a single line of code until they’d had the experience I just had. Shadowing would be every new recruit’s initiation. And it would be non-negotiable.

EXCERPTED FROM THE WORLDS I SEE. COPYRIGHT © 2023 BY DR. FEI-FEI LI. EXCERPTED BY PERMISSION OF MOMENT OF LIFT BOOKS, AN IMPRINT OF FLATIRON BOOKS, A DIVISION OF MACMILLAN PUBLISHERS. NO PART OF THIS EXCERPT MAY BE REPRODUCED OR REPRINTED WITHOUT PERMISSION IN WRITING FROM THE PUBLISHER.