Meet the American spies who helped mammograms save more lives | Fortune

So I was taken aback when a retired CIA officer I know recently told me a fascinating fact:  Modern mammography was invented with help from American spies. Or, more precisely, by people who do the lab work for spies, technologists inside an intelligence agency so secret the U.S. government didn’t even admit it existed until 1992. 

The surprising origin of computer-aided mammography is a particularly high-stakes example of how government tech spending has shaped private-sector businesses. In this case, it helped launch an $11-billion-a-year medical industry and changed the lives of millions of American families—with most of them never knowing about the connection with Uncle Sam.

Hunting for breakthroughs

The story begins in 1994, when a public health doctor at the Department of Health and Human Services, Susan Blumenthal, went hunting for breast cancer breakthroughs in Washington, D.C. The search was personal for Blumenthal, who’d lost her mother to breast cancer just before becoming a doctor.

“In my last year of medical school, it metastasized to her spine,” Blumenthal told Fortune recently. “This beautiful, brilliant woman could no longer walk. Metastasis was such a brutal way to die.  And so I vowed then and there that no other woman should have to suffer the way she did.” 

CQ Roll Call—Getty Images

As an assistant Surgeon General, Blumenthal knew the U.S. government had some of the world’s most advanced digital imaging technology. But at the time, mammography was a 40-year-old procedure that remained stubbornly analog—with doctors peering at sheets of X-ray film through jeweler’s loupes. It seemed far from the state-of-the-art, computerized stuff Blumenthal kept hearing about from other parts of the federal government. As she told me, “We could see the surface of Mars, we can track missiles in outer space, why couldn’t we find small tumors right here on Earth?”

So Blumenthal called up the then-director of the Central Intelligence Agency, James Woolsey, and asked for help taking on a problem that was killing some 45,000 Americans per year. (The fact that as a federal public-health officer she technically held a military rank likely weighed in favor of the CIA director taking the call). 

Woolsey recalled the conversation in an oral history of the Clinton presidency: “Jim, this is a strange request,” he recalled Blumenthal saying, but could the intelligence community “be of any assistance in improving diagnosis of mammograms?’  

“It sounds kind of unlikely to me,” Woolsey told her, “but maybe not impossible.’”

The sharpest eyes at the Pentagon

Mammograms, in essence, were just pictures that sharp-eyed experts scrutinized for hidden trouble. So Woolsey called up Jeffrey Harris, head of an intelligence agency full of some of the sharpest-eyed experts at the Pentagon: the National Reconnaissance Office. The NRO builds America’s fleet of spy satellites and studies the pictures they take. The U.S. government had officially admitted the agency’s existence only two years prior. 

Alex Wong—Getty Images

Harris—the first NRO director allowed to publicly admit he held the job—oversaw teams that pioneered some of the earliest software and hardware for processing digital images, like the chunky, steampunk-ish IDEX workstation on display at the Smithsonian.

 ”The intelligence community over time spent probably a couple hundred million dollars in several generations inventing [the technology] and getting it fielded,” Harris told Fortune. “We were inventing the mathematics to manipulate digital imagery.”

After his own conversation with Blumenthal, Harris recalled, he decided, “ Let’s just go do a little study on behalf of the women of America to see if the algorithms that we’re perfecting to find Soviet / Russian deployed missile fields work for mammograms.”

Harris had some guys in a lab who thought they could help. One of them was a physicist and image researcher named Sam Grant—who had some adjusting to do when handed an assignment he was actually supposed to talk about at home.

“You work with a group of people who work for an organization that doesn’t exist,” Grant told me, laughing, “You get into that culture.”

Grant’s first job was to enlist his spouse to help him learn how the images got made. He was immediately taken aback to learn that the process required first squashing the breast flat. “ I went with my wife for a mammogram and saw how they collected the data and I thought, my goodness,” he said. “I mean they put her breast in a vice, essentially.”

The NRO team also got help from researchers from Massachusetts General Hospital in Boston, who shared some 200,000 mammogram images, many of which were known positives for cancer. The NRO researchers scanned and digitized these x-ray films, then ran them through an early version of artificial intelligence—a neural network that had been trained to search for missile launchers in digitized satellite photos of forested regions in Russia. 

“ The algorithm gets trained by humans and says, ‘I’m seeing an interesting pattern in the middle of a primordial forest—is this of interest?’” Harris said.

One interesting pattern the algorithm had picked up on was that sharp-edged objects like missile launchers almost always could be found along roadways, which showed up as long, linear features in the satellite images. Sharp edges and roadways helped the system locate its quarry even when partially obscured by tree cover.

 ”That was something that the AI could latch onto,” Harris said.

From missiles to microcalcifications

As it turned out, the method translated well to mammograms. In early stages, breast cancer can sometimes show up as microcalcifications, tiny deposits of calcium in breast tissue. Not all microcalcifications turn out to be cancerous, but cancer is much more strongly associated with deposits that cluster along a distinctly linear feature inside the breast: milk ducts.

“ What excited the algorithm was, the duct work looked like dirt roads through a forest,” Harris said.

The NRO algorithm helped find calcium deposits that were much more likely to be cancerous—which cut down on false positives that would require women to undergo needless biopsies or follow-up smashing in the mammogram machine.

While this early version of AI had a different architecture than, say, the Chat GPT app on your phone, it shared some of the same mathematical DNA: neural nets trained to find patterns in high-dimensional data. And much like modern, computer-aided mammography, today’s large language models got a boost from U.S. government funding, which helped keep the underlying technology going when commercial interest wavered. For example, consider the 2010 NSF grant that helped ImageNet spark the deep-learning boom, or the $25 million DARPA contract that same year, which buoyed Nvidia at a moment when non-gaming uses for GPUs looked like a non-starter.

Far beyond basic research, Washington has long played fairy godmother to the commercial technologies we use every day—from the Small Business Administration-backed investment that helped launch Apple to the government spy tech that put GPS and Google Earth in your phone. 

These technologies grow out of what economist Mariana Mazzucato calls “the entrepreneurial state”—the often-forgotten government funding that drives innovation throughout the lifecycle of technology, far beyond basic research and development.

“What actually got us all the technology that makes the iPhone smart and not stupid was government finance.” Mazzucato told Fortune. “If you really think government should have just done basic R&D, throw out your iPhones!”  

Taking mammograms from analog to digital

Or, perhaps, throw out your digital mammogram. Digital mammography and computer-aided breast cancer diagnosis were in their infancy in the 1990s. The task force of cancer doctors and intelligence analysts that Blumenthal assembled helped to change that. 

When researchers at Mass General piloted a system employing a version of the NRO’s spy photo algorithm as a “second reader” alongside human doctors, Grant said, “ They saw 15% reduction in false positives.” The takeaway was clear: The combination of digital imagery and a pattern-recognition algorithm could make mammograms more accurate. 

Blumenthal says this breakthrough and others helped pave the way for the huge, and life-saving, digital mammography and computer-aided diagnosis industry we know today.  Now more than 34 million U.S. women get annual mammograms and they—or their insurance companies—spend more than $11 billion yearly for the privilege. Perhaps more importantly, since 1989, one study estimated that mammograms have saved as many as 600,000 lives.

That’s likely high on the list of reasons why CIA Director Woolsey mentioned the program—which became known as Missiles to Mammograms—when tallying his achievements  with presidential historians in 2010. And it’s the reason Sam Grant, who spent most of his career never breathing a word about his NRO work to anyone, decided to speak with Fortune. He said, “ I’ve been proud of it ever since.”