A picture tells a thousand words:

Prof. Sandy Chen’s big image data plus machine learning
allow clinicians to skip biopsies and predict outcomes

Cheng Yu Chen

  • Professor, Department of Radiology, School of Medicine, College of Medicine, Taipei Medical University
  • Chairman, Department of Medical Imaging, Taipei Medical University Hospital
  • Director, Translational imaging research center, Taipei Medical University Hospital

Details of his 21 major research projects and 231 scholarly publications can be found at:


Most patients have undergone routine or diagnostic medical imaging, and they may think this experience hasn’t changed much over the decades as they briefly hold their breath for an X-ray or scan. As TMU radiologist and medical imaging researcher Prof. Sandy Cheng-Yu Chen notes, CT scans have been both useful and increasingly affordable for 30 years, and MRIs have been used for 20 years.

Yet what has changed vastly, especially with computing advances, is how these images are used: “What’s new is that with big data we can extract features from these images, and use machine learning to develop algorithms that are useful in many ways.”

Tall and energetic, Prof. Chen has a kindly manner when reassuring anxious patients; he notes modestly that “After all, we [radiologists] are doctors too.” It might be a radiologist, not an oncologist or surgeon, who performs a tricky needle biopsy of a lung mass while watching its position carefully on a display screen.

But his wry sense of humor also emerged in a recent discussion of imaging research advances at TMU. He joked that high-tech imaging and AI that compares results and prognoses may cost many doctors their jobs. As such diagnoses are increasingly assisted by computers by comparing vast databases of cases, he said many radiologists may concern about their jobs being sidelined.

“We physicians love the new artificial intelligence techniques,” Prof. Chen said. “Yet we also see how they might be able to do our jobs in the future – and radiology is likely to be one of the first [medical subfields] to be mechanized.”

¹ A CT scan, also known as CAT scans, are computed axial tomography scans that combine many X-ray measurements from different angles to produce cross-sectional (or tomographic) images that show “slices” of a scanned area, allowing the user to see inside without cutting.
² Magnetic resonance imaging (MRI) uses the magnetic properties of different tissues and bones to produce detailed images without X-rays. MRIs require massive machinery and cannot be used on patients with metal stents or artificial joints because of the powerful magnets used to make the images.

Just like a fortune teller

Why are these computerized results superior to decades of human clinical experience? “AI offers precise deductions from thousands of images, rather than of relying on human judgments as in all past medical eras. Two doctors can look at an image and see different things – but with a library of ten thousand images, the machine can assist in drawing consistent conclusions.”

And it is both computing power and imaging detail that have made such conclusions reliable. “We can now go to the ‘pixel level’ to observe the smallest details of anatomical structure changes confered by genomic and molecular alternations,” Prof. Chen said. “We can observe these changes to predict histopathology-alike diagnoses based on evidence and algorithms, not our human experience.”

TMU’s research in this area is widely noted by both research funders and scientific journals. He explained that his team’s brain tumor study will soon be published in a level-10 impact factor journal, Clinical Cancer Research: “It was accepted in part because this government-funded study correlates multi-center’s data in new and useful ways. … What this model offers is predictive power based on both retrospective and prospective studies.

“If we follow hundreds or thousands of people through the course of a disease, or who experience similar types of trauma, then we will know how long they lived. We analyze their images, genomic, survival data and other case factors to look for patterns.”

Many patients can thank this new technology for sparing them from uncomfortable and invasive diagnostic techniques such as biopsies that previously were needed to predict tumor growth and response to various treatments: “With this [imaging] data, we don’t even need to biopsy to understand what a tumor is likely to behavior, based on the facts of how long patients with similar neoplasms survived. I can extrapolate outcomes – just like a fortune teller! – based on separate groups of similar cases.”

TMU ranked #1 in imaging grant

Prof. Chen says TMU is a leader in this area because of its in-deep research background in both cancer and trauma, as well as because of its leadership in applying these advanced imaging technologies in artificial intelligence.

“It takes a big project to collect and analyze big data – and Taiwan’s Ministry of Science and Technology selected the top three institutes to lead this [imaging and artificial intelligence] research. TMU received the highest rating of around thirty applicants. … This project came to TMU based on our history of productive work in this area, but also based on our access to a huge archive of data resources. It’s a huge, heavy artificial intelligence job!”

Yet cancer remains a fearsome opponent, and Prof. Chen voiced frustration at the low survival rates for brain, lung and pancreatic cancer, noting that these have hardly improved over the past four decades.

In 1976, the US National Cancer Institute published the major cancer survival rates based on data available then. He said, “The five-year survival rate was very low for brain and lung cancers, but it was much better for prostate cancer – maybe a 67.8 percent five-year survival rate – and for other types of cancers.

“In 2012 a similar study showed that prostate cancer now has reached a 99 percent five-year survival rate. Yet the five-year survival rate for brain and lung cancers improved by barely ten percent over what it had been four decades earlier and remains very low, as does five-year survival for pancreatic cancer.”

Prof. Chen hopes the new AI tools will enable more progress against the deadliest cancers. “Now we also have immunotherapy after many decades of relying on radiation, surgery and chemotherapy as our only weapons against cancer. And imaging can potentially guide us where the T cells and natural killer cells fail to kill residual cancer.”

Investigating tumor margins

He said most challenging aspect of image analysis is “locus-specific correlation” because of tissue heterogeneity. This means that a tumor is not just one thing: it has different kinds of tissues in different places within a single tumor. The margins of the tumor are those battle fields that our immune system fights with the cancer cells. “

There is no discrete edge on a brain tumor by eyeballing, he explained. “The brain is a soft substance, and we can’t just keep removing tissue to try to ensure that we have taken enough – the more brain tissue that is removed, the more patients lose vital cognitive and body functions.

“But why do brain tumors almost always grow back, when we think we have left no visible traces? We are studying what happens on the edges of these tumors, trying to find the heterogeneity by precision-mapping our tissue samples and corresponding imaging to study tumor microenvironments.”

If this is the case, readers might wonder how soon such diagnoses will be the rule rather than the exception (i.e., the standard care). Given TMU’s experience in full-spectrum “bench to bedside” research, many patients at the university’s affiliated hospitals enjoy a comparative clinical advantage with the government grants sponsoring their testing.

Prof. Chen noted that “Our patients are getting more precision with their medical treatment, because we can use the information from these thousands of images. … Imaging now can tell benign from malignant masses in some cancers that formerly required invasive biopsies. And certain genetic profiles, such as the IDH1 mutation, offer clues as to who will respond well to chemotherapy – expensive laboratory-based analysis might be spared, just images.”

“With this data, we don’t even need to biopsy to understand what a tumor is likely to do, based on the facts of how long patients with similar neoplasms survived. I can predict outcomes – just like a fortune teller! – based on separate groups of similar cases.”

Tools lead to new standards

TMU’s progress helps patients at other care institutions as well.  Asked if hospitals with vast image libraries are logically where cancer patients should seek the best treatment, he said the scientific publication process ensures that advances can quickly benefit all patients: “The purpose of research is to share knowledge, and everyone has to share their data as part of the publication process, both for replicability and credibility.”

However, he said TMU enjoys some competitive advantages in clinical practice because of this high-tech imaging research: “We have the most experienced neurosurgeons, because of our long and deep work in neurotrauma, stroke and other areas. And with the government’s research support we can offer patients cost-free genetic analyses.”

These results would otherwise easily cost each patient 100,000 Taiwan dollars, since tissue samples are taken from several sites in the tumor, and each site biopsy costs 20,000 Taiwan dollars to analyze.

“When we schedule their surgery, we ask them to authorize us to remove the tissue samples just before the tumor itself is treated or removed,” Prof. Chen said. “And because of our protocols, we then have precise images and reliable measurements of where these biopsy samples were taken, and from what kinds of tumors.”

Images clarify brain trauma

Although Prof. Chen is eager to fight the most refractory cancers, he is also proud of TMU’s work with brain trauma imaging. “Our research into mild traumatic brain injury (mTBI) has been very fruitful, in part because we and our partner hospitals have followed six hundred trauma patients for six years. With so many scooters in Taiwan we have a lot of concussion patients, even while the mandatory helmet law limits skull fractures and brain hemorrhages.

“And while concussion seems mild in terms of treatment and immediate outcomes, with maybe a week of rest prescribed as treatment, there is great concern about later onset dementia” and other serious after-effects that may take years or decades to emerge. At TMU we also analyze brain injuries in rats, notably by using a very powerful imaging tool, a seven-Tesla MRI.”

This record of leadership in trauma and cancer research is why TMU is building on Prof. Chen’s research legacy by prioritizing artificial intelligence. Imaging and big data offer insights that can lead to new treatments for impact and blast injuries, as well as for cancers and other diseases.

“This project came to TMU based on our history of productive work in this area, but also based on our access to a huge archive of data resources. It’s a huge, heavy artificial intelligence job.”

  • For interviews or a copy of the paper, contact Office of Global Engagement via global.initiatives@tmu.edu.tw.
  • Read the short version on QS WOWNews