Digital Twin

Patient twin: Why Sophia is no longer afraid of cancer

Our technology vision shows how artificial intelligence and patient twinning could revolutionize cancer therapy. Meet Sophia, a patient from the future.
10min
Katja Gäbelein
Published on September 2, 2024
Cancer remains one of the greatest threats to human health. Yet within the next ten years, the entire cancer treatment process could be revolutionized and patient outcomes significantly improved – thanks to networked key technologies, artificial intelligence (AI), and digital patient twins based on accurate data. Find out how in our technology vision.

Portrait photo of Peter Aulbach against an orange background. He wears a full beard and a suit and smiles into the camera.

Curious to learn more? Find out how the future of cancer therapy could look thanks to developments in medical technology. Follow the journey of our fictitious patient, Sophia. While our innovation team, of course, considered many different types of cancer and treatment options, only ONE possible scenario is presented here to illustrate how a future product and solution portfolio could improve cancer care.

Sophia has been having a tough time. Two years ago, the 58-year-old was diagnosed with lung cancer – more specifically, with non-small-cell lung cancer (NSCLC). [1] A few months ago, further metastases, so-called oligometastases, formed in her liver and spine – a shock for Sophia and her family.

Non-small-cell lung cancer, also called bronchial carcinoma or lung carcinoma.

Initial treatment of all tumors has proven successful. That's why Sophia is now able to not focus on her illness in everyday life. She feels quite reassured. Her health is monitored through regular check-ups with her medical team of oncologists and radiologists, via special apps on her smartphone, and by modern medical technology – all with the help of artificial intelligence, of course.

All medical data collected over the course of Sophia's treatments flow into the digital model of the patient: her digital patient twin. AI continuously evaluates her data in real time so that it can make predictions about the development of Sophia's health. Large language models help to structure disordered data and convert this information into medical reports in language appropriate for Sophia's family doctor, her oncologist, or for Sophia herself.

AI-based language models that have the ability to communicate text-based information between humans and machines. They can generate, summarize, and translate texts – tasks for which they are trained with large volumes of data.
Still 3D rendering. We see a semi-long shot of a petrol-colored, fluid body with human outlines, representing a digital patient twin. Orange bubbles, symbolizing a stream of data, fly through the image towards the body.

The data are also used in preparing the medical equipment that Sophia needs for her examinations, applying the optimal settings for her individual needs. This not only results in more precise treatment, but also saves time and capacity for medical staff.

The digital patient model knows the course of Sophia's treatment and the onward plan that she has agreed with her doctors. It knows Sophia's personal preferences and family situation. Via an app on her smartphone, the digital model reminds Sophia when an examination or treatment is coming up. Based on scientific recommendations, her digital twin offers tips and tricks on what she can do before, during, and after a particular treatment to enhance her general condition and well-being.

The data from Sophia's digital patient twin not only help Sophia herself; she has agreed to donate her data ─ in anonymized form, of course. The data is fed into a giant database, that can be used to train AI algorithms, allowing the AI-based decision-making to be constantly improved.

Ren-Yi Lo heads our Big Data Office and, together with her team, "conducts" huge amounts of data. Our scientists need these data to develop new AI solutions for medical technology.
Learn more

Artificial intelligence helps medical centers simulate therapeutic successes so that the best next step can be chosen. Early warnings based on the data collected allow medical centers not only to schedule appointments for check-ups, diagnostics and follow-up treatment, but also to predict bottlenecks in therapy capacity and adjust their human resources accordingly.

Once a week, Sophie performs a liquid biopsy in the morning after brushing her teeth, a test aimed at detecting cancer cells in the blood. She draws the blood sample herself by simply pricking her finger.

Femke de Theije and her team have developed Atellica VTLi, a handheld biosensor device that delivers highly sensitive troponin I results in just eight minutes, enabling faster diagnosis and treatment of heart attack patients.
Learn more

Today, an alarm has sounded in Sophia's portable biosensor device: the biosensors have detected abnormalities in her blood counts that could indicate that her cancer has progressed. At the same time, the AI sends a warning message to Sophia's medical team and suggests a strategy for further action. Sophia is worried.

The lead oncologist calls Sophia directly and reassures her. Following the AI's recommendation, she decides together with Sophia that the next diagnostic step should be a computed tomography (CT) scan. The AI in the patient twinning app on Sophia's smartphone identifies the best CT device for her examination. The AI checks available resources and arranges an appointment with Sophia for the CT scan, including the exact location and travel options. Sophia is even told who will meet her to carry out the CT scan.

Medium close-up portrait photo of Ulrike Attenberger wearing half-long, dark brown hair, a doctor's white lab coat and smiling into the camera. A blurred image of the Bonn University Hospital building is visible in the background.

What do physicians think about digital twinning in cancer therapy?

“Digital twins will be essential to make precision medicine a reality,” says Ulrike Attenberger. Here you can read an interview with Professor Attenberger, who heads the Department of Radiology at Bonn University Hospital.


Two days later, Sophia makes her way to her CT scan. Based on the data provided by the patient twin, which has already stored the updated data from Sophia's blood test, the scanner unit is optimally adjusted to her specific needs – even before she enters the medical facility.

The radiology technologist onsite simply has to check and confirm the automatically adjusted settings on the equipment. Radiology technologists could even do this from their computer at home, or while on the go using a tablet. State-of-the-art technology makes it possible to carry out complex medical examinations at any time, even in remote areas or regions where medical staff are in short supply.

Clinical staff working from home? Thanks to our intelligent technology, this is already possible. Read Stefanie Hajduga's story, a radiology technologist.
Read more

The CT scanner that the AI recommended for Sophia performs photon-counting computed tomography. Its advantages include a reduced radiation dose and improved image quality. [4] The scanner's particularly high-resolution and high-contrast images help medical staff to recognize small details and differences in tissues, and thus delineate the tumor invasion. In combination with AI, radiomics allows further key information about the tumor to be obtained from the data.

This technology already exists today: with photon-counting CT, Siemens Healthineers has developed a radically new technology for clinical practice.
Learn more

A team of medical specialists looks after Sophia, with oncologists from various disciplines working together from different locations. They meet virtually in an online tumor board – a kind of digital round table. The aim is to make a reliable diagnosis for Sophia based on all the data collected so far from her digital patient twin, and to evaluate further treatment options.

All the information is presented visually in the form of a patient avatar. Virtual reality (VR) and augmented reality (AR) headsets help medical staff better understand and interact with the multi-layered information.

Unfortunately, the initial suspicion is confirmed. By comparing images from Sophia's earlier CT and magnetic resonance imaging (MRI) scans as well as scans from a large pool of data from other patients, the photon-counting CT working in tandem with AI identifies patterns that the human eye cannot yet detect: a new tumor is growing in Sophia's left lung.

Still 3D rendering. In a semi-close-up shot, we see the abstractly depicted digital twin of the lungs within the fluid body. In the right lung, inactive tumors are shown as transparent vesicles. In the left lung you can see the digital twin of an active tumor, which is shown in black and red with fire glow.
For extended reality (XR) senior key expert Anton Ebert, these two areas are inextricably linked. Read about the exciting applications that already exist or that could one day exist.
Learn more

Fortunately, the tumor is still at a very early stage. The avatar shows exactly where it is located. On request, the physicians also receive further data, for example about Sophia's vital signs, laboratory tests, and genomics. All this information is automatically stored in Sophia's electronic health record (EHR).

An EHR is a database for storing patients' health data, including their prior medical history, treatments, medications, and allergies, etc.

The AI uses the avatar to present possible therapeutic approaches so that the medical team can quickly decide on further treatment: among other things, the AI has created a digital twin of the new tumor based on genetic patterns in complex calculations involving radiomics.

This procedure reduces the number of tissue samples required to be taken from the patient. Therapy can be provided in a faster, more targeted manner. Based on the digital twin of the tumor and patient, the AI predicts how likely it is that the tumor will respond to the various forms of therapy and with what success. Can cryoablation [6] or radiation therapy currently better help Sophia?

A medical procedure in which extremely cold temperatures are used to freeze and destroy tumors, for example. It is usually performed when surgical tumor removal is not possible.

Based on the calculated probabilities and taking into account the expected side effects as well as cost efficiency, the AI provides the medical team with therapy recommendations.

Once the physicians have agreed a course of action, Sophia will be brought into the conference. With the help of a large language model, the AI has prepared the complex medical information in a way that is easy for the patient to understand. The doctors inform Sophia about the next recommended treatment steps and the related risks. This gives Sophia clarity and reduces her uncertainty. Sophia ultimately opts to undergo AI-assisted radiation therapy.

Tumor still image

Before AI-optimized workflows were introduced, radiation therapy planning was a lengthy process that often left patients waiting several weeks. With the support of AI, treatment planning and Sophia's radiation therapy can now be done within a few hours on the same day, so no precious time is lost.

The AI plans capacity, coordinates appointments, assists with the detailed planning of radiation therapy, and adjusts the settings of all devices precisely to Sophia's needs based on her patient data.

An interdisciplinary team is already working intensively with AI to optimize the entire workflow of complex radiation therapy planning with the aim of drastically shortening waiting times. Read our concept developer Fernando Vega's story.
Learn more

The AI then generates a digital twin specifically for the planned radiation therapy. It uses the data from Sophia's avatar, such as data from the current CT and MRI scans as well as from the AI-supported recognition (autocontouring) of the organs at risk that need to be protected during irradiation. Determining the boundaries between tumorous and healthy tissue as precisely as possible is critical in the subsequent success of the treatment.

Based on precise data, the AI calculates the amount and distribution of the radiation dose. The radiation oncology specialist checks all the results generated by the AI before the actual treatment starts.

Autocontouring is performed with the aid of specialized software algorithms that can automatically identify and segment organs and other anatomical structures captured on the medical scans. Artificial intelligence can already help in this area today.
Learn more

The AI has calculated that Sophia should receive a total radiation dose of 25 fractions, spread over several weeks. Her first radiation treatment will start immediately. Based on the treatment protocol data, the linear accelerator (linac) settings have already been adjusted to deliver exactly the calculated dose. 

The linac has a built-in imaging function that generates more medical images during treatment. It recognizes how the patient's soft tissues and organs move so that it can compensate for these movements. The sensor technology and AI integrated in the linac take into account Sophia's breathing and breathing pauses during irradiation. With the help of this imaging and AI, the radiation dose can be precisely concentrated on the tumor while sparing the surrounding tissue. During quality control, physicians can see whether the treatment was successful, even while it is still ongoing.

Among other applications, linacs use electricity to generate high-energy gamma rays. One of their most common uses is to treat cancer by destroying cancer cells.
Learn more about our radiotherapy systems:

For the second radiation dose a few days later, the AI has already evaluated the digital twin for radiation therapy for the results of the first fraction. It recommends an adjustment of the treatment's prescribed dose. The radiation oncologist then decides whether to accept the proposed adjustment to the radiation dose for the second fraction. After consulting with Sophia, he approves the protocol.

Sophia's liver tumor needed treatment about a year ago. At that time, her doctors selected a transcatheter arterial chemoembolization (TACE) on the advice of the AI.

In conjunction with other key technologies, the AI has also supported TACE, firstly by recommending to the medical team the appropriate guidewires and catheters to use as well as the best embolization material for targeted vascular occlusion. The AI simultaneously checked whether that material was available in the hospital's inventory. For the angiography to embolize the blood supply to the tumor, the AI recommended the correct injection site, access routes, and the best angle for the C-Arm.

Even further in the future, patients like Sophia could perhaps be helped by a surgical angio robot in a specialized medical center. Under constant control and monitoring by the medical professionals, who can intervene remotely at any time, the surgical angio robot could automatically guide the catheter through the blood vessels and inject the necessary drugs as well as materials for the vascular occlusion of the tumor. This would allow difficult, rarely performed and time-critical procedures to be offered in remote, rural areas as well.

A C-arm is an X-ray machine mounted on an articulated joint that allows it to rotate around the patient. Angiography can then be used to create precise images of the blood vessels in order to guide the catheter.

If Sophia needs a minimally invasive surgical procedure, the physicians could use virtual reality to stage and simulate the procedure in advance. In this way, they could save valuable time in the operating theater and minimize the risk of complications during the procedure. In the operating theater itself, augmented reality headsets would be used so that doctors have all the important information about Sophia's health literally “in view” at all times.

The information in their headsets would navigate the surgeons through the procedure. Experts who cannot be onsite could be linked in via the headset to give their assessment. During the procedure, AI would provide real-time support with decision-making. In this way, the process and progress of the operation could be simulated and recommendations or warnings could be given at an early stage if the AI detects deviations from the planned procedure or an impending complication.

How is Sophia doing a few weeks later? Her radiation therapy has been completed. The current evaluations of her avatar confirm that the treatment has successfully combated the tumor in her lung.

Sophia's life goes on. But her cancer is a chronic disease that she herself and her doctors are monitoring closely: Sophia regularly shares her health status with her treatment team via the app on her smartphone, reliably attends her follow-up examinations based on reminders in the app, and takes her weekly blood test at home.

Such an app already exists today: Noona is an app for managing patient outcomes that enables patients to actively participate in the therapy they're undergoing, such as by reporting symptoms in real-time and maintaining direct contact with the treatment team.
Learn more

Sophia feels well cared for: If even the slightest sign of a deterioration in her condition should appear, her digital twin would immediately warn her and the doctors treating her. The AI, together with her medical team, would then find an evidence-based, personalized treatment option for Sophia – and continue to help her manage her health.


We need more than just innovative technological developments and visions if digital patient twins and intelligently networked key technologies are to one day become reality: It will take reliable digital infrastructure, for example, that enables a continuous exchange of medical data. The foundations for this would need to be established in today's healthcare systems and hospitals so that patients and medical staff can benefit from them in the future.


By Katja Gäbelein
An interdisciplinary team of over 200 experts led by Peter Aulbach collaborated co-creatively in a series of workshops to develop a technology vision. Aulbach is an engineer with a doctorate in health sciences and works in the Innovation Strategy and Ecosystem (TE ISE) unit. He leads the Siemens Healthineers Innovation Center in Erlangen, Germany. Katja Gäbelein has applied this technology vision to create a patient story, which she tells here. Gäbelein is an editor for multimedia content at Siemens Healthineers and specializes in technology and innovation topics.