Title:
Learning Patient-Specific Models for Image-Guided Intervention and Surgical Planning
Abstract:
Image-guided intervention and surgical planning begin with medical image analysis and end with a physical act on a specific patient. This talk presents work from our lab on using learning to build patient-specific models from multimodal images for such procedures. The talk begins with MRI–ultrasound fusion for prostate biopsy without external tracking hardware or expert manual alignment. We developed a content-based pipeline that reconstructs 3D ultrasound from untracked freehand sweeps, registers live 2D frames into that volume in real time, and aligns MRI to ultrasound through learned similarity and cross-modal attention, turning a hardware-and-expertise problem into a representation-learning problem. The second half of the talk concerns orthognathic surgery planning, where the surgeon decides where the bone will move and the patient wants to know what the face will look like before the procedure. I will present learning-based estimation of a patient’s target facial appearance, fast simulation of facial soft-tissue outcome driven by planned bony movement, including correspondence-attention and physics-informed approaches that aim for finite-element fidelity at interactive speed, and the automated pipeline that makes these methods usable in practice. I will close with the open challenges in this domain that motivate our current work.
Biography:
Dr. Pingkun Yan is Professor and Department Head of Biomedical Engineering at Rensselaer Polytechnic Institute (RPI), where he directs the Deep Imaging Analytics Lab (DIAL). His research develops artificial intelligence and machine learning methods for medical image computing, multimodal healthcare data analysis, image-guided intervention, and patient-specific surgical planning, with an emphasis on clinically translatable systems developed in close collaboration with physicians. Before joining RPI in 2017, Dr. Yan was a Senior Scientist at Philips Research North America, embedded at the National Institutes of Health Clinical Center, where he contributed to the first FDA-cleared MRI–ultrasound fusion-guided prostate biopsy system. He has authored more than 120 peer-reviewed articles, is an inventor on 17 granted U.S. patents, and has received the NSF CAREER Award, the NIBIB Trailblazer Award, and several best paper awards. He serves on the Board of Directors of the MICCAI Society, chairs the IEEE EMBS Technical Committee on Biomedical Imaging and Image Processing, and was named an IEEE EMBS Distinguished Lecturer for 2025–26. He is a Fellow of the American Institute for Medical and Biological Engineering and a Senior Member of IEEE and the National Academy of Inventors.