Yueming Jin Assistant Professor Department of Biomedical Engineering Department of Electrical and Computer Engineering National University of Singapore (NUS) Office: Block E7, #05-01H, NUS, Singapore 119276 Email: ymjin AT nus.edu.sg

Biography

I am an Assistant Professor at Department of Biomedical Engineering and Department of Electrical and Computer Engineering at National University of Singapore. Before joining NUS, I was a senior research fellow at the Department of Computer Science, University College London, working with Prof. Danail Stoyanov. I received my Ph.D. degree in the Department of Computer Science and Engineering, The Chinese University of Hong Kong, advised by Prof. Pheng-Ann Heng and Prof. Chi-Wing Fu. I am listed at Forbes 30 Under 30 Asia, Class 2024.

My research interests include artificial intelligence and its applications on healthcare, with an emphasized application to medical image computing and robotic surgical data science. I recently work on multi-modality medical foundation model, video-language learning in surgery, resource efficient learning, and smart agent for robotic surgery with 3D reconstruction.

*Opening!* I am looking for PhD(2025Spring&Fall)/Postdoc/RA/Interns/Visiting students supported by CSC, who are interested in machine learning, medical image analysis, surgical 2D/3D vision, surgical robotic intelligence. Students with strong self-motivation and publication record are preferred. Welcome to email me after reading the details.

If you are an NUS student interested in doing research with me, also feel free to drop me an email!

News

Selected Publications [Find full list at Google Scholar]

• Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization
  Yunpeng Zhao, Cheng Chen, Qing You Pang, Quanzheng Li, Carol Tang, Beng-Ti Ang, Yueming Jin
  (We present a universal model to tackle missing modality in both train and test stages for brain tumor segmentation)
  Preprint, 2024.
  
• Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
  Junde Wu, Yu Zhang, Rao Fu, Huihui Fang, Yuanpei Liu, Zhaowei Wang, Yanwu Xu, Yueming Jin
  (We present a simple yet effective adaptation technique, to extend SAM to medical domain)
  Preprint, 2023.
  

[code][Medical Adapter Zoo]

• Think Step by Step: Chain-of-Gesture Prompting for Error Detection in Robotic Surgical Videos
  Zhimin Shao, Jialang Xu, Danail Stoyanov, Evangelos B Mazomenos, Yueming Jin
  IEEE Robotics and Automation Letters (RA-L), 2024.

[code]

• Surgical SAM 2: Real-time segment anything in surgical video by efficient frame pruning
  Haofeng Liu, Erli Zhang, Junde Wu, Mingxuan Hong, Yueming Jin
  Neural Information Processing Systems, Advancements In Medical Foundation Models, 2024.

[code]

• Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splatting
  Shuojue Yang, Qian Li, Daiyun Shen, Bingchen Gong, Qi Dou, Yueming Jin
  Medical Image Computing and Computer Assisted Intervention (MICCAI), 2024.

  (Early accept)

[code]

• Energy-Induced Explicit Quantification for Multi-modality MRI Fusion
  Xiaoming Qi, Yuan Zhang, Tong Wang, Guanyu Yang†, Yueming Jin†, Shuo Li
  European Conference on Computer Vision (ECCV), 2024.
• S^2Former-OR: Single-Stage Bimodal Transformer for Scene Graph Generation in OR
  Jialun Pei, Diandian Guo, Jingyang Zhang, Manxi Lin, Yueming Jin†, Pheng-Ann Heng
  IEEE Transactions on Medical Imaging (IEEE TMI), 2024.

[code]

• Video-instrument synergistic network for referring video instrument segmentation in robotic surgery
  Hongqiu Wang, Guang Yang, Shichen Zhang, Jing Qin, Yike Guo, Bo Xu, Yueming Jin†, Lei Zhu
  IEEE Transactions on Medical Imaging (IEEE TMI), 2024.

[code]

• SimCol3D -- 3D Reconstruction during Colonoscopy Challenge
  Anita Rau†, Sophia Bano†, Yueming Jin†, Pablo Azagra, Javier Morlana, Edward Sanderson, Bogdan J. Matuszewski, Jae Young Lee, Dong-Jae Lee, Erez Posner, Netanel Frank, Varshini Elangovan, Sista Raviteja, Zhengwen Li, Jiquan Liu, Seenivasan Lalithkumar, Mobarakol Islam, Hongliang Ren, José M.M. Montiel, Danail Stoyanov
  Medical Image Analysis, 2024.
  
• One-Prompt to Segment All Medical Images
  Junde Wu, Jiayuan Zhu, Yueming Jin, Min Xu
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
  

[code]

• LLM-Assisted Multi-Teacher Continual Learning for Visual Question Answering in Robotic Surgery
  Kexin Chen, Yuyang Du, Tao You, Mobarakol Islam, Ziyu Guo, Yueming Jin†, Guangyong Chen, Pheng-Ann Heng
  IEEE International Conference on Robotics and Automation (ICRA), 2024.
  

[Video demo]

• Calibrate the Inter-observer Segmentation Uncertainty via Diagnosis-first Principle
  Junde Wu, Yu Zhang, Huihui Fang, Lixin Duan, Mingkui Tan, Weihua Yang, Chunhui Wang, Huiying Liu, Yueming Jin†, Yanwu Xu†
  IEEE Transactions on Medical Imaging (TMI), 2024.
  
• Medsegdiff-v2: Diffusion based medical image segmentation with transformer
  Junde Wu, Wei Ji, Huazhu Fu, Min Xu, Yueming Jin, Yanwu Xu
  Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2024.
  
• FedDP: Dual Personalization in Federated Medical Image Segmentation
  Jiacheng Wang, Yueming Jin, Danail Stoyanov, Liansheng Wang.
  IEEE Transactions on Medical Imaging (TMI), 2023.
  
• Keep Your Eye on the Best: Contrastive Regression Transformer for Skill Assessment in Robotic Surgery
  Dimitrios Anastasiou, Yueming Jin, Danail Stoyanov, Evangelos Mazomenos.
  IEEE Robotics and Automation Letters (RA-L), 2023.
  
• Trans-SVNet: Hybrid Embedding Aggregation Transformer for Surgical Workflow Analysis
  Yueming Jin, Yonghao Long, Xiaojie Gao, Danail Stoyanov, Qi Dou, Pheng-Ann Heng.
  International Journal for Computer Assisted Radiology and Surgery (IJCARS), 2022.
  (1st Prize of Best Paper Award of IJCARS-MICCAI 2021)
  

[code][award] [press coverage]

• Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation
  Yueming Jin, Yang Yu, Cheng Chen, Zixu Zhao, Pheng-Ann Heng, Danail Stoyanov.
  IEEE Transactions on Medical Imaging (TMI), 2022.
  

[code][video]

• Personalizing Federated Medical Image Segmentation via Local Calibration
  Jiacheng Wang*, Yueming Jin*, Liansheng Wang.
  European Conference on Computer Vision (ECCV), 2022.
  

[code]

• Temporal Memory Relation Network for Workflow Recognition from Surgical Video
  Yueming Jin, Yonghao Long, Cheng Chen, Zixu Zhao, Qi Dou, Pheng-Ann Heng.
  IEEE Transactions on Medical Imaging (TMI), 2021.
  

[code]

• Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learning
  Zixu Zhao, Yueming Jin†, Pheng-Ann Heng.
  International Conference on Computer Vision (ICCV), 2021.
  

[video][poster]

• Real-time Landmark Detection for Precise Endoscopic Submucosal Dissection via Shape-aware Relation Network
  Jiacheng Wang*, Yueming Jin*, Shuntian Cai, Hongzhi Xu, Pheng-Ann Heng, Jing Qin, Liansheng Wang.
  Medical Image Analysis (MedIA), 2021.
  
• One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video
  Zixu Zhao, Yueming Jin†, Bo Lu, Chi-Fai Ng, Qi Dou, Yun-Hui Liu, Pheng-Ann Heng.
  IEEE International Conference on Robotics and Automation (ICRA), 2021.
  

[video]

• Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis
  Yueming Jin, Huaxia Li, Qi Dou, Hao Chen, Jing Qin, Chi-Wing Fu, Pheng-Ann Heng.
  Medical Image Analysis (MedIA), 2020.
  (Ranked 1st place in MICCAI 2019 EndoVis Surgical Workflow Challenge on phase segmentation)
  

[code]

• Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search
  Xiaojie Gao, Yueming Jin, Qi Dou, Pheng-Ann Heng.
  International Conference on Robotics and Automation (ICRA), 2020.
  
• Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video
  Yueming Jin, Keyun Cheng, Qi Dou, Pheng-Ann Heng.
  Medical Image Computing and Computer Assisted Intervention (MICCAI), 2019.

  (Oral)

[code]

• SV-RCNet: Workflow Recognition from Surgical Videos using Recurrent Convolutional Network
  Yueming Jin, Qi Dou, Hao Chen, Lequan Yu, Jing Qin, Chi-Wing Fu, Pheng-Ann Heng.
  IEEE Transactions on Medical Imaging (TMI), 2018.
  (Ranked 1st place in MICCAI M2CAI 2016 Surgical Workflow Challenge)
  

[code]

• Reconfigurable Interlocking Furniture
  Peng Song, Chi-Wing Fu, Yueming Jin, Hongfei Xu, Ligang Liu, Pheng-Ann Heng, Daniel Cohen-Or.
  ACM Transactions on Graphics (ACM TOG), SIGGRAPH Asia, 2017.
  (Results were selected to be included in the technical papers trailer)
  

[project]

• 3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes
  Qi Dou, Hao Chen, Yueming Jin, Lequan Yu, Jing Qin, Pheng-Ann Heng.
  Medical Image Computing and Computer Assisted Intervention (MICCAI), 2016.
  (Young Scientist Award Finalist)
  

* indicates equal contribution; † indicates corresponding authorship.

Selected Awards

Forbes 30 Under 30 Asia List, Class 2024 [NUS press coverage]

1st Prize of Best Paper Award of IJCARS-MICCAI 2021 [NUS press coverage]

Best Paper Award in Medical Robotics at ICRA 2021

Best Paper Award of MedIA-MICCAI 2017 , 2017

Young Scientist Publication Impact Award Finalist at MICCAI, 2021

Young Scientist Award Runner Up at MICCAI, 2016

NIH Awards at MICCAI 2020

Champion, SimSurgSkill Challenge on Objective Surgical Skills Assessment (MICCAI 2021)

Champion, ROBUST-MIS Challenge on Multi-instance Segmentation (MICCAI 2019)

Champion, EndoVis Surgical Workflow Challenge on Phase Segmentation (MICCAI 2019)

Champion, Surgical Workflow Challenge (MICCAI M2CAI 2016)

Teaching Assistant of Merit, 2016

Hong Kong PhD Fellowship, 2015-2019

Team

Current Students:

Qian Li (B.E., Ph.D., HIT)(Research Fellow)

Haofeng Liu (M.S. SUSTech)(PhD Student, 2024 Spring)

Shuojue Yang (M.S. JHU)(PhD Student, 2024 Spring)

Yibing Fu (B.E. M.S. Beihang)(PhD Student, 2024 Fall)

Borui Kang (B.E. THU)(PhD Student, 2024 Fall)

Guanyi Qin (B.E. SCUT M.S. THU)(PhD Student, 2024 Fall)

Daiyun Shen (B.E. THU)(PhD Student, 2024 Fall)

Zhitao Zeng (B.E. Nankai M.S. CAS)(PhD Student, 2024 Fall)

Erli Zhang (B.E. NTU)(PhD Student, 2024 Fall)

Tianyi Zhang (B.E. Beihang M.S. NTU)(PhD Student, 2024 Fall)

Mingxuan Hong (B.E. CUHK)(MEng Student, 2023-)

Yunpeng Zhao (B.E. Beihang)(MEng Student, 2023-)

Zheng Fang (B.E. ZJU-UIUC)(MEng Student, 2023-)

Ziyue Wang (B.E. HIT-SZ)(MEng Student, 2024-)

Chang Han Low (B.E. NUS)(MEng Student, 2024-)

Bochong Zhang (B.E. Nankai)(MEng Student, 2024-)

Exchange PhD students: Xiaoming Qi (Ph.D. SEU), Shi Yin (Ph.D. CSU)

Exchange B.E. students: Jiaan Zhang (B.E. ZJU), Bian Shujun (B.E. ZJU)

Alumni:

Junde Wu (2023-24 RA)(M.S. HIT -> PhD at Oxford)

Zhimin Shao (2023 Summer Intern) (B.E. M.S. at THU -> PhD at JHU)

KinHei Lee (2024 Intern), (B.E. at CUHK -> Mres at ICL)

Previous Mentees:

Dimitrios Anastasiou (Ph.D. UCL)

Zixu Zhao (2022 Ph.D. CUHK), Current: Amazon

Xueying Shi (2021 Ph.D. CUHK), Current: Huawei

Professional Services

Conference Services:
* Area Chair of MICCAI 2023 2024, IPCAI 2023 2025
* Special Programmes Co-Chair of ICBME 2024, Singapore
* Publicity Co-Chair of IEEE-RAS RoboSoft 2023
* Oral Session Chair of MICCAI 2024, ICRA 2021
* Organization Committee:
IJCAI Workshop on Trustworthy Machine Learning in Healthcare, 2024
ICLR Workshop on Trustworthy Machine Learning in Healthcare, 2023
MICCAI Workshop on Cancer Prevention through early detecTion, 2022, 2023, 2024
MICCAI Challenge on 3D Reconstruction During Colonoscopy, 2022
MICCAI Challenge on Pre-operative to Intra-operative Laparoscopy Fusion, 2022
* Regular Conference reviewer:
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
IEEE International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
IEEE International Conference on Robotics and Automation (ICRA)
IEEE International Conference on Intelligent Robots and Systems (IROS)
Association for the Advancement of Artificial Intelligence (AAAI)
Medical Image Computing and Computer Assisted Intervention (MICCAI)
ACM International Conference on Multimedia (ACM MM)

Regular Journal Reviewer:
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
IEEE Transactions on Medical Imaging (TMI)
Medical Image Analysis (MedIA)
Nature Communications Medicine
International Journal of Surgery
IEEE Journal of Biomedical and Health Informatics (JBHI)
IEEE Robotics and Automation Letters (RA-L)
Medical Physics
Neurocomputing
International Journal of Computer Assisted Radiology and Surgery (IJCARS)
Computer Methods and Programs in Biomedicine (CMPB)

Selected Talks

Towards AI-powered Robotic Surgical Vision.
Woman in Medical Image Understanding and Analysis, University of Cambridge, UK, Jul. 2022.

Towards AI-powered Robotic Surgical Vision.
IUPESM World Congress, Singapore, Jun. 2022.

Dynamic Surgical Video Analysis for Intelligent Robotic Surgery.
CMIC Seminar at University College London (UCL), Sep. 2021.

Dynamic Surgical Video Analysis for Intelligent Robotic Surgery.
The Hong Kong Polytechnic University (PolyU), Nov. 2020.

Towards Intelligent Surgery: Dynamic Surgical Visual Perception with Deep Learning.
Theme-based Research Scheme Symposium on Image-guided Robotic Surgery, CUHK, Nov. 2019.

Towards Intelligent Surgery: Dynamic Surgical Video Analysis with Deep Learning.
The 5th World Chinese Microbial Ecology and Intelligent Endoscopy Summit, Xiamen, Nov. 2019.

Teaching

2024	Fall	BN5211 Medical Robotic Intelligence
2024	Fall	BN5212 Advanced Machine Learning for Biomedical Engineering and Sciences
2024	Spring	EE2211 Introduction to Machine Learning
2023	Fall	BN5211 Medical Robotics
TA at CUHK:
2015	Fall	ESTR1934 Digital Logic and Systems
2016	Spring	CSCI1930 Data Structures
2016	Fall	CSCI3260 Principles of Computer Graphics
2017	Spring	CSCI1930 Data Structures
2017	Fall	CSCI3260 Principles of Computer Graphics

© Yueming Jin | Last updated: Aug 2024