Dongdong Chen

I'm a Principal Research Manager at Microsoft GenAI, WA, and I am leading the post-training team for Microsoft's Phi family of small language models (SLMs). I am the main model developer for many impactful Microsoft Product features, including but not limited to BackgroundRemoval, Generative Eraser and PPT summarization in Office Copilot.

My research interests includes large-scale multi-modality and single-modality pretraining \& post-training, deep generative models (e.g., GAN/Diffusion, Image-to-Image translation), general representation learning (such as fundamental network structure design), and AI security (e.g., adversarial learning and model IP protection).

I served as the Area Chairs of NeurIPs 23/24/25, CVPR 22/23, ECCV 2022, ICPR 22/24, WACV 24/25, SPC of AAAI 2022, and the Associated Editor of Pattern Recognition and IEEE TMM.

Email  /  CV  /  Google Scholar  

profile photo

News!

  • Latest multimodal Phi-4 is released, feel free to try it here.
  • 4 papers are accepted by CVPR 25.
  • SGEdit is accepted by SIGGRAPH Asia 2024.
  • Phi-3 SLM is released, feel free to try it here!!!
  • 1 paper is accepted by ECCV 24.
  • 2 papers are accepted by CVPR 24.
  • 2 papers are accepted by NeurIPs 23
  • 3 papers are accepted by ICCV 23
  • 7 papers are accepted by CVPR 23
  • 1 paper (X-Paste) is accepted by ICML 23
  • 2 papers are accepted by NeurIPs 22
  • 2 papers are accepted by ECCV 22
  • 9 papers are accepted by CVPR 22

Selected Publications

clean-usnob Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
Microsoft GenAI, I am leading the post-training part.
clean-usnob Olympus: A Universal Task Router for Computer Vision Tasks
Yuanze Lin, Yunsheng Li, Dongdong Chen, Weijian Xu, Ronald Clark, Philip HS Torr
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)
clean-usnob SmartEraser: Remove Anything from Images using Masked-Region Guidance
Longtao Jiang, Zhendong Wang, Jianmin Bao, Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, Houqiang Li
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)
clean-usnob UNICL-SAM: Uncertainty-Driven In-Context Segmentation with Part Prototype Discovery
Dianmo Sheng, Dongdong Chen, Zhentao Tan, Qiankun Liu, Qi Chu, Tao Gong, Bin Liu, Jing Han, Wenbin Tu, Shengwei Xu, Nenghai Yu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)
clean-usnob Show and Segment: Universal Medical Image Segmentation via In-Context Learning
Yunhe Gao, Di Liu, Zhuowei Li, Yunsheng Li, Dongdong Chen, Mu Zhou, Dimitris N. Metaxas
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)
clean-usnob SmartEraser: Remove Anything from Images using Masked-Region Guidance
Longtao Jiang, Zhendong Wang, Jianmin Bao, Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, Houqiang Li
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)
clean-usnob SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing
Zhiyuan Zhang, Dongdong Chen, Jing Liao
Siggraph Asia 2024
clean-usnob Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Microsoft GenAI, I am leading the multimodal post-training part.
clean-usnob Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation
Zixin Zhu, Xuelu Feng, Dongdong Chen, Junsong Yuan, Chunming Qiao, and Gang Hua
European Conference on Computer Vision (ECCV2024)
clean-usnob OmniViD: A Generative Framework for Universal Video Understanding
Junke Wang, Dongdong Chen, Chong Luo, Bo He, Lu Yuan, Zuxuan Wu, Yu-Gang Jiang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)
clean-usnob Towards More Unified In-context Visual Understanding
Dianmo Sheng, Dongdong Chen, Zhentao Tan, Qiankun Liu, Qi Chu, Jianmin Bao, Tao Gong, Bin Liu, Shengwei Xu, Nenghai Yu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)
clean-usnob Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, Kwan-Yee K. Wong
Thirty-Seventh Conference on Neural Information Processing Systems (NeurIPs 2023)
clean-usnob HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending
Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Gang Hua, Nenghai Yu
International Conference on Computer Vision (ICCV 2023)
clean-usnob AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control
Ruixiang Jiang, Can Wang, Jingbo Zhang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao
International Conference on Computer Vision (ICCV 2023)
clean-usnob X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion
Hanqing Zhao, Dianmo Sheng, Jianmin Bao, Dongdong Chen, et al., Nenghai Yu
International Conference on Machine Learning (ICML 2023)
clean-usnob Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning
Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Lu Yuan, Yu-Gang Jiang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2023)
clean-usnob Look Before You Match: Instance Understanding Matters in Video Object Segmentation
Junke Wang, Dongdong Chen, Zuxuan Wu, Chong Luo, Chuanxin Tang, Xiyang Dai, Yucheng Zhao, Yujia Xie, Lu Yuan, Yu-Gang Jiang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2023)
clean-usnob Streaming Video Model
Yucheng Zhao, Chong Luo, Chuanxin Tang, Dongdong Chen, Noel C Codella, Zheng-Jun Zha
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2023)
clean-usnob Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles
Shuquan Ye, Yujia Xie, Dongdong Chen, Yichong Xu, Lu Yuan, Chenguang Zhu, Jing Liao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2023)
clean-usnob MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining
Xiaoyi Dong, Jianmin Bao, Yinglin Zheng, Ting Zhang, Dongdong Chen, Hao Yang, Ming Zeng, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2023)
clean-usnob Peco: Perceptual codebook for bert pre-training of vision transformers
Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu
Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023), AAAI 2023 Top-3 Influential paper!!!
clean-usnob i-Code: An Integrative and Composable Multimodal Learning Framework
Ziyi Yang, Yuwei Fang, Chenguang Zhu, Reid Pryzant, Dongdong Chen, et.al, Xuedong Huang
Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023)
clean-usnob OmniVL: One Foundation Model for Image-Language and Video-Language Tasks
Junke Wang, Dongdong Chen, Zuxuan Wu, Chong Luo, Luowei Zhou, Yucheng Zhao, Yujia Xie, Ce Liu, Yu-Gang Jiang, Lu Yuan
Thirty-sixth Conference on Neural Information Processing Systems (NeurIPs 2022)
clean-usnob REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering
Yuanze Lin, Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, Lu Yuan
Thirty-sixth Conference on Neural Information Processing Systems (NeurIPs 2022)
clean-usnob Bootstrapped Masked Autoencoders for Vision BERT Pretraining
Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu
European Conference on Computer Vision (ECCV 2022)
clean-usnob BEVT: BERT Pretraining of Video Transformers
Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Yugang Jiang, Luowei Zhou, Lu Yuan
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
clean-usnob Large Scale Pre-training for Person Re-identification with Noisy Labels
Dengpan Fu, Dongdong Chen, Hao Yang, Jianmin Bao, Lu Yuan, Lei Zhang, Houqiang Li, Dong Chen, Fang Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
clean-usnob HairCLIP: Design Your Hair by Text and Reference Image
Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Zhentao Tan, Lu Yuan, Weiming Zhang, Nenghai Yu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
clean-usnob CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields
Can Wang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
clean-usnob General Facial Representation Learning in a Visual-Linguistic Manner
Yinglin Zheng, Hao Yang, Ting Zhang, Jianmin Bao, Dongdong Chen, Yangyu Huang, Lu Yuan, Dong Chen, Ming Zeng, Fang Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022 Oral)
clean-usnob Vector Quantized Diffusion Model for Text-to-Image Synthesis
Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, Baining Guo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022 Oral)
clean-usnob Mobile-Former: Bridging MobileNet and Transformer
Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Xiaoyi Dong, Lu Yuan, Zicheng Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022 Oral)
clean-usnob CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, Baining Guo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022), CVPR2022 Top-10 Influential paper!!!
paper-image Reduce Information Loss in Transformers for Pluralistic Image Inpainting
Qiankun Liu, Zhentao Tan, Dongdong Chen, Qi Chu, Xiyang Dai, Yinpeng Chen, Mengchen Liu, Lu Yuan, Nenghai Yu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
paper-image Bringing Old Films Back to Life
Ziyu Wan, Bo Zhang, Dongdong Chen, Jing Liao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
paper-image High-Fidelity Pluralistic Image Completion with Transformers
Ziyu Wan, Jingbo Zhang, Dongdong Chen, Jing Liao
International Conference on Computer Vision (ICCV 2021)
paper-image Improve Unsupervised Pretraining for Few-label Transfer
Suichan Li, Dongdong Chen, Yinpeng Chen, Lu Yuan, Lei Zhang, Qi Chu, Bin Liu, Nenghai Yu
International Conference on Computer Vision (ICCV 2021)
paper-image Unsupervised Pre-training for Person Re-identification
Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, Dong Chen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021)
clean-usnob Dynamic Convolution via Matrix Decomposition
Yunsheng Li, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Dongdong Chen, Ye Yu, Lu Yuan, Zicheng Liu, Mei Chen, Nuno Vasconcelos
International Conference on Learning Representations 2021
clean-usnob Improved Image Matting via Real-time User Clicks and Uncertainty Estimation
Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Hanqing Zhao, Weiming Zhang, Nenghai Yu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021)
clean-usnob Diverse Semantic Image Synthesis via Probability Distribution Modeling
Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu, Bin Liu, Gang Hua, Nenghai Yu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021)
clean-usnob Efficient Semantic Image Synthesis via Class-Adaptive Normalization
Zhentao Tan, Dongdong Chen, Qi Chu, Menglei Chai, Jing Liao, Mingming He, Lu Yuan, Gang Hua Nenghai Yu
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI 2021)
clean-usnob Passport-aware Normalization for Deep Model Protection
Jie Zhang, Dongdong Chen, Jing Liao, Weiming Zhang, Gang Hua, Nenghai Yu
Advances in Neural Information Processing Systems (NeurIPs 2020)
clean-usnob MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu, Lu Yuan, Sergey Tulyakov, Nenghai Yu
ACM Transactions on Graphics (SIGGRAPH) 2020
clean-usnob Bringing Old Photos Back to Life
Ziyu Wan, Bo Zhang, Dongdong Chen, Pan Zhang, Dong Chen, Jing Liao, Fan Wen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2020 Oral), Over 15.3K github stars!!!
clean-usnob Density-Aware Graph for Deep Semi-Supervised Visual Recognition
Suichan Li, Bin Liu, Dongdong Chen, Qi Chu, Lu Yuan, Nenghai Yu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2020)
clean-usnob Model Watermarking for Image Processing Networks
Jie Zhang*, Dongdong Chen, et al., Nenghai Yu
Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020)
clean-usnob A General Decoupled Learning Framework for Parameterized Image Operators
Qingnan Fan*, Dongdong Chen*, Lu Yuan, Gang Hua, Nenghai Yu, Baoquan Chen (*Equal Contribution)
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI 2019)
clean-usnob Progressive Color Transfer with Dense Semantic Correspondences
Mingming He, Jing Liao, Dongdong Chen*, Lu Yuan, Pedro V Sander
ACM TOG 2018
clean-usnob Decouple Learning for Parameterized Image Operators
Qingnan Fan*, Dongdong Chen*, Lu Yuan, Gang Hua, Nenghai Yu, Baoquan Chen (* Equal Contribution)
European Conference on Computer Vision 2018
clean-usnob Deep Exemplar-based Colorization
Mingming He*, Dongdong Chen*, Jing Liao, Pedro V.Sander, Lu Yuan (* Equal Contribution)
ACM Transactions on Graphics (SIGGRAPH) 2018
clean-usnob Stereoscopic Neural Style Transfer
Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, Gang Hua
IEEE Conference on Computer Vision and Pattern Recognition 2018
clean-usnob Coherent Online Video Style Transfer
Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, Gang Hua
International Conference on Computer Vision 2017
clean-usnob StyleBank: An Explicit Representation for Neural Image Style Transfer
Dongdong Chen, Lu Yuan, Jing Liao, Nenghai Yu, Gang Hua
IEEE Conference on Computer Vision and Pattern Recognition 2017

Awards and honors

  • Outstanding Reviewer Award of CVPR 20/21, ICCV 21
  • Chinese Academy of Sciences President Award (Special), July 2019
  • National Scholarship for Graduate Students, Nov 2018
  • (3/2950 Teams) FashionAI Global Challenge—Attributes Recognition of Apparel 2018
  • (7/2322 Teams) FashionAI Global Challenge—Key Points Detection of Apparel 2018
  • National ScholarShip, Nov 2012/2013.
  • Grand Prize (1/1438 teams) in CCF National Youth Innovation Contest of Big Data 2015
  • Top 5 in Tianyi Algorithm Contest of Big Data 2016
  • Best Demo Prize of Di-Tech Algorithm Contest of Big Data 2016

website template borrowed from source code. Thanks!