Sucheng (Oliver) Ren

Hi, I am Sucheng Ren (ไปป่‹ๆˆ), a Computer Science Ph.D. student at Johns Hopkins University, where I am fortunate to be advised by Bloomberg Distinguished Professor Alan Yuille and Prof. Cihang Xie. I received my B.S. and M.S. degree in Computer Science from South China University of Technology. Previously, I spent great time at Microsoft Research Asia (MSRA), Tsinghua University and National University of Singapore.

My research lies at the Self-Supervised Learning and Multimodal Learning.

I'm currently looking for research intern roles for Summer 2024!

Email  |  CV  |  Scholar  |  Github  | 

profile photo
News
  • [Aug. 2023] Join Johns Hopkins University as a PhD student!
  • [Jul. 2023] SG-Former got accepted by ICCV2022!๐ŸŽ‰
  • [Feb. 2023] TinyMIM got accepted by CVPR2023!๐ŸŽ‰
  • [Jun. 2022] Working with Dr. Han Hu and Fangyun Wei at Microsoft Research Asia (MSRA)!๐Ÿ‘ฉโ€๐Ÿ’ป
  • [Feb. 2022] Three first author papers got accepted by CVPR2022 including one Oral paper!๐ŸŽ‰
  • [Dec. 2021] Working with Prof. Jiashi Feng at National University of Singapore!๐Ÿ‘ฉโ€๐Ÿ’ป
  • [Jun. 2021] Working with Prof. Cihang Xie and Prof. Alan Yuille at John Hopkins University!๐Ÿ‘ฉโ€๐Ÿ’ป
  • [Jul. 2021] Two papers got accepted by ICCV2021 !๐ŸŽ‰
  • [May. 2021] Working with Prof. Cihang Xie and Prof. Alan Yuille at John Hopkins University!๐Ÿ‘ฉโ€๐Ÿ’ป
  • [Mar. 2021] Three papers got accepted by CVPR2021 including two first author papers!๐ŸŽ‰
  • [Dec. 2020] Working with Prof. Hang Zhao in Tsinghua as a research intern!๐Ÿ‘ฉโ€๐Ÿ’ป
Selected Publications
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models

Sucheng Ren, Fangyun Wei, Zheng Zhang, Han Hu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023
[paper] [code] [bibtex]

We explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones.

Shunted Self-Attention via Multi-Scale Token Aggregation

Sucheng Ren, Daquan Zhou, Shengfeng He, Jiashi Feng, Xinchao Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (Oral), 2022
[paper] [code] [bibtex]

Integrating the capability of capturing multiscale objects in each attention layer by adaptively merging tokens.

SG-Former: Self-guided Transformer with Evolving Token Reallocation

Sucheng Ren, Xingyi Yang, Songhua Liu, Xinchao Wang
International Conference on Computer Vision (ICCV), 2023
[paper] [code] [bibtex]

Integrating the capability of capturing multiscale objects in each attention layer by adaptively merging tokens.

Co-advise: Cross Inductive Bias Distillation

Sucheng Ren, Zhengqi Gao, Tianyu Hua, Zihui Xue, Yonglong Tian, Shengfeng He, Hang Zhao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[paper] [code] [bibtex]

The first work delves into the influence of models inductive biases in knowledge distillation

A Simple Data Mixing Prior for Improving Self-Supervised Learning

Sucheng Ren, Huiyu Wang, Zhengqi Gao, Shengfeng He, Alan Yuille, Yuyin Zhou, Cihang Xie
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[paper] [bibtex]

A generic training strategy in data mixing that can improve the self-supervised representation learning of both CNNs and ViTs

Multimodal Knowledge Expansion

Zihui Xue, Sucheng Ren, Zhengqi Gao, Hang Zhao
International Conference on Computer Vision (ICCV), 2021
[paper] [website] [bibtex]

A knowledge distillation-based framework to effectively utilize multimodal data without requiring labels.

Learning from the Master: Distilling Cross-modal Advanced Knowledge for Lip Reading
Sucheng Ren, Yong Du, Jianming Lv, Guoqiang Han, and Shengfeng He
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[paper] [bibtex]

Training a master to learn how to teach a better student.

Reciprocal Transformations for Unsupervised Video Object Segmentation

Sucheng Ren, Wenxi Liu, Yongtuo Liu, Haoxin Chen, Guoqiang Han and Shengfeng He
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[paper] [bibtex] [code]

Jointly learning salient objects, moving objects, recurring objects for Unsupervised Video Object Segmentation.

TENet: Triple Excitation Network for Video Salient Object Detection

Sucheng Ren, Chu Han, Xin Yang, Guoqiang Han and Shengfeng He
European Conference on Computer Vision (ECCV), 2020
(Spotlight, Acceptance Rate 5.0%)
[paper] [bibtex]