Qingnan Fan (樊庆楠)

I am a Lead Researcher in the 3D Photography Research Center of VIVO. Our group is the core algorithm team responsible for advancing the photographic quality in the flagship smartphones with the cutting-edge technologies (3D, AIGC, etc).

I was a Senior Researcher in the Visual Computing Center of Tencent AI Lab between 2021 to 2023.

I was a Postdoctoral Researcher in Stanford University supervised by Prof. Leonidas Guibas between 2019 to 2021.

I obtained my PhD degree in the Computer Science and Technology School of Shandong University at 2019. I was supervised by Prof. Baoquan Chen.

If you are interested in the internship in our group for either publishing academic papers or landing the latest technologies, feel free to drop me an email.

Email  /  CV  /  Biography  /  Google Scholar  /  LinkedIn  /  Twitter  /  Github


I am the host of VVTalk, which invites the expert speakers in the fields of computer vision, graphics and computational photography. The mission of VVTalk is to promote the communication between academic and industrial communities, and explore the possibilities of landing the most innovative research ideas into any AI-powered industrial applications. If you are insterested in joining this family to give a talk, feel free to let me know.

2023-09-08: Xintao Wang, Tencent, Visual generation and editing via diffusion models

2023-08-15: Huan Ling, Nvidia, Generative models and vision perception using diffusion models

2023-07-25: Jianmin Bao, MSRA, 2D & 3D visual synthesis and manipulation via diffusion models


My research focus lies in computer graphics, 3D vision, image processing, and human-computer interaction. My recent effort has been spent on pushing the limit of 3D vision and reinforcement learning technologies to implement an intelligent embodied agent in both forms of physical robots and digital humans.

Scene-aware Activity Program Generation with Language Guidance
Zejia Su, Qingnan Fan, Xuelin Chen, Oliver van Kaick, Hui Huang, Ruizhen Hu.
SIGGRAPH Asia, 2023 & TOG, 2023
project page / supp file / bibtex

We address the problem of scene-aware activity program generation, which requires decomposing a given activity task into instructions that can be sequentially performed within a target scene to complete the activity.

C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
Zhiyang Dou, Xuelin Chen, Qingnan Fan, Taku Komura, Wenping Wang.
SIGGRAPH Asia, 2023
arXiv / project page / video / bibtex

We present C·ASE, an efficient and effective framework that learns conditional Adversarial Skill Embeddings for Elite physics-based characters.

3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification
Jiazhao Zhang*, Liu Dai*, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, He Wang.
CVPR, 2023
arXiv / project page / bibtex

We propose a framework for the challenging 3D-aware Object goal navigation task based on two straightforward sub-policies. The two sub-polices, namely corner-guided exploration policy and category-aware identification policy, simultaneously perform by utilizing online fused 3D points as observation.

DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Object Manipulation
Yan Zhao*, Ruihai Wu*, Zhehuan Chen, Yourong Zhang, Qingnan Fan, Kaichun Mo, Hao Dong.
ICLR, 2023
arXiv / project page / video / bibtex

We propose a novel learning framework, DualAfford, to learn collaborative affordance for dual-gripper manipulation tasks. The core design of the approach is to reduce the quadratic problem for two grippers into two disentangled yet interconnected subtasks for efficient learning.

Towards Accurate Active Camera Localization
Qihang Fang*, Yingda Yin*, Qingnan Fan, Fei Xia, Siyan Dong, Sheng Wang, Jue Wang, Leonidas Guibas, Baoquan Chen.
ECCV, 2022
arXiv / codes / video / supp file / bibtex

In this work, we explicitly model the camera and scene uncertainty components to solve the problem of active camera localization by reinforcement learning. Our algorithm improves over the state-of-the-art Markov Localization based approaches by a large margin on the fine-scale camera pose accuracy.

AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions
Yian Wang*, Ruihai Wu*, Kaichun Mo*, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, Hao Dong.
ECCV, 2022
arXiv / project page / codes / video / bibtex

In this paper, we propose a novel framework, named AdaAfford, that learns to perform very few test-time interactions for quickly adapting the affordance priors to more accurate instance-specific posteriors.

Multi-Robot Active Mapping via Neural Bipartite Graph Matching
Kai Ye*, Siyan Dong*, Qingnan Fan, He Wang, Li Yi, Fei Xia, Jue Wang, Baoquan Chen.
CVPR, 2022
arXiv / codes / video / supp file / poster / bibtex

We propose a novel multi-robot active mapping algorithm by reducing the problem to bipartite graph matching, solved by the proposed multiplex graph neural network (mGNN) via reinforcement learning.

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Shift in Semantic Segmentation
Yanchao Yang*, Hanxiang Ren*, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas.
CVPR, 2022 (Oral)
arXiv / bibtex

We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects
Ruihai Wu*, Yan Zhao*, Kaichun Mo*, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, Hao Dong.
ICLR, 2022
arXiv / project page / codes / video / bibtex

We design an interaction-for-perception framework, VAT-MART, to learn actionable visual representations for more effective manipulation of 3D articulated objects.

CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds
Yijia Weng*, He Wang*, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, Leonidas Guibas.
ICCV, 2021 (Oral)
arXiv / project page / codes / video / bibtex

For the first time, we propose a unified framework that can handle 9-DoF pose tracking for novel rigid object instances as well as per-part pose tracking for 3D articulated objects.

Contrastive Multimodal Fusion with TupleInfoNCE
Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi.
ICCV, 2021
arXiv / bibtex

We propose a novel contrastive learning objective, TupleInfoNCE. It contrasts tuples based not only on positive and negative correspondences, but also by composing new negative tuples using modalities describing different scenes.

Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments
Siyan Dong*, Qingnan Fan*, He Wang, Ji Shi, Li Yi, Thomas Funkhouser, Baoquan Chen, Leonidas Guibas.
CVPR, 2021 (Oral)
arXiv / codes / video / bibtex

A novel outlier-aware neural tree to tackle the camera localization challenges in dynamic indoor environments. It achieves the best performance in the RIO-10 benchmark.

Generating Manga from Illustrations via Mimicking Manga Creation Workflow
Lvmin Zhang, Xinrui Wang, Qingnan Fan, Yi Ji, ChunPing Liu.
CVPR, 2021
project page / bibtex

A data-driven framework to convert a digital illustration into three corresponding components: manga line drawing, regular screentone, and irregular screen texture. These components can be directly composed into manga images and can be further retouched for more plentiful manga creations.

A General Decoupled Learning Framework for Parameterized Image Operators
Qingnan Fan*, Dongdong Chen*, Lu Yuan, Gang Hua, Nenghai Yu, Baoquan Chen.
TPAMI, 2021
arXiv / codes / bibtex

A journal extension of our ECCV 2018 paper. We further propose a cheap parameter-tuning version of the decouple learning framework that enables real-time alternation between different image operators.

Generative 3D Part Assembly via Dynamic Graph Learning
Jialei Huang*, Guanqi Zhan*, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas Guibas, Hao Dong.
NeurIPS, 2020
arXiv / project page / codes / bibtex / press (机器之心,AI科技评论)

A dynamic graph learning algorithm for autonomous part assembly. It learns to reason an assembly-oriented dynamically-evolved relation graph, which indicates the assembly process which is guided by the major parts (chair leg&seat).

Controllable Image Processing via Adaptive FilterBank Pyramid
Dongdong Chen, Qingnan Fan, Jing Liao, Angelica I. Aviles-Rivero, Lu Yuan, Nenghai Yu, Gang Hua.
TIP, 2020

we propose a new plugin module, “Adaptive Filterbank Pyramid”, which can be inserted into a backbone network to support multiple operators and continuous parameter tuning.

RainFlow: Optical Flow under Rain Streaks and Rain Veiling Effect
Ruoteng Li, Robby T. Tan, Loong-Fah Cheong, Angelica I. Aviles-Rivero, Qingnan Fan, Carola-Bibiane Schönlieb.
ICCV, 2019

A deep-learning based optical flow approach designed to handle heavy rain.

GraphXNET - Chest X-Ray Classification Under Extreme Minimal Supervision
Angelica Aviles-Rivero, Nicolas Papadakis, Ruoteng Li, Philip Sellars, Qingnan Fan, Robby Tan, Carola-Bibiane Schönlieb.
MICCAI, 2019
arXiv / bibtex

A novel semi-supervised framework for X-ray classification which is based on a graph-based optimisation model. A new multi-class classification functional that strengthens the synergy between the limited number of labels and the huge amount of unlabelled data.

Mirror, Mirror, on the Wall, Who's Got the Clearest Image of Them All? - A Tailored Approach to Single Image Reflection Removal
Daniel Heydecker*, Georg Maierhofer*, Angelica Aviles-Rivero*, Qingnan Fan, Dongdong Chen, Carola-Bibiane Schönlieb, Sabine Süsstrunk.
TIP, 2019
arXiv / bibtex

A simple and tractable user interactive tool for single image reflection removal, which is facilitated with a spatially-aware prior term solved by an efficient half-quadratic splitting optimization approach.

Gated Context Aggregation Network for Image Dehazing and Deraining
Dongdong Chen, Mingming He, Qingnan Fan, Jing Liao, Liheng Zhang, Dongdong Hou, Lu Yuan, Gang Hua.
WACV, 2019
arXiv / codes / bibtex

A novel end-to-end gated context aggregation network GCANet that outperforms all the existing appraoches by a large margin on both image dehazing and deraining tasks.

Image Smoothing via Unsupervised Learning
Qingnan Fan, Jiaolong Yang, David Wipf, Baoquan Chen, Xin Tong.
SIGGRAPH Asia, 2018 & TOG, 2018
arXiv / codes / supp file / bibtex

Treat deep learning as an optimization tool to minimize the proposed image smoothing objective function in an unsupervised manner. Multiple different smoothing effects can be easily learned by adaptively changing the proposed objective function.

Decouple Learning for Parameterized Image Operators
Qingnan Fan*, Dongdong Chen*, Lu Yuan, Gang Hua, Nenghai Yu, Baoquan Chen.
ECCV, 2018
arXiv / codes / supp file / poster / bibtex

The first decouple learning framework that is capable of successfully incorporating many different parameterized image operators into a single network without requirement of retraining or fintuning any other networks.

Revisiting Deep Intrinsic Image Decompositions
Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf.
CVPR, 2018 (Oral)
arXiv / codes / slides / supp file / poster / presentation (start from 36:44) / bibtex

The first demonstration of a single basic deep architecture capable of achieving state-of-the-art results when applied to each of the major intrinsic benchmarks.

A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing
Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf.
ICCV, 2017
arXiv / codes / supp file / poster / bibtex

An advanced deep architecture for low-level vision tasks; A novel reflection image synthesis approach which enables outstanding generalization ability to real images with trained newtork.

JumpCut: Non-Successive Mask Transfer and Interpolation for Video Cutout
Qingnan Fan, Fan Zhong, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen.
SIGGRAPH Asia, 2015 & TOG, 2015
codes / slides / video / supp file / dataset / bibtex

An interactive real-time video segmentation algorithm. Significantly improve the video cutout accuracy and efficiency.

Build-to-Last: Strength to Weight 3D Printed Objects
Lin Lu, Andrei Sharf, Haisen Zhao, Yuan Wei, Qingnan Fan, Xuelin Chen, Yann Savoye, Changhe Tu, Daniel Cohen-Or, Baoquan Chen.
SIGGRAPH, 2014 & TOG, 2014
video / bibtex

Reduce the material cost and weight of a given object while providing a durable printed model that is resistant to impact and external forces.

Interesting research projects
Single image reflection removal via learning with multi-image constraints
Yingda Yin*, Qingnan Fan*, Dongdong Chen, Yujie Wang, Angelica Aviles-Rivero, Ruoteng Li, Carola-Bibiane Schönlieb, Baoquan Chen.

Our algorithm works by learning a deep neural network to optimize the target with joint constraints enhanced among multiple input images during the training phase, but is able to eliminate reflections only from a single input for evaluation.

RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment
Mingxin Yu*, Lin Shao*, Zhehuan Chen, Tianhao Wu, Qingnan Fan, Kaichun Mo, Hao Dong.
Arxiv / Project page

We formulate the part assembly task as a concrete reinforcement learning problem and propose a pipeline for robots to learn to assemble a diverse set of chairs.

IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration
Ziyi Wu, Yueqi Duan, He Wang, Qingnan Fan, Leonidas Guibas.
Arxiv / Codes

We propose an IF-Defense framework to directly optimize the coordinates of input points with geometry-aware and distribution-aware constraints, to address the 3D adversarial attacks.

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding
Yunze Liu, Li Yi, Shanghang Zhang, Qingnan Fan, Thomas Funkhouser, Hao Dong.

We propose contrasting “pairs of point-pixel pairs”, where positives include pairs of RGB-D points in correspondence, and negatives include pairs where one of the two modalities has been disturbed and/or the two RGB-D points are not in correspondence.

Work experience
Lead Researcher
Tencent AI Lab
Senior Researcher
Stanford University
Postdoctoral Researcher
Supervised by Prof. Leonidas Guibas
Shandong University
Ph.D. student
Supervised by Prof. Baoquan Chen
Shandong University
Undergraduate student
Research experience
Beijing Film Academy
Research Intern
Supervised by Prof. Baoquan Chen
University of Cambridge
Visiting Student
Supervised by Prof. Carola-Bibiane Schönlieb
Microsoft Research Asia
Research Intern
Supervised by Gang Hua, Xin Tong, Jiaolong Yang and David Wipf
Tel Aviv University
Visiting Student
Supervised by Prof. Daniel Cohen-Or
The Hebrew University of Jerusalem
Visiting Student
Supervised by Prof. Dani Lischinski

APR. 2022: Active 3D scene understanding and its applications
“三维视觉与智能图形”前沿论坛, 图图名师讲堂, China

OCT. 2021: Visual Localization
Embodied AI Workshop, Valse, China

JAN. 2019: Deep Learning in Computational Photography
USC ICT/UW Reality Lab/Berkeley/Stanford/Google/MSR, US

DEC. 2018: Deep Learning for Single Image Artifact Removal
ACCV Tutorial 2018, Australia

DEC. 2018: Image Smoothing via Unsupervised Learning
SIGGRAPH Asia 2018, Japan; GAMES Webinar, China

AUG. 2018: Discovering Unsupervised Learning in Image Processing
CIA, Cambridge University, UK

JUN. 2018: Revisiting Deep Intrinsic Image Decomposition
CVPR 2018, USA

NOV. 2015: Interactive Real-time Video Segmentation
SIGGRAPH Asia 2015, Japan


2022: Tencent Outstanding Contributor

2020: CCF Doctorial Dissertation Award Nominee (CCF 优博提名)

2019: Outstanding Academic Achievement Award of Shandong University

2018: Academic Star Nominee of Shandong University (10/20000)

2018: National Scholarship

2016: Outstanding Academic Achievement Award of Shandong University

2015: Presidential Scholarship of Shandong University (35/20000) (Highest honor for students in SDU, only 35 elected among around 20000 candidates)

2015: National Scholarship

2015: Pacemaker to Outstanding Graduate Student of Shandong University