Publications
My research focus lies in computer graphics, 3D vision, image processing, and human-computer interaction. My recent effort has been spent on pushing the limit of 3D vision and reinforcement learning technologies to implement an intelligent embodied agent in both forms of physical robots and digital humans.
|
|
3D Vision for Embodied AI
|
|
3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification
Jiazhao Zhang*,
Liu Dai*,
Fanpeng Meng,
Qingnan Fan,
Xuelin Chen,
Kai Xu,
He Wang.
CVPR, 2023
arXiv
/
project page
/
bibtex
We propose a framework for the challenging 3D-aware Object goal navigation task based on two straightforward sub-policies. The two sub-polices, namely
corner-guided exploration policy and category-aware identification policy, simultaneously perform by utilizing online fused 3D points as observation.
|
|
DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Object Manipulation
Yan Zhao*,
Ruihai Wu*,
Zhehuan Chen,
Yourong Zhang,
Qingnan Fan,
Kaichun Mo,
Hao Dong.
ICLR, 2023
arXiv
/
project page
/
video
/
bibtex
We propose a novel learning framework, DualAfford, to learn collaborative affordance for dual-gripper manipulation tasks. The core design of the approach is
to reduce the quadratic problem for two grippers into two disentangled yet interconnected subtasks for efficient learning.
|
|
Towards Accurate Active Camera Localization
Qihang Fang*,
Yingda Yin*,
Qingnan Fan,
Fei Xia,
Siyan Dong,
Sheng Wang,
Jue Wang,
Leonidas Guibas,
Baoquan Chen.
ECCV, 2022
arXiv
/
codes
/
video
/
supp file
/
bibtex
In this work, we explicitly model the camera and scene uncertainty components to solve the problem of active camera localization by reinforcement learning. Our algorithm improves over the state-of-the-art Markov Localization based approaches by a large margin on the fine-scale camera pose accuracy.
|
|
AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions
Yian Wang*,
Ruihai Wu*,
Kaichun Mo*,
Jiaqi Ke,
Qingnan Fan,
Leonidas Guibas,
Hao Dong.
ECCV, 2022
arXiv
/
project page
/
codes
/
video
/
bibtex
In this paper, we propose a novel framework, named AdaAfford, that learns to perform very few test-time interactions for quickly adapting the affordance priors to more accurate instance-specific posteriors.
|
|
Multi-Robot Active Mapping via Neural Bipartite Graph Matching
Kai Ye*,
Siyan Dong*,
Qingnan Fan,
He Wang,
Li Yi,
Fei Xia,
Jue Wang,
Baoquan Chen.
CVPR, 2022
arXiv
/
codes
/
video
/
supp file
/
poster
/
bibtex
We propose a novel multi-robot active mapping algorithm by reducing the problem to bipartite graph matching, solved by the proposed multiplex graph neural network (mGNN) via reinforcement learning.
|
|
VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects
Ruihai Wu*,
Yan Zhao*,
Kaichun Mo*,
Zizheng Guo,
Yian Wang,
Tianhao Wu,
Qingnan Fan,
Xuelin Chen,
Leonidas Guibas,
Hao Dong.
ICLR, 2022
arXiv
/
project page
/
codes
/
video
/
bibtex
We design an interaction-for-perception framework, VAT-MART, to learn actionable visual representations for more effective manipulation of 3D articulated objects.
|
|
CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds
Yijia Weng*,
He Wang*,
Qiang Zhou,
Yuzhe Qin,
Yueqi Duan,
Qingnan Fan,
Baoquan Chen,
Hao Su,
Leonidas Guibas.
ICCV, 2021 (Oral)
arXiv
/
project page
/
codes
/
video
/
bibtex
For the first time, we propose a unified framework that can handle 9-DoF pose tracking for novel rigid object instances as well as per-part pose tracking for 3D articulated objects.
|
|
Contrastive Multimodal Fusion with TupleInfoNCE
Yunze Liu,
Qingnan Fan,
Shanghang Zhang,
Hao Dong,
Thomas Funkhouser,
Li Yi.
ICCV, 2021
arXiv
/
bibtex
We propose a novel contrastive learning objective, TupleInfoNCE. It contrasts tuples based not only on positive and negative correspondences, but also by composing new negative tuples using modalities describing different scenes.
|
|
Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments
Siyan Dong*,
Qingnan Fan*,
He Wang,
Ji Shi,
Li Yi,
Thomas Funkhouser,
Baoquan Chen,
Leonidas Guibas.
CVPR, 2021 (Oral)
arXiv
/
codes
/
video
/
bibtex
A novel outlier-aware neural tree to tackle the camera localization challenges in dynamic indoor environments. It achieves the best performance in the RIO-10 benchmark.
|
|
Generative 3D Part Assembly via Dynamic Graph Learning
Jialei Huang*,
Guanqi Zhan*,
Qingnan Fan,
Kaichun Mo,
Lin Shao,
Baoquan Chen,
Leonidas Guibas,
Hao Dong.
NeurIPS, 2020
arXiv
/
project page
/
codes
/
bibtex
/
press (机器之心,AI科技评论)
A dynamic graph learning algorithm for autonomous part assembly. It learns to reason an assembly-oriented dynamically-evolved relation graph, which indicates the assembly process which is guided by the major parts (chair leg&seat).
|
|
Build-to-Last: Strength to Weight 3D Printed Objects
Lin Lu,
Andrei Sharf,
Haisen Zhao, Yuan Wei,
Qingnan Fan, Xuelin Chen,
Yann Savoye,
Changhe Tu,
Daniel Cohen-Or,
Baoquan Chen.
SIGGRAPH, 2014 & TOG, 2014
video
/
bibtex
Reduce the material cost and weight of a given object while providing a durable printed model that is resistant to impact and external forces.
|
|
Image and Video Processing
|
|
ADeLA: Automatic Dense Labeling with Attention for Viewpoint Shift in Semantic Segmentation
Yanchao Yang*,
Hanxiang Ren*,
He Wang,
Bokui Shen,
Qingnan Fan,
Youyi Zheng,
C. Karen Liu,
Leonidas Guibas.
CVPR, 2022 (Oral)
arXiv
/
bibtex
We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.
|
|
Generating Manga from Illustrations via Mimicking Manga Creation Workflow
Lvmin Zhang,
Xinrui Wang,
Qingnan Fan,
Yi Ji,
ChunPing Liu.
CVPR, 2021
project page
/
bibtex
A data-driven framework to convert a digital illustration into three corresponding components: manga line drawing, regular screentone, and irregular screen texture. These components can be directly composed into manga images and can be further retouched for more plentiful manga creations.
|
|
A General Decoupled Learning Framework for Parameterized Image Operators
Qingnan Fan*,
Dongdong Chen*,
Lu Yuan,
Gang Hua,
Nenghai Yu,
Baoquan Chen.
TPAMI, 2021
arXiv
/
codes
/
bibtex
A journal extension of our ECCV 2018 paper. We further propose a cheap parameter-tuning version of the decouple learning framework that enables real-time alternation between different image operators.
|
|
Controllable Image Processing via Adaptive FilterBank Pyramid
Dongdong Chen,
Qingnan Fan,
Jing Liao,
Angelica I. Aviles-Rivero,
Lu Yuan,
Nenghai Yu,
Gang Hua.
TIP, 2020
bibtex
we propose a new plugin module, “Adaptive Filterbank Pyramid”, which can be inserted into a backbone network to support multiple operators and continuous parameter tuning.
|
|
RainFlow: Optical Flow under Rain Streaks and Rain Veiling Effect
Ruoteng Li,
Robby T. Tan,
Loong-Fah Cheong,
Angelica I. Aviles-Rivero,
Qingnan Fan,
Carola-Bibiane Schönlieb.
ICCV, 2019
bibtex
A deep-learning based optical flow approach designed to handle heavy rain.
|
|
GraphXNET - Chest X-Ray Classification Under Extreme Minimal Supervision
Angelica Aviles-Rivero,
Nicolas Papadakis,
Ruoteng Li,
Philip Sellars,
Qingnan Fan,
Robby Tan,
Carola-Bibiane Schönlieb.
MICCAI, 2019
arXiv
/
bibtex
A novel semi-supervised framework for X-ray classification which is based on a graph-based optimisation model. A new multi-class classification functional that strengthens the synergy between the limited number of labels and the huge amount of unlabelled data.
|
|
Mirror, Mirror, on the Wall, Who's Got the Clearest Image of Them All? - A Tailored Approach to Single Image Reflection Removal
Daniel Heydecker*,
Georg Maierhofer*,
Angelica Aviles-Rivero*,
Qingnan Fan,
Dongdong Chen,
Carola-Bibiane Schönlieb,
Sabine Süsstrunk.
TIP, 2019
arXiv
/
bibtex
A simple and tractable user interactive tool for single image reflection removal, which is facilitated with a spatially-aware prior term solved by an efficient half-quadratic splitting optimization approach.
|
|
Gated Context Aggregation Network for Image Dehazing and Deraining
Dongdong Chen,
Mingming He,
Qingnan Fan,
Jing Liao,
Liheng Zhang,
Dongdong Hou,
Lu Yuan,
Gang Hua.
WACV, 2019
arXiv
/
codes
/
bibtex
A novel end-to-end gated context aggregation network GCANet that outperforms all the existing appraoches by a large margin on both image dehazing and deraining tasks.
|
|
Image Smoothing via Unsupervised Learning
Qingnan Fan,
Jiaolong Yang,
David Wipf,
Baoquan Chen,
Xin Tong.
SIGGRAPH Asia, 2018 & TOG, 2018
arXiv
/
codes
/
supp file
/
bibtex
Treat deep learning as an optimization tool to minimize the proposed image smoothing objective function in an unsupervised manner. Multiple different smoothing effects can be easily learned by adaptively changing the proposed objective function.
|
|
Decouple Learning for Parameterized Image Operators
Qingnan Fan*,
Dongdong Chen*,
Lu Yuan,
Gang Hua,
Nenghai Yu,
Baoquan Chen.
ECCV, 2018
arXiv
/
codes
/
supp file
/
poster
/
bibtex
The first decouple learning framework that is capable of successfully incorporating many different parameterized image operators into a single network without requirement of retraining or fintuning any other networks.
|
|
Revisiting Deep Intrinsic Image Decompositions
Qingnan Fan,
Jiaolong Yang,
Gang Hua,
Baoquan Chen,
David Wipf.
CVPR, 2018 (Oral)
arXiv
/
codes
/
slides
/
supp file
/
poster
/
presentation (start from 36:44)
/
bibtex
The first demonstration of a single basic deep architecture capable of achieving state-of-the-art results when applied to each of the major intrinsic benchmarks.
|
|
A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing
Qingnan Fan,
Jiaolong Yang,
Gang Hua,
Baoquan Chen,
David Wipf.
ICCV, 2017
arXiv
/
codes
/
supp file
/
poster
/
bibtex
An advanced deep architecture for low-level vision tasks; A novel reflection image synthesis approach which enables outstanding generalization ability to real images with trained newtork.
|
|
JumpCut: Non-Successive Mask Transfer and Interpolation for Video Cutout
Qingnan Fan,
Fan Zhong,
Dani Lischinski,
Daniel Cohen-Or,
Baoquan Chen.
SIGGRAPH Asia, 2015 & TOG, 2015
codes
/
slides
/
video
/
supp file
/
dataset
/
bibtex
An interactive real-time video segmentation algorithm. Significantly improve the video cutout accuracy and efficiency.
|
|