Projects

This survey reviews and outlooks the current audio-visual learning field from different aspects. We hope it can provide researchers with a better understanding of this area.

This project focus on the multi-modal perception and understanding with the motivation of cognitive neuroscience.

The Young Elite Scientists Sponsorship Program is a national young talents program first initiated in October 2015. It mainly supports young scientific and technological talents under the age of 32 with great innovation ability and development potential, and helps them become high-level leading talents and reserve forces of high-level innovation teams.

This project is oriented to complex dynamic audio-visual scenes. With a cross-domain matching model to mine the consistent representation of audiovisual modalities, we establish a dialogue mechanism and improve the speaker tracking and diarization performance.

This project is proposed to transfer cross-modal knowledge for the unconstrained videos. We consider the heterogeneous audio-visual correspondence in in-the-wild videos to improve the performance of cross-modal knowledge distillation.

GeWu Lab is working on how to develop more effectively cross-modal encoding scheme for the disable people with AI technique and this project is trying to help more blind-friends in China Mainland.

Based on Mi CyberDog, the project is dedicated to narrowing gaps in the Sim2Real process of Audio-Visual Navigation.