Zhaokai Wang

(王肇凯)

Biography

I am a third-year Ph.D. candidate at Shanghai Jiao Tong University, supervised by Prof. Jifeng Dai. I obtained my bachelor’s degree from Beihang University in 2022, where I worked with Prof. Si Liu. I also have a double bachelor’s degree in economics from Peking University. Currently, I am an intern at OpenGVLab of Shanghai AI Laboratory. Previously I interned at SenseTime and Sea AI Lab.

Interests

Artificial Intelligence
Computer Vision
Music Generation

Education

Ph.D. (Joint Program with Shanghai AI Lab), 2022-

Department of EE, Shanghai Jiao Tong University
B.A. in Economics (Double Major), 2019-2022

National School of Development, Peking University
B.Eng. in Computer Science, 2018-2022

Shenyuan Honors College, Beihang University

Working Experience

Research Intern

Shanghai AI Lab

Nov 2022 – Present Shanghai

OpenGVLab

Research Intern

SenseTime

Feb 2022 – Oct 2022 Beijing

Fundamental Vision Group

Research Intern

Sea AI Lab

Aug 2021 – Feb 2022 Beijing

Research on music AI

News

2024.10: ⭐️ Our paper ItiNeta on LLM for urban itinerary generation is accepted by EMNLP 2024.
2024.9: ⭐️ Our paper PIIP on efficient vision backbone is accepted by NeurIPS 2024 as Spotlight.
2024.8: 🏆 Our paper ItiNera on LLM for urban itinerary generation is awarded the Best Paper Award of KDD Urban Computing Workshop (UrbComp) 2024!
2024.2: ⭐️ Our paper Auto MC Reward on LLM for Minecraft RL agents is accepted by CVPR 2024.

Publications

Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jifeng Dai, Yu Qiao, Xizhou Zhu (2024). Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training. Preprint.

PDF Cite Project

Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, Jinhua Zhao (2024). Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Composite Spatial Reasoning. Preprint.

Cite

Yihong Tang, Zhaokai Wang, Ao Qu, Yihao Yan, Zhaofeng Wu, Dingyi Zhuang, Jushi Kai, Kebing Hou, Xiaotong Guo, Jinhua Zhao, Zhan Zhao, Wei Ma (2024). ITINERA: Integrating Spatial Optimization with Large Language Models for Open-domain Urban Itinerary Planning. In EMNLP 2024.

PDF Cite Blog

Xizhou Zhu, Xue Yang, Zhaokai Wang, Hao Li, Wenhan Dou, Junqi Ge, Lewei Lu, Yu Qiao, Jifeng Dai (2024). Parameter-Inverted Image Pyramid Networks. In NeurIPS 2024 (Spotlight).

PDF Cite Code Blog

Yihong Tang, Zhaokai Wang, Ao Qu, Yihao Yan, Kebing Hou, Dingyi Zhuang, Xiaotong Guo, Jinhua Zhao, Zhan Zhao, Wei Ma (2024). Synergizing Spatial Optimization with Large Language Models for Open-domain Urban Itinerary Planning. In KDD UrbComp 2024 (Best Paper Award).

PDF Cite Blog

Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, Jifeng Dai (2023). Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft. In CVPR 2024.

PDF Cite Demo Blog

Le Zhuo, Zhaokai Wang, Baisen Wang, Yue Liao, Chenxi Bao, Stanley Peng, Songhao Han, Aixi Zhang, Fei Fang, Si Liu (2022). Video Background Music Generation: Dataset, Method and Evaluation. In ICCV 2023.

PDF Cite Demo

Shangzhe Di, Zeren Jiang, Si Liu, Zhaokai Wang, Leyan Zhu, Zexin He, Hongming Liu, Shuicheng Yan (2021). Video Background Music Generation with Controllable Music Transformer. In ACM MM 2021 (Best Paper Award).

PDF Cite Code Colab Notebook Demo