Publications

Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jifeng Dai, Yu Qiao, Xizhou Zhu (2024). Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training. Preprint.

PDF Cite Project

Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, Jinhua Zhao (2024). Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Composite Spatial Reasoning. Preprint.

Cite

Yihong Tang, Zhaokai Wang, Ao Qu, Yihao Yan, Zhaofeng Wu, Dingyi Zhuang, Jushi Kai, Kebing Hou, Xiaotong Guo, Jinhua Zhao, Zhan Zhao, Wei Ma (2024). ITINERA: Integrating Spatial Optimization with Large Language Models for Open-domain Urban Itinerary Planning. In EMNLP 2024.

PDF Cite Blog

Xizhou Zhu, Xue Yang, Zhaokai Wang, Hao Li, Wenhan Dou, Junqi Ge, Lewei Lu, Yu Qiao, Jifeng Dai (2024). Parameter-Inverted Image Pyramid Networks. In NeurIPS 2024 (Spotlight).

PDF Cite Code Blog

Yihong Tang, Zhaokai Wang, Ao Qu, Yihao Yan, Kebing Hou, Dingyi Zhuang, Xiaotong Guo, Jinhua Zhao, Zhan Zhao, Wei Ma (2024). Synergizing Spatial Optimization with Large Language Models for Open-domain Urban Itinerary Planning. In KDD UrbComp 2024 (Best Paper Award).

PDF Cite Blog

Hao Li, Xue Yang, Zhaokai Wang, Xizhou Zhu, Jie Zhou, Yu Qiao, Xiaogang Wang, Hongsheng Li, Lewei Lu, Jifeng Dai (2023). Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft. In CVPR 2024.

PDF Cite Demo Blog

Le Zhuo, Zhaokai Wang, Baisen Wang, Yue Liao, Chenxi Bao, Stanley Peng, Songhao Han, Aixi Zhang, Fei Fang, Si Liu (2022). Video Background Music Generation: Dataset, Method and Evaluation. In ICCV 2023.

PDF Cite Demo

Shangzhe Di, Zeren Jiang, Si Liu, Zhaokai Wang, Leyan Zhu, Zexin He, Hongming Liu, Shuicheng Yan (2021). Video Background Music Generation with Controllable Music Transformer. In ACM MM 2021 (Best Paper Award).

PDF Cite Code Colab Notebook Demo

Zhaokai Wang, Renda Bao, Qi Wu, Si Liu (2021). Confidence-aware Non-repetitive Multimodal Transformers for TextCaps. In AAAI 2021.

PDF Cite Code