ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large Vision-and-Language Models

Yuqi Liu, Liangyu Chen, Jiazhen Liu, Mingkang Zhu, Zhisheng Zhong, Bei Yu, Jiaya Jia

Project Page

• ViSurf (Visual Supervised-and-Reinforcement Fine-Tuning) is a unified post-training paradigm that integrates the strengths of both SFT and RLVR within a single stage.

VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning

Yuqi Liu^* , Tianyuan Qu^* , Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, Jiaya Jia

Project Page

• VisionReasoner is a unified framework for visual perception tasks.
• Through carefully crafted rewards and training strategy, VisionReasoner has strong multi-task capability, addressing diverse visual perception tasks within a shared model.

Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement

Yuqi Liu^* , Bohao Peng^* , Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, Jiaya Jia

Project Page

• Seg-Zero exhibits emergent test-time reasoning ability. It generates a reasoning chain before producing the final segmentation mask.
• Seg-Zero is trained exclusively using reinforcement learning, without any explicit supervised reasoning data.
• Compared to supervised fine-tuning, our Seg-Zero achieves superior performance on both in-domain and out-of-domain data.

Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition

Zhisheng Zhong^*, Chengyao Wang^*, Yuqi Liu^*, Senqiao Yang,Longxiang Tang, Yuechen Zhang, Jingyao Li, Tianyuan Qu, Yanwei Li, Yukang Chen, Shaozuo Yu, Sitong Wu, Eric Lo, Shu Liu, Jiaya Jia

Project Page

• Stronger performance: Achieve SOTA results across a variety of speech-centric tasks.
• More versatile: Support image, video, speech/long-speech, sound understanding and speech generation.
• More efficient: Less training data, support faster training and inference.

Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrieval

Yang Du^*, Yuqi Liu^*, Qin Jin

• A benchmark aims to evaluate temporal understanding of video retrieval models.

Token Mixing: Parameter-Efficient Transfer Learning from Image-Language to Video-Language

Yuqi Liu, Luhui Xu, Pengfei Xiong, Qin Jin

Project Page

• We study how to transfer knowledge from image-language model to video-language tasks.
• We have implemented several components proposed by recent works.

TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval

Yuqi Liu, Pengfei Xiong, Luhui Xu, Shengming Cao, Qin Jin

Project Page

• TS2-Net is a text-video retrieval model based on CLIP.
• We propose our token shift transformer and token selection transformer.