PhD Dissertation Proposal by CHENG Yu Tong | Conversational Search for Video Retrieval

ABOUT THE TALK

As digital video libraries expand exponentially, our search engines remain trapped in a passive, "one-shot" paradigm that forces users through exhausting loops of visual browsing fatigue. This talk introduces a transition in search paradigm: transforming video retrieval from a rigid query-ranking mechanism into an active, cooperative human-machine dialogue grounded in information theory. Moving beyond traditional system limitations, we will explore a conversational framework designed to dynamically narrow down the search space in real time. We will examine how training-free active concept pruning optimizes textual search by breaking away from the pitfalls of classical feedback loops. From there, we will discuss about how entropy can be use as a "central controller" to dynamically orchestrate cross-modal interaction modalities—like visual exemplars and fluid open questions—without the constraints of a static vocabulary. Finally, we will preview how this framework extends active reasoning directly into fine-grained scene layouts using intelligent spatial layout questioning. Ultimately demonstrating how bridging multimodal representation with rigorous information theory can fundamentally redefine how we navigate the massive visual landscapes of tomorrow.

ABOUT THE SPEAKER

CHENG Yu Tong is a Ph.D. candidate supervised by Prof. NGO Chong Wah at Singapore Management University. His research primarily focuses on the field of multimedia retrieval, with a specialization in the known-item search task. He also has an extensive background in system design and competitive benchmarking, having participated in numerous iterations of the Video Browser Showdown (VBS). Notably, in the most recent VBS competition, his team's retrieval system, VIREO, achieved first place in the visual known-item search category. His current research investigates the development of smart conversational search systems, aiming to leverage interactive querying to streamline user interaction and improve retrieval accuracy in large-scale video datasets like TRECVid.