PhD Dissertation Defense by MA Zhixin | Interactive Known-Item Search in Large Video Corpora

ABOUT THE TALK

The surge in video volume makes it challenging to locate a specific target with a single query using automatic video retrieval systems. The interactive video retrieval offers a solution by enabling users to iteratively refine a search. Nevertheless, existing systems often present users with an overwhelming number of similar videos, which can lead to mental fatigue while inspecting results and increase difficulty in providing feedback. This dissertation studies known-item video search and addresses four key challenges. First and foremost, as the link between users and the system, the interaction must be both efficient and effective. To ensure effectiveness, the user's workload should be minimized, as excessive effort can hinder their ability to thoroughly review the results. Efficiency, in this context, means reducing the number of iterations, as users are likely to abandon the task after a few unsuccessful attempts. Second, bridging the information gap between user intention and feedback is not trivial. As it is not realistic to assume that user intentions can always be clearly conveyed, there is always an information gap between user intention and system-received feedback. Third, resilience to noise or irrelevant search results is critically important, especially when narrowing the search space for more efficient search and browsing. A robust system should be able to identify and filter out noise to prevent from steering the search towards irrelevant results. Lastly, recruiting human evaluators to interact with a search system for training data collection is often impractical. Instead, designing a model to simulate user interactions is a common practice. The user simulation must be feasible and closely replicate human behavior. This dissertation contributes to develop an efficient, noise-resilient interactive video search system.

ABOUT THE SPEAKER

Zhixin MA is now a Ph.D. candidate in Computer Science at Singapore Management University, advised by Prof. Chong-Wah NGO. His general research interests lie in the field of multimedia computing, interactive retrieval, and video understanding. Now, he mainly focuses on interactive video retrieval, large-scale video processing, user simulation, and cross-modal LLMs.