PhD Dissertation Proposal by WANG Qing | Domain Adaptation and Causal Inference for Food Computing

Food computing encompasses a wide range of tasks and applications related to food, including food recognition, food recipe retrieval and more.

Food recognition is a fundamental task because it enables automatic logging of food intake for nutrition estimation. However, food recognition is challenging because real-world food datasets tend to be long-tailed distributed and domain shifted. To this end, we will present a unified framework to address the domain shift and class imbalance in food recognition.

Recipe retrieval aims to match images of dishes with corresponding recipes. In real world, a food image may not equally capture every detail in a recipe, due to factors such as the cooking process, dish presentation, which makes the current retrieval learning tends to capture dominant visual-text alignment. We model such bias in cross-modal representation learning using causal theory and utilize a simple backdoor adjustment to alleviate the bias. We empirically prove the oracle performance of retrieval and propose a new neural architecture to approach the theoretical upbound performance for this problem.
 

Wang Qing is a Ph.D. candidate at SMU SCIS, supervised by Prof. NGO Chong Wah. Qing's research focuses on addressing the complexities of food recognition and retrieval, particularly in the presence of domain gaps and biases related to food ingredients.