PhD Dissertation Proposal by ANG Meng Kiat Gary | Learning Dynamic Multimodal Networks

Capturing and modelling relationships between node entities and attributes associated with node entities is an important challenge in network or graph learning. In this dissertation research, we focus on modelling an important class of networks in many real-world applications.  These networks involve i) attributes from multiple modalities, also known as multimodal attributes; ii) multimodal attributes that are not static but time-series information, i.e. dynamic multimodal attributes, and iii) relationships that evolve across time, i.e. dynamic networks. An example of networks with multimodal attributes are networks of design objects in user interfaces (UI), e.g., links between UI screens and their constituent UI elements, where the design objects may be associated with visual UI screen and element images, textual UI descriptions and code, as well as numerical UI rating attributes. An example of dynamic networks and dynamic multimodal attributes are relationships between companies that evolve across time, e.g., commercial relationships between companies, where the companies may be associated with time-series of numerical stock prices, textual news and categorical event attributes. While there has been significant progress in the area of network or graph learning, most existing works do not focus on modelling such dynamic multimodal networks. Hence, in this proposal, we discuss key challenges associated with learning dynamic multimodal networks, and discuss completed and planned research on approaches that capture and model multimodal attributes, dynamic multimodal attributes and dynamic networks.  For the completed research, we describe our proposed models and report the experiment results that highlight our research contributions.

Gary Ang is a PHD candidate in the School of Computing and Information Systems, Singapore Management University, supervised by Professor Lim Ee Peng. His research interests include network and time-series modelling in domains such as user interfaces and finance.