PhD Dissertation Proposal by GOKARN Ila Nitin | Enabling Saliency-Aware Optimised Machine Perception at the Edge

Cyber-physical systems and applications have fundamentally changed people and processes in the way they interact with the physical world, ushering in the fourth industrial revolution. Supported by a variety of sensors, hardware platforms, artificial intelligence and machine learning models, and systems frameworks, CPS systems aim to automate and ease the burden of repetitive, laborious, or unsafe tasks borne by humans. However, efficient and effective machine perception remains a formidable challenge in sustaining high fidelity and high throughput of perception tasks on affordable edge devices. This is especially given the continuing increase in resolution of sensor streams (e.g., video input streams generated by 4K/8K cameras) and computational complexity of Deep Neural Network (DNN) models, which overwhelms embedded platforms, adversely impacting machine perception efficiency. Given the insufficiency of the available computation resources, a question then arises on whether parts of the perception task can be prioritized (and executed preferentially) to achieve highest task fidelity while adhering to the resource budget. This thesis explores the concept of saliency-awareness from the field of cognitive psychology in the context of machine perception pipelines on resource-constrained platforms, in guiding perception pipelines and systems on ``what" to pay attention to in the sensing field and ``when", to maximize overall perception fidelity under computational constraints.

Specifically, this thesis explores attention-based processing of multiple concurrent video streams to differentially process with higher priority those regions/objects in the sensing field that are determined to have higher saliency across timescales, thereby reducing the total computational volume needed. With novel pipelines such as spatio-temporal saliency mapping, canvas-based processing and attention scheduling, this thesis demonstrates that it is possible to achieve both high throughput and high accuracy, across multiple concurrent video streams at the resource-constrained edge, for real-time multi-sensor machine perception tasks, such as in surveillance and autonomous vehicle navigation.

GOKARN Ila Nitin is a PhD candidate in Computer Science, advised by Prof. Archan Misra. She works primarily in the field of pervasive computing with a focus on cognitive edge computing paradigms and platforms. She received her Bachelor’s of Science in Information Systems at Singapore Management University in 2015. Prior to starting the PhD program, she worked as a Software Systems Engineer at Arista Networks and Cisco Systems where she specialised in software systems, cloud management platforms, edge computing, and IoT ecosystems for smart cities.