Dissertation Proposal by Dhanuja Tharith Wanniarachchi | Enhancing Edge Intelligence for Multi-View Sensing via Collaborative Inferencing and Actuation

Deep Neural Network( DNN) based AI models have resulted in significant enhancements in machine perception (especially for vision-based tasks), but their high computational complexity makes it challenging to execute them efficiently on resource-constrained edge devices. Current approaches for edge-based execution of DNNs for vision tasks, such as object detection, typically involve the creation of simpler, less-demanding DNN models but these usually imply a loss in task accuracy. Our work seeks to overcome this tradeoff by introducing the "ComAI" framework, a collaborative system designed to augment the detection accuracy of simple DNN-based detectors by leveraging cues from peer detectors and intended for use in networked, multi-camera deployments. ComAI enables real-time operations on resource-constrained edge devices by utilizing less complex DNN models while maintaining accuracy levels comparable to complex DNN-based detectors. ComAI achieves this breakthrough by utilizing intermediate states from the execution on one DNN pipeline to create hints that enhance the confidence of outputs generated by a peer DNN pipeline. The framework's distinctive trait lies in its ability to efficient learn deployment-specific characteristics, efficiently select collaborative pairs and regions, and dynamically adapt sensor attributes (such as pose) to maximize the beneficial impacts of collaborative inferencing. We additionally propose to extend the framework to support heterogeneous deployments involving a mix of vision and LIDAR sensors. A significant advantage of the ComAI framework is its versatility, enabling the use of off-the-shelf detectors without extensive training efforts.

Dhanuja Tharith Wanniarachchi is a Ph.D. candidate in Computer Science at Singapore Management University, supervised by Prof Archan MISRA. His research focuses on Pervasive Sensing and systems. Dhanuja holds a bachelor's degree in Electronic and Telecommunication Engineering. Prior to his academic journey, he worked as a Research Engineer at Living Analytics Research Centre in SMU.