Pre-Conference Talk by Amit | AudioSense - Sound Based Shopper Behavior Analysis System

AudioSense - Sound Based Shopper Behavior Analysis System

Shoppers view, touch, and pick various items while they shop. Such interaction with items strongly reflects shoppers’ needs and preferences. Accordingly, real-time monitoring of shopper-item interaction will enable useful new applications. For this purpose, we present AudioSense, the system to monitor user-item interactions inside a store hence enabling precisely customized promotions. A shopper’s smartwatch emits sound every time the shopper picks up or touches an item inside a store. This sound is then localized, in 2D space, by calculating the angles of arrival captured by multiple microphones deployed on the racks. Lastly, the 2D location is mapped to specific items on the rack based on the rack layout information. In our initial experiments conducted with a simulated single rack with 16 compartments, we could localize the shopper’s smartwatch with a median estimation error of 15.9 cm in 2-dimensional space.

This is a pre-conference talk for the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UBICOMP 2017).