Researchers from Carnegie Mellon University's Systems, Networking and Energy Efficiency Lab (Synergy) presented several multiyear studies on their work around ubiquitous sensing during the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp). Led by School of Computer Science Associate Professor Yuvraj Agarwal, the Synergy Lab aims to develop more energy-efficient computing in buildings, improve the security and privacy of Internet of Things (IoT) devices, and advance mobile systems.

The team unveiled innovative sensing systems — Mites, VAX and TAO — and explained how the data they collect can be converted into useful insights without sacrificing the privacy of the people in monitored spaces.

While the three systems work independently of each other, researchers said there is potential to use the features of TAO and/or VAX to further strengthen the Mites end-to-end hardware-software system. Each system could offer building managers and occupants valuable insight and functionality while preserving the privacy of individuals working in spaces outfitted with the sensors.

"Among many other capabilities, the Mites stack enables users to build custom machine learning models that convert low-level featurized sensor data into activities and inferences they are interested in. However, this is still a manual process, requiring the users to give labels or examples for those activities," Agarwal said. "VAX can help reduce the labeling effort for privacy-preserving modalities and helps expand the scope of our sensors by training the model using audio and video. Last but not least, TAO enables us to convert these activities into an even higher-level semantic abstraction of context, ultimately supporting smart building applications toward improving occupant wellness and productivity."

View the full story and learn more about each technology on the CMU CyLab website.