Together with the ETH AI Center, I explored a self-supervised learning method to address the issue of missing data in wearables. Wearable devices such as smartwatches suffer from a high rate of missing data, e.g., when the sensor loses contact with the skin. This poses a big problem for downstream tasks, such as detecting a user’s activity or state, e.g. stress, since most data processing methods require complete data.

To address this issue, I trained a transformer deep learning model to complete (“impute”) the missing data sequences using masked self-supervised learning techniques. Then, I extensively compared the trained model against standard missing data completion methods, such as linear interpolation or nearest neighbours, and evaluated the effect of the missing data completion methods on downstream tasks, such as fatigue detection.

The interesting finding is that different data sources require different impuation methods: For short missing sequences, linear interpolation suffices but for longer sequences, the transformer model outperforms the other methods. Also, for more static signals standard imputation methods are often adequate, but the transformer becomes advantageous in high-frequency signals.

Publication: https://arxiv.org/abs/2401.05437

Repository: https://github.com/Yahnnosh/Sensor-Based-Modeling-of-Fatigue-Using-Transformer-Model