Sensor-based emotion recognition in software development: facial expressions as gold standard

Early identification of emotions of software developers can enable timely intervention in order to support developers’ well-being and prevent burnout. We present a machine learning experiment aimed at recognizing emotions during programming tasks using wearable biometric sensors, tracking electrodermal activity and heart-related metrics. As a gold standard for supervised learning, we rely on a state-of-the-art tool for emotion recognition based on facial expression analysis. We design, implement and evaluate an approach that combines the output of two classifiers for neutral valence recognition and positive/negative polarity classification. Our findings suggest that biometric sensors in a wristband can be used to identify emotions whose recognition would otherwise need an intrusive webcam.

In 10th International Conference on Affective Computing and Intelligent Interaction