Although masks protect us from COVID-19, they cover a substantial portion of the face and thus eliminate important facial social cues during face-to-face interactions, not only leading to misinterpretation but also lowering trust in communication.

 

EmoMask is an affective mask that senses and displays the wearer’s expressions in mask-wearing conditions. By leveraging the microcontroller embedded in the mask, the system is able to extract MFCC features of the speech, categorize 4 emotional expressions using the long short-term memory (LSTM) model with 73% accuracy, and show them on a frontal LED display.

 

EmoMask was specifically designed for the general public during the pandemic era, as well as in other specific situations where wearing masks is commonplace, such as in factories and hospitals.

Github: Speech Emotion Recognition using TinyML
Video: An Interactive Mask that Mediates Communication