Analysis of Ambient Sound in HINT

Title	Analysis of Ambient Sound in HINT
Summary	Machine Learning applied to sound classification. Creation (recoding and annotation) of new a sound database and a baseline system for sound event detection and/or localization
Keywords
TimeFrame
References	Dekkers, G., Vuegen, L., van Waterschoot, T., Vanrumste, B., & Karsmakers, P. (2018). DCASE 2018 Challenge-Task 5: Monitoring of domestic activities based on multi-channel acoustics. arXiv preprint arXiv:1807.11246. Dekkers, G., Lauwereins, S., Thoen, B., Adhana, M. W., Brouckxon, H., Van den Bergh, B., ... & Karsmakers, P. (2017). The SINS database for detection of daily activities in a home environment using an acoustic sensor network. Detection and Classification of Acoustic Scenes and Events 2017.
Prerequisites	Machine Learning, Python
Author
Supervisor	Wagner de Morais and Tiago Cortinhal
Level
Status

BACKGROUND State-of-the-art methods for home-based health monitoring and assistance use wearable sensors and computer vision methods. However, wearable sensors need to be attached to the body and at-home camera setups lack user acceptability due to privacy concerns and the feeling of being watched and intruded. Sounds in the home capture the context of daily routines. Thus, sound recognition of activities of daily living and emergency sounds can overcome those limitations by using advanced methods to automate anonymity and avoid communication of sensitive data [1]. In 2013, researches in signal processing and machine learning created community called DCASE, which stands for Detection and Classification of Acoustic Scenes and Events. The project Analysis of Ambient Sound in Home Environments concerns the techniques and methods for the automatic detection of sound events and classification of acoustic scenes in real home settings. The main purpose of the project is to recognize daily activities and emergency sounds in the home of older and disabled people living alone. In our design, we plan address challenges associated with technical and operational requirements to collect and process data, legal aspects and protection of personal data. We believe that doing data computation in the home can meet these requirements by using Artificial Intelligence to automate anonymity and avoid communication of sensitive data. PROJECT PROPOSAL Typically, systems for Environmental Sound Recognition (ESR) include modules for 1) detecting the sound and separating and localizing its source, 2) extracting features from the registered sound signal, and 3) for classifying the sound based on its features. State-of-the-art methods for Environmental Sound Recognition typically use 1) advanced signal processing techniques, such as Mel-frequency coefficients, for feature extraction, and 2) deep Learning techniques, such as Convolutional Neural Network (vector modeling) and Recurrent Neural Network (sequence modeling), as classifiers. [2, 3, 4, 5, 6, 7] To train and evaluate systems, there are a few openly available sounds databases [8, 9]. At Halmstad University, the Halmstad Intelligent Home (HINT)is a realistic home environment that can be used not only to develop and test ESR systems, but also to create sound databases. In this project, students will work in the creation (recoding and annotation) of new a sound database and a baseline system for sound event detection and/or localization. The sound recording will be performed at HINT and will contain sound data of 1) predefined and 2) spontaneous activities, as well as when HINT is not occupied. The sound database is complemented by events measured by other sensors deployed in HINT, such as motion and contact sensors.

REFERENCES 1. Dekkers, G., Vuegen, L., van Waterschoot, T., Vanrumste, B., & Karsmakers, P. (2018). DCASE 2018 Challenge-Task 5: Monitoring of domestic activities based on multi-channel acoustics. arXiv preprint arXiv:1807.11246. 2. Turpault, N., Wisdom, S., et al. (2020). Improving Sound Event Detection In Domestic Environments Using Sound Separation. arXiv preprint arXiv:2007.03932. Tech. Rep. DCASE Challenge. 3. Serizel, R., Turpault, N., Shah, A., & Salamon, J. (2020, May). Sound event detection in synthetic domestic environments. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 86-90). IEEE. 4. Turpault, N., Serizel, R., Salamon, J., & Shah, A. P. (2019). Sound event detection in domestic environments with weakly labeled data and soundscape synthesis. Tech. Rep. DCASE Challenge. 5. Miyazaki, K., et al. (2020). Convolution-Augmented Transformer for Semi-Supervised Sound Event Detection. Tech. Rep. DCASE Challenge. 6. Hao, J., Hou, Z., & Peng, W. (2020). Cross-domain sound event detection: from synthesized audio to real audio. Tech. Rep. DCASE Challenge. 7. Koh, C. Y., et al. (2020). Sound event detection by consistency training and pseudo-labeling with feature-pyramid convolutional recurrent neural networks. Tech. Rep. DCASE Challenge. 8. Foster, P., Sigtia, S., Krstulovic, S., Barker, J., & Plumbley, M. D. (2015, October). CHiME-home: A dataset for sound source recognition in a domestic environment. In 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (pp. 1-5). IEEE. 9. Dekkers, G., Lauwereins, S., Thoen, B., Adhana, M. W., Brouckxon, H., Van den Bergh, B., ... & Karsmakers, P. (2017). The SINS database for detection of daily activities in a home environment using an acoustic sensor network. Detection and Classification of Acoustic Scenes and Events 2017.