Multi-channel source separation with video data
Date
2024
Authors
Mosayyebpour, Sahand
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This research introduces a supervised multi-channel audio source separation system that integrates a video-based face detection system. The face detector identifies the nose position, aiding the multi-channel processing in isolating the primary speaker while suppressing environmental background noise and distracting secondary speakers. It is demonstrated that in far-field applications, multi-channel processing struggles with distracting secondary speakers when the primary speaker position is unknown. Utilizing video data provides valuable insights to identify the target speaker and assists the audio source separation system in directing its focus towards the target speaker. Furthermore, it is shown that multi-channel processing benefits from speaker position information to improve noise reduction in noisy reverberant environments.