As proven by the recent EU’s General Data Protection Regulation (GDPR), our society is becoming increasingly concerned with how cloud-based applications use their client’s data. Among other data types, speech has received growing interest due to recent advances in speech recognition that have boosted the development of voice-based virtual assistants. However, current speech-based machine learning models are already able to determine characteristics about an individual as private as his/her physical and mental health. It is thus important to find ways to protect the privacy of an individual’s voice, and prevent sensitive information from being leaked. In the midway between simple speech applications and speech recognition lies an enabling technology that not only serves as an intermediate step in complex applications, but is also useful on its own. Automatic Speaker Diarization (ASD), the problem of determining “who spoke when” in a speech recording, is the middle step that can be used to contextualize transcriptions, improve the results of speech recognition systems, and allow users to search and filter speech segments of a specific speaker. In this exploratory project we aim to develop a proof-of-concept for private ASD, which could be applied to scenarios where data cannot be delegated to an external party, due to legal or ethical constraints, such as criminal investigations, medical interviews and courtroom hearings. This project addresses this privacy issue by combining state-of-the-art diarization methods (based on speaker embeddings obtained from the hidden layers of deep neural networks) with cryptographic techniques such as homomorphic encryption and secure multi-party computation. Notwithstanding its own merits, a private solution for ASD will also provide a basis on which future research can be built upon and will push the current state-of-the-art forward in the direction of fully private end-to-end speech processing applications.
- Isabel Trancoso (P.I., INESC-ID)
- Bhiksha Raj (P.I., CMU)
- Alberto Abad (Co-P.I., INESC-ID)
- Francisco Teixeira (Ph.D. student)
- Daniel Oliveira (M.Sc. student)
Project Timeline (January-December 2021)
For further information about our project, please contact us at email@example.com.