Abstract

People with hearing or cognitive impairments including user of hearing aids and cochlear implants often have problems with intelligibility of speech in noisy environments (cf. cocktail party effect). One current approach to mitigate this speech-in-noise problem is to support intelligibility with orthographic transcriptions giving users subtitles of a conversation partner’s utterances. Subtitles are displayed on devices for augmented reality, i.e., AR-glasses, which have recently become comfortable to wear, due to reduction of weight. In this thesis, we plan to (i) evalute the performance of state-of-the-art automated speech recognition with data collected using AR-glasses in adversarial environments, and (ii) explore possibilities of language simplification using a large language model.

Your Tasks

• literature review
• making of speech audio recordings using AR-glasses
• orthographic transcription of the recordings
• training and testing of state-of-the-art approaches to real-time automatic speech recognition
• use of a language model to simplify the language
• implementation and testing with AR-glasses
• reporting your results (thesis writing, optional: paper writing)

Your Profile

• good knowledge of speech recognition and language modelling
• good knowledge of Python and relevant packages

Additional information

• https://brilliant.xyz

Contact

Philipp Aichinger (philipp.aichinger@meduniwien.ac.at) Martin Hagmüller (hagmueller@tugraz.at or 0316/873 4377)