Welcome!
In 2000, the Signal Processing and Speech Communication Laboratory (SPSC Lab) of Graz University of Technology (TU Graz) was founded as a research and education center in nonlinear signal processing and computational intelligence, algorithm engineering, as well as circuits & systems modeling and design. It covers applications in wireless communications, speech/audio communication, and telecommunications.
If you want to learn more about Signal Processing, click: What is Signal Processing?
The Research of SPSC Lab addresses fundamental and applied research problems in five scientific areas:
- Audio and Acoustics
- Intelligent Systems
- Nonlinear Signal Processing
- Speech Communication
- Wireless Communications
Profiles
Result of the Month
Stochasticity in Tokenisation Improves Robustness
The widespread adoption of large language models (LLMs) has increased concerns about their robustness. Vulnerabilities in perturbations of tokenisation of the input indicate that models trained with a deterministic canonical tokenisation can be brittle to adversarial attacks. Recent studies suggest that stochastic tokenisation can deliver internal representations that are less sensitive to perturbations.
In this paper, we analyse how stochastic tokenisations affect robustness to adversarial attacks and random perturbations. We systematically study this over a range of learning regimes (pre-training, supervised fine-tuning, and in-context learning), data sets, and model architectures. We show that pre-training and fine-tuning with uniformly sampled stochastic tokenisations improve robustness to random and adversarial perturbations.
Read the full article.Contact: Sophie Steger