Trustworthy Representation Learning via Information Funnels and Bottlenecks
- Published
- Mon, Dec 01, 2025
- Tags
- rotm
- Contact
Ensuring trustworthiness in machine learning—by balancing utility, fairness, and privacy—remains a critical challenge, particularly in representation learning. In this work, we investigate a family of closely related information-theoretic objectives, including information funnels and bottlenecks, designed to extract invariant representations from data.
We introduce the Conditional Privacy Funnel with Side-information (CPFSI), a novel formulation within this family, applicable in both fully and semi-supervised settings. Given the intractability of these objectives, we derive neural-network-based approximations via amortized variational inference. We systematically analyze the trade-offs between utility, invariance, and representation fidelity, offering new insights into the Pareto frontiers of these methods. Our results demonstrate that CPFSI effectively balances these competing objectives and frequently outperforms existing approaches. Furthermore, we show that by intervening on sensitive attributes in CPFSI’s predictive posterior enhances fairness while maintaining predictive performance.
Our paper is published open-access in Machine Learning (Springer).
Browse the Results of the Month archive.