Alistarh Group
Deep Algorithms and Systems Lab (DASLab)
Artificial Intelligence has made massive progress over the past decade, with breakthroughs across several applications and tasks. Yet, the sustainability of this pace of progress is in question: the computation required to train and deploy state-of-the-art AI models has been rising exponentially, potentially hindering innovation, and leading to inequalities in terms of expertise and economic benefits.
The Alistarh group works to remove these barriers to the democratization of AI, by creating training and inference algorithms that are significantly more efficient than conventional ones. For this, we develop new algorithms for learning over compressed (e.g., sparse or quantized) representations, as well as efficient systems implementations that can leverage compression gains in practice.
Group Leader
Administrative Support
Christine Francois
Assistant to Professors
+43 2243 9000 1098
Team
Current Projects
Efficient Training and Inference for Massive AI Models | Large-scale distributed machine learning | Adaptive concurrent data structures | Fundamental limits of distributed computation
Publications
Talaei S, Ansaripour M, Nadiradze G, Alistarh D-A. 2025. Hybrid decentralized optimization: Leveraging both first- and zeroth-order optimizers for faster convergence. Proceedings of the39th AAAI Conference on Artificial Intelligence. 39(19), 20778–20786. View
Wu D, Modoranu I-V, Safaryan M, Kuznedelev D, Alistarh D-A. 2024. The iterative optimal brain surgeon: Faster sparse recovery by leveraging second-order information. 38th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 37. View
Modoranu I-V, Safaryan M, Malinovsky G, Kurtic E, Robert T, Richtárik P, Alistarh D-A. 2024. MICROADAM: Accurate adaptive optimization with low space overhead and provable convergence. 38th Conference on Neural Information Processing Systems. , Advances in Neural Information Processing Systems, vol. 37. View
Ashkboos S, Mohtashami A, Croci ML, Li B, Cameron P, Jaggi M, Alistarh D-A, Hoefler T, Hensman J. 2024. QuaRot: Outlier-free 4-bit inference in rotated LLMs. 38th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 37. View
Malinovskii V, Mazur D, Ilin I, Kuznedelev D, Burlachenko K, Yi K, Alistarh D-A, Richtarik P. 2024. PV-tuning: Beyond straight-through estimation for extreme LLM compression. 38th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 37. View
ReX-Link: Dan Alistarh
Career
Since 2022 Professor, Institute of Science and Technology Austria (ISTA)
2017 – 2022 Assistant Professor, Institute of Science and Technology Austria (ISTA)
2016 – 2017 “Ambizione Fellow”, Computer Science Department, ETH Zurich, Switzerland
2014 – 2016 Researcher, Microsoft Research, Cambridge, UK
2014 – 2016 Morgan Fellow, Downing College, University of Cambridge, UK
2012 – 2013 Postdoc, Massachusetts Institute of Technology, Cambridge, USA
2012 PhD, EPFL, Lausanne, Switzerland
Selected Distinctions
2023 ERC Proof of Concept Grant
2018 ERC Starting Grant
2015 Awarded Swiss National Foundation “Ambizione” Fellowship
2014 Elected Morgan Fellow at Downing College, University of Cambridge
2012 Postdoctoral Fellowship of the Swiss National Foundation
2011 Best Paper Award at the International Conference on Distributed Computing and Networking
