Neural Magic, AI Startup with ISTA & MIT Roots, Acquired by Red Hat

Neural Magic, a pioneer AI startup has been acquired by Red Hat, the world’s leading provider of open source solutions. As a founding employee and leader of Machine Learning research, ISTA researcher Dan Alistarh played a key role in advancing the company’s technology, alongside MIT scientists and co-founders Alex Matveev and Nir Shavit. This acquisition marks a significant milestone for the Institute of Science and Technology Austria (ISTA), as Neural Magic is the first startup to collaborate with the institute and successfully achieve a major exit.

Neural Magic is a leader in AI deployment, tackling the rising energy and infrastructure costs of large language models (LLMs). Its software boosts generative AI performance while cutting both environmental and financial footprints.

Alistarh underlines the impact of his research on the ‘democratization’ of AI: “The key motivation behind this work is to bring the power of efficient machine learning to companies, researchers, and enthusiasts, enabling them to run models affordably on their own hardware or in the cloud.”

Making LLMs leaner and accessible for small enterprises

While generative AI is at the forefront of technological advancement, the LLMs underpinning these systems continue to increase in size, driving up technology’s resource demands. Building, deploying, and scaling these models requires immense computing power, specialized hardware, and operational expertise—barriers that often exclude smaller enterprises from realizing AI’s potential.

Neural Magic’s solutions address these barriers, by redefining how LLMs operate, developing software that accelerates generative AI inference workloads. In collaboration with ISTA, Neural Magic developed innovative LLM compression research and shared their findings with the open source community, including the state-of-the-art GPTQ and SparseGPT techniques, in addition to strategies that facilitate the adoption of its technology across various corporate environments.

This technology enables the deployment of state-of-the-art AI models with incredible efficiency, drastically reducing hardware costs and energy consumption, thus making AI deployment more cost-effective and environmentally sustainable. As a result, Neural Magic attracted the attention of several big players, and raised venture capital funding from investors such as Andreessen Horowitz, Comcast Ventures, and Ridgeline Ventures.

Accelerating a hybrid cloud-ready gen AI

With the acquisition, Red Hat plans to leverage Neural Magic’s expertise in vLLM to build an enterprise-grade inference stack that enables customers to optimize, deploy and scale LLM workloads across hybrid cloud environments with full control over infrastructure choice, security policies and model lifecycle.
Together with Neural Magic’s work on optimization research, LLM Compressor and repository maintenance of pre-optimized models their software are expected to become a cornerstone of Red Hat’s strategy to enable customers to lower AI’s costs, skill barriers and deploy LLMs across hybrid cloud environments with a ready-made, highly-optimized and open inference stack.

In its announcement on the acquisition, Red Hat president and CEO Matt Hicks said, “AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs.”

ISTA’s Role in Nurturing Innovation

Neural Magic’s journey from academic research to commercial success exemplifies ISTA’s growing role as a hub for scientific innovation. By supporting curiosity-driven basic research as well as entrepreneurial researchers like Dan Alistarh, ISTA strengthens its mission to bridge cutting-edge science and societal impact.

ISTA’s commitment to ensuring that research creates meaningful impact is further exemplified by their daughter company, xista innovation. A company specifically set up to support the commercialisation of technologies and know-how developed at the Institute, working closely with researchers like Dan Alistarh to establish strategies for transferring technologies to industry, whether in the context of collaborations with established companies or by out-licensing technologies or by setting up new spin off companies.

As ISTA continues to expand its focus on entrepreneurship, this achievement underscores the institute’s pivotal role in shaping Austria’s life sciences and technology landscape.