Johannes Brandstetter

Johannes Brandstetter

Chief Researcher @ NXAI, Assistant Professor @ JKU Linz

Johannes Kepler University, Linz, Institute for Machine Learning

About me

I am leading a group “AI for data-driven simulations” at the Institute for Machine Learning at the Johannes Kepler University (JKU) Linz. Additionally, I am a Chief Researcher at NXAI - our new European AI hub in Linz (Austria).

I have obtained my PhD after working several years at the CMS experiment at CERN. During this time, I had the privilege of learning from brilliant minds from all around the world, and got the chance to co-author seminal papers in the realm of Higgs boson physics. In 2018, after completing my PhD, my career trajectory shifted towards machine learning, and I was fortunate to join the research group of Mr LSTM Sepp Hochreiter in Linz. Under Sepp’s mentorship, I delved into the intricacies of machine learning and modern deep learning over a span of 2.5 years.

From 2021 to 2023, I had the pleasure of spending three remarkable years in Amsterdam. Initially, I was part of the Amsterdam Machine Learning Lab lead by Max Welling, and subsequently joined Microsoft Research for 2 years. During this period, my passion for Geometric Deep Learning, particularly involving Geometric (Clifford) algebras, and my interest in partial differential equations (PDEs), with a particular focus on developing neural surrogates for (PDEs), became profound. Most importantly, I pivoted towards large-scale PDEs, including weather and climate modeling, which culminated in Aurora.

My years in Amsterdam have shaped my research vision. I am firmly convinced that AI is on the cusp of disrupting simulations at industry-scale. Every day thousands and thousands of compute hours are spent on turbulence modeling, simulations of fluid or air flows, heat transfer in materials, traffic flows, and many more. Many of these processes follow similar underlying patterns, but yet need different and extremely specialized softward to simulate. Even worse, for different parameter settings the costly simulations need to be run at full length from scratch.

This is what I want to change! Therefore, I have started a new group at JKU Linz which has strong computer vision, numerical simulation, and engineering components. We want to advance data-driven simulations at industry-scale, and place the Austrian industry engine Linz as a center for doing that.

News

[Feb 2024] I have started as Chief Researcher at NXAI.

[Oct 2023] I have re-joined Sepp Hochreiter’s group, starting my own group “AI for data-driven simulations”

NXAI

Mission statement of NXAI

With NXAI, we are building an European AI hub that comprises world-class research, close ties to local universities, and strong industrial know-how and support. Our mission is to create a large-scale entrepreneurial framework to transform latest scientific developments into industrial-ready applications.

At its core, NXAI is dedicated to independent research, which we believe is our greatest strength. However, we aspire to go further. We are building a pipeline that converts our research into high-impact industrial applications. Our goal is to become a nimble AI powerhouse at the forefront of the AI revolution in Europe.

Our research focus is on two key areas. The first is AI4Simulation, where we are building foundational models for industrial simulations. The second is Large Language Models (LLMs), highlighted by our flagship project xLSTM. In both domains, we have world-leading experts committed to unlocking the vast potential for creating downstream applications for engineering and industrial use cases

More information can be found at our webpage https://www.nx-ai.com/.

Research

Oct 1, 2023
Data-Driven Simulations
Data-Driven Simulations

I am firmly convinced that AI is on the cusp of disrupting simulations at industry-scale. Therefore, I have started a new group at JKU Linz which has strong computer vision, simulation, and engineering components. My vision is shaped by experience both from university and from industry.

Jan 1, 2021
Geometric Deep Learning
Geometric Deep Learning

My passion for Geometric Deep Learning can be unmistakenly traced back to my physics background. I have contributed to the fields of graph neural networks, equivariant architectures, and neural PDE solvers. Furthermore, I have lead efforts to introduce Lie Point Symmetries, and, most recently, Clifford (Geometric) Algebras into the Deep Learning community.

Jul 1, 2018
General Deep Learning
General Deep Learning

After switching from High Energy Physics to Deep Learning, I started working in Reinforcement Learning before pivoting towards Associative Memories and modern Transformer networks. Recent years have shown that scalable ideas, improving the datasets, and clever engineering are the ingredients for ever better Deep Learning models. This totally coincides with my experience, and – needless to say – I will continue working on general large-scale Deep Learning directions.

Jan 1, 2014
High Energy Physics
High Energy Physics

I have spent five years (including my PhD) working in the CMS Collaboration at CERN. Most of my research was dedicated to Higgs boson physics.

Recent (selected) Publications

Vision-LSTM -- xLSTM as Generic Vision Backbone
We introduce Vision-LSTM (ViL), an adaption of the xLSTM building blocks to computer vision.
Benedikt Alkin, Maximilian Beck, Korbinian Poeppel, Sepp Hochreiter, Johannes Brandstetter
PDF Cite Code Project
Vision-LSTM -- xLSTM as Generic Vision Backbone
Aurora -- A foundation Model of the Atmosphere
Aurora leverages the strengths of the foundation modelling approach to produce operational forecasts for a wide variety of atmospheric prediction problems, including those with limited training data, heterogeneous variables, and extreme events
Cristian Bodnar, Wessel W. Bruinsma, Ana Lucic, Megan Stanley, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan Weyn, Haiyu Dong, Anna Vaughan, Jayesh K. Gupta, Kit Tambiratnam, Alex Archibald, Elizabeth Heider, Max Welling, Rich E. Turner, Paris Perdikaris
PDF Cite
Aurora -- A foundation Model of the Atmosphere
xLSTM -- Extended Long Short-Term Memory
How far do we get in language modeling when scaling LSTMs to billions of parameters, leveraging the latest techniques from modern LLMs, but mitigating known limitations of LSTMs?
Maximilian Beck, Korbinian Poeppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter
PDF Cite Code
xLSTM -- Extended Long Short-Term Memory
Geometry-Informed Neural Networks
We introduce geometry-informed neural networks (GINNs) to train shape generative models without any data.
Arturs Berzins, Andreas Radler, Sebastian Sanokowski, Sepp Hochreiter, Johannes Brandstetter
PDF Cite Code Project
Geometry-Informed Neural Networks
Universal Physics Transformers -- A Framework For Efficiently Scaling Neural Operators
We introduce Universal Physics Transformers (UPTs), an efficient and unified learning paradigm for a wide range of spatio-temporal problems. UPTs operate without grid- or particle-based latent structures, enabling flexibility and scalability across meshes and particles.
Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter
PDF Cite Code Project
Universal Physics Transformers -- A Framework For Efficiently Scaling Neural Operators
Mim-refiner -- A contrastive learning boost from intermediate pre-trained representations
We introduce MIM (Masked Image Modeling)-Refiner, a contrastive learning boost for pre-trained MIM models.
Benedikt Alkin, Lukas Miklautz, Sepp Hochreiter, Johannes Brandstetter
PDF Cite Code Project
Mim-refiner -- A contrastive learning boost from intermediate pre-trained representations
We identify particle clustering originating from tensile instabilities as one of the primary pitfalls. Based on these insights, we enhance both training and rollout inference of GNN-based simulators with varying components from standard SPH solvers, including pressure, viscous, and external force components.
Smoothed particle hydrodynamics (SPH) is omnipresent in modern engineering and scientific disciplines. SPH is a class of Lagrangian …
Artur P. Toshev, Jonas A. Erbesdobler, Nikolaus A. Adams, Johannes Brandstetter
PDF Cite Code Project Poster
We identify particle clustering originating from tensile instabilities as one of the primary pitfalls. Based on these insights, we enhance both training and rollout inference of GNN-based simulators with varying components from standard SPH solvers, including pressure, viscous, and external force components.
Lie Point Symmetries and Physics-Informed Networks
We present how to use Lie Point Symmetries of PDEs to improve physics-informed neural networks. Published at NeurIPS 2023.
Tara Akhound-Sadegh, Laurence Perreault-Levasseur, Johannes Brandstetter, Max Welling, Siamak Ravanbakhsh
PDF Cite
Lie Point Symmetries and Physics-Informed Networks
PDE-Refiner - Achieving Accurate Long Rollouts with Neural PDE Solvers
PDE-Refiner is an iterative refinement process that enables neural operator training for accurate and stable predictions over long time horizons. Published at NeurIPS 2023 (Spotlight).
Phillip Lippe, Bastiaan S. Veeling, Paris Perdikaris, Richard E. Turner, Johannes Brandstetter
PDF Cite Code Project Poster Slides Video
PDE-Refiner - Achieving Accurate Long Rollouts with Neural PDE Solvers
Learning Lagrangian Fluid Mechanics with E(3)-Equivariant Graph Neural Networks
We introduce E(3)-equivariant GNNs to two well-studied fluid-flow systems, namely 3D decaying Taylor-Green vortex and 3D reverse Poiseuille flow. Published at GSI 2023.
Artur P. Toshev, Gianluca Galletti, Johannes Brandstetter, Stefan Adami, Nikolaus A. Adams
PDF Cite Code
Learning Lagrangian Fluid Mechanics with E(3)-Equivariant Graph Neural Networks
See all publications

Open Positions

[Nov 2023] Open positions can be found here.