Efficient Systems for Foundation Models

Code it, run it, crash it–restart it.

➡️ ES-FoMO is back for ICML 2025! Submissions are live on OpenReview!

The deadline is May 19, 2025 11:59PM UTC time.

🔥 the gist

what? A workshop to bring together interdisciplinary experts working on the emerging research questions and challenges associated with foundation model training and inference.
when & where?
- Join us at ICML 2025 in Vancouver!
questions? Contact us at esfomo.workshop@gmail.com.

🦾 the pitch

As models increase in size and training budget, they not only systematically improve in upstream quality, but also exhibit novel emergent capabilities. This increase in scale raises proportionate difficulties for practitioners: foundation model training and inference lie at a unique interdisciplinary crossroad, combining open problems in algorithms, system design, and software engineering.

Machine learning practitioners are key stakeholders here: on the one hand, researchers may contribute algorithmic insights and novel methods to improving training and inference of large models; on the other hand, novel research findings may be best demonstrated at scale—which may require training models as efficiently as possible to make the best use of available resources.

The goal of this workshop is to bring together interdisciplinary experts working on the emerging research questions and challenges associated with foundation model training and inference. We welcome submissions around training and inference systems/algorithms for foundation models, focusing on scaling-up or on reducing compute, time, memory, bandwidth, and energy requirements. Notably, we encourage submissions concerning the entire spectrum of foundation models: from BERT-sized Transformers, to large models with 100B+ parameters. Topics include but are not limited to (see our 📝 call for papers for details):

Training and inference systems, either distributed at large scale or in resource-constrained scenarios;
Algorithms for improved training and inference efficiency;
Systems for foundation models, such as novel programming languages or compilers.

This is the third installment of ES-FoMo; we are bringing further focus in our sessions and talks on two trends observed in 2024 and early 2025:

Test-time compute, popularized by Open AI o1 and DeepSeek r1
Emergence of new modeling paradigms and modalities sucha s real-time video and decentralized training

We look forward to continuing to grow this community at ICML 2025!