> For the complete documentation index, see [llms.txt](https://brindha.gitbook.io/mylearning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://brindha.gitbook.io/mylearning/tools/olmo.md). # OLMo OLMo: The Allen Institute for AI's Fully Open Language Model **What is OLMo?** OLMo — Open Language Model — is a family of large language models developed by the Allen Institute for AI (Ai2), a Seattle-based nonprofit research institute founded in 2014 by the late Microsoft co-founder Paul Allen. First released in February 2024, OLMo is distinguished from virtually all other open-weight models by its commitment to full openness: not just weights, but training data, training code, intermediate checkpoints, evaluation tools, and training logs — everything needed to reproduce, study, and build upon the model from scratch. Ai2 believes that full access to open language models for the research community is critical to the scientific study of these models, their strengths and weaknesses, and their biases and risks. Accordingly, OLMo provides the whole framework from data to training to evaluation tools, including multiple training checkpoints across multiple hardware types, training logs, and exact datasets used, with a permissive license. *** **The Dolma Dataset** OLMo's training data comes from Dolma, Ai2's own curated open dataset. OLMo 2 uses the OLMo-Mix-1124, a collection of approximately 3.9 trillion tokens sourced from DCLM, Dolma, StarCoder, and Proof Pile II in its first training stage, before transitioning to a curated mixture of high-quality web data and domain-specific content including academic content, Q\&A forums, instruction data, and math workbooks. Dolma itself has been publicly released, enabling other researchers to build on and study the pretraining data directly. *** **Evolution of OLMo** **OLMo 1.0 (February 2024)** The inaugural release, a 7B parameter model released with full training transparency. Supported by Databricks, Harvard's Kempner Institute, AMD, and the Lumi Supercomputer. Released under Apache 2.0 with a February/March 2023 data cutoff based on the Dolma dataset. **OLMo April 2024 (OLMo 1.7)** Extended the context length from 2,048 to 4,096 tokens and trained on the improved Dolma 1.7 dataset. Achieved 52 on MMLU, surpassing LLaMA 2-7B and approaching LLaMA 2-13B performance. **OLMoE (September 2024)** A mixture-of-experts variant and the first fully open model among its peers to sit on the Pareto frontier of performance versus size. **OLMo 2 (November 2024)** Available in 7B and 13B sizes, trained on up to 5 trillion tokens. OLMo 2's architecture leverages RMSNorm, rotary positional embeddings, and Z-loss regularization to enhance model robustness. OLMo 2 7B and 13B are the best fully-open models at their release, often outperforming open-weight models of equivalent size on English academic benchmarks. **OLMo 2 1B (May 2025)** The smallest member of the OLMo 2 family, outperforming other small models in its class including Gemma 3 1B and Llama 3.2 1B. **OLMo 3 (November 2025)** Ai2 unveiled OLMo 3, a collection of open language models that outperforms fully open models such as Stanford's Marin and commercial open-weight models like Meta's Llama 3.1. With OLMo 3, Ai2 expanded its focus from purely scientific tools to powerful, efficient, and transparent systems suitable for real-world use including commercial applications. OLMo 3 supports inputs up to 65,000 tokens. The OLMo 3 family includes several variants. OLMo 3 Instruct 7B and 32B are tuned to produce shorter, more direct responses, optimized for general chat and function calling, surpassing Qwen 2.5, Gemma 3, IBM Granite 3.3, and Llama 3 at comparable sizes. OLMo 3 Think is a reasoning model trained on multi-step problems spanning math, code, and general problem solving, competitive with Qwen 3 32B on reasoning benchmarks. OLMo 3 RL-Zero is a variant trained using RL directly from the base model, enabling researchers to study how base model data affects RL performance. On math benchmarks, OLMo 3-Think 7B matches Qwen 3 8B on MATH and leads all comparison models on HumanEvalPlus for coding, demonstrating particular strength in code-intensive reasoning. *** **Training Infrastructure and Datasets** OLMo 3 introduced Dolma 3, a new pretraining data mix encompassing carefully sampled natural data from crawled sources, a midtraining mix of high-quality data designed to jump-start reasoning, and a large collection of science-focused PDF documents enabling long-context support. Post-training uses Dolci, a new suite covering step-by-step reasoning for supervised fine-tuning, high-quality contrastive data for preference tuning, and challenging prompts for reinforcement learning. In August 2025, Ai2 was selected by the National Science Foundation and NVIDIA for a $152 million initiative to build fully open multimodal AI models for scientific research. *** **Strengths** * The most comprehensively open LLM family available: weights, data, code, checkpoints, logs, and recipes * Strong scientific research utility — enables studying training dynamics, memorization, concept acquisition * Competitive benchmark performance against open-weight models at equivalent sizes * OLMo 3 Think provides capable reasoning at 7B scale matching much larger models * Apache 2.0 license with no restrictions on commercial use * Nonprofit mission with no commercial pressure to restrict openness **Limitations** * Trails frontier closed models on complex reasoning and knowledge-intensive tasks * Smaller parameter ceiling than models from large corporate labs * Less focus on multimodal, speech, and vision capabilities compared to Granite or Nemotron * Primarily English-focused; limited multilingual depth * Smaller deployment ecosystem and enterprise tooling than commercially backed models *** **Why Choose OLMo?** OLMo is the definitive choice for researchers, academics, and organizations that need a model whose entire development pipeline is auditable, reproducible, and scientifically documented. It is the reference model for studying language model behavior from first principles — and with OLMo 3, it is now also a genuinely competitive choice for real-world deployment.