> For the complete documentation index, see [llms.txt](https://brindha.gitbook.io/mylearning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://brindha.gitbook.io/mylearning/tools/granite.md). # Granite What is Granite? IBM Granite is a family of open-source AI models purpose-built for enterprise use, developed by IBM Research and released under the permissive Apache 2.0 license. Unlike many AI models developed primarily for research or general-purpose use, Granite was designed from the ground up with production deployment, enterprise governance, and real-world customization in mind. The family spans language, vision, speech, embedding, and safety models — making it one of the most comprehensive open enterprise AI portfolios available. *** #### The Granite Philosophy IBM built Granite around three consistent principles across all generations: performance at small scale, transparency in data and training, and safety for enterprise deployment. Every Granite model release includes full data clearance documentation, governance and compliance evaluations, cryptographic signatures, and ISO certification — standards that distinguish it sharply from many other open-weight models that provide weights but little else in terms of auditability. *** #### Evolution of the Granite Family **Granite 1.x and 2.x — Early Foundations** IBM's earliest Granite models established the project's identity as an enterprise-first LLM family, focused on code generation, text summarization, and classification tasks for business workflows. These early models were deployed internally within IBM's consulting and software products before being made more broadly available. **Granite 3.0 (October 2024)** Granite 3.0 marked IBM's first major public generational leap. The flagship 8B and 2B language models were designed as workhorse models for enterprise AI, delivering strong performance for tasks such as retrieval-augmented generation, summarization, classification, and code generation. The models can outperform or match similarly sized models from leading providers on many academic and industry benchmarks, while maintaining strong transparency and safety profiles. Released under Apache 2.0, Granite 3.0 also introduced support for key agentic capabilities including advanced reasoning and a structured prompting style for implementing tool-use workflows. IBM also announced that Granite would become the default model of Consulting Advantage, its AI-powered delivery platform used by 160,000 IBM consultants. **Granite 3.2 (February 2025)** Granite 3.2 introduced three significant new capabilities: reasoning, vision, and guardrails. The standout innovation was programmable chain-of-thought reasoning — IBM gave developers the ability to toggle reasoning on or off programmatically. For simple tasks, the model can skip reasoning to reduce unnecessary compute overhead. For complex tasks, reasoning can be enabled and further scaled using inference-time scaling techniques. Granite 3.2 also introduced updated time-series models offering long-range forecasting with fewer than 10 million parameters, alongside verbalized confidence in safety models for nuanced risk assessment. All models were released under Apache 2.0 and made available on watsonx.ai, Ollama, Replicate, and LM Studio. **Granite 4.0 (October 2025)** Granite 4.0 was IBM's most technically ambitious release at the time, introducing a hybrid architecture that blends transformer and Mamba-2 layers. The Mamba-2 blocks efficiently process global context, periodically passing information through transformer blocks for more nuanced local context parsing via self-attention — combined in a 9:1 ratio of Mamba-2 to transformer blocks. This hybrid approach makes Granite 4.0 models significantly faster and more memory-efficient than comparably sized standard transformer models. The Granite 4.0 family includes several size variants: * Granite-4.0-H-Small — a workhorse model for strong, cost-effective performance on enterprise workflows like multi-tool agents and customer support automation * Granite-4.0-H-Tiny — a hybrid MoE with 7B total parameters and 1B active parameters * Granite-4.0-H-Micro — a dense hybrid model with 3B parameters * Granite-4.0-Micro — a 3B dense model with a conventional transformer architecture for platforms that do not yet support hybrid architectures Select enterprise partners including EY and Lockheed Martin were given early access to test Granite 4.0 at scale. **Granite 4.0 Nano (October 2025)** Alongside the main Granite 4.0 release, IBM pushed the same hybrid Mamba-2 plus transformer architecture down to sub-2B scales for edge and on-device workloads. The Nano family ranges from 350 million to approximately 1.5 billion parameters. The 350M variants can run on a modern laptop CPU with 8 to 16GB of RAM, while the 1.5B models typically require a GPU with at least 6 to 8GB of VRAM. These models carry the same governance and provenance story as the larger Granite models, are Apache 2.0 licensed, ISO 42001 aligned, cryptographically signed, and runnable on vLLM, llama.cpp, and MLX. *** #### Granite 4.1 — The Latest Generation Granite 4.1 is IBM's most expansive model release to date, covering language, vision, speech, embedding, and Guardian safety models — all tailored for enterprise workloads and released in early May 2025. **Language Models** Granite 4.1 language models are available in three sizes: 3B, 8B, and 30B parameters, each in both base and instruction-tuned variants, with optional FP8 quantization for efficient deployment. They use a dense decoder-only transformer architecture with Grouped Query Attention, Rotary Position Embeddings, SwiGLU activations, RMSNorm, and shared input/output embeddings. Training used a five-phase pipeline on approximately 15 trillion tokens, culminating in long-context extension to 512K tokens. Post-training combined supervised fine-tuning on roughly 4.1 million high-quality curated samples with a multi-stage reinforcement learning pipeline using on-policy GRPO with DAPO loss, covering multi-domain RL, RLHF, identity and knowledge calibration, and math RL stages. A standout result: the Granite 4.1 8B instruct model consistently matches or outperforms the Granite 4.0 32B Mixture-of-Experts model, while using a simpler dense architecture that is more flexible for downstream fine-tuning. **Vision — Granite Vision 4.1 4B** Granite Vision 4.1 is a vision-language model delivering frontier-level performance on structured document extraction tasks including chart extraction, table extraction, and semantic key-value pair extraction — in a compact 4B parameter footprint. It integrates seamlessly with Docling, IBM's document processing pipeline, and supports chart-to-CSV conversion, table extraction to JSON or HTML, and semantic KVP extraction across diverse document layouts. The model was trained on IBM's Blue Vela supercomputing cluster using 32 NVIDIA H100 GPUs for approximately 200 hours. **Speech — Granite Speech 4.1** Granite Speech 4.1 2B achieves a 5.33% word error rate, placing it among the top models on the OpenASR Leaderboard. Three variants are available: the standard 2B model, Granite Speech 4.1 2B Plus with richer transcription features, and Granite Speech 4.1 2B NAR — a non-autoregressive variant that generates entire sequences at once rather than one token at a time, resulting in considerably better GPU utilization and much higher throughput. IBM plans to extend this non-autoregressive format to more models in the future. **Guardian — Safety Models** Granite Guardian provides harm detection capabilities across the Granite ecosystem, offering content safety evaluation, toxicity detection, and risk assessment for enterprise AI deployments. *** #### Technical Highlights Across the Family * Context window up to 512K tokens (Granite 4.1) * Apache 2.0 license across all releases, enabling free commercial use * Cryptographic signatures and ISO 42001 alignment for auditability * Full data clearance documentation and GRC evaluations * Multilingual support: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese * Compatible with vLLM, llama.cpp, MLX, Ollama, and Hugging Face transformers * Available on IBM watsonx.ai, Red Hat RHEL AI, Ollama, Replicate, LM Studio, and Hugging Face *** #### Enterprise Applications Granite models are deployed across IBM's own product lines, including watsonx.ai, watsonx Orchestrate, and Consulting Advantage. IBM has also positioned Granite as the foundation for agentic AI workflows, with the 4.1 models specifically optimized for tool calling, instruction following, and multi-step reasoning pipelines. Industry partners including NASDAQ, JetBlue, Block, and Accenture have explored Granite-based applications, and enterprise partners like EY and Lockheed Martin participated in early access programs for Granite 4.0. *** #### Strengths * Among the most thoroughly documented and auditable open-weight model families available * Apache 2.0 license with no usage restrictions for commercial deployment * Full model family spanning language, vision, speech, embedding, and safety under one umbrella * Programmable reasoning toggle allows compute optimization per task * Hybrid Mamba-2 architecture in Granite 4.0 delivers superior inference efficiency * Nano variants enable genuine edge and on-device deployment * 512K context window in Granite 4.1 for long-document enterprise workflows * Deep integration with IBM's enterprise platform ecosystem *** #### Limitations * Smaller models trail frontier models like GPT-4o and Claude on complex open-ended reasoning * Hybrid Mamba-2 architecture requires ecosystem support not yet universal across all inference frameworks * Primarily designed for enterprise workflows rather than general consumer use cases * Multilingual coverage, while present, is not as broad as purpose-built multilingual models like Aya *** #### Why Choose Granite? Granite is the right choice for organizations that need a fully auditable, enterprise-grade AI model family with strong governance, flexible deployment from edge to cloud, and deep integration into a unified data and AI platform. Its combination of open licensing, transparent training documentation, safety tooling, and a comprehensive multi-modal family makes it one of the most complete open enterprise AI offerings available today.