> For the complete documentation index, see [llms.txt](https://brindha.gitbook.io/mylearning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://brindha.gitbook.io/mylearning/tools/dbrx.md). # DBRX What is DBRX? DBRX is an open, general-purpose large language model created by Databricks, released on March 27, 2024, under the Databricks Open Model License. It was built by the Databricks AI Research Team with a strong emphasis on efficiency, performance, and enterprise applicability. At the time of its release, DBRX set a new state-of-the-art across a range of standard benchmarks for established open LLMs. *** #### Architecture DBRX is a transformer-based, decoder-only LLM built on a fine-grained mixture-of-experts (MoE) architecture. It has 132 billion total parameters, but only 36 billion parameters are active for any given input — specifically 4 out of 16 experts are activated per token. This design allows the model to deliver high quality while remaining computationally efficient at inference time. The MoE structure means DBRX inference throughput is 2 to 3 times higher than a comparable 132B non-MoE dense model, making it significantly faster in practice than its total parameter count would suggest. *** #### Training DBRX was trained on 3,072 NVIDIA H100 GPUs connected by 3.2 Tbps InfiniBand, at a reported cost of approximately US$10 million. The full training process — including pretraining, post-training, evaluation, red-teaming, and refinement — took approximately three months. The entire training pipeline was built using Databricks' own tools: * Training data was governed and managed using Unity Catalog * Data was processed and cleaned using Apache Spark and Databricks notebooks * Exploratory data work used the newly acquired Lilac AI * Training used open-source libraries: MegaBlocks, LLM Foundry, Composer, and Streaming * Large-scale training across thousands of GPUs was managed via Mosaic AI Training * Results were tracked using MLflow * Human feedback for quality and safety was collected through Databricks Model Serving and Inference Tables *** #### Performance DBRX surpasses GPT-3.5 and is competitive with Gemini 1.0 Pro. On coding, it outperforms specialized models like CodeLLaMA-70B, achieving a 70.1% pass rate on the HumanEval benchmark. On the Databricks open-source Gauntlet — a suite of over 30 state-of-the-art benchmarks — DBRX outperforms LLaMA2-70B, Mixtral-8x7B, and Grok-1 across language understanding, programming, math, and logic. Inference speed reaches up to approximately 150 tokens per second via the Foundation Model APIs. *** #### Model Variants DBRX is available in two versions: **DBRX Base** — the pretrained foundation model, available for further fine-tuning and customization on proprietary or domain-specific data. **DBRX Instruct** — a fine-tuned variant optimized for instruction following, conversational AI, customer service automation, and other task-specific applications. Both versions are available on GitHub and Hugging Face for research and commercial use. *** #### Context Window DBRX was trained with inputs of up to 32,768 tokens (32K), making it well-suited for long-context applications such as retrieval-augmented generation (RAG), where the model needs to locate and reason over relevant information buried deep within large documents. *** #### Deployment and Access DBRX is accessible via: * Databricks Mosaic AI Foundation Model APIs with pay-as-you-go pricing * Databricks AI Playground for interactive experimentation * Provisioned throughput option for production applications with performance guarantees * GitHub and Hugging Face for open-weight downloads * Self-hosted or private cloud deployments via Databricks Model Serving To run DBRX in its standard configuration independently, a setup with at least four NVIDIA H100 GPUs — or hardware totaling around 320GB of GPU memory — is required, making self-hosting impractical for most individual developers outside of enterprise environments. *** #### Enterprise Positioning DBRX was built entirely on Databricks' own infrastructure, which means every enterprise using the Databricks platform can use the same tools and techniques to build or customize their own models. This was a deliberate design choice — to demonstrate that full-stack model development from raw data to production inference is possible inside a single unified platform. Enterprises can interact with DBRX, leverage its long-context capabilities in RAG systems, and build custom DBRX models on their own private data, all within the Databricks ecosystem. Early enterprise adopters included JetBlue, Block, NASDAQ, and Accenture. *** #### Strengths * Fine-grained MoE architecture delivering high throughput with far fewer active parameters than total count suggests * Competitive benchmark performance against closed models at the time of release * Especially strong on code generation and programming tasks * 32K token context window well-suited for RAG workloads * Built and deployable entirely within the Databricks platform for unified governance * Open weights available on Hugging Face and GitHub for research and commercial use * Training pipeline fully reproducible using publicly available Databricks open-source libraries *** #### Limitations * Requires substantial hardware (approx. 320GB GPU memory) for self-hosted deployment, limiting accessibility for individual developers * Companies with more than 700 million active users face usage restrictions comparable to Meta's Llama terms * Has been surpassed by newer and more capable models since its April 2024 release * Primarily optimized for English, with limited multilingual depth compared to purpose-built multilingual models * Most practical to use as a Databricks customer; standalone deployment outside the platform is technically demanding *** #### Why Choose DBRX? DBRX is a strong choice for organizations that: * Are already using or planning to use the Databricks Data Intelligence Platform * Want a capable open-weight LLM with strong coding and RAG performance * Need to customize a foundation model on their own proprietary data with enterprise governance * Require a production-ready model with provisioned throughput, compliance, and security guarantees * Value a model whose entire training pipeline — from data to deployment — is documented and reproducible *** DBRX represents Databricks' statement that enterprise-grade, open, and efficient LLM development is achievable using a unified data and AI platform — and that the same tools available to Databricks customers can produce a model competitive with the best closed APIs of its time.