> For the complete documentation index, see [llms.txt](https://brindha.gitbook.io/mylearning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://brindha.gitbook.io/mylearning/tools/dolly.md).

# Dolly

#### Databricks Dolly: The First Commercially Licensed Instruction-Following LLM

**What is Dolly?**

Dolly is an open-source instruction-following language model released by Databricks, with Dolly 2.0 released in April 2023 as what Databricks claimed was the first open-source, commercially usable instruction-following LLM. While earlier instruction-tuned models like Stanford Alpaca were based on LLaMA with non-commercial restrictions, and used synthetic data generated from proprietary OpenAI models — which raised additional licensing concerns — Dolly 2.0 distinguished itself on both counts: it was built on EleutherAI's Pythia, which carries an Apache 2.0 license, and fine-tuned on a dataset crowdsourced entirely from Databricks employees with no external proprietary data involved.

***

**Versions**

**Dolly 1.0** — Released in March 2023, a 6B parameter model fine-tuned from EleutherAI's GPT-J on an Alpaca-style dataset. A proof of concept demonstrating that surprisingly good instruction-following behavior could emerge from fine-tuning on a small, curated dataset.

**Dolly 2.0** — Released in April 2023, a 12 billion parameter model fine-tuned from Pythia-12B on the databricks-dolly-15k dataset — 15,000 high-quality instruction-following records written entirely by Databricks employees. This dataset covered eight categories of tasks: brainstorming, classification, closed-form question answering, generation, information extraction, open-form question answering, summarization, and creative writing. Both the model weights and the dataset were released under open licenses permitting commercial use.

***

**The Databricks-Dolly-15k Dataset**

The databricks-dolly-15k dataset is one of Dolly's most enduring contributions. Because it was created entirely by human authors — Databricks employees — without relying on outputs from proprietary models like ChatGPT or GPT-4, it is free of the licensing ambiguities that affected datasets like Alpaca's training data. The dataset was released under Creative Commons Attribution license, making it freely usable for commercial instruction tuning. It became a widely used component in subsequent open-source instruction-tuning projects.

***

**Performance and Positioning**

Dolly 2.0 was positioned not as a frontier model but as a demonstration of principle — that high-quality, commercially licensable instruction tuning was achievable using entirely open and human-generated resources. In terms of raw benchmark performance, it was surpassed by models like Vicuna and WizardLM, but its clean licensing story made it uniquely valuable for organizations needing to deploy an instruction-following model without legal uncertainty. Training Dolly 2.0 cost approximately $30 — even lower than Alpaca's $600.

***

**Legacy**

Dolly's commercial licensing story had lasting influence on the open-source LLM ecosystem. It demonstrated that the combination of permissively licensed base models and human-generated (rather than AI-generated) instruction data could produce commercially deployable models. The databricks-dolly-15k dataset continued to be used as a component in many subsequent fine-tuning projects. Databricks subsequently moved beyond Dolly with the acquisition of MosaicML and the development of DBRX.

***

**Strengths**

* First genuinely commercially usable open instruction-following model
* databricks-dolly-15k dataset is human-generated, commercially licensed, and widely reused
* Extremely low training cost ($30) demonstrates accessibility of fine-tuning
* Built on Apache 2.0 licensed Pythia with no proprietary base model restrictions
* Clean licensing story with no ambiguity around synthetic data from proprietary models

**Limitations**

* Performance trails Vicuna, WizardLM, and other contemporaries
* No longer actively developed — superseded by MPT and DBRX within Databricks
* Primarily English
* 12B scale limits capability on complex tasks
* No multimodal capabilities