> For the complete documentation index, see [llms.txt](https://brindha.gitbook.io/mylearning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://brindha.gitbook.io/mylearning/tools/deepseek-coder.md).

# DeepSeek-Coder

**What it is**

DeepSeek Coder comprises a series of code language models trained from scratch on both 87% code and 13% natural language in English and Chinese, with each model pre-trained on 2 trillion tokens. Models range from 1B to 33B versions.

**Performance**

DeepSeek-Coder-Base-33B significantly outperforms existing open-source code models. Compared with CodeLlama-34B, it leads by 7.9%, 9.3%, 10.8%, and 5.9% on HumanEval Python, HumanEval Multilingual, MBPP, and DS-1000 respectively. DeepSeek-Coder-Instruct-33B after instruction tuning outperforms GPT-3.5-turbo on HumanEval and achieves comparable results on MBPP. 

**Repository-level training**

DeepSeek-Coder made the first attempt to incorporate repository-level data construction during the pre-training phase, enabling the model to understand code at the project level, not just individual files. The models exhibit proficiency in understanding 87 programming languages. 

**DeepSeek-Coder-V2**

DeepSeek-Coder-V2 substantially enhances coding and mathematical reasoning capabilities, expanding support for programming languages from 86 to 338 and extending context length from 16K to 128K. In standard benchmark evaluations, it achieves superior performance compared to closed-source models such as GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks. 

**Efficiency**

DeepSeek-Coder-V2 is introduced with 16B and 236B parameters based on the MoE framework, with activation parameters of only 2.4B and 21B, efficiently supporting diverse computation needs.