> For the complete documentation index, see [llms.txt](https://brindha.gitbook.io/mylearning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://brindha.gitbook.io/mylearning/tools/stablelm.md). # StableLM **StableLM: Stability AI's Open-Source Language Model Suite** **What is StableLM?** StableLM is a suite of state-of-the-art language models developed by Stability AI, designed to meet the needs of a wide range of businesses across numerous industries. It provides a stable and reliable foundation for natural language processing needs. The models are open-source and showcase the power of small, efficient models that can generate high-performing text and code locally on personal devices. *** **Background and Development** Stability AI had prior success in open-sourcing language models in collaboration with EleutherAI, a non-profit research hub, and their release of StableLM builds on that experience. The project was designed with a clear philosophy: Stability AI builds models to support users, not replace them, focusing on efficient, specialized, and practical AI performance rather than a quest for general superintelligence. The tools are meant to help everyday people and firms use AI to unlock creativity, boost productivity, and open up new economic opportunities. *** **Model Variants and Sizes** StableLM has been released in several generations and sizes: The original StableLM-Base-Alpha is a suite of 3B and 7B parameter decoder-only language models pre-trained on a diverse collection of English datasets with a sequence length of 4096, pushing beyond the context window limitations of existing open-source language models at the time. Stable LM 2 was later released as a 1.6B parameter language model trained on 2 trillion tokens of text data from seven languages, capable of running on common laptop computers. It comes in two versions: a base model and an instruction-tuned version called Stable LM 2 Zephyr. Stability AI further extended the family with a 12B parameter model, Stable LM 2 12B, pre-trained on 2 trillion tokens of diverse multilingual and code datasets. It offers solid benchmark results on zero-shot and few-shot tasks compared to models like Mixtral, Llama2, Qwen 1.5, Gemma, and Mistral. *** **Architecture and Training** Stable LM 2 1.6B is an auto-regressive language model based on the transformer decoder architecture. It was trained on the Stability AI cluster across 512 NVIDIA A100 40GB GPUs and uses flash-attention as well as SwiGLU and Rotary Embedding kernels from FlashAttention-2. The base models are pre-trained in mixed-precision (FP16), optimized with Adam, and use the NeoX tokenizer with a vocabulary size of 50,257. *** **Key Capabilities** StableLM models demonstrate strong language understanding and generation capabilities. They can engage in open-ended conversations, answer questions, summarize information, and generate creative content like stories and poems. Specific use cases include: Content generation for creative writing, article generation, and chatbots; chatbots and virtual assistants that can be fine-tuned to engage in intelligent, contextual conversations; and question answering based on provided context. The instruction-tuned version is particularly well-suited for retrieval-augmented generation (RAG) systems, due to its strong performance in tool usage and function calling. *** **Multilingual Support** The Stable LM 2 base model was trained on data in Dutch, French, German, Italian, Portuguese, Spanish, and English, and Stability AI used recent algorithmic advancements in language modeling to give the small model capabilities rivaling larger models. *** **Licensing** Base model checkpoints are licensed under the Creative Commons license (CC BY-SA-4.0). Under this license, users must give credit to Stability AI, provide a link to the license, and indicate if changes were made. The models are intended to be used as foundational models for application-specific fine-tuning without strict limitations on commercial use. *** **Current Status** The StableLM language models, including StableLM 2 1.6B and StableLM Zephyr 3B, have reached deprecated status and are no longer actively maintained. They are not recommended for production applications. For language tasks, more capable alternatives such as Meta Llama or Microsoft Phi are now recommended. Stability AI has since refocused its core efforts on generative models for image, video, and audio. *** **Summary** StableLM was a significant contribution to the open-source AI ecosystem, demonstrating that small, efficient language models could deliver meaningful performance on consumer hardware. While no longer actively developed, it laid important groundwork for the broader movement toward transparent, accessible, and locally runnable language models.