After Llama, Meta open sources Code Llama for programming tasks

Facebook-parent Meta has open sourced Code Llama 70B. Introduced first in August, Code Llama is the large language model (LLM) specifically for coding tasks. Meta calls the 70 billion parameter version (70B) its largest and best-performing models in the Code Llama family and is on par with OpenAI’s ChatGPT.

To be sure, Code Llama is a code-specialised version of Meta’s Llama 2 model for which the latter has been trained on code-specific datasets. Code Llama can hence ‘generate code, and natural language about code, from both code and natural language prompts’, the company’s blog said.

Code Llama comes in three forms — CodeLlama - 70B, the foundational code model; CodeLlama - 70B - Python, 70B specialised for Python; and Code Llama - 70B - Instruct 70B, which is fine-tuned for understanding natural language instructions.

Notably, Meta became one of the very few firms to open source their LLMs when it first made Llama model generally available. Other LLMs like GPT (which forms the basis for ChatGPT) are only accessible through application programming interfaces (API). This has helped several companies to fine-tune on Llama and release their own models built for specific tasks and requirements. This allows developers to create more advanced and natural language interactions for chatbots and virtual assistants. Sarvam AI’s Indic language model OpenHathi, for example, is fine-tuned on Llama. 

With the latest announcement, Meta seems to adopt similar approach for Code Llama, unlike tools like GitHub Copilot which is not available in open source. “The generative AI space is evolving rapidly, and we believe an open approach to today’s AI is the best one for developing new AI tools that are innovative, safe, and responsible. We are releasing Code Llama under the same community license as Llama 2,” the company blog said.