Meta and Microsoft Unveil the Next Generation of 'Llama'

Content Highlights:

Meta announces the launch of Llama 2 today! This is the latest version of our open-source large language model. The good news, it’s completely free for both research and commercial use.
Meta is also thrilled to say that their relationship with Microsoft is growing stronger. They’ve come on board as the preferred partner for Llama 2, showing their confidence in this innovative tool.
Opening access to Llama 2 is a collective effort. Meta has got a fantastic support system of tech companies, academic institutions, and policy advocates who also value open innovation in the current AI landscape.
Meta firmly believes in responsible development and use of technology. With that in mind, we’re providing resources to ensure those who use Llama 2 can do so in a responsible manner. Get ready for an exciting journey with Llama 2!

Meta – that’s Facebook to most of us – has just rolled out the commercial version of its AI Large Language Model, Llama-v2. This smart tool can create text, images, and code. Pretty impressive, right?

Llama’s journey hasn’t been smooth. Its first version was released back in February, and it was only meant for a select group of researchers and organizations. But, as you may guess, it didn’t take long for the code to leak online.

Meta didn’t just stand back and watch this happen. The company tried to reclaim the rogue program by filing take-down orders to sites like GitHub and Hugging Face. But when the code started popping up everywhere on the web, Meta decided to embrace the situation and go with the flow.

Both Llama versions learned from a variety of sources such as Common Crawl, GitHub, Wikipedia, Project Gutenberg, ArXiv, and Stack Exchange. While Microsoft and OpenAI’s ChatGPT might have been stealing the show in the headlines, many open-source developers quietly turned to Llama.

But here’s where Meta’s offer gets even more enticing: they shared their model’s weights – the learned parameters during training. This is a big deal because it makes it a lot easier to create and run your own AI programs. This is something other major LLMs, like GPT, haven’t done, and it definitely set Llama apart.

Llama isn’t just another AI program, it’s actually the first major open-source LLM. It’s been trained on a massive amount of data – think 2 trillion tokens! It’s fine-tuned models have been trained on over 1 million human annotations, and its parameters range from 7 to 70 billion.

In an open letter, Meta’s researchers explained why they’ve chosen to open up Llama-v2: “While many companies have opted to build AI behind closed doors, we are releasing Llama 2 openly to encourage responsible AI innovation. Based on our experience, an open approach draws upon the collective wisdom, diversity, and ingenuity of the AI practitioner community to realize the benefits of this technology. Collaboration will make these models better and safer.”

Now, whether Llama-v2 is truly open-source might be up for debate – the license hasn’t been officially approved by the Open Source Initiative (OSI). However, Meta has made it available for both research and commercial use free of charge, and is including model weights and starting code.

As Amanda Brock, the CEO of OpenUK, put it, “Not an OSI approved license but a significant release of Open Technology … This is a step to moving AI from the hands of the few to the many, democratizing technology and building trust in its use and future through transparency. No, it’s not perfect, and yes, there is more work to be done, but this bold move sets the tone for AI-open innovation with a responsible but light-touch principles-based approach to regulating the use and development of AI.”

Meta isn’t just going it alone, either. The company has named Microsoft as its preferred Llama 2 partner, so you can now find Llama 2 in the Azure AI model catalog. Llama 2 is also optimized to run locally on Windows, and is available through Amazon Web Services (AWS), Hugging Face, and other providers.

They’re not just focused on availability – they want Llama to be safe and well-behaved, too. To make it more secure, they’ve conducted Red-Teaming (Security Testing) exercises. They’ve also published a Developer’s Guide for Safe and Responsible Use and a Llama Acceptable Use Policy.

What’s the end goal for Meta? Well, they hope to surpass OpenAI. As a Google AI engineer recently put it, “The uncomfortable truth is, we aren’t positioned to win this [Generative AI] arms race, and neither is OpenAI. While we’ve been squabbling, a third faction has been quietly eating our lunch.” That third group is the open-source community. Watch out, world!