Grok, an AI chatbot developed by xAI, is now available for public download on GitHub, as per Elon Musk’s announcement on March 11.
In order to compete with alternative technologies like OpenAI, Meta, Google, and other companies, this will enable scientists and developers to improve upon the model and influence how xAI updates Grok in the future.
The “base model weights and network architecture” of the “314 billion parameter Mixture-of-Experts model, Grok-1” are included in this open release, according to a business blog post. The model is from a snapshot from October of last year, it says, and it hasn’t been adjusted “for any specific application, such as dialogue.”
This week, @xAI will open source Grok
— Elon Musk (@elonmusk) March 11, 2024
According to VentureBeat, it is being made available for commercial usage under the Apache 2.0 license; however, connections to X for real-time data and the training data are not included. The LLM Grok was “developed over the last four months,” according to a post by xAI published in November 2023. Its intended applications include coding creation, creative writing, and question-answering.
Following his acquisition of Twitter (now X), Elon Musk publicly chastised businesses that refuse to make their AI model publicly available and eventually made the code underlying the company’s algorithms available. This includes OpenAI, which he assisted in forming and is currently suing on the grounds that it violated an open-source founding agreement.
In order to gather input from other researchers on how to make their models better, companies have released limited or open-source models. The most popular models are either closed-sourced or have a restricted open license, despite the fact that there are numerous fully open-source AI foundation models, such as Mistral and Falcon. For instance, Meta’s Llama 2 offers its research for free, but its 700 million daily users must pay a subscription, and the company forbids others from building upon Llama 2.
Upon its inception, the Grok chatbot was only accessible with an X subscription, which is essentially a paid blue check. Grok presented itself as a more modern, irreverent chatbot than Google’s Gemini or OpenAI’s ChatGPT. Instead, it lacked anything that would have made it distinguishable from other chatbots that were more potent and sophisticated in our early testing. It was also comical.
Very nice to see Grok go open-source & open-weights: https://t.co/RTlvQarPzy
— François Chollet (@fchollet) March 17, 2024
here’s your DEEP DIVE into @grok‘s architecture!
I just went through the https://t.co/8Y5cjeImg6, for this 314B open source behemoth with *no strings attached*.— Andrew Kean Gao (@itsandrewgao) March 17, 2024
Grok weights are out under Apache 2.0: https://t.co/9K4IfarqXK
It’s more open source than other open weights models, which usual come with usage restrictions.
It’s less open source than Pythia, Bloom, and OLMo, which come with training code and reproducible datasets. https://t.co/kxu2anrNiP pic.twitter.com/UeNew30Lzn
— Sebastian Raschka (@rasbt) March 17, 2024
