Meta has released Llama 3, the most advanced open source large-scale language model available today. This work builds on the foundation laid by its predecessor, his Llama 2, which was a surprise considering the rumors that the release will take place next month.
With open source roots, Llama-2 contributed to the simultaneous development of other powerful models such as Mixtral, Alpaca, Vicuna, and WizardLM. Now, Llama-3 promises to take these capabilities even further, offering capabilities comparable to OpenAI’s current flagship AI model, GPT-4.
Meta hailed Thursday’s release as “the next generation of our cutting-edge open source large-scale language model.” The tech giant is so confident in its capabilities that Llama 3 powers Meta AI, which has been added to almost all of the company’s hugely popular apps (Instagram, Facebook, WhatsApp) . Although it is available in some countries, users in other regions can also access it via his VPN.
Meta AI’s chatbot interface is comparable to ChatGPT Plus and is free.
“We are upgrading MetaAI with the new cutting-edge Llama 3 AI model that we are open sourcing,” Mark Zuckerberg said in a Facebook post. “We believe this new model will make Meta AI the most intelligent AI assistant at our disposal.”
Decryption We were able to test the new AI and found it to be functionally equivalent to ChatGPT-Plus without a paid subscription. It can generate images and animations, generate code, and provide consistent, context-relevant responses. New chatbots can also access the internet, but they still don’t come close to the capabilities of specialized solutions like Perplexity.
Perhaps the only drawback is that Llama-3’s current context window is limited to 8K tokens (approximately 6,000 words).
Meta has released a 70 billion parameter Llama-3 model, which requires a large amount of computing power (possibly an entire rack of GPUs) to use. According to comprehensive benchmarks, this model outperforms Gemini 1.5 Pro and Claude 3 Sonnet.
An 8 billion parameter model is also available and can be run locally on consumer-grade GPUs. This outperforms Google’s Gemma and Mistral 7B on various synthetic benchmarks. Since this model is not yet listed on LLM Arena, we do not yet have a subjective ELO score for him to report.
Image: Meta
Both models can also be run on cloud instances at low cost.
“We are committed to developing Llama 3 in a responsible manner and are providing a variety of resources to help others use Llama 3 responsibly,” Mehta said. . This includes the introduction of new reliability and safety tools such as Llama Guard 2, Code Shield, and CyberSec Eval 2.
Meta says it plans to introduce new features, longer context windows, additional model sizes, and enhanced performance in the coming months. Llama 3 research papers will also be shared.
“Built with Llama 3 technology, Meta AI is one of the world’s leading AI assistants that increases intelligence and reduces workload, helping you learn, get work done, create content, and make the most of every moment. It helps you connect,” Mehta said.
Meta added that it is also training a large-scale 400 billion parameter model that is expected to be released later this year. This model is likely to be comparable to the latest versions of Claude Opus or GPT-4.5, and could be the most powerful open source model to date. If history repeats itself, it will surpass his Llama-3 in overall quality, serve as the basis for a new generation of fine-tuned models, and encourage competition with models closer to the main source. Sho.
riding a llama
Decryption I tested Llama-3 within Meta AI to see if it’s as good as Zuck says it is. In short, Llama-3 introduces many notable features and should be a good foundational model for the open source community to iterate on.
content moderation
Llama-3 demonstrates a strong commitment to content moderation. Consistently refused to produce harmful racial content even when faced with common jailbreak techniques.
For example, when asked for instructions on how to seduce a woman, the model gave a generic but informative answer. However, when asked about how to seduce his best friend’s wife, the model flatly refused to provide an answer.
images and animations
Similar to ChatGPT-Plus, Meta AI with Llama-3 can generate images. However, we take this feature a step further by providing the option to animate. This feature is not available in ChatGPT or Gemini.
Images produced by Meta AI using Llama-3 are more realistic than those produced by Dalle-3, but they fall short of the quality of images produced by Google’s upcoming ImageFX.
Coding function
Llama-3 has proven to be very skilled at coding. When presented with a unique and poorly described game idea, the model allowed him to generate the necessary Python code in two tries, resulting in a working game. The first shot gave us a rough idea of how to create the game, but once it became clear that we needed it in Python, we wrote working code.
The game was functional, but it was missing some small details, such as restarting after the player won. However, the same thing happened with other chatbots.
We found Claude 3 Sonnet to be the best tool for this task, followed by Llama 3. GPT-4 falls back to his third place. However, different users may have different results.
Here is a pastebin with generated source code for those who want to test Llama3, Claude, and ChatGPT.
political neutrality
This model aims for political neutrality, as evidenced by the answers to questions about capitalism and communism. The answers were similar in structure and provided an overview, advantages, and disadvantages of each system.
This pattern of neutrality was also observed in responses to questions such as “What is a human being?” and “What is a woman?”
Still, the reaction is slightly pro-capitalist and left-leaning, which is not surprising since this is the most common political trend among large language models.
logical reasoning
Rama-3 demonstrated strong logical reasoning abilities. When tested with complex LSAT questions that often confuse users, the model not only provided the correct answers, but also provided clear and reasonable explanations.
Long prompt limitations
Despite Llama-3’s many advantages, it struggles with long prompts. When presented with a long prompt with about a page and a half of context, which can be captured by models such as GPT-4, Claude, and Mistral, the model returned an error his message.
language understanding
This model shows a deep understanding of different languages. When asked to translate a slogan in Spanish, they not only provided an accurate translation, but also provided context to better understand the slogan.
conclusion
As a chatbot interface, Meta AI (powered by Llama3) can compete with ChatGPT Plus and is an overall better choice.
On a more technical level, LLama3 as an LLM is sufficient to compete with GPT-4 in various scenarios, but it also has token context capabilities and search extension generation (basically extracting information from a specific dataset provided by the user). You’re just losing out in terms of (obtaining). While this may be important for tech-savvy users, it may not be a big deal for the average person.
If you primarily use ChatGPT to generate images in Dall-E, you may want to consider canceling your subscription as Llama-3 has comparable image and animation generation capabilities . However, if you also need support for long prompts, Llama-3 may not be the best choice and you may want to consider using ChatGPT-Plus.
In some cases, users may find that Llama-3 meets their needs without requiring a paid membership.
For tasks that require a lot of internet research, ChatGPT Plus or Perplexity may be better suited.
Finally, if your focus is on coding, Llama-3 can be a good alternative, but there are other specialized tools available. The fact that Llama-3 is free is a huge advantage.
Edited by Ryan Ozawa.
