From Digital Age to Nano Age. WorldWide.

Tag: Open

Robotic Automations

Tumblr launches its semi-private Communities in open beta | TechCrunch


Tumblr, the blogging site acquired twice, is launching its “Communities” feature in open beta, the Tumblr Labs division has announced. The feature offers a dedicated space for users to connect with others on different topics, outside of Tumblr’s main dashboard. The open beta comes six months after Communities launched in closed beta and represents a shift […]

© 2024 TechCrunch. All rights reserved. For personal use only.


Software Development in Sri Lanka

Robotic Automations

Despite recent successes, IPO market still won't fully open until 2025 | TechCrunch


This year already proved that startups are willing to go public in a less-than-ideal market — and get rewarded for it, too. But bankers, lawyers and investors said the recent IPO successes aren’t enough to foster more than a dozen tech IPOs this year.

“I don’t think we will have the floodgates open like I might have thought,” Greg Martin, co-founder and managing director at Rainmaker Securities, told TechCrunch. “The trickle was delayed; I thought it would happen sooner in Q1. Because of that, I think the floodgates can’t open til 2025, but we could have a healthy flow of 10 to 15 companies for the year.”

Jeremy Glaser, a lawyer and co-chair of Mintz’s venture capital and emerging companies practice, said that despite how the recent IPOs have performed thus far, people need more data than just a few weeks, or a month, of trading to feel confident.

Looking at how Klaviyo and Instacart are performing today shows why people remain cautious. Klaviyo is currently trading at a $5.94 billion market cap, down from its $9.2 billion IPO price. Instacart is faring better, but still trading under its initial IPO price of $9.9 billion. It’s currently trading at $9.47 billion.

“I’ve lived through a lot of IPO cycles, you really do need an extended period of time where you are seeing multiple IPOs staying above the IPO price,” Glaser said. “I don’t know if we are there yet. We have some positive signs but we need to see more companies staying above the IPO price for an extended amount of time.”

Timing plays a big factor here, too, due to the election. If a couple of companies had come out and made their public debuts at the beginning of the year — and had they done well — it might have given other companies enough time and confidence to get through a full S-1 process before the election. But due to the timing of the recent IPOs, companies would be crunched for time.

Martin added that despite the successes, he’s not sure this is really a good market to go out in anyway. Interest rates aren’t being cut the way many predicted and were hoping for this year, and Martin isn’t convinced that the economy is fully in the clear yet after 2022’s bear market — especially with uncertainty about how the markets will react after the election in November.

“I still feel like recession is not out of the woods yet,” Martin said. “We had, what, 1% growth in Q1? Mostly macro economic factors, it feels like the market is sensing relative stability right now but there [are] a lot of things that could turn that. I’m hopeful [the market] remains stable. I’m remaining optimistic at this point.”

The sentiment from Glaser and Martin seems to align with what other folks in the market are saying, too. A top-tier venture fund recently told TechCrunch that it was advising all of its portfolio companies that could potentially IPO to wait until next year. Colin Stewart, Morgan Stanley’s global head of technology equity markets, recently told CNBC that he thinks 10 to 15 companies could go public this year — right in line with Martin’s prediction — and that 2025 will be better.

Investors weren’t sure what to think about the IPO market heading into 2024. Some thought that activity would start to pick back up while others thought it would be another quiet year, according to a TechCrunch survey. The one thing they all seemed to agree on was that any rise in activity wasn’t likely until the second half of the year.

But then Astera Labs filed to go public in February, and Reddit followed shortly after. Ibotta was next in March, followed by Rubrik just a week later. All four have since floated and popped on their first day of trading. While the respective stocks retreated since then, they are all currently trading above their IPO prices — which were all priced above their initial target ranges.

Watching these four stocks hit the market successfully makes us wonder: Were investors wrong about the timeline of the return of IPOs? But based on sentiment from folks like Martin and Glaser, probably not.

This means that VCs likely have to wait another year for the IPO market to be a meaningful source of liquidity. However, exits aren’t fully off the table this year. Glaser said that he isn’t working on IPOs, but his M&A practice has been the busiest it’s been in a long time. For investors looking for returns this year, that’s good news.


Software Development in Sri Lanka

Robotic Automations

Why code-testing startup Nova AI uses open source LLMs more than OpenAI


It is a universal truth of human nature that the developers who build the code should not be the ones to test it. First of all, most of them pretty much detest that task. Second, like any good auditing protocol, those who do the work should not be the ones who verify it.

Not surprisingly, then, code testing in all its forms  –  usability,  language- or task-specific tests, end-to-end testing – has been a focus of a growing cadre of generative AI startups. Every week, TechCrunch covers another one like  Antithesis (raised $47 million); CodiumAI (raised $11 million) QA Wolf (raised $20 million). And new ones are emerging all the time, like new Y Combinator graduate Momentic.

Another is year-old startup Nova AI, an Unusual Academy accelerator grad that’s raised a $1 million pre-seed round. It is attempting to best its competitors with its end-to-end testing tools by breaking many of the Silicon Valley rules of how startups should operate, founder CEO Zach Smith tells TechCrunch.

Whereas the standard Y Combinator approach is to start small, Nova AI is aiming at mid-size to large enterprises with complex code-bases and a burning need now. Smith declined to name any customers using or testing its product except to describe them as mostly late-stage (series C or beyond) venture-backed startups in ecommerce, fintech or consumer products, and “heavy user experiences. Downtime for these features is costly.”

Nova AI’s tech sifts through its customers’ code to automatically build tests automatically using GenAI. It is particularly geared toward continuous integration and continuous delivery/deployment (CI/CD) environments where engineers are constantly shipping bits and pieces into their production code.

The idea for Nova AI came from the experiences Smith and his cofounder Jeffrey Shih had when they were engineers working for big tech companies. Smith is a former Googler who worked on cloud-related teams that helped customers use a lot of automation technology. Shih had previously worked at Meta (also at Unity and Microsoft before that) with a rare AI speciality involving synthetic data. They’ve since added a third cofounder, AI data scientist Henry Li.

Another rule Nova AI is not following: while boatloads of AI startups are building on top of OpenAI’s industry leading GPT, Nova AI is using OpenAI’s Chat GPT-4 as little as possible, only to help it generate some code and to do some labeling tasks. No customer data is being fed to OpenAI.

While OpenAI promises that the data of those on a paid business plan is not being used to train its models, enterprises still do not trust OpenAI, Smith tells us. “When we’re talking to large enterprises, they’re like, ‘We don’t want our data going into OpenAI,” Smith said.

The engineering teams of large companies are not the only ones that feel this way. OpenAI is fending off a number of lawsuits from those who don’t want it to use their work for model training, or believe their work wound up, unauthorized and unpaid for, in its outputs.

Nova AI is instead heavily relying on open source models like Llama developed by Meta and StarCoder (from the BigCoder community, which was developed by ServiceNow and Hugging Face), as well as building its own models. They aren’t yet using Google’s Gemma with customers, but have tested it and “seen good results,” Smith says.

For instance, he explains that a common use for OpenAI GPT4 is to “produce vector embeddings” on data so LLM models can use the vectors for semantic search. Vector embeddings translate chunks of text into numbers so the LLM can perform various operations, such as cluster them with other chunks of similar text. Nova AI is using OpenAI’s GPT4 for this on the customer’s source code, but is going to lengths not to send any data into OpenAI.

“In this case, instead of using OpenAI’s embedding models, we deploy our own open-source embedding models so that when we need to run through every file, we aren’t just sending it to OpenAi,” Smith explained.

While not sending customer data to OpenAI appeases nervous enterprises, open source AI models are also cheaper and more than sufficient for doing targeted specific tasks, Smith has found. In this case, they work well for writing tests.

“The open LLM industry is really proving that they can beat GPT 4 and these big domain providers, when you go really narrow,” he said. “We don’t have to provide some massive model that can tell you what your grandma wants for her birthday. Right? We need to write a test. And that’s it. So our models are fine-tuned specifically for that.”

Open source models are also progressing quickly. For instance, Meta recently introduced a new version of Llama that’s earning accolades in technology circles and that may convince more AI startups to look at OpenAI alternatives.


Software Development in Sri Lanka

Robotic Automations

Flipboard deepens its ties to the open source social web (aka the fediverse) | TechCrunch


Flipboard, a Web 2.0-era social magazine app that is reinventing itself to capitalize on the renewed push toward an open social web, is deepening its ties to the fediverse, the social network of interconnected servers that includes apps like Mastodon, Pixelfed, PeerTube and, in time, Instagram Threads, among others. On Thursday, the company announced it’s expanding its fediverse integrations to 400 more Flipboard creators and introducing fediverse notifications in the Flipboard app itself.

The latter will allow Flipboard users to see their new followers and other activity around the content they share in the fediverse directly in the Flipboard app. This follows last year’s introduction of a Mastodon integration in the app, replacing Twitter, and the introduction of support for ActivityPub, the social networking protocol that powers the open source, decentralized social networks that include Mastodon and others.

In February, Flipboard announced it would begin to add its creators and their social magazines to the fediverse as well, meaning that the curated magazines of links and other social posts that its creators typically share within the Flipboard app could now find a broader audience. By sharing creators’ posts and links with the wider fediverse, Flipboard’s publishing partners gained their own native ActivityPub feeds so they could be discovered by Mastodon users and those on other federated social apps. That initial push toward federation was started with 1,000 Flipboard magazines and today adds 400 more. In total, Flipboard says there are now over 11,000 curated Flipboard magazines available to federated social networking users.

“This is a major step toward fully federating our platform,” noted Flipboard CEO Mike McCue in an announcement. “We’re not just making curated content on Flipboard viewable, but enabling two-way communication so users can see activity and engage with fediverse communities. Personally, it has made my curation even more exciting as I know it’s reaching new people who may share my interests.”

The expanded set of accounts includes public accounts with one or two public magazines that have activity curated in the past 30 days and don’t have any trust and safety violations. They’ve also participated in Flipboard community programs. Accounts will be alerted to their federated status via email.

While Flipboard is working toward federating its users’ accounts by default, people will be able to “unfederate” by toggling off the “Federate” button in their Flipboard settings.

In addition to the newly federated magazines, Flipboard is also bringing a more integrated fediverse experience to its own app. With the version arriving Thursday (ver. 4.3.25), Flipboard users will be able to see their new followers from the fediverse in their Flipboard profile, while their Flipboard notifications will now include fediverse reactions and conversations.

This notification window will now contain three sections: Replies, Activity and News. In Replies, users will be able to see and reply to posts from people both on Flipboard and in the fediverse, as well as any other fediverse @mentions. When they respond, their reply is also sent back to the fediverse, making Flipboard more of a fediverse client app than before. The Activity tab, meanwhile, will show users the likes, follows and boosts (the fediverse’s take on the retweet), along with other Flipboard activity. The News section (previously called Content) will now showcase breaking news and other stories recommended by Flipboard’s editorial team.

The company had already begun curating content for fediverse users across a handful of “news desks” (dedicated fediverse accounts) that directed users to interesting articles and links across topics. There is a broader news desk, plus those dedicated to TechCulture and Science. This existing curation can help fuel the newly rebranded News section in the Flipboard app.


Software Development in Sri Lanka

Robotic Automations

This Week in AI: When 'open source' isn't so open | TechCrunch


Keeping up with an industry as fast-moving as AI is a tall order. So until an AI can do it for you, here’s a handy roundup of recent stories in the world of machine learning, along with notable research and experiments we didn’t cover on their own.

This week, Meta released the latest in its Llama series of generative AI models: Llama 3 8B and Llama 3 70B. Capable of analyzing and writing text, the models are “open sourced,” Meta said — intended to be a “foundational piece” of systems that developers design with their unique goals in mind.

“We believe these are the best open source models of their class, period,” Meta wrote in a blog post. “We are embracing the open source ethos of releasing early and often.”

There’s only one problem: the Llama 3 models aren’t really “open source,” at least not in the strictest definition.

Open source implies that developers can use the models how they choose, unfettered. But in the case of Llama 3 — as with Llama 2 — Meta has imposed certain licensing restrictions. For example, Llama models can’t be used to train other models. And app developers with over 700 million monthly users must request a special license from Meta. 

Debates over the definition of open source aren’t new. But as companies in the AI space play fast and loose with the term, it’s injecting fuel into long-running philosophical arguments.

Last August, a study co-authored by researchers at Carnegie Mellon, the AI Now Institute and the Signal Foundation found that many AI models branded as “open source” come with big catches — not just Llama. The data required to train the models is kept secret. The compute power needed to run them is beyond the reach of many developers. And the labor to fine-tune them is prohibitively expensive.

So if these models aren’t truly open source, what are they, exactly? That’s a good question; defining open source with respect to AI isn’t an easy task.

One pertinent unresolved question is whether copyright, the foundational IP mechanism open source licensing is based on, can be applied to the various components and pieces of an AI project, in particular a model’s inner scaffolding (e.g. embeddings). Then, there’s the mismatch between the perception of open source and how AI actually functions to overcome: open source was devised in part to ensure that developers could study and modify code without restrictions. With AI, though, which ingredients you need to do the studying and modifying is open to interpretation.

Wading through all the uncertainty, the Carnegie Mellon study does make clear the harm inherent in tech giants like Meta co-opting the phrase “open source.”

Often, “open source” AI projects like Llama end up kicking off news cycles — free marketing — and providing technical and strategic benefits to the projects’ maintainers. The open source community rarely sees these same benefits, and, when they do, they’re marginal compared to the maintainers’.

Instead of democratizing AI, “open source” AI projects — especially those from big tech companies — tend to entrench and expand centralized power, say the study’s co-authors. That’s good to keep in mind the next time a major “open source” model release comes around.

Here are some other AI stories of note from the past few days:

  • Meta updates its chatbot: Coinciding with the Llama 3 debut, Meta upgraded its AI chatbot across Facebook, Messenger, Instagram and WhatsApp — Meta AI — with a Llama 3-powered backend. It also launched new features, including faster image generation and access to web search results.
  • AI-generated porn: Ivan writes about how the Oversight Board, Meta’s semi-independent policy council, is turning its attention to how the company’s social platforms are handling explicit, AI-generated images.
  • Snap watermarks: Social media service Snap plans to add watermarks to AI-generated images on its platform. A translucent version of the Snap logo with a sparkle emoji, the new watermark will be added to any AI-generated image exported from the app or saved to the camera roll.
  • The new Atlas: Hyundai-owned robotics company Boston Dynamics has unveiled its next-generation humanoid Atlas robot, which, in contrast to its hydraulics-powered predecessor, is all-electric — and much friendlier in appearance.
  • Humanoids on humanoids: Not to be outdone by Boston Dynamics, the founder of Mobileye, Amnon Shashua, has launched a new startup, Menteebot, focused on building bibedal robotics systems. A demo video shows Menteebot’s prototype walking over to a table and picking up fruit.
  • Reddit, translated: In an interview with Amanda, Reddit CPO Pali Bhat revealed that an AI-powered language translation feature to bring the social network to a more global audience is in the works, along with an assistive moderation tool trained on Reddit moderators’ past decisions and actions.
  • AI-generated LinkedIn content: LinkedIn has quietly started testing a new way to boost its revenues: a LinkedIn Premium Company Page subscription, which — for fees that appear to be as steep as $99/month — include AI to write content and a suite of tools to grow follower counts.
  • A Bellwether: Google parent Alphabet’s moonshot factory, X, this week unveiled Project Bellwether, its latest bid to apply tech to some of the world’s biggest problems. Here, that means using AI tools to identify natural disasters like wildfires and flooding as quickly as possible.
  • Protecting kids with AI: Ofcom, the regulator charged with enforcing the U.K.’s Online Safety Act, plans to launch an exploration into how AI and other automated tools can be used to proactively detect and remove illegal content online, specifically to shield children from harmful content.
  • OpenAI lands in Japan: OpenAI is expanding to Japan, with the opening of a new Tokyo office and plans for a GPT-4 model optimized specifically for the Japanese language.

More machine learnings

Image Credits: DrAfter123 / Getty Images

Can a chatbot change your mind? Swiss researchers found that not only can they, but if they are pre-armed with some personal information about you, they can actually be more persuasive in a debate than a human with that same info.

“This is Cambridge Analytica on steroids,” said project lead Robert West from EPFL. The researchers suspect the model — GPT-4 in this case — drew from its vast stores of arguments and facts online to present a more compelling and confident case. But the outcome kind of speaks for itself. Don’t underestimate the power of LLMs in matters of persuasion, West warned: “In the context of the upcoming US elections, people are concerned because that’s where this kind of technology is always first battle tested. One thing we know for sure is that people will be using the power of large language models to try to swing the election.”

Why are these models so good at language anyway? That’s one area there is a long history of research into, going back to ELIZA. If you’re curious about one of the people who’s been there for a lot of it (and performed no small amount of it himself), check out this profile on Stanford’s Christopher Manning. He was just awarded the John von Neuman Medal; congrats!

In a provocatively titled interview, another long-term AI researcher (who has graced the TechCrunch stage as well), Stuart Russell, and postdoc Michael Cohen speculate on “How to keep AI from killing us all.” Probably a good thing to figure out sooner rather than later! It’s not a superficial discussion, though — these are smart people talking about how we can actually understand the motivations (if that’s the right word) of AI models and how regulations ought to be built around them.

The interview is actually regarding a paper in Science published earlier this month, in which they propose that advanced AIs capable of acting strategically to achieve their goals, which they call  “long-term planning agents,” may be impossible to test. Essentially, if a model learns to “understand” the testing it must pass in order to succeed, it may very well learn ways to creatively negate or circumvent that testing. We’ve seen it at a small scale, why not a large one?

Russell proposes restricting the hardware needed to make such agents… but of course, Los Alamos and Sandia National Labs just got their deliveries. LANL just had the ribbon-cutting ceremony for Venado, a new supercomputer intended for AI research, composed of 2,560 Grace Hopper Nvidia chips.

Researchers look into the new neuromorphic computer.

And Sandia just received “an extraordinary brain-based computing system called Hala Point,” with 1.15 billion artificial neurons, built by Intel and believed to be the largest such system in the world. Neuromorphic computing, as it’s called, isn’t intended to replace systems like Venado, but to pursue new methods of computation that are more brain-like than the rather statistics-focused approach we see in modern models.

“With this billion-neuron system, we will have an opportunity to innovate at scale both new AI algorithms that may be more efficient and smarter than existing algorithms, and new brain-like approaches to existing computer algorithms such as optimization and modeling,” said Sandia researcher Brad Aimone. Sounds dandy… just dandy!


Software Development in Sri Lanka

Robotic Automations

Meta releases Llama 3, claims it's among the best open models available | TechCrunch


Meta has released the latest entry in its Llama series of open source generative AI models: Llama 3. Or, more accurately, the company has open sourced two models in its new Llama 3 family, with the rest to come at an unspecified future date.

Meta describes the new models — Llama 3 8B, which contains 8 billion parameters, and Llama 3 70B, which contains 70 billion parameters — as a “major leap” compared to the previous-gen Llama models, Llama 2 8B and Llama 2 70B, performance-wise. (Parameters essentially define the skill of an AI model on a problem, like analyzing and generating text; higher-parameter-count models are, generally speaking, more capable than lower-parameter-count models.) In fact, Meta says that, for their respective parameter counts, Llama 3 8B and Llama 3 70B — trained on two custom-built 24,000 GPU clusters — are are among the best-performing generative AI models available today.

That’s quite a claim to make. So how is Meta supporting it? Well, the company points to the Llama 3 models’ scores on popular AI benchmarks like MMLU (which attempts to measure knowledge), ARC (which attempts to measure skill acquisition) and DROP (which tests a model’s reasoning over chunks of text). As we’ve written about before, the usefulness — and validity — of these benchmarks is up for debate. But for better or worse, they remain one of the few standardized ways by which AI players like Meta evaluate their models.

Llama 3 8B bests other open source models like Mistral’s Mistral 7B and Google’s Gemma 7B, both of which contain 7 billion parameters, on at least nine benchmarks: MMLU, ARC, DROP, GPQA (a set of biology-, physics- and chemistry-related questions), HumanEval (a code generation test), GSM-8K (math word problems), MATH (another mathematics benchmark), AGIEval (a problem-solving test set) and BIG-Bench Hard (a commonsense reasoning evaluation).

Now, Mistral 7B and Gemma 7B aren’t exactly on the bleeding edge (Mistral 7B was released last September), and in a few of benchmarks Meta cites, Llama 3 8B scores only a few percentage points higher than either. But Meta also makes the claim that the larger-parameter-count Llama 3 model, Llama 3 70B, is competitive with flagship generative AI models including Gemini 1.5 Pro, the latest in Google’s Gemini series.

Image Credits: Meta

Llama 3 70B beats Gemini 1.5 Pro on MMLU, HumanEval and GSM-8K, and — while it doesn’t rival Anthropic’s most performant model, Claude 3 Opus — Llama 3 70B scores better than the weakest model in the Claude 3 series, Claude 3 Sonnet, on five benchmarks (MMLU, GPQA, HumanEval, GSM-8K and MATH).

Image Credits: Meta

For what it’s worth, Meta also developed its own test set covering use cases ranging from coding and creating writing to reasoning to summarization, and — surprise! — Llama 3 70B came out on top against Mistral’s Mistral Medium model, OpenAI’s GPT-3.5 and Claude Sonnet. Meta says that it gated its modeling teams from accessing the set to maintain objectivity, but obviously — given that Meta itself devised the test — the results have to be taken with a grain of salt.

Image Credits: Meta

More qualitatively, Meta says that users of the new Llama models should expect more “steerability,” a lower likelihood to refuse to answer questions, and higher accuracy on trivia questions, questions pertaining to history and STEM fields such as engineering and science and general coding recommendations. That’s in part thanks to a much larger data set: a collection of 15 trillion tokens, or a mind-boggling ~750,000,000,000 words — seven times the size of the Llama 2 training set. (In the AI field, “tokens” refers to subdivided bits of raw data, like the syllables “fan,” “tas” and “tic” in the word “fantastic.”)

Where did this data come from? Good question. Meta wouldn’t say, revealing only that it drew from “publicly available sources,” included four times more code than in the Llama 2 training data set, and that 5% of that set has non-English data (in ~30 languages) to improve performance on languages other than English. Meta also said it used synthetic data — i.e. AI-generated data — to create longer documents for the Llama 3 models to train on, a somewhat controversial approach due to the potential performance drawbacks.

“While the models we’re releasing today are only fine tuned for English outputs, the increased data diversity helps the models better recognize nuances and patterns, and perform strongly across a variety of tasks,” Meta writes in a blog post shared with TechCrunch.

Many generative AI vendors see training data as a competitive advantage and thus keep it and info pertaining to it close to the chest. But training data details are also a potential source of IP-related lawsuits, another disincentive to reveal much. Recent reporting revealed that Meta, in its quest to maintain pace with AI rivals, at one point used copyrighted ebooks for AI training despite the company’s own lawyers’ warnings; Meta and OpenAI are the subject of an ongoing lawsuit brought by authors including comedian Sarah Silverman over the vendors’ alleged unauthorized use of copyrighted data for training.

So what about toxicity and bias, two other common problems with generative AI models (including Llama 2)? Does Llama 3 improve in those areas? Yes, claims Meta.

Meta says that it developed new data-filtering pipelines to boost the quality of its model training data, and that it’s updated its pair of generative AI safety suites, Llama Guard and CybersecEval, to attempt to prevent the misuse of and unwanted text generations from Llama 3 models and others. The company’s also releasing a new tool, Code Shield, designed to detect code from generative AI models that might introduce security vulnerabilities.

Filtering isn’t foolproof, though — and tools like Llama Guard, CybersecEval and Code Shield only go so far. (See: Llama 2’s tendency to make up answers to questions and leak private health and financial information.) We’ll have to wait and see how the Llama 3 models perform in the wild, inclusive of testing from academics on alternative benchmarks.

Meta says that the Llama 3 models — which are available for download now, and powering Meta’s Meta AI assistant on Facebook, Instagram, WhatsApp, Messenger and the web — will soon be hosted in managed form across a wide range of cloud platforms including AWS, Databricks, Google Cloud, Hugging Face, Kaggle, IBM’s WatsonX, Microsoft Azure, Nvidia’s NIM and Snowflake. In the future, versions of the models optimized for hardware from AMD, AWS, Dell, Intel, Nvidia and Qualcomm will also be made available.

And more capable models are on the horizon.

Meta says that it’s currently training Llama 3 models over 400 billion parameters in size — models with the ability to “converse in multiple languages,” take more data in and understand images and other modalities as well as text, which would bring the Llama 3 series in line with open releases like Hugging Face’s Idefics2.

Image Credits: Meta

“Our goal in the near future is to make Llama 3 multilingual and multimodal, have longer context and continue to improve overall performance across core [large language model] capabilities such as reasoning and coding,” Meta writes in a blog post. “There’s a lot more to come.”

Indeed.


Software Development in Sri Lanka

Robotic Automations

Meta confirms that its Llama 3 open source LLM is coming in the next month | TechCrunch


At an event in London on Tuesday, Meta confirmed that it plans an initial release of Llama 3 — the next generation of its large language model used to power generative AI assistants — within the next month.

This confirms a report published on Monday by The Information that Meta was getting close to launch.

“Within the next month, actually less, hopefully in a very short period of time, we hope to start rolling out our new suite of next-generation foundation models, Llama 3,” said Nick Clegg, Meta’s president of global affairs. He described what sounds like the release of several different iterations or versions of the product. “There will be a number of different models with different capabilities, different versatilities [released] during the course of this year, starting really very soon.”

The plan, Meta Chief Product Officer Chris Cox added, will be to power multiple products across Meta with Llama 3.

Meta has been scrambling to catch up to OpenAI, which took it and other big tech companies like Google by surprise when it launched ChatGPT over a year ago and the app went viral, turning generative AI questions and answers into everyday, mainstream experiences.

Meta has largely taken a very cautious approach with AI, but that hasn’t gone over well with the public, with previous versions of Llama criticized as too limited. (Llama 2 was released publicly in July 2023. The first version of Llama was not released to the public, yet it still leaked online.)

Llama 3, which is bigger in scope than its predecessors, is expected to address this, with capabilities not just to answer questions more accurately but also to field a wider range of questions that might include more controversial topics. It hopes this will make the product catch on with users.

“Our goal over time is to make a Llama-powered Meta AI be the most useful assistant in the world,” said Joelle Pineau, vice president AI Research. “There’s quite a bit of work remaining to get there.” The company did not talk about the size of the parameters it’s using in Llama 3, nor did it offer any demos of how it would work. It’s expected to have about 140 billion parameters, compared to 70 billion for the biggest Llama 2 model.

Most notably, Meta’s Llama families, built as open source products, represent a different philosophical approach to how AI should develop as a wider technology. In doing so, Meta is hoping to play into wider favor with developers versus more proprietary models.

But Meta is also playing it more cautiously, it seems, especially when it comes to other generative AI beyond text generation. The company is not yet releasing Emu, its image generation tool, Pineau said.

“Latency matters a lot along with safety along with ease of use, to generate images that you’re proud of and that represent whatever your creative context is,” Cox said.

Ironically — or perhaps predictably (heh) — even as Meta works to launch Llama 3, it does have some significant generative AI skeptics in the house.

Yann LeCun, the celebrated AI academic who is also Meta’s chief AI scientist, took a swipe at the limitations of generative AI overall and said his bet is on what comes after it. He predicts that will be joint embedding predicting architecture (JEPA), a different approach both to training models and producing results, which Meta has been using to build more accurate predictive AI in the area of image generation.

“The future of AI is JEPA. It’s not generative AI,” he said. “We’re going to have to change the name of Chris’s product division.”


Software Development in Sri Lanka

Back
WhatsApp
Messenger
Viber