From Digital Age to Nano Age. WorldWide.

Tag: hopes

Robotic Automations

Symbolica hopes to head off the AI arms race by betting on symbolic models | TechCrunch

In February, Demis Hassabis, the CEO of Google‘s DeepMind AI research lab, warned that throwing increasing amounts of compute at the types of AI algorithms in wide use today could lead to diminishing returns. Getting to the “next level” of AI, as it were, Hassabis said, will instead require fundamental research breakthroughs that yield viable alternatives to today’s entrenched approaches.

Ex-Tesla engineer George Morgan agrees. So he founded a startup, Symbolica AI, to do just that.

“Traditional deep learning and generative language models require unimaginable scale, time and energy to produce useful outcomes,” Morgan told TechCrunch. “By building [novel] models, Symbolica can accomplish greater accuracy with lower data requirements, lower training time, lower cost and with provably correct structured outputs.”

Morgan dropped out of college at Rochester to join Tesla, where he worked on the team developing Autopilot, Tesla’s suite of advanced driver-assistance features.

While at Tesla, Morgan says that he came to realize that current AI methods — most of which revolved around scaling up compute — wouldn’t be sustainable over the long term.

“Current methods only have one dial to turn: increase scale and hope for emergent behavior,” Morgan said. “However, scaling requires more compute, more memory, more money to train and more data. But eventually, [this] doesn’t get you significantly better performance.”

Morgan isn’t the only one to reach that conclusion.

In a memo this year, two executives at TSMC, the semiconductor fabricator, said that, if the AI trend continues at its current pace, the industry will need a 1-trillion-transistor chip — a chip containing 10x as many transistors as the average chip today — within a decade.

It’s unclear whether that’s technologically feasible.

Elsewhere, a report (unpublished) co-authored by Stanford and Epoch AI, an independent AI research Institute, finds that the cost of training cutting-edge AI models has increased substantially over the past year and change. The report’s authors estimate that OpenAI and Google spent around $78 million and $191 million, respectively, training GPT-4 and Gemini Ultra.

With costs poised to climb higher still — see OpenAI’s and Microsoft’s reported plans for a $100 billion AI data center — Morgan began investigating what he calls “structured” AI models. These structured models encode the underlying structure of data — hence the name — instead of trying to approximate insights from enormous data sets, like conventional models, enabling them to attain what Morgan characterizes as better performance using less overall compute.

“It’s possible to produce domain-tailored structured reasoning capabilities in much smaller models,” he said, “marrying a deep mathematical toolkit with breakthroughs in deep learning.”

Symbolic AI isn’t exactly a new concept. They date back decades, rooted in the idea that AI can be built on symbols that represent knowledge using a set of rules.

Traditional symbolic AI solves tasks by defining symbol-manipulating rule sets dedicated to particular jobs, such as editing lines of text in word processor software. That’s as opposed to neural networks, which try to solve tasks through statistical approximation and learning from examples. Symbolica aims to leverage the best of both worlds.

Neural networks are the cornerstone of powerful AI systems like OpenAI’s DALL-E 3 and GPT-4. But, Morgan claims, scale is not the end-all be-all; marrying mathematical abstractions with neural networks might in fact be better positioned to efficiently encode the world’s knowledge, reason their way through complex scenarios, and “explain” how they arrive at an answer, Morgan argues.

“Our models are more reliable, more transparent and more accountable,” Morgan said. “There are immense commercial applications of structured reasoning capabilities, particularly for code generation — i.e. reasoning over large codebases and generating useful code — where existing offerings fall short.”

Symbolica’s product, designed by its 16-person team, is a toolkit for creating symbolic AI models and models pre-trained for specific tasks, including generating code and proving mathematical theorems. The exact business model is in flux. But Symbolica might provide consulting services and support for companies that wish to build bespoke models using its technologies, Morgan said.

“The company will work closely with large enterprise partners and customers, building custom structured models with significantly improved reasoning capabilities — tailored to individual customer needs,” Morgan said. “They’ll also develop and sell state-of-the-art code synthesis models to large enterprise customers.”

Today marks Symbolica’s launch out of stealth, so the company doesn’t have customers — at least none that it’s willing to talk about publicly. Morgan did, however, reveal that Symbolica landed a $33 million investment earlier this year led by Khosla Ventures. Other investors included Abstract Ventures, Buckley Ventures, Day One Ventures and General Catalyst.

Indeed, $33 million is no small figure; Symbolica’s backers evidently have confidence in the startup’s science and roadmap. Vinod Khosla, Khosla Ventures’ founder, told me via email that he believes Symbolica is “tackling some of the most important challenges facing the AI industry today.”

“To enable large-scale commercial AI adoption and regulatory compliance, we need models with structured outputs that can achieve greater accuracy with fewer resources,” Khosla said. “George has amassed one of the best teams in the industry to do just that.”

But others are less convinced that symbolic AI is the right path forward.

Os Keyes, a PhD candidate at the University of Washington focusing on law and data ethics, notes that symbolic AI models depend on highly structured data, which makes them both “extremely brittle” and dependent on context and specificity. Symbolic AI needs well-defined knowledge to function, in other words — and defining that knowledge can be highly labor-intensive.

“This could still be interesting if it marries the advantages of deep learning and symbolic approaches,” Keyes said, referring to DeepMind’s recently published AlphaGeometry, which combined neural networks with a symbolic AI-inspired algorithm to solve challenging geometry problems. “But time will tell.”

Morgan rebutted by saying that current training methods soon won’t be able to meet the needs of companies that wish to harness AI for their purposes, making any promising alternatives worth investing in. And, he claimed, Symbolica is strategically well-positioned for this future, given that it has “several years” of runway with its latest funding tranche and its models are relatively small (and therefore cheap) to train and run.

“Tasks like automating software development, for example, at scale will require models with formal reasoning capabilities, and cheaper operating costs, to parse large code databases and produce and iterate on useful code,” he said. “Public perception around AI models is still very much that ‘scale is all you need.’ Thinking symbolically is absolutely necessary to make progress in the field — structured and explainable outputs with formal reasoning capabilities will be required to meet demands.”

There’s not much to prevent a big AI lab like DeepMind from building its own symbolic AI or hybrid models and — setting aside Symbolica’s points of differentiation — Symbolica is entering an extremely crowded and well-capitalized AI field. But Morgan’s anticipating growth all the same, and expects San Francisco-based Symbolica’s staff to double by 2025.

Software Development in Sri Lanka

Robotic Automations

French deep tech spinout Diamfab crystallizes hopes for diamond semiconductors to support green transition | TechCrunch

As more funding flows into deep tech to address difficult global problems like climate change, PhD entrepreneurs coming out of Europe’s top universities and labs are increasingly turning their research into companies.

French spinout Diamfab, founded in 2019, is one example. Its co-founders, CEO Gauthier Chicot and CTO Khaled Driche, both PhDs in nanoelectronics and recognized researchers in the field of semiconducting diamond, left Institut Néel, a laboratory of the French National Center for Scientific Research (CNRS), with two licensed patents under their belt.

Since then, Chicot and Driche have registered more patents and brought on a third co-founder, Ivan Llaurado, as their chief revenue officer and partnership director. They also raised an €8.7 million round of funding from Asterion Ventures, Bpifrance’s French Tech Seed fund, Kreaxi, Better Angle, Hello Tomorrow and Grenoble Alpes Métropole.

This interest comes because the paradigm around semiconducting diamonds has changed in the last two years. “Diamonds are no longer a laboratory subject: They have become an industrial reality, with startups, with manufacturers interested in this field and with the partners we have around us,” Chicot told TechCrunch.

Getting out of the lab

Silicon is still the most widely used semiconductor material in electronics because it’s ubiquitous and cheap. But there’s hope other options could someday outperform it, and not just in labs. Tesla’s decision to use silicon carbide instead of silicon was an important step in that direction, and diamond could be next.

Because diamond is naturally more resistant to high temperatures and more energy-efficient, Diamfab envisions a future in which a given component will need a much smaller surface of synthetic diamond than of silicon carbide, which will make it competitive on price.

The firm’s long-term goal is to make more efficient semiconductors with a lower carbon footprint, while also supporting what Chicot refers to as “the electrification of society,” starting with transportation.

Diamond-based electronics open the door to applications in the field of power electronics — think of smaller batteries and chargers with more autonomy, because less temperature control is required, which is particularly relevant for the automotive sector and electric mobility. But diamond wafers could also be leveraged for nuclear batteries, space tech and quantum computing, too.

The case for diamond as a better alternative to silicon doesn’t come out of nowhere; Diamfab is building on the Institut Néel’s 30 years of R&D into synthetic diamond growth. Its founders wanted to take this technology out of the lab. “We wanted to be useful pioneers,” Chicot said.

Being awarded the Jury’s Grand Prize of i-Lab in 2019 was a turning point for the firm. Co-organized by French institutions, it brought grants and a sense of validation that helped the team inwards and outwards.

With this seal of approval, “banks trust you even if you don’t generate any sales,” Chicot said. “It was a real plus in the beginning to get this award. And it was partly because we have great technology, and partly because it’s technology that’s crucial for the world.”

Diamond promises

French public sector investment bank Bpifrance, one of the organizers of the i-Lab awards, is doubling down on Diamfab with funding from the French Tech Seed fund, which Bpifrance manages on behalf of the French government as part of the France 2030 plan.

When silicon has become a commodity, Diamfab’s high-value-added diamond wafers could be made in Europe and sold at a premium warranted by their higher efficiency, which also ties into the green transition. Decarbonization is one key goal of France 2030, and diamonds could help.

Their carbon footprint would be lighter because of the smaller surface that diamond requires compared to silicon carbide, but also because Diamfab synthetizes its diamonds from methane. In the future, this source could be biomethane, giving a commercial outlet to this byproduct of recycling.

Image Credits: Diamfab

Most of this, however, is still in the future. Diamfab is not decades away from its goals, but says it will need five years for its technology to be able to support the mass production of diamond wafers that fit industry requirements. This means taking its know-how in growing and doping diamond layers on one-inch wafers, and applying it to the four-inch wafers that silicon carbide already works on. Even with enough funding to support a small pilot production line, this will take a few years.

This five-year horizon made Diamfab a no-go for some VCs; while these may be sympathetic to the idea of reindustrializing Europe with cutting-edge innovation, their liquidity cycles make these types of investments more difficult. But Chicot ultimately managed to round up the €8.7 million that will help the startup go through its pre-industrialization phase.

Grenoble, a deep tech hub

The group of investors that have rallied around Diamfab is “balanced,” Chicot said, including public players, evergreen fund Asterion Ventures, and supporters of Diamfab’s region, Auvergne-Rhône-Alpes, and its city of Grenoble.

While there’s warranted hype around AI in Paris, Grenoble may be the closest to a French Silicon Valley. In no small part thanks to Nobel Prize-winning physicist Louis Néel, the Alpine city’s focus on electronics turned it into a deep tech hub that’s now also part of the conversation on both green tech and sovereign tech.

Grenoble startups that pop to mind include Verkor, which secured more than €2 billion for its gigafactory in Northern France, and Renaissance Fusion, which raised $16.4 million last year to build nuclear fusion technology in Europe. But Diamfab may benefit more from its partnerships with larger players with local ties, including CEA, Schneider Electric, Soitec and STMicroelectronics.

There’s no doubt that more semiconductors will come out of the French Alps. As both the EU and the U.S. adopted Chip Acts to reduce their dependency in Asia, France is set to provide €2.9 billion in aid for the upcoming joint factory of STMicroelectronics and GlobalFoundries, and Soitec recently opened a fourth factory nearby. Now Diamfab hopes it can play a part, too, and unleash the full potential of diamond in semiconductors.

Software Development in Sri Lanka