Venture Bytes #106: Photonic Chips - Next Leg Up in Semiconductor Innovation

Photonic Chips - Next Leg Up in Semiconductor Innovation

In February, Sam Altman called for a $5-$7 trillion global investment to produce more powerful chips for the next generation of AI platforms. The amount, which is almost a quarter of the US federal budget, highlights the critical importance of semiconductor innovation in driving forward AI capabilities.

While some companies, like Groq, are focusing on inference chips to meet specific AI requirements, another cohort of startups is tackling a pressing issue that underpins the entire AI ecosystem: moving data fast and efficiently enough across AI systems. Start-ups like Celestial AI, Lightmatter, Ayar Labs, and Salience Labs are at the forefront of this revolution.

Currently, data typically moves via electrical signals within computers and between chips in data centers during AI algorithm training. Although some segments of these interconnections may be converted to fiber-optic links to accommodate high bandwidth requirements, the conversion process between optical and electrical signals introduces a communication bottleneck. Photonic chips offer a solution by reducing heat generation, enabling faster data transmission, and increasing resilience to environmental factors. This enhances training performance for AI algorithms.

These companies are delivering a new paradigm in chip architecture and the next transition for large-scale computing by using light to transfer data. For instance, Celestial AI’s point-of-compute Photonic Fabric architecture uses a proprietary, thermally stable modulation technology and other tricks to enable bandwidth delivery “beyond the beachfront” and directly to the point of compute within the chip. This contrasts with the beachfront architecture of most optical-interconnect approaches, where optical-to-electrical conversion occurs at the boundaries of GPUs or ASICs, leading to low bandwidth. The beachfront architecture also leads to a “memory wall” that can create substantial amounts of “stranded memory,” a situation that in turn can degrade efficiency and affect AI performance. For instance, in Azure, up to 25% of memory is stranded after the servers' cores have been rented to VMs, according to a Microsoft report.

Memory transaction energy overhead is another key benefit of Celestial AI’s technology. The company, which recently raised its $175 million Series C round from investors including AMD Ventures and Samsung, claims its memory transaction energy overhead to be about 6.2 picojoules per bit versus about 62.5 via Nvidia’s NVLink, NVSwitch for a remote memory transaction. With power consumption and cooling costs being significant concerns for data center operators, solutions like Photonic Fabric that can reduce energy usage while maintaining or improving performance are likely to see strong adoption. Further, by allowing the disaggregation of compute and memory, the Photonic Fabric can enable a significant performance kick with as much as a 25-fold boost in off-package bandwidth compared to “other state-of-the-art technologies.

Lightmatter, another Series C startup, emerges as a compelling contender with its flagship offering, Passage that enables direct interfacing with the transistors on silicon chips like GPUs. The company claims that its technology enables data transfer between chips with an astounding 100-fold increase in bandwidth compared to traditional methods. To put this into perspective, consider the monumental scale of GPT-4 which is speculated to have run on over 20,000 GPUs. Lightmatter's Passage, due to be ready by 2026, promises to orchestrate 1+ million GPUs to collaborate seamlessly in parallel during AI training sessions, signaling a significant leap forward in AI infrastructure capabilities.

Ayar Labs is another start-up that stands out. Collaborations with major players such as NVIDIA, Lockheed Martin, and Lumentum underscore Ayar Labs' pivotal role in shaping future technological landscapes. The California-based start-up recently unveiled a second generation of its SuperNova light™ source, powering Ayar Labs' TeraPHY optical I/O chiplet, facilitating substantial advancements in bi-directional bandwidth capabilities.

Based in the UK, Salience Labs is taking a different approach. The company is developing ultra high-speed multi-chip processors that integrate photonics chips with standard electronics. These hybrid photonic chips merge conventional silicon chip technology with a photonic layer, ensuring compatibility with existing systems while also being well-positioned to meet future demands.

With the photonic integrated circuit market expected to reach $38.35 billion by 2029, growing at a CAGR of 20.47% between 2024 and 2029, per Mordor Intelligence, the opportunity for these start-ups is huge. As these startups continue to push the boundaries of technological advancement, they stand poised to play a pivotal role in shaping the future AI landscape.

Thermal Batteries Poised to Reshape Energy Storage

There are no silver bullets for climate change, but certain technologies edge close to being transformative. Thermal batteries, though relatively under the radar, hold immense promise.

Thermal batteries function by converting electricity into heat, storing it for extended periods, and releasing it when needed, reaching temperatures of up to 1,500-1,700 °C with an impressive 95% round-trip efficiency.  These batteries can meet over 90% of the industrial process heat demand met with combustible fuels. This capability makes them a compelling choice for large-scale energy storage, offering advantages such as lower costs, longer lifespan, and increased efficiency.

The shifting economics of clean energy, coupled with a favorable regulatory push, have propelled thermal battery technology into the spotlight. Investors are taking notice, with VC investments in thermal battery startups surging 5x between 2020 and 2023, with this year already matching last year's total funding, per Pitchbook. Median post-money valuation also surged 81.9% y-o-y in 2023, reaching $35.4 million. The overall energy storage space is also having a great time. VCs put $9.2 billion into 86 deals in energy storage in 2023, a historic high and up 59% from 2022. Notably, 38% of these deals focused on battery technology, per Mercom Capital Group.

Although the sector is nascent, it is starting to produce some notable companies. Thermal battery start-ups are currently in the early commercialization stages, which may require policy incentives or funding for some deployments. However, near-term cost estimates and long-term forecasts provided by multiple thermal battery companies suggest that the cost of a thermal battery (excluding installation) is projected to fall considerably due to due to economies of scale and technology improvements. The prices are likely to range from $85 to $210 per kWh by 2025, decrease to $40 to $110 per kWh by 2030, and further down to $20 to $55 per kWh by 2035, according to the Center for Climate and Energy Solutions (CCES) and Renewable Thermal Collaborative (RTC).

Thermal batteries can have varied applications. Industries remain the biggest target market though with US industries consuming 1.02 trillion kWh of electricity in 2022, per Energy Information Administration. Manufacturing accounts for 36% of all greenhouse emissions, according to Rhodium Group. Just last month, the US Department of Energy announced up to $6 billion to decarbonize energy-intensive industries such as chemicals, steel, cement, and food production. These industries require reliable, constant sources of cheap energy and can therefore be difficult to decarbonize. Thermal batteries could lower the cost of electricity for industrial heating by 50%-63%, fundamentally changing businesses’ decision-making around electrifying process heat equipment.

California-based Antora Energy, which recently raised its $170 million Series B round, is a frontrunner, using its heat-to-power thermophotovoltaic technology to fulfill large industrial operations demands. Rondo Energy, which raised a Series B round in 2023, is another emerging player in the industrial decarbonization space. With the global industrial thermal energy storage market expected to reach $4.5 billion by 2034, growing at a 75% CAGR between 2024 and 2034 (per IDTechEx), these start-ups can benefit immensely.

Renewable-thermal energy emerges as a cost-competitive alternative to natural gas for industrial heating, particularly in regions abundant in renewables, according to another analysis by CCES and RTC. Assuming a $6/MMBtu natural gas cost and a $50/tonne GHG value, renewable-thermal supply proves economically viable. The declining cost of renewable energy and the strategic application of thermal storage technology are two key factors behind the economic viability of renewable thermal storage.

Additionally, large commercial buildings can use thermal batteries to cut heating and cooling expenses. Thermal batteries can also stabilize temperatures for greenhouses or crop drying. Massachusetts-based Fourth Power Energy is developing localized thermal grids, optimizing energy distribution, and reducing costs through shared resources. The company claims its battery to be more than 10X cheaper than lithium-ion batteries.

Founded in 2018, Malta addresses a crucial challenge. The outdated US power grid struggles to accommodate surplus energy. According to BloombergNEF, to achieve a clean grid and progress towards net-zero emissions by 2050, we need around 7.7 terawatt-hours of electricity storage globally, averaging 257 gigawatt-hours installed per year. This staggering demand for energy storage underscores the urgency for innovative solutions like Malta's, which aim to address grid inefficiencies and facilitate the transition to a sustainable energy future.

There are still hurdles to clear including high initial investment and the search for materials that can withstand high temperatures and offer high specific heat capacity. However, as technology evolves and becomes integrated into the energy ecosystem, thermal battery start-ups are poised to grow immensely.