Venture Bytes #105: GPU-as-a-Service Primed for Massive Growth

GPU-as-a-Service Primed for Massive Growth

‍

In the next decade, the integration of LLMs and generative AI will enhance every human endeavor.  Achieving  this  necessitates making GPU computing as ubiquitous as electricity, calling for billions of dollars of investment in computing power. Data from IDC underscores this trend, with its forecast of AI spending reaching $40.1 billion in 2024 and $151.1 billion in 2027.

The GPU-as-a-Service market is expected to grow to $25.5 billion by 2030 from $2.4 billion in 2022, according to Fortune Business  Insights,  underscoring  the growing demand for cloud-based GPU services. To put this into perspective, the training  of  GPT-4  reportedly  involved approximately 25,000 Nvidia A100 GPUs over a span of 90–100 days. Each A100 GPU costs around $10,000-$25,000, according to various estimates. Recognizing that not all startups possess the financial capacity to purchase such an extensive array of chips, the reliance on cloud GPU providers becomes  imperative  for  cost-effective access to advanced computing resources.

By offering a combination of affordability, flexibility, and cutting-edge technology, start-ups are poised to play a crucial role in  the  AI  revolution.  California-based Together.ai, which raised a $102 million

Series A round three months ago, claims to offer 4x lower cost than AWS with a renewal rate of more than 95% on its clusters.

The value proposition offered by startups in this domain is stark. Startup cloud providers specializing in AI are gaining a foothold against larger hyperscalers by emphasizing affordability  and  specialized  resources. They attract customers by offering 50% to 70% savings on GPU hours, particularly with advanced Nvidia A100 and the newest H100 GPUs. For instance, Lambda Labs’ A100 80GB GPUs cost $1.50 per hour, which  is  a  significant  discount  from offerings from the hyperscalers: Microsoft Azure is at $3.67 per hour, Google Cloud at $5.07, and Amazon AWS at $5.12 per hour.

Lambda Labs also has the largest cluster of H100 GPUs among private start-ups, even surpassing Google and Oracle, per data from the State of AI Report Compute Index. Lambda, which raised a $320 million Series C round in February, reached $250 million in revenue in 2023 and expects to double that in 2024. The company was one of the first cloud providers to make Nvidia H100 Tensor Core GPUs available on-demand in the public cloud and is also one of Nvidia’s partners offering its    200 GPU with enhanced memory.

‍

Moreover, these startups shine by offering a wider range of GPUs. For instance, CoreWeave’s offering includes over a dozen SKUs of Nvidia GPUs including the H100, A100, V100, A40, A6000, A5000, A4000, and various models of the Quadro RTX, outstripping the offerings of tech giants AWS, Azure, Oracle, and Google. The Nvidia-backed start-up also signed a multi-year deal with Microsoft in 2023. Similarly, FluidStack provides not only the A100 and A40 but also diversifies with the A6000, A5000, and a range of Quadro RTX and GeForce options. This breadth of GPU availability underscores the startups' commitment to catering to a wide array of computing needs.

Additionally, their adaptability extends to providing robust multi- cloud support and effective edge computing capabilities. This strategic positioning allows startups to meet specific customer needs more flexibly and cost-effectively, distinguishing them in the competitive AI-focused cloud computing landscape. These companies have adeptly harnessed partnerships with behemoths like AWS, Azure, GCP, and Nvidia, paving the way for a versatile and scalable computing environment.**

Industrial Metaverse Finds a Strong Footing

‍

As the industrial economy goes digital, the industrial metaverse is finding a strong footing. While applications of the consumer metaverse in real-life scenarios are still in their infancy, the industrial metaverse has surged ahead on the adoption curve, aligned with tangible business challenges, and driven by practical implementation strategies. Notably, Florida-based Magic Leap recently redefined its roadmap for 2024, targeting its XR devices toward enterprise and industrial clients.

Amazon Robotics and Mercedes-Benz are prime examples, with both using digital twins - a core building block of the industrial metaverse - and AI to optimize operations and empower employees with data-driven insights. Confectionery and pet food manufacturer Mars is combining AI with digital twins to simulate operations in over 100 of their manufacturing facilities globally.

BMW is expanding its use of the NVIDIA Omniverse platform for building and operating industrial metaverse applications across its production network around the world. Data from ABI Research indicates that revenue generated by the industrial metaverse is poised to reach nearly $100 billion by 2030, surpassing both consumer ($50 billion) and enterprise ($30 billion) segments.

As spatial technologies gain traction in industrial settings, businesses are increasingly prioritizing digital twins, spatial simulation, augmented work instructions, and collaborative digital environments to enhance safety and efficiency in factories and operations. An analysis from Accenture indicates a substantial rise in industrial metaverse deployments since 2018, at a CAGR of 47%.

The industrial metaverse will have applications across multiple industries including aerospace and defense, automotive and transport, banking, energy, health, industrial, insurance, retail, travel, and software and platforms. Of these, energy, healthcare, automotive, and industrial present the largest opportunity for metaverse adoption. The metaverse can help energy companies simulate and plan energy infrastructure, from power plants to distribution networks, reducing energy waste and providing resilient systems. Digital twins assist with remote monitoring and predictive maintenance to improve the efficiency of operations and reduce the need for physical on-site visits.

Novo Nordisk partnered with Microsoft to implement HoloLens 2 in their manufacturing facilities, enhancing employee training and onboarding with augmented 3D instructions. This initiative delivered remarkable results, including a 30% reduction in managerial oversight, a 75% decrease in rework leading to $13,680 in annual savings per worker, and an impressive 80% reduction in consumable costs associated with training. Pharmaceutical giants like AstraZeneca, Pfizer, Roche, Novartis, and Sanofi are already leveraging digital twins to refine drug development processes. Unlearn,  a California-based  startup,  builds  machine-learning platforms to create biological digital twins for the healthcare and pharmaceutical industries.

‍

Further, the industrial Metaverse can digitalize every phase of the automotive product life cycle. Advanced simulations help to optimize vehicle performance, as well as develop and test vehicle design by mimicking real-life driving conditions, leading to the development, and testing of autonomous vehicles. Additionally, twins can help simulate the benefits of autonomous driving, including on the environment. For example, MIT model led the impact of the introduction of 4,000 autonomous taxis in NYC, using current NYC taxi data. That algorithm could reduce car trips by 98% within 3 minutes, saving gas and pollution.

‍

[The metaverse is] immersive, making users feel as if they’re in a real environment; collaborative in real time; open enough for different applications to seamlessly interact; and trusted by the individuals and businesses that participate. - Annika Hauptvogel, Head of Technology and Innovation Management, Siemens ‘‘

‍

In consumer product manufacturing, the integration of digital twins, coupled with AI and automation, presents a compelling prospect of elevated quality and accelerated development timelines. Serving as a visual conduit, digital twins empower consumer product companies to seamlessly transfer information and knowledge across  digital  platforms.  Sight Machine,  a California-based startup offering digital twin solutions, helped Asian Paints, an

‍

Indian decorative coating company to optimize around 12 minutes of production cycle time in their 3-hour batch. Moreover, Augury, a New York-based digital twin solutions provider helped Frito-Lay, a subsidiary of PepsiCo, in augmenting manufacturing capacity by 4,000 hours annually, translating to a substantial increase in production.

Looking ahead, advancements in high-fidelity 3D assets and immersive hardware for extended reality technologies are poised to further streamline operations. This progress is expected to pave the way for an operationalized spatial web, facilitating accelerate workflows and simplifying processes across various industries.**