Venture Bytes #107: AI to AGI- the Path Goes Through Vector Databases and RAG

AI to AGI- the Path Goes Through Vector Databases and RAG

Generative AI is phenomenal, but it is not yet without a critical problem. It hallucinates and often gives inaccurate answers. Large language models (LLMs) have no inherent intelligence and make the best inference from words, images, speech, and music from the pre-trained data. Teaching AI to reason is the next frontier and a pivotal cog in the transition towards artificial general intelligence (AGI). Vector databases and refined AI architectures such as RAG (Retrieval Augmented Generative) models will be the key enablers in our opinion.

The inherent challenge lies in the statelessness of LLMs during the inference phase. This means that every time a request is made to LLM, the model's response is determined solely by the specific input data and parameters provided. It can't remember previous questions or the context of the conversation, meaning that the model cannot retain information from previous interactions to inform its responses in subsequent interactions. Overcoming this challenge is crucial to unlock the full potential of AI.

Current databases including MySQL, MongoDB, and Cassandra have long been the cornerstone of business data management, storing structured data in tables or documents. While efficient for fixed-schema data, they struggle with unstructured or multi-dimensional data like images and text, making them unsuitable for AI applications.

In response to these challenges, vector databases offer optimized storage and querying capabilities for embeddings. Due to their suitability for AI applications, vector databases have been the fastest-growing databases in popularity over the last 12 months, per DB-Engines Ranking. By indexing vector embeddings, these databases enable rapid retrieval and similarity searches, addressing the diverse needs of machine learning, natural language processing, and recommendation systems. Moreover, they provide context and long-term memory to LLMs, thereby enhancing the accuracy of techniques like RAG.

One of the key advantages of vector databases lies in their ability to facilitate in-context learning. Instead of transmitting extensive document collections with each API call, developers can leverage vector databases to selectively retrieve the most relevant data for any given query, streamlining processes and enabling scalable enterprise applications. Furthermore, vector databases seamlessly integrate with LLM operations, effectively handling AI workloads and improving overall efficiency.

In tandem with vector databases, the emergence of RAG architecture represents a paradigm shift in AI model design. RAG, an innovative framework, augments LLMs by supplementing them with real-time information from extern knowledge bases, increasing the accuracy and relevance of their responses. This dynamic augmentation empowers LLMs to overcome the constraints of fixed datasets, enabling them to generate up-to-date and contextually relevant outputs.

According to Menlo Ventures’ The State of Generative AI in the Enterprise report for 2023, RAG is becoming a standard framework among AI practitioners. With 31% of surveyed participants embracing this approach, it signifies a shift in how organizations leverage AI technologies to enhance productivity and drive innovation.

Research from Pinecone, a California-based vector database start-up, reveals that increasing the amount of data available for context retrieval significantly enhances the performance of LLMs. Even when scaling the data size to 1 billion, irrespective of the LLM used, the addition of sufficient data alongside LLMs like GPT-4 with RAG led to a notable 13% improvement in answer quality, effectively reducing unhelpful responses by 50%.

Various startups are emerging to capitalize on the burgeoning vector database market opportunity, which is projected to grow from $1.5 billion in 2023 to $4.3 billion by 2028, at a CAGR of 23.3%, per MarketsandMarkets. Leading the pack is Pinecone, dubbed as the "Snowflake of AI", boasting over 1,500 paying enterprise customers, including industry giants like Microsoft, Shopify, Notion, Klarna, Gong, HubSpot, and Accenture. Pinecone's recent launch of its cloud-native and serverless vector database marks a significant milestone, enabling companies to scale limitlessly and develop high-performance applications such as RAG with unprecedented speed. With this, many enterprises have cut their database expenditures by up to 50 times.

Zilliz, another California-based vector database startup, is trusted by over 5,000 enterprises across the globe including Walmart, Nvidia, Tencent, Intuit, and eBay. It recently launched Milvus 2.4, a vector database management system, offering enhanced vector search capabilities.

Weviate, a Netherlands-based vector database startup, has over 4 million downloads. The company has raised $67.7 million from investors including New Enterprise Associates, Battery Ventures, and Index Ventures. It has partnerships with AWS, Google Cloud, and Snowflake – a testament to its credibility and market influence.

On the open-source front, Qdrant has experienced exponential growth in user adoption, surpassing 5 million downloads in recent years. The Series A start-up, founded in 2021, provides vector similarity search services with an API to store, index, and manage massive embedding vectors.

Rising Regulatory Focus on Climate Disclosures Amplifies Startup Roles

‍

In March 2024, the US Securities and Exchange Commission (SEC) adopted rule changes requiring companies to disclose certain climate-related information, ranging from greenhouse gas emissions to expected climate risks to transition plans. The move is aimed at providing investors with consistent, comparable, and decision-useful information for making investment decisions, and consistent and clear reporting obligations for issuers. This rule change is propelling start-ups that offer carbon accounting and analytics services to address the complex and time-consuming process of measuring carbon emissions across a company’s operations, energy consumption, and suppliers.

Carbon disclosure regulations are surging around the world. In September 2020, New Zealand led the global stage by becoming the first country with an announcement to mandate climate disclosures. In 2021, the Canadian Securities Administrators proposed a climate-related disclosure requirement for financial institutions and ESG-related requirements for large and listed entities. In January 2023, the EU’s new Corporate Sustainability Reporting Directive entered into force. In June 2023, the Australian Treasury released a Climate-Related Financial Disclosure Consultation Paper, outlining the requirements that certain Australian companies may have to follow in the future related to climate disclosures – as soon as 2024. In January 2025, new climate-related disclosure requirements for issuers under the Hong Kong Stock Exchange will come into effect.

‍

With surging carbon disclosure regulations, the accurate measurement of a company’s carbon footprint has never been more critical. In our view, carbon and climate disclosures are expected to emerge as one of the most significant compliance markets, given their vital role in helping businesses make informed decisions for a sustainable future. However, carbon and climate disclosures are much more complex than other significant compliances, hence requiring specialized solutions.

In 2023, 23,202 companies – representing a staggering $67 trillion in market capitalization - disclosed their environmental performance data to the Climate Disclosure Project, a non-profit organization that helps companies, cities, states, regions, and public authorities disclose their environmental impact. This marked a remarkable 140% increase from 2020, indicating a shift towards environmental disclosure among businesses.

Investors have become a significant voice in the call for climate change risk disclosure. According to an analysis of comment letters of 320 institutional investors by Ceres, a Boston-based non-profit sustainability advocacy organization, 97% support the requirement of disclosures in form 10-K, while 100% supported aligning the required disclosures with the recommendations of the Task Force on Climate-related Financial Disclosures.

Further, the surge in demand for ESG reporting tech startups is additionally catalyzed by the rapid evolution of AI technology. This evolution allows companies to harness AI for more accurate reporting of value chain emissions, addressing data gaps and refining estimations in this challenging category. The carbon accounting market size will be the biggest beneficiary of the strong momentum in climate disclosures and is likely to reach $61.4 billion in 2029 from $16.98 billion in 2024, growing at a CAGR of 29.3%, per Mordor Intelligence.

This wave is propelling start-ups that offer carbon accounting services to address the complex and time-consuming process of measuring carbon emissions across a company’s operations, energy consumption, and suppliers. Leading the pack is Arizona-based SaaS startup Persefoni. As first-to-market with category-creating end-to-end digital calculation capabilities for financed emissions, Persefoni has quickly cemented a leadership position helping asset managers, banks, and other financial institutions calculate their financed emissions footprint. The company, which raised its $50 million Series C round in August 2023, offers a plug-and-play software that can be implemented quickly and bypasses the status quo of hiring specialist consultant firms to calculate a footprint.

‍

Similarly, California-based enterprise climate platform Watershed is poised to benefit from the increasing adoption of emissions reporting. The startup recently raised a $100 million series C round at a valuation of $1.8 billion, an 80% markup from the previous round. Watershed has formed partnerships with major players like Google Cloud, KPMG, ISS, and ERM to enhance its platform capabilities and go-to-market reach.

Valued at $311 million, Patch offers solutions that include removal, estimation, and education about carbon emissions. Backed by investors such as Energize Capital, Andreessen Horowitz, and Coatue, Patch has rapidly grown its customer base to over 100 companies and expanded its team by 400% in the past year, highlighting accelerating demand for its carbon credit platform.