Agentic search doesn't find pages, it assembles answers

Jeff Wang explains that EXA is a search engine built from scratch for agents, but its basic architecture still resembles conventional web search: crawl trillions of pages, scrape and store the data, then identify relevant information for a query. He describes search as a database with two especially important components: a vector database for semantic retrieval and an inverted index mapping words to documents. Unlike a typical database, search also handles continuous large-scale writes as the web changes, alongside an unusually high volume of reads.

Wang says agentic search differs from human search first through scale. Humans—roughly eight billion people—make a few trillion searches per year, while the coming ecosystem may contain trillions of agents, each searching more frequently than a person. Agents also search differently: instead of finding one listicle about technology podcasts, an agent might request 1,000 podcasts and then run additional searches about every one. Search infrastructure therefore must support large parallel operations that assemble answers from many pages rather than return a single page for a human to read.

Ryan asks whether individual agents can receive the benefit of Google’s pre-cached internet without storing petabytes themselves. Wang says most agent search systems already depend on some cached representation of the web, because creating an index requires knowing the pages and their contents. But each AI company could theoretically build its own crawler and index, at enormous cost. He calls the operation “not for the faint of heart”: it requires continuous crawling, processing, updating, and serving, unlike a language model’s fixed weights after training.

Wang says EXA built its own vector database partly because, when the company started in 2021, today’s specialized providers were not available, and partly because retrieval is central enough that outsourcing it would limit experimentation. A performant system must return results at low latency and high volume. EXA also trains its own embeddings because search embeddings must match short, information-seeking queries to the contents of URLs, a more specific task than comparing generic sentences, paragraphs, or words. Wang calls embeddings the “heart and soul” of the search algorithm.

EXA’s search does not simply return every relevant page. After an initial search, the system examines a large result set and uses internally trained transformer-based extraction models to select relevant tokens across documents, deduplicating overlapping information. Wang says ten pages may contain only the equivalent of one page of distinct text. Ryan connects this to duplicated AI-generated material and “AI slop”; Wang says EXA tries to produce the best possible index and “de-slop” results so agents receive high-trust information rather than repeated versions of the same claims.

Wang describes 2023-era retrieval-augmented generation as a workaround for roughly 4,000-token context windows: engineers chunked documents, selected small passages, and inserted them into prompts, often producing partial answers. With million-token windows now common, systems can place much larger documents into context, although token efficiency still saves money and avoids polluted context. He believes needle-in-a-haystack problems can still exist, but says current models are good enough that he does not personally observe them often, even though model performance still varies in evaluations.

Wang says search is the most common tool call made by language models because, like people without a computer or phone, models need outside information to do much. He divides agent development into three eras: single-step agents, multi-step systems such as coding agents that read and edit files, and general agents that can execute broad tasks. He expects agent swarms next, citing reports of 10,000 agents tackling a math problem. EXA has already powered products including Cursor and Cognition, as well as unusual agent social networks and personal agents searching for restaurants.