Literature Retrieval in Parallax: Set Algebra, Full-Corpus Recall, and Jev Screening

Motivation

AI papers are being published faster than ever, and literature search has become a real challenge. When I used general-purpose or coding agents for literature search (finding related work, or mapping out a field), I kept running into two limitations:

1. General agents search the open web through black-box search APIs. They usually only get the top-k results, so they can't be exhaustive, and they can't answer questions like "how many papers are there in this area?"
2. Results from search APIs are noisy. They cover the whole internet, so extra effort goes into cleaning and organizing the data.

Because of this, agents tend to stop early when searching for papers, or they end up slow and expensive. Earlier this year I vibe-coded a pipeline for myself: arXiv metadata crawling -> embedding -> keyword or semantic recall -> LLM screening. It was still fairly expensive, and screening was slow, so it never felt good to use.

In mid-September Jev came out, and I realized it was a great fit for screening and classifying papers after recall. So I started rebuilding my literature infrastructure on top of Jev. This GIF gives a sense of how fast screening is:

The rest of this post covers how the retrieval system is designed, and some of the optimizations needed to run it in production.

Retrieval structure

I worked on recommender systems for a while but hadn't done much with agentic search, and I carried some of those habits over. My first idea was a funnel: recall, coarse ranking, fine screening. It turned out that paper search is quite different from feed recommendation:

• It needs expansion: papers cite each other, and following the citations of an early paper is a very common way to find more work.
• It needs supplementing: whether a human or an agent does the search, things get missed. After the first pass you need a review step to check whether anything unsuitable got in, or whether something you noticed earlier was left out.

A funnel clearly isn't enough for this. Add the need to merge and deduplicate multiple recall channels, and to remove false positives in bulk, and sets turn out to be a good data structure for the problem. So I rewrote the whole retrieval layer around a set algebra interface.

The basic unit the agent works with is a set of papers (with the usual properties: well-defined, distinct elements, unordered). It supports union, intersection and difference, plus a few practical objects and operators:

• Virtual roots: a virtual object that represents a starting point, such as the whole corpus or the papers in one day's arXiv announcement. Virtual roots avoid persisting huge numbers of members and taking up too much storage.
• Rule filters: filter papers by arXiv category, time window, resource type, embedding similarity, keyword match, and so on.
• AI screening: use Jev to judge each paper against criteria that the agent writes based on the user's intent.
• AI classification: use Jev's multi-label probability interface to classify papers. A paper can belong to more than one category, so categories are not mutually exclusive.

The whole retrieval system is visualized as a node graph, built with ReactFlow on the frontend. A node graph is flexible enough to handle all kinds of real-world needs.

A typical workflow looks like the one above: several keyword and semantic recall channels, a union node to merge and deduplicate them into a large candidate set, an AI screening node that calls Jev, and then an AI classification node that sorts papers by topic.

Most of the time the agent follows this typical workflow. For larger and more complex requests, though, it may find gaps during review, build new sets to fill them, and iterate until it has a "final" set before moving on to classification. The resulting workflow can get as complex as a circuit diagram.

General and coding agents usually keep track of results in text files, or in XML/JSON processed by scripts. Keeping sets in a database with indexes scales much further and is far more efficient. Handing this kind of large-scale processing to deterministic code also guards against model hallucination and against details being lost when the context gets compressed. The agent only holds handles to sets; all member-level operations happen inside the database.

Another practical issue is recall depth. A keyword match is either a hit or not, but semantic recall is a ranking: rank 300 may still be relevant while rank 3000 may already be off-topic, and it's hard to pick K in advance. So before creating a set, the agent first runs a cheap probe that returns only the total count, a few samples and the distribution. For semantic recall the probe also includes a rank-vs-similarity curve, and the agent can decide where to cut based on the elbow in that curve.

In early versions the probe and the set creation were two separate computations. For the same top-10000 semantic query, the probe took 2 minutes and creating the set recomputed everything for another 37 minutes. Now the probe's full ranking is stored, and creating the set just takes the top N from it, with no recomputation.

Recall performance

The idea above is quite natural. Low-code node-graph tools like ComfyUI are already widely known, and many people have probably thought of something similar for literature search. The hard part of putting such a system into production is not the data abstraction. It's performance at scale, and balancing speed, cost and quality.

Parallax currently harvests metadata updates for arXiv cs.* categories since 2020 through the OAI interface, about 820,000 papers. The metadata and its indexes take about 7.8 GB. For semantic recall we use Perplexity's pplx-embed-v1-0.6b to embed each abstract into a 1024-dimensional vector. At float32 that's 4 KB per vector, about 3.3 GB in theory for 820,000 papers; as halfvec (half precision) it's 2 KB each, about 1.65 GB total. That's just the raw data, though. In PostgreSQL + pgvector we store vectors in more than one form so we can be both exact and fast, and with indexes the total comes to about 17 GB.

Keyword recall

The usual way to do fuzzy matching in PostgreSQL is a trigram index: split text into three-character pieces, build an inverted index, use the pieces to find candidates, then go back to the table to check the original text. It performs badly on common phrases, though. On a corpus of about 690,000 papers, searching abstracts for motion generation did use the trigram index, but took 34 seconds, 32 of which were spent waiting on IO. Pieces like mot, ion and gen are just too common: the index produced 25,000 candidates, only 776 actually matched, and the tens of thousands of table lookups were wasted.

Our fix was to add an index of adjacent-word fragments. For every pair of adjacent words in a title or abstract, we take the last 3 characters of the first word and the first 3 characters of the second (plus 5-character versions) and join them into a fragment. motion generation, for example, produces ion:gen and otion:gener. These go into a GIN array index. Two specific words appearing next to each other is much rarer than a single three-character piece, so the candidate set shrinks a lot. When words are long enough we prefer the 5-character version, because combinations like ion:gen are still fairly common.

The fragments are only used as a necessary condition. The final check is still the original per-field ILIKE against the source text, so the result set is identical to the old implementation and the returned count is exact. After the change, the same single-phrase query takes about 30 ms, and a seven-phrase query the agent commonly runs went from 23 seconds to 0.09 seconds, with the old and new paper IDs matching one by one on the same snapshot.

Semantic recall

Vector search always involves a trade-off between speed and accuracy. Exact search scans every candidate vector. Approximate nearest neighbor (ANN) indexes like HNSW are much faster but miss results, and miss more as K grows. We ended up keeping two paths and choosing between them automatically.

The approximate path uses partitioning + half precision + reranking:

• A 1024-dim float32 HNSW index is 4.4 GB. When it isn't cached, one query reads hundreds of MB of index pages and takes over twenty seconds even with the index. So the index uses halfvec, which halves its size, and is only used to find candidates.
• We fetch several times more candidates than needed, then rescore and sort them with the original float32 vectors to correct the error from half precision.
• Vectors are partitioned by year, each partition with its own HNSW graph. When the user specifies a time window, irrelevant years are skipped.

The key ANN parameters are the search width ef_search and the candidate pool multiplier. We compared settings on 32 real semantic queries from production, using exact top-K as ground truth:

The missed papers almost all sit right around rank K, with similarity only a few thousandths below the K-th result, so the effect on the actual results is smaller than the recall numbers suggest.

For wide time ranges, large K, or when ANN times out, we switch to the exact path. Running a single exact recall SQL over the whole corpus has unpredictable runtime, and if it's interrupted there's no checkpoint, so it has to start over. Instead we split the candidates into batches, compute the top-K of each batch, then merge and take the overall top-K.

This loses no precision. If a paper is in the global top K, fewer than K papers in the whole corpus score higher than it, so fewer than K papers in its own batch score higher either. It must be in its batch's top K and can't be dropped during the merge. As long as every batch uses the same scoring and the same ordering (ties broken by paper ID), the batched result is exactly the same as sorting everything at once.

Batching adds some time overhead, but reliability and resource control both improve. Each batch is saved when it finishes, so a restarted process continues from where it stopped; a batch that times out is split into two smaller batches; and large jobs yield resources between batches so they don't block other users' small queries. We ran an acceptance test on the production database: 810,000 candidates from the full corpus, K=5000, done in 407 batches in about 5 minutes. Compared against a one-shot float32 exact query on the same snapshot, all 5000 results matched in ID, order and score.

Screening and classification

Compared with the DeepSeek-Flash screening we started with, after switching to Jev in late September, the new pipeline screened 36,564 papers on the production server in about 5 minutes. That's 10+ times faster than the LLM version, at less than a fifth of the cost.

Why is the LLM more expensive and slower? Because our pipeline collects audit information for each decision (a probability or a reason). The LLM doesn't just output true or false, it outputs JSON with a decision and a reason. With CoT reasoning turned on, the reasoning length is unpredictable, so latency is unpredictable. With CoT turned off, batch processing often produces malformed JSON, and we either throw away and retry the whole batch, or collect the failed items into a new batch.

Jev's interface instead takes a context (state) and a set of questions, and returns calibrated probabilities directly. Both screening and classification are built on Jev's Noul primitive. For screening, it's one paper plus the include/exclude criteria written by the agent, and the paper is included if the probability is at least 0.5. For classification, each category gets its own Noul and is judged independently: at least 0.8 counts as a hit, and 0.2 to 0.8 is marked as uncertain for the user and agent to review. The output is structured by design, so malformed output isn't a problem, and every paper's probability on every criterion is stored in the database.

Input format for multi-label classification

In multi-label classification each paper has to answer a dozen or so category questions. The original format repeated the paper's title and abstract in every question, which wasted a lot of tokens. The improvement was to put the paper in the shared state and have all category questions refer to it. Going one step further: could we put five papers in the state and ask all categories for all five in one request? We compared these options on 120 real papers with twelve categories:

On speed and cost alone, five papers per request wins. But running the same grouping twice changed the result for only 1 paper, while shuffling the 120 papers into new groups changed the labels of about 20. For example, one paper whose abstract explicitly mentions graph neural networks got 0.95 for the "graph learning" category when judged alone, and 0.78 when grouped with four other papers, falling below the hit threshold.

In other words, the other papers in the group affect how the model judges this one, which isn't acceptable for classification. So by default each request contains a single paper, with all twelve categories judged in that request. That still uses about 66% fewer input tokens than the original format, without interference between papers.

Batching and concurrency

A single Jev request takes only a few hundred milliseconds, but getting through tens of thousands of papers reliably involves a lot of work outside the model:

• Adaptive packing: the batch size for screening isn't fixed. It's computed from Jev's context window and the token length of the metadata in the current batch. The first implementation tried papers one at a time and re-estimated the whole batch's tokens each time, so larger batches got slower. Switching to a binary search for the largest prefix that fits cut packing overhead significantly, while the requests sent stay exactly the same.
• Rate limiting: the account has quotas for requests per second and tokens per second. We use the GCRA algorithm to advance on theoretical send times, so lock waits and thread scheduling no longer eat into the quota. When rate limits are the bottleneck, actual throughput went from 73% of the quota to 98%.
• Fairness: all jobs share one account quota, and when queuing, users with fewer requests in flight go first, so small jobs aren't starved by large ones.

Profiling also showed that, for a while, throughput was limited by neither the network nor the model, but by our own worker process maxing out the CPU. Each HTTP request involved about 120 SQL statements, covering cost reservation, pre-send checks, settlement and result persistence, all there to make every call recoverable, never double-charged, and traceable when something goes wrong. We cut the quota-related SQL from about 36 statements per request to about 15, and grouped results that finish around the same time into a single commit, before throughput could go up further.

Measured by input tokens, screening one paper takes a little over three hundred tokens, so screening ten thousand papers costs roughly one yuan (about fifteen US cents). With screening this cheap, the agent's behavior changed as well. It used to shrink the candidate set as much as possible before screening; now it can recall broadly, pull in everything possibly relevant, and let Jev judge each paper.

Results

In a search on "memory mechanisms for LLM agents", the agent first ran cheap probes on the corpus based on the user's hints to understand the rough distribution of papers. It then created 6 keyword and 5 semantic recall channels, which merged into over thirty thousand candidates. The agent wrote the criteria for Jev, what kinds of papers to include and what to exclude, and submitted them for concurrent batch judging in the background.

In another task, "a review of August work on multimodal understanding and unified multimodal models", the agent got 466 candidates from three keyword and semantic recall channels, and screening kept 428 papers on topic. The agent then read the titles in the set, drafted a category list, and called the Jev classification node. The result was 10 categories, including benchmarks and datasets, embodied AI, safety and privacy, and unified understanding and generation.

The user pointed out that the benchmark category still had over a hundred papers and could be split further, so the agent explored those entries and divided the benchmark subset into 11 subcategories.

During classification, Jev gives each paper an inclusion probability for each category's criteria. These probabilities are all stored in the database for the user and agent to review.

Why Parallax?

Why build Parallax? Until recursive self-improvement really arrives and AI papers lose their value entirely, arXiv papers remain the first-hand source of information about the field for researchers, students and developers outside the frontier labs. But as more and more papers are published and they collectively contain more and more insight, it's getting harder for people to gain understanding of the field at a pace that matches the growth in papers.

I want to find all the relevant papers and see what has happened across a whole field, or a niche I care about, over space and time. The only way to do that is a fast recall scan over the full corpus; the top-K results from Google Search or arXiv Search can't get you there. The demand for "finding everything" is obvious from the countless awesome paper lists on GitHub.

A year and a half ago, when I was still working in the game industry, I attended a talk by our user research team about how they watched new genres take shape. Steam tags are created by players, and by continuously tracking reviews and tags you can see a phrase go from scattered mentions to something most players use, until it becomes a new "genre". "Soulslike" and "Metroidvania" started that way: originally players just meant "this game is as hard as Dark Souls and tells its story the same way" or "this plays like Metroid and Castlevania".

The explosion of arXiv papers has put AI research into a high-frequency, fast-moving period, and Jev showed me a much cheaper and more flexible way to watch how the field is changing.

Parallax is live. You're welcome to sign up and try it at https://parallax.ink.