I Want Flexible Queries, Not RAG

By on •

There is a lot of excitement around retrieval augmented generation or “RAG.” Roughly the idea is: some of the deficiencies in current generative AI or large language models (LLMs) can be papered over by augmenting their hallucinations with links, references, and extracts from definitive source documents. I.e.: knocking the LLM back into the lane.

LLMs do have a lot of value. However, in my opinion, much of it is in the flexible natural language query interface and not in the celebrated generative stage. I’ve come to the feeling that I want the non-brittle flexible natural language query interface, but do not want the spammy LLM output.

Here is an example.

My mother remembers growing up with a Sicilian dish that was primarily rice baked in an egg mixture. Roughly a “rice frittata.” With ChatGPT (purely generative, not RAG- thanks for the correction!) I get the following specious answer.

Yes, Sicily is “not a million miles away” from Naples.

My wife spent some time with search engines looking for the recipe. Most of her time was dealing with the brittleness of the query interface (depending on word matching and source popularity), spam, and locked down sources. However, in the end she delivered the following great result.

She found this.

The following bookmarks are the recipes she found personally interesting (not all matching the search).

And here are the two candidate recipes (one matching the search, one she just wanted):

I then prepared one of the recipes for my mother, and she said it matched her memory! I fail to see how an LLM summarizing the material would be an improvement. I wanted the retrieval of a good recipe, not an amalgam of “things that plausibly look like recipes.”

To me, LLM query management and retrieval is much more valuable than response generation. It reminds me of the aphorism (from an early low point in combinatorial chemistry): “you don’t find the needle in the haystack by harvesting more hay.”