This, specially the nlp part, strikes me as a primitive version of the RAG-LLM pair

obligatory mention of cp-alogs: https://cp-algorithms.com/index.html

Love cp-algos. Great quality, also love the sample code.

One of my favourite write-ups: https://cp-algorithms.com/string/z-function.html

the first thing that comes to mind is CLIP: https://github.com/openai/CLIP maybe combining it with some sort of knowledge graph is what you're looking for