The author, Greg[0], wanted to use pgvector in a Postgres services, so he created a PR[1] in our Postgres repo. He then reached out and we decided it would be fun to collaborate on a project together, so he helped us build a "ChatGPT" interface for the supabase docs (which we will release tomorrow).

This article explains all the steps you'd take to implement the same functionality yourself.

I want to give a shout-out to pgvector too, it's a great extension [2]

[0] Greg: https://twitter.com/ggrdson

[1] pgvector PR: https://github.com/supabase/postgres/pull/472

[2] pgvector: https://github.com/pgvector/pgvector

    create table documents (
      id bigserial primary key,
      content text,
      embedding vector (1536)
    );
 

You can combine the search results into a prompt, and send that to GPT for a "ChatGPT-like" interface, where it will generate an answer from the documents provided

[0] https://platform.openai.com/docs/guides/embeddings

How does this scale with the number of rows in the database? My first thoughts are that this is O(n). Does the pgvector have a smarter implementation that allows performing k-nearest neighbor searches efficiently?

They mention this paper in the references: https://dl.acm.org/doi/pdf/10.1145/3318464.3386131

My experiments so far have involved storing the embeddings as binary strings representing the floating point arrays in a SQLite blob column: openai-to-sqlite is my tool for populating those: https://datasette.io/tools/openai-to-sqlite

I then query them using an in-memory FAISS vector search index using my datasette-faiss plugin: https://datasette.io/plugins/datasette-faiss

I love your writing, Simon. Have you blogged about this yet?

Postgres can do a lot of things, but for large enough datasets and/or when you want to add filtering into the mix along with vector search, then it becomes slow. And at that point you want to use a dedicated vector search database.

It's similar to how Postgres can also do full text search, but for large datasets and/or you want to add typo tolerance, faceting, grouping, filtering, synonyms, etc - the usual features you'd need when implementing a search feature - then it becomes slow to do this in pg and you'd then use a dedicated search engine.

In Typesense, we've now combined Vector Search along with filtering based on attributes in your documents, so you get the best of both worlds [2], and we made it fast with an in-memory index.

[1] https://typesense.org/

[2] https://typesense.org/docs/0.24.0/api/vector-search.html

[1] https://github.com/facebookresearch/faiss/wiki/Guidelines-to...

What does this mean?

  def average_embeddings(embeddings):
    return [
      sum([embedding[x] for embedding in embeddings]) / len(embeddings)
      for x in range(len(embeddings))
    ]

So if you're querying a document for predatory birds and you are using the string "Eagles" as your target you will be accidentally digging into embedding space for sports teams (go birds). But if your target is `average_embeddings([eagles_embedding, hawks_embedding, falcons_embedding])` you'll end up with less noise.

Curious about both performance/QPS and scale/# of vectors.

https://towardsdatascience.com/milvus-pinecone-vespa-weaviat...

But a "source document" could also be a natural language query that you need to convert to an embedding - if you want to enable natural language search on your corpus? (maybe along with handling queries in French getting good semantic hits in English etc?)?

Redis has approximate nearest-neighbors vector similarity search: https://redis-py.readthedocs.io/en/stable/examples/search_ve...

Generate the embeddings on a rented GPU, push to Redis then do a similarity search. Store vectors in Redis using ndarray.tobytes()

as an aside, Pinecone looks great

Used it to index 40M text snippets in the legal domain. Allows incremental adding.

I love how it just works. You know, doesn’t ANNOY me or makes a FAISS. ;-)

P.S. Many people seem to think that for vector search you need a GPU. You don't.

https://www.postgresql.org/docs/current/cube.html

The cube datatype also stores arrays of numbers (cubes or vectors, same thing) and has functions/operators for computing distance. Granted, they're slightly different, with no direct substitute for cosine distance, but on the unit sphere the cosine distance, inner product, and Euclidean distance have direct correspondences.

Is it correct that if OpenAI alters the embeddings model, such as they did with Ada on 22 December, you need to generate all your embeddings again?

Mind you that this was all running in WSL and I'm very new to vector search so I might not know what I'm doing.

I'm wondering if maybe if makes sense to use pgvector? or sqlite? instead

while pgvector seems to be doing cosine distance matching.

im not sure which is better, but tsvector will support the popular RUM plugin for cosine similarity.

any thoughts here ?

I love also the practicality with basic postgres, very very powerful stuff here.

I'm onboarding folks, and if you have Algolia, Gitbook, Readme or most common docs platform it takes minutes to set up.

It also integrates with your Slack and Discord (both in terms of letting people ask questions and get responses from the assistant in those tools, and marking conversations you want the assistant to know about for the future).

Would love more folks to try it out if anyone's interested, and think their product would benefit from it!