I looked at Annoy at the time but from memory it required rebuilding the index every time a new vector is inserted, which was an issue for the project I was working on.
I assume that Vlad takes care of the reindexing in the background, which would save a lot of the work.
Main author and architect of Weaviate (https://github.com/semi-technologies/weaviate) here. This real-time requirement was one of the major design principles from the get-go in Weaviate.
In Weaviate, any imported vector is immediately searchable, you can update and delete your objects or the vectors attached to the objects and all results are immediately reflected. In addition every write is written to a Write-Ahead-Log, so that writes are persisted, even if the app crashes.
We wanted to make sure that Weaviate really combines the advantages of AI-based search with the comfort and guarantees you would expect from an "old school" database or search engine.
I assume that Vlad takes care of the reindexing in the background, which would save a lot of the work.