Exactly. Training models, generating embeddings and building databases all can take days to run, and hundreds of Dollars in server costs. It’s painful to have done all that only to realize that one has gone down the wrong path. It pays to test your pipeline with a smaller batch first. Most likely you will discover some issues in your process. This iterative cycle is much faster if you work with a smaller test set first and do not switch over to your main corpus prematurely.