I found this tutorial helpful for getting started with fine-tuning https://www.youtube.com/watch?v=74NSDMvYZ9Y

This guy used gradient.ai and he has a Google Collab to try it

While you're experimenting, worth checking out https://gradient.ai/ -- they're basically the OpenAI API but with llama2

In this post, we explore problems involved in LLM deployment, from GPU shortages to bottlenecks in model performance. These problems have inspired recent developments in distributed training frameworks commonly used to train LLMs, notably ZeRO-Offload. Here we give an overview of ZeRO-Offload, and in future posts we describe its benefits in depth.

RAG is great for pulling some additional knowledge, but if you combine it with fine-tuning (i.e., the LLM 'understands' the domain-specific terminology better) it becomes a lot more effective

Looking exactly into this, any research on this topic?

Research: https://arxiv.org/abs/2005.11401

An implementation on Hugging Face: https://huggingface.co/docs/transformers/model_doc/rag

Txs but I meant fine tuning and RAG combined

https://gradient.ai/ API for inference and fine-tuning open-source LLMs

not quite self-hosted but gradient.ai gives you access to llama2 via CLI

https://gradient.ai/ is doing that with llama2

Looks really promising. I wonder if the similar pricing to OpenAI means that Gradient is also(?) bleeding money even if they get a good customer base. Or are these prices sustainable over time?

Good question, esp as Gradient fine-tuning is so much cheaper than Open AI's

Yeah it's even cheaper. Although it looks like it's about the same in proportion to approx model size/expected quality? They haven't launched any >13B model yet, although they plan to.