Is 48GB enough to run 70B models ? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		moonchrome on Aug 9, 2023 \| parent \| context \| favorite \| on: Testing Intel’s Arc A770 GPU for Deep Learning Is 48GB enough to run 70B models ?

sacred_numbers on Aug 9, 2023 [–]

Yes, with 4 bit quantization.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact