Most LLMs lose the ability to track facts over about 20k words of content, the best can manage maybe 40k words.

Look for "needle" benchmark tests, as in needle-in-haystack.

Not to mention the memory requirements of such a huge context like 128k or 1M tokens. Only people with enterprise servers at home could run that locally.