get_flat_ccs_offset() reads the base of the flat CCS storage from the
hardware, scales it by the number of enabled L3 nodes, and rounds the
result up to 128K. Everything below that offset is then ...
It will still run necessary calculations much slower than a gpu, so you will need to run it much longer for the same result, which is not only inconvenient but will likely use more power in the end anyway.
But isn’t it as bad as running a graphically intensive AAA game on your computer
That’s still just moving the impact from data centers to end users’ machines.
Gamers (especially those that own 5090) are a tiny minority compared to llm users.
CPU?
This will just waste even more energy since cpus are less efficient at this kind of task.
LLMs require insane amounts of compute, period. The hardware to do that needs energy and emits heat, there is no way around.
What if it the CPU was energy efficient?
It will still run necessary calculations much slower than a gpu, so you will need to run it much longer for the same result, which is not only inconvenient but will likely use more power in the end anyway.