RAM <-> VRAM paging?

[Hexorg]

I have 12GB GPU and 128GB CPU. I can do ~64 Tok/s on GPU, but as soon as one layer is on CPU it drops down to ~12 Tok/s.

I couldn't find any discussion/approaches for dynamic paging of model layers - e.g. load first 12 layers, compute 12th layer output, load next 12 layers, compute 24th layer output - all on GPU. Is doing such paging really slower than letting the CPU crunch through numbers?