Gemma2 KoboldCPP ROCm - Does 7950X3d extra CPU cores help with CPU inference?

Conclusion-with 70B models and below, you will be limited more by your dual RTX 3090 GPU setup, Focus on VRAM. Its not until you hit Threadrippers or Xeons and server motherboards with eight-channel RAM that you can get serious with CPU inference speed. Trident Z5 Neo 6000 MhZ overclocked RAM.

17th september 2024
Mirrored on Odysee
Bluesky social(Twitter FOSS):https://bsky.app/profile/newjersey14.bsky.social
Revolt server(Discord equal):https://rvlt.gg/8h3PzaWt
Dlive:https://dlive.io?ref=asleepinglaffey

Crypto donation addresses if for some reason you want to back me up without ads (It's better with that 9 USD or even 60 USD compared to Youtube constantly pushing Youtube premium on you for 9 USD monthly while intentionally causing a war with Ublock Origin and Freetube,or trying to view my videos constantly to support me with trashy ad monetization rates, trust me!)

Bitcoin:bc1qzl8p4pn8ujvpyreedv0a69k0ks86du3lhkdrynnjp6ypsse9wjtspnm3du
ETH/Dai Stablecoin:0x922480ca4a9bcbcbd49668d3c871f57d9e747928
TRX/USDT(Probably does not work with Dogecoin or USDC):TLdGipgMC31cTcyQXfQbEHRbPRhmTssCYi
Monero/XMR(Liquidity issues as much as i like it):43rnZ8a5nFi8SGf6kePRSPKyvvN6cSxyJeKivmWca
Cun5wA1feZ4Gr88EdoHgcGfcNMj4q4hpEKBwYfjt