A LocalLLaMA user listed a 32-core EPYC system with 512GB of RAM and 44GB of combined GPU memory, asking what models and speeds it could handle and which software to use.
The post asks which software and model-running approaches would suit the system.
The user says speed is less important than running larger models and asks whether the setup remains useful, mentioning Project Maya’s ability to stream from an NVMe drive.