A university course organizer is asking for AMD-compatible inference software to host Qwen 27B or another Qwen model for students, with support for multiple users at once.
Key points
The planned free endpoint is for students to test pipelines
The system has two AMD 7900 XTX GPUs, a 12-core Zen 4 CPU and 64 GB of RAM
The organizer currently uses llama.cpp and is seeking alternatives
The Reddit poster says the endpoint is intended for a university course and should let multiple students use it at the same time. They are considering Qwen 27B or another Qwen model and ask which inference frameworks work well with AMD GPUs.