CUDA+ROCm dual-backends LLM inference for mixed-vendor GPUs (up to 48% faster)
Reported by r/LocalLLaMA1 source ↓
A community developer says a Project MAYA fork lets an RTX 5090 and Strix Halo iGPU cooperate on GLM-5.3-Flash inference using CUDA and ROCm backends.
Key points
- The fork runs prefill and decode across an RTX 5090 and Strix Halo iGPU
- The CUDA GPU handles attention, dense/shared layers and expert cache; ROCm runs complementary experts
- Project MAYA’s creator plans to test the design before deciding whether to merge it
Sources
Details
The developer describes the work as a fork of Project MAYA that runs GLM-5.3-Flash Maya-M GGUF inference cooperatively on an RTX 5090 and AMD Strix Halo integrated GPU. The GPUs contribute during both prefill and decode in the same process.
In the described setup, the RTX 5090 uses CUDA for attention, dense and shared layers, and expert cache. The Strix Halo uses ROCm to execute complementary experts. The developer links to a GitHub pull request and says Project MAYA’s creator plans to test the design before deciding whether to merge it.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh