In the past we have seen [1]Llama.cpp with Vulkan outperforming AMD's ROCm compute stack in some of the large language model (LLM) AI benchmarks. Curious if anything has changed given the recent [2]ROCm 7.1 release , I ran some benchmarks of an up-to-date Llama.cpp using the AMD ROCm back-end compared to the Vulkan back-end with the latest RADV driver. For this round of testing the [3]Radeon AI PRO R9700 graphics card was used.

[4]

Curious how the ROCm 7.1 versus Vulkan performance is looking with Llama.cpp when using the latest ROCm and RADV drivers, I ran some fresh benchmarks the past few days for seeing if the ROCm performance is anymore decisive or how well RADV/Vulkan is looking for AI these days using the Mesa Git driver stack.

The AMD Radeon AI PRO R9700 running with the Threadripper PRO 9995WX was used for these straight-forward benchmarks of comparing the two back-ends on the same hardware.



[1] https://www.phoronix.com/review/llama-cpp-windows-linux

[2] https://www.phoronix.com/news/AMD-ROCm-7.1-Released

[3] https://www.phoronix.com/search/Radeon+AI+PRO+R9700

[4] https://www.phoronix.com/image-viewer.php?id=rocm-71-llama-cpp-vulkan&image=rocm_71_vulkan_llama_cpp_lrg