News: 0001592861

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

AMD ROCm 7.1 vs. RADV Vulkan For Llama.cpp With The Radeon AI PRO R9700

([Display Drivers] 76 Minutes Ago 6 Comments)


In the past we have seen [1]Llama.cpp with Vulkan outperforming AMD's ROCm compute stack in some of the large language model (LLM) AI benchmarks. Curious if anything has changed given the recent [2]ROCm 7.1 release , I ran some benchmarks of an up-to-date Llama.cpp using the AMD ROCm back-end compared to the Vulkan back-end with the latest RADV driver. For this round of testing the [3]Radeon AI PRO R9700 graphics card was used.

[4]

Curious how the ROCm 7.1 versus Vulkan performance is looking with Llama.cpp when using the latest ROCm and RADV drivers, I ran some fresh benchmarks the past few days for seeing if the ROCm performance is anymore decisive or how well RADV/Vulkan is looking for AI these days using the Mesa Git driver stack.

The AMD Radeon AI PRO R9700 running with the Threadripper PRO 9995WX was used for these straight-forward benchmarks of comparing the two back-ends on the same hardware.



[1] https://www.phoronix.com/review/llama-cpp-windows-linux

[2] https://www.phoronix.com/news/AMD-ROCm-7.1-Released

[3] https://www.phoronix.com/search/Radeon+AI+PRO+R9700

[4] https://www.phoronix.com/image-viewer.php?id=rocm-71-llama-cpp-vulkan&image=rocm_71_vulkan_llama_cpp_lrg



Please go away.