Intel LLM-Scaler vllm-0.14.0-b8.2 Released With Official Arc Pro B70 Support
([Intel] 3 Hours Ago
LLM-Scaler vllm-0.14.0-b8.2)
- Reference: 0001628950
- News link: https://www.phoronix.com/news/Intel-LLM-Scaler-vllm-0.14-b8.2
- Source link:
As part of Intel's [1]LLM-Scaler initiative for AI inferencing on Intel Arc hardware, out today is their vllm-0.14.0-b8.2 update that includes officially supporting the Arc Pro B70 graphics card.
Intel LLM-Scaler provides the Docker-ized approach for deploying large language models on Intel Arc hardware with a particular focus on latest-generation Battlemage graphics hardware, including multi-GPU configurations as part of the [2]Project Battlematrix initiative that began last year.
With today's update for their LLM-Scaler stack with vLLM, they have updated the platform image to intel/llm-scaler-platform:26.18.8.2. The only other listed change is now officially supporting the Intel Arc Pro B70 GPU, the BMG-G31 graphics card that recently debuted with 32GB of vRAM and a sub-$1000 price point.
The Intel [3]Arc Pro B70 continues running well in tests at Phoronix and I will have out some more benchmarks soon.
The new release is tagged on [4]GitHub and also available via Docker Hub. Though since the release announcement notification, they've dropped mentioning the highlights and the Arc Pro B70 support, presumably some release process snafu.
[1] https://www.phoronix.com/search/llm-scaler
[2] https://www.phoronix.com/search/Project+Battlematrix
[3] https://www.phoronix.com/search/Arc+Pro+B70
[4] https://github.com/intel/llm-scaler/releases/tag/vllm-0.14.0-b8.2
Intel LLM-Scaler provides the Docker-ized approach for deploying large language models on Intel Arc hardware with a particular focus on latest-generation Battlemage graphics hardware, including multi-GPU configurations as part of the [2]Project Battlematrix initiative that began last year.
With today's update for their LLM-Scaler stack with vLLM, they have updated the platform image to intel/llm-scaler-platform:26.18.8.2. The only other listed change is now officially supporting the Intel Arc Pro B70 GPU, the BMG-G31 graphics card that recently debuted with 32GB of vRAM and a sub-$1000 price point.
The Intel [3]Arc Pro B70 continues running well in tests at Phoronix and I will have out some more benchmarks soon.
The new release is tagged on [4]GitHub and also available via Docker Hub. Though since the release announcement notification, they've dropped mentioning the highlights and the Arc Pro B70 support, presumably some release process snafu.
[1] https://www.phoronix.com/search/llm-scaler
[2] https://www.phoronix.com/search/Project+Battlematrix
[3] https://www.phoronix.com/search/Arc+Pro+B70
[4] https://github.com/intel/llm-scaler/releases/tag/vllm-0.14.0-b8.2