TurnkeyML 6.0 Released With OpenAI-Compatible Server, Other Changes
- Reference: 0001531294
- News link: https://www.phoronix.com/news/TurnkeyML-6.0-Released
- Source link:
TurnkeyML aims to make it easy to leverage the tools within the ONNX ecosystem via "no-code CLIs and low-code APIs" with their Turnkey optimizer and Lemonade SDK for large language models (LLMs) with the OnnxRuntime GenAI (OGA).
With the release of TurnkeyML 6.0, they are introducing a new OpenAI-compatible server. TurnkeyML has replaced their previous "serve" tool with a OpenAI-compatible server and the developers are also pursuing Ollama compatibility too.
TurnkeyML 6.0 also adds support for Quark quantization in a new "quark" tool, benchmark tooling improvements, and other improvements. Downloads and more details on TurnkeyML 6.0 via [2]onnx/turnkeyml on GitHub .
[1] https://www.phoronix.com/news/ONNX-TurnkeyML
[2] https://github.com/onnx/turnkeyml/releases/tag/v6.0.0
timofonic