DeepSeek Develops Linux File-System For Better AI Training & Inference Performance
([Linux Storage] 3 Hours Ago
DeepSeek 3FS)
- Reference: 0001531065
- News link: https://www.phoronix.com/news/DeekSeek-3FS-File-System
- Source link:
Chinese AI company DeepSeek made public this week 3FS, a Linux FUSE-based file-system intended for allowing better AI training and inference performance.
The DeepSeek 3FS Fire-Flyer File-System is a distributed file-system designed to leverage modern solid-state drives and RDMA networks for a better AI training and inference experience. 3FS relies on FUSE for user-space based file-system support and aims to make it easier to deploy large-scale AI training/inference clusters with shared storage.
DeepSeek's Fire-Flyer File System has a disaggregated architecture, strong consistency, and other optimizations in the name of AI performance and usability.
Besides leveraging FUSE, this Linux file-system also relies on the Rust programming language, Apple's FoundationDB distributed database, and other modern dependencies.
Those curious about DeepSeek's 3FS open-source file-system can find the code and documentation via [1]GitHub . The 3FS code is available under an MIT license.
[1] https://github.com/deepseek-ai/3FS?tab=readme-ov-file
The DeepSeek 3FS Fire-Flyer File-System is a distributed file-system designed to leverage modern solid-state drives and RDMA networks for a better AI training and inference experience. 3FS relies on FUSE for user-space based file-system support and aims to make it easier to deploy large-scale AI training/inference clusters with shared storage.
DeepSeek's Fire-Flyer File System has a disaggregated architecture, strong consistency, and other optimizations in the name of AI performance and usability.
Besides leveraging FUSE, this Linux file-system also relies on the Rust programming language, Apple's FoundationDB distributed database, and other modern dependencies.
Those curious about DeepSeek's 3FS open-source file-system can find the code and documentation via [1]GitHub . The 3FS code is available under an MIT license.
[1] https://github.com/deepseek-ai/3FS?tab=readme-ov-file
onlyLinuxLuvUBack