The growth of AI is driving exponential growth in computing power and a doubling of networking speeds every few years. Less well-known is that it’s also putting new demands on storage.Â
Training new models typically requires high-bandwidth networked access to petabytes of data, while inference with the latest types of retrieval augmented generation (RAG) requires low-latency access to hundreds of terabytes of storage. New models also run training, indexing, and retrieval on rich image and video data. Many of the new AI inference indices and search tools rely on vector databases and must also preserve extensive metadata about all embedded content. Â
Traditional file storage often doesn’t work well for these new workloads, and traditional storage hardware is not the most efficient or performant way to support AI. Often, object storage is the best architecture for large amounts of data. New storage solutions are needed that offer better performance and efficiency, relative to both price and energy consumption. Â
Enter the DPU for storage
Supermicro has long offered many JBOF (just a bunch of flash) solutions deployed as direct-attached storage or networked file or object storage. Now they are launching a new JBOF that is powered by the NVIDIA BlueField data processing unit (DPU).Â
Instead of using a traditional storage server design with separate CPU, memory, and networking cards, the new JBOF uses the DPU to run the software, connect to the network, support remote management, and accelerate critical networking, storage, and security functions in customized silicon.Â
The DPU card replaces the CPU, network cards, DRAM modules, CPU PCIe switch, crypto accelerators, BMC, and remote management port. External PCIe switches may still be required to connect to SSDs.Â
BlueField is the NVIDIA market-leading DPU and it is optimized to offload and accelerate networking, storage, security, and management features. Because the DPU unifies networking, CPU, memory controllers, PCIe switch, and traffic accelerators on one chip, data can move directly between SSDs and the high-speed network ports without needing to be processed by an external CPU or cross a separate PCIe bus multiple times. This enables low-latency storage access and makes the system more efficient in terms of price, performance, and power efficiency. Â
The BlueField DPU storage controller card supports up to 400 Gb/s of network traffic and can also accelerate the NVMe over Fabrics (NVMe-oF) storage protocol and other RDMA-based storage traffic. It also acts as the PCIe root complex to manage the SSDs and uses its Arm cores to run the storage software allowing the JBOF to present as block, file, or object storage and deploy as part of a scale-out storage solution.Â
BlueField also provides security offloads and remote management functionality—including its own baseboard management controller (BMC) and separate management ports—typically required for large-scale data center deployments and cloud operations.Â
New Supermicro JBOF
Supermicro’s new 2RU JBOF showcases a flexible design, supporting either 36 E3.S SSDs or 24 U.2 SSDs, with a raw capacity of up to 1.44 PB, later supporting up to 2 PB with newer 60-TB SSDs. It holds two controller canisters, each capable of supporting up to two BlueField-3 DPUs and one NVIDIA GPU.
The JBOF can be deployed with two canisters for active-active or active-passive high availability within each JBOF chassis or with just one canister for maximum efficiency in cloud storage situations where redundancy and failover are handled by software across many JBOFs.Â
Figure 2 shows that each controller canister houses up to two NVIDIA BlueField-3 DPUs and one NVIDIA GPU card. Customers can deploy two canisters per JBOF for high availability or one canister for higher cost efficiency and reduced power consumption.
The dense design is ideal for both scale-up and scale-out file and object storage, while the high networking throughput—up to 800Gb/s per JBOF—supports the needs of AI training and HPC workloads.
Testing of the Supermicro JBOF showed that under a storage workload with one BlueField DPU installed, it saturates the 400-Gb/s network connections. The BlueField-based JBOF also shows 13% lower latency for a small-block (4 KB) random read workload, clocking in at 86 
The use of a single DPU card to replace the CPU, memory, networking card, and BMC results in a significant power savings of up to 50% for the non-SSD subsystem or 10-15% power savings for the entire JBOF (including the SSDs). In a large, scale-out storage deployment, this can mean kilowatts of power savings across the data center.
Solution partners
This JBOF is ideal for cloud deployments and object storage. As such, Supermicro is teaming up with multiple partners including Cloudian, Hammerspace, Kioxia, and Micron.  Â
Cloudian delivers scale-out, high-performance object storage optimized for fast access to large content, including video, audio, photos, and vector databases. The Cloudian software runs on both the storage clients and on the BlueField DPU in the Supermicro JBOF, taking advantage of the faster performance and higher efficiency in the JBOF.Â
Hammerspace supports a scale-out parallel file system for optimum shared access to billions of files and petabytes of data. Many AI and HPC workloads require file storage, and the Hammerspace solution provides a global namespace, parallel access for higher performance, and data orchestration services to automate the flow of data. The server side of the Hammerspace software runs on the BlueField DPU within the Supermicro JBOF. Â
Kioxia and Micron deliver innovative SSDs supporting different capacities, performance levels, and cost points, available in both the U.2 and E3.S form factors. Supermicro includes the optimum SSD type for each customer’s storage workload. Â
Supermicro is working with additional software, SSD, and systems partners to integrate this new JBOF into additional scalable storage solutions. Â
See it at the OCP Global Summit
The new Supermicro BlueField-powered JBOF will be on display in the Supermicro booth (B21) at the 2024 OCP Global Summit, October 15-17 in San Jose, California. In addition, Supermicro and NVIDIA are presenting a session on DPU-powered AI storage on Wednesday, October 16 at 1:10 p.m. (Concourse Level, 210DH).
For the complete list of NVIDIA-related sessions, see NVIDIA at OCP Summit 2024.