For a long time, the role of SSDs was relatively clear. Its core task was to replace mechanical hard drives, improving the overall responsiveness and data throughput of the system. Whether in consumer computers, enterprise servers, or data centers, the technological evolution of SSDs basically revolved around several fixed goals: higher sequential read/write speeds, larger capacity, lower cost, and better reliability. However, with the rapid development of artificial intelligence, especially large models and deep learning, the workloads faced by storage systems have changed significantly. In AI scenarios, data is no longer simply “read in, processed, and written out.” Instead, it exhibits several new characteristics:
First, data volume has exploded. Whether it’s the massive raw data used in the model training phase or the model parameters and vector databases that need repeated access during the inference phase, the data size far exceeds that of traditional applications.
Second, the access pattern has fundamentally changed. AI workloads often involve a large number of small, random, high-concurrency data block accesses, rather than the large-block sequential reads and writes that traditional storage systems are better at handling.
Third, storage has begun to directly impact computational efficiency. In AI servers, the computing power of GPUs or other accelerators is growing very rapidly. If the storage system cannot keep up with the data supply speed, “computing idling” occurs, actually reducing the overall system efficiency.
It is against this backdrop that the concept of AI SSD began to be frequently mentioned and gradually moved from concept to specific products and technical roadmaps.
What is AI SSD?
To many newcomers to this concept, “AI SSD” can easily be misunderstood as a product that integrates AI algorithms inside the solid-state drive, capable of “learning” or “optimizing” on its own. However, according to the mainstream industry definition, this understanding is not accurate. The core of AI SSD is not about “whether there is AI inside the SSD,” but about “whether the SSD is born for AI workloads.” More precisely, an AI SSD is a type of solid-state storage device deeply optimized for AI training, inference, and data service scenarios. This optimization includes both hardware-level architecture design and changes in firmware, protocols, and system collaboration methods. In terms of functional positioning, AI SSD is still a storage device; it will not replace the computational role of GPUs or CPUs. But its goal is very clear: to minimize the performance limitations imposed by storage on AI systems.
Practical Bottlenecks of Traditional SSDs in AI Scenarios
To understand the value of AI SSD, one must first be clear about the problems traditional SSDs face in AI scenarios.
- Latency becomes a more critical metric than bandwidth. In traditional applications, sequential read/write bandwidth is often an important indicator for evaluating SSD performance, such as 7GB/s or 14GB/s. But in AI scenarios, the importance of latency often surpasses peak bandwidth. This is because AI tasks involve a large number of fine-grained data requests. If each access requires waiting for tens of microseconds, even if the amount of data per request is small, the cumulative effect will significantly slow down the overall progress. The typical access latency for many enterprise SSDs is between 40-100 microseconds, which is acceptable in database or virtualization scenarios, but appears relatively high in large-scale AI inference or training.
- IOPS is no longer just “good enough.” IOPS (Input/Output Operations Per Second) has long been one of the core metrics for SSDs, but in traditional businesses, reaching hundreds of thousands of IOPS was often sufficient. AI workloads are completely different. Scenarios like vector retrieval, parameter loading, and model sharding access generate an extremely large number of random read requests. In such cases, the IOPS of traditional SSDs quickly becomes a system bottleneck. This is why discussions about AI SSD technology often mention targets of millions or even tens of millions of IOPS, which was very rare in the past.
- The CPU becomes a “bottleneck for transfer.” In the classic server architecture, the data path between the SSD and the GPU is usually: SSD → CPU → Memory → GPU. This model worked well in the era of general-purpose computing but exposes obvious problems in AI servers. On one hand, the CPU needs to handle a large amount of data movement; on the other hand, this path itself introduces additional latency. As GPU computing power continues to increase, this data path that “goes around the CPU to reach the GPU” is becoming a constraint on overall system efficiency.
Positioning Differences Between AI SSD and Traditional SSD
To understand the difference between the two more intuitively, we can compare them from the perspective of their “design starting point.”
| Comparison Dimension | Traditional SSD | AI SSD |
|---|---|---|
| Primary Goal | Balance between general-purpose performance and cost | Extreme efficiency for AI workloads |
| Optimization Focus | Sequential read/write, capacity, reliability | Low latency, high IOPS, high concurrency |
| Typical Applications | PC, Server, Database | AI Training, Inference, Vector Search |
| Data Access Pattern | Mixed, primarily sequential | Small-block, random, high-concurrency |
| System Collaboration | CPU-centric | Closer to GPU/Accelerator |
It is important to emphasize that AI SSD is not meant to replace traditional SSDs. In the vast majority of general-purpose scenarios, traditional SSDs remain a more reasonable and cost-effective choice. The existence of AI SSD is to serve systems that are already “pushed to the limit” by AI workloads.
The Core Positioning of AI SSD
From a system perspective, the essential role of AI SSD can be summarized in one sentence: its task is not just to store data itself, but to provide data to the AI computing unit efficiently, stably, and continuously. To achieve this goal, AI SSDs typically focus on optimization in the following directions:
- Extremely low access latency
- Very high random IOPS capability
- Internal architecture more suited to AI data access patterns
- Tighter system-level collaboration methods
These characteristics are not achieved by simply stacking parameters, but often require rethinking the SSD’s controller design, flash memory management strategies, and even system interface methods.
Key Technical Features and Architectural Approaches of AI SSD
AI SSD is not simply about taking an existing enterprise SSD, boosting the controller performance, adding more flash memory, and maximizing interface speed to naturally suit AI scenarios. The real difficulty lies in the structural difference between the access patterns of AI workloads and traditional storage applications. The technological evolution of AI SSD is essentially a redesign centered around data access patterns.
Extremely Low Latency
In AI systems, storage latency often directly determines the utilization rate of computing resources. Taking the GPU as an example, its computing power is increasing much faster than storage systems. If the GPU is idle while waiting for data, then even if the theoretical computing power of the GPU is high, the actual throughput will drop significantly. In this case, average latency is not sufficient; tail latency is even more critical. A single IO operation with abnormal latency can slow down the execution of an entire batch.
The access latency of traditional enterprise NVMe SSDs is typically at the level of tens of microseconds, which is a very mature and stable result. But in AI scenarios, the industry is trying to further compress latency to the level of ten microseconds or even close to single-digit microseconds. Achieving this is not just about increasing interface speed; it requires systematic optimization in the following areas:
- Reducing interrupts and context switches in the control path
- Optimizing flash memory access scheduling strategies
- Shortening the internal processing path of data within the controller
It can be said that latency optimization is a systematic project that runs through the entire design process of AI SSD.
Ultra-High IOPS
During model training and inference, data access often exhibits “fragmented” characteristics. For example:
- Model parameters are split into numerous small blocks.
- Vector databases require frequent access to indexes and features.
- Multiple models or tasks run in parallel.
In these scenarios, the SSD faces not a few large, continuous requests, but a massive number of concurrent small requests. This makes IOPS a key indicator determining the performance ceiling. In traditional enterprise SSDs, several hundred thousand IOPS is already considered high-end. In the planning of AI SSDs, common targets are millions, several million, or even tens of millions of IOPS. It is important to note that the IOPS referred to here is not just a peak value under laboratory conditions, but a sustainable capability under high concurrency and low latency constraints. Improving IOPS is not a problem that can be solved simply by “opening more queues.” When the number of concurrent requests is extremely large, the following problems quickly emerge:
- Increased complexity of queue management
- Uneven load between flash memory channels
- Interference from write amplification and garbage collection
Therefore, AI SSDs often need to introduce more aggressive concurrent scheduling strategies at the firmware level, while also managing flash memory resources more finely.
System Co-Design for GPUs and Accelerators
In classic server architecture, there is usually no direct data path between the SSD and the GPU. Data must first pass through the CPU and system memory before being sent to the GPU. The problems of this architecture are amplified in AI servers. As the number of GPUs increases and single-card computing power continues to improve, the efficiency issues of this indirect path become more pronounced. To solve this problem, AI SSDs are beginning to make new attempts at the system level, such as:
- Supporting GPU-direct or near-direct data access modes.
- Reducing unnecessary CPU involvement.
- Optimizing the data transmission path between storage and accelerators.
These designs do not necessarily mean completely bypassing the CPU, but rather reducing data copying and transfer times in appropriate scenarios to improve overall efficiency. In this architecture, storage is no longer just a “passive data warehouse,” but more like an active data service node within the AI system. It needs to understand the upper-level access patterns, respond quickly to concurrent requests, and form efficient collaboration with computing units. This is also why the design of AI SSDs often requires deep collaboration with the entire machine system, and even GPU manufacturers.
Large Capacity and High Density
As large model parameter sizes continue to grow, with individual models often reaching hundreds of GB or even several TB, storage systems must have sufficient capacity and density to support actual deployment. Furthermore, AI systems often need to store multiple versions of models, training data, intermediate results, vector databases, indexes, etc. This makes high capacity and high density another important characteristic of AI SSDs. However, high capacity is not “free.” Increasing flash density often comes with costs:
- Increased access latency per flash die.
- Limited concurrent performance.
- Challenges to endurance and reliability.
Therefore, while pursuing capacity, AI SSDs also need to minimize the performance loss caused by high density through architectural and scheduling designs.
Industry Status and Future Trends of AI SSD
If viewed from the perspective of the technology lifecycle, AI SSD is still in a stage of “early adoption and rapid exploration.” On one hand, AI SSD is not just a marketing term that remains at the conceptual level; AI workloads already exist and are growing rapidly in data centers, cloud computing platforms, and large enterprises. On the other hand, AI SSD has not yet formed a completely unified and standardized product form. True SSDs designed entirely around AI are still mainly limited to customized, pre-research, and high-end enterprise products, and are some distance away from widespread adoption.
In the advancement of AI SSD, the roles within the traditional storage industry chain are changing. In the past, SSD manufacturers focused more on the performance and reliability of single devices. In the AI scenario, they need to participate earlier in system-level design, collaborate with server, GPU, and cloud platform providers, and perform deep optimization for specific AI workloads. This is blurring the boundaries between storage original equipment manufacturers, controller manufacturers, and system integrators.
Future AI SSDs will likely no longer be just “a device plugged into a PCIe slot,” but will be more deeply co-designed with computing resources at the system level. This may be reflected in: more direct data paths, fewer intermediate copies, and tighter software-hardware collaboration. This will further weaken the boundary between storage and computation. On the other hand, as AI systems scale, “handing all computation to the GPU” is not necessarily the optimal solution. In some scenarios, having the storage device handle some data processing or preprocessing tasks can help reduce the overall system load. Although such ideas are still in the exploratory stage, they have become a focus of industry attention.
AI SSD is not a new species that fundamentally changes what storage is, but a reshaping of the storage role centered around AI workloads. Its emergence stems from the fundamental changes AI brings to data access patterns; its value lies not in the exaggerated improvement of individual metrics, but in the continuous improvement of system efficiency; its future will not replace all SSDs, but will become an indispensable part of AI infrastructure. If the GPU is the “engine” of the AI system, then the AI SSD is more like the fuel system that provides stable, continuous supply. It may not be the most conspicuous, but once it falls behind, the entire system is affected.
