60us latency: Huawei OceanStor M900 shares KV cache to double AI coding throughput
New Context Memory Storage platform shares KV cache across clusters to reduce latency and improve throughput for large-scale AI workloads.
Key points
- Huawei introduced OceanStor M900 at HUAWEI CONNECT 2026 in Shanghai.
- The platform extends key-value cache into a shared storage pool using UnifiedBus.
- Claims include 60-microsecond latency and doubled token throughput for coding tasks.
Huawei launched the OceanStor M900 Context Memory Storage platform at HUAWEI CONNECT 2026 in Shanghai, aiming to resolve memory constraints in large-scale AI inference. The product targets organizations moving toward longer conversational agents and complex reasoning tasks that strain existing infrastructure.
The system addresses the growing demand for key-value cache storage by extending capacity beyond expensive GPU and system memory. Huawei positions the platform to support hyperscale datacenters and its Ascend SuperPoD infrastructure, creating a dedicated layer for shared cache resources.
Technical architecture
The OceanStor M900 uses Huawei’s UnifiedBus high-speed interconnect to share key-value cache resources across a cluster. Huawei states the system can provide up to 64 petabytes of capacity, offering substantially more context per neural processing unit than traditional setups.
The platform employs an innovative ASU disk that integrates CPU, networking, and storage controller functions. Combined with native key-value semantics, this design allows processors to access stored cache data in a single hop, bypassing conventional protocol conversion and CPU forwarding delays.
Performance and cost claims
According to Huawei, this architectural shift reduces access latency to 60 microseconds. The company claims this improvement can double token throughput and halve the time to first token in typical AI coding workloads, addressing the bottleneck where accelerators wait for data.
Huawei also highlights cost efficiency through KV-aware adaptive storage technology, which manages data placement based on cache lifecycles. The vendor claims drive endurance of up to 24 writes per day, which is 16 times higher than conventional solid-state drives, potentially lowering replacement costs.
Who is affected
This development primarily impacts hyperscale datacenter operators and enterprises deploying large-scale AI inference workloads. Organizations using Huawei’s Ascend SuperPoD infrastructure are the direct targets, as the platform is designed to integrate with these specific computing clusters.
Smaller businesses or those not running intensive, multi-turn AI agent workloads may see less immediate relevance. The solution addresses specific economic and performance pressures associated with expanding context windows and autonomous agent deployments in high-volume environments.
Defender's checklist
Monitor vendor announcements for compatibility requirements with existing AI hardware. Assess current infrastructure for latency bottlenecks during peak inference loads. Evaluate the total cost of ownership when comparing traditional storage upgrades to specialized memory solutions.
Operational impact
Teams usually need to assess whether current storage architectures can support expanding context windows without significant latency. Check if existing key-value cache management strategies align with growing computational demands. Inform infrastructure and finance teams about potential hardware shifts. Verify supplier roadmaps for compatibility with new storage layers. Ensure procurement plans account for specialized drive endurance requirements.
Background: AI agents
An AI agent is a software system that perceives its environment, makes decisions, and takes actions to achieve specific goals without constant human input. Unlike passive tools, agents execute tasks, browse data, and interact with other systems autonomously.
Read the full guide: AI Agents Defined: Autonomy, Risk, and Real-World Use
What to do and how to stay safe: Huawei
- Review current AI inference workloads for memory bottlenecks and latency issues.
- Evaluate vendor claims regarding throughput improvements against internal performance metrics.
- Assess compatibility of existing infrastructure with new specialized storage technologies.
- Plan for potential changes in data placement and cache lifecycle management strategies.
Step-by-step guide: Implementing AI Usage Policies at Work: A Step-by-Step Framework
General security guidance from the InfoSec Ledger newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What problem does the OceanStor M900 solve?
It addresses memory bottlenecks in AI inference by extending key-value cache into shared storage.
What is the claimed latency for the new platform?
Huawei claims access latency is reduced to 60 microseconds using its UnifiedBus interconnect.
How does the system improve drive endurance?
KV-aware adaptive storage technology claims endurance of up to 24 writes per day, 16 times conventional SSDs.
