Dr. Leo

Four Layers of AI Infrastructure

1. GPU & NVLink

Modern GPUs provide immense compute power, but their true value emerges when linked via NVLink into coherent clusters. This high-bandwidth, low-latency interconnect transforms isolated devices into a unified compute fabric—critical for training large-scale models like LLMs. From my perspective, this trend marks a fundamental shift in computing: from linear pipelines to two-dimensional, matrix-like computation, where performance depends not just on raw power, but on the architecture of connections themselves.

2. RDMA & Networking

Traditional networking protocols like TCP/IP were never designed for the demands of modern AI. RDMA fundamentally redefines this layer by enabling memory-to-memory transfers without CPU involvement—achieving microsecond latencies and full link saturation at 200 Gbps+. But more than a performance hack, RDMA represents a shift toward spatial communication models. In two-dimensional computing, data is no longer passed linearly, but flows across a fabric—emphasizing topology, synchronization, and bandwidth symmetry as first-class design principles.

3. Kubernetes & Scheduling

Kubernetes brought scale to the cloud, but AI workloads require more than generic orchestration—they demand hardware-aware, interconnect-sensitive scheduling. My work pushes K8s toward a topology-driven model, where GPU placement, NVLink awareness, and RDMA path optimization are tightly integrated. This aligns with the broader move from time-based to space-based computation: where resource geometry and inter-node relationships directly impact performance and scalability.

4. AI-Native Application Stack

AI applications today aren’t just services—they're dynamic, emergent systems. LLM inference, multimodal fusion, token streaming—all rely on real-time orchestration of compute, memory, and network. To support this, infrastructure must become more reactive, adaptive, and tensor-native. In my view, this signals a new layer of computing logic—one rooted in matrix structures and flow fields, where software doesn't just run on hardware, it resonates with it.