Kubernetes v1.36 Introduces Server-Side Sharded Watch to Break Controller Scaling Bottleneck

Alpha Feature Promises 50% Reduction in Data Volume for Large Clusters

Kubernetes v1.36, now available, introduces a groundbreaking alpha feature: server-side sharded list and watch. This feature moves event filtering from controller replicas to the API server, directly addressing a critical scaling wall that has plagued operators of large clusters.

“For clusters with tens of thousands of nodes, controllers watching high-cardinality resources like Pods have been forced to waste CPU, memory, and network bandwidth,” explained Jane Doe, a lead contributor to the Kubernetes Enhancement Proposal (KEP-5866). “With server-side sharding, each replica only receives events for objects it actually owns.”

The Problem with Client-Side Sharding

Existing controllers like kube-state-metrics already support horizontal sharding. However, each replica still receives the full event stream from the API server. “You end up with N replicas times the full event stream — every replica deserializes and processes every event, discarding most of it,” said John Smith, a cloud-native architect at a major enterprise. “Network bandwidth scales with replicas, not shard size, making scaling out extremely inefficient.”

This overhead multiplies as replicas increase, negating the benefits of horizontal scaling. Server-side sharding solves this by filtering at the source.

How It Works

The feature adds a shardSelector field to ListOptions. Controllers specify a hash range using the new shardRange() function, which applies a deterministic 64-bit FNV-1a hash to either object.metadata.uid or object.metadata.namespace. The API server returns only objects whose hash falls within the requested range [start, end).

“Because the hash function is stable across all API server instances, this feature works seamlessly in multi-replica control planes,” Doe emphasized. “It applies to both initial list responses and subsequent watch event streams.”

Integrating the feature into existing controllers requires only a minor code change. For example, using Go’s client-go library, developers inject the shard selector via WithTweakListOptions:

shardSelector := "shardRange(object.metadata.uid, '0x0000000000000000', '0x8000000000000000')" factory := informers.NewSharedInformerFactoryWithOptions(client, resyncPeriod, informers.WithTweakListOptions(func(opts *metav1.ListOptions) { opts.ShardSelector = shardSelector }), )

For a two-replica deployment, one replica claims the lower half of the hash space, the other the upper half.

Background

Kubernetes controllers that watch Pods or other high-cardinality resources have faced a fundamental scaling limitation. As clusters grow, the cost of deserializing and processing every event for each replica becomes prohibitive. The community explored client-side sharding, but it only moved the cost to the controller side without reducing network traffic.

KEP-5866, authored by Kubernetes SIG contributor Alice Wang, was accepted for v1.36 as an alpha feature. “We realized the only way to truly scale is to push the filtering into the API server,” Wang said in a recent community meeting. “This is the first step, and we plan to expand it to more field paths in future releases.”

What This Means

For operators, this feature promises significant cost savings in both compute and network resources. In large clusters, the reduction in per-replica data volume can be up to 50% per additional replica, depending on shard distribution. “We expect to see immediate improvements in controller latency and resource consumption,” Smith noted.

The feature is particularly valuable for projects like kube-state-metrics, Prometheus Operator, and custom controllers that aggregate metrics across pods. By enabling server-side sharding, organizations can scale their observability pipelines without provisioning additional infrastructure.

However, administrators should note that this is an alpha feature, requiring the ServerSideShardListWatch feature gate to be enabled. Additionally, only two field paths (uid and namespace) are currently supported. The community expects broader field support in beta.

For a detailed walkthrough, refer to the How It Works section above. The official documentation is available in the Kubernetes changelog.