Kubernetes Networking Without Sidecars Using Cilium + Hubble — From iptables to eBPF Datapath

This post is aimed at developers who are running or considering adopting a Kubernetes cluster. If kube-proxy or CNI is unfamiliar territory, reading through the Kubernetes networking basics documentation first will make this much easier to follow.

Honestly, I have to admit — when I first heard the phrase "sidecar-free service mesh," I dismissed it as marketing speak. After adopting Istio, I took it for granted that an Envoy container would attach to every pod, and I thought that was just the nature of a service mesh. It wasn't until I actually watched 500 sidecars spin up alongside 500 pods in a real cluster — each consuming roughly 50–100 MB of memory — that I started looking for another way.

Cilium is that other way. It leverages eBPF technology running inside the Linux kernel to handle network policy, load balancing, and L7 traffic control outside the pod. The thousands of iptables rules managed by kube-proxy are replaced by eBPF hash maps, and the per-pod proxies injected by Istio are consolidated into a single shared Envoy per node. This post covers the process of migrating from an iptables-based cluster to an eBPF datapath, and how to inspect the entire network flow without changing a single line of code using Hubble.

As of 2025, Cilium has been adopted as the default CNI for GKE Autopilot and as an official integration option for AKS, and it has become a CNCF Graduated project. Having moved from "adventurous choice" to "industry standard," it's now worth properly understanding how it works and where you need to be careful.

Core Concepts

The Limits of iptables and the Problem eBPF Solves

Consider how kube-proxy handles service routing. Every time a Service is added, dozens of iptables rules are appended, and when a packet arrives, those rules are scanned sequentially from the beginning. With 100 services in a cluster that's fine, but with thousands, every packet has to traverse tens of thousands of rules. This is the moment O(n) complexity becomes something you actually feel.

eBPF (Extended Berkeley Packet Filter) approaches this problem in a completely different way. Without modifying kernel source code or adding modules, it can execute sandboxed programs inside the kernel itself. Service routing is handled via hash-map-based O(1) lookups. And thanks to eBPF's Socket LB (socket-level load balancing), traffic between services on the same node is intercepted at the socket level before it ever touches the network stack — eliminating the network hop entirely.

eBPF Verifier: eBPF programs loaded into the kernel must pass the kernel's built-in verifier before they can run. It performs static analysis to ensure infinite loops and invalid memory accesses are impossible, so programs execute safely without the risk of crashing the kernel.

Cilium's Architecture — How It Handles L7 Without Sidecars

eBPF operates at the packet level — so how does it handle L7 parsing? I found this puzzling at first. The answer lies in layer separation.

When L7 policy is needed, Cilium transparently redirects that traffic to a single Envoy instance (DaemonSet) per node. Rather than attaching a sidecar to every pod, there's one proxy per node. Even with 1,000 pods, the number of Envoy instances only equals the number of nodes.

Istio Ambient Mode is heading in a similar direction — also eliminating sidecars — but Cilium differs in that it uses eBPF as the network datapath itself. Ambient strips out sidecars while preserving the Istio control plane ecosystem, whereas Cilium is architected from scratch around eBPF, from CNI through service mesh. Neither is strictly better; they simply start from different premises.

One more point worth noting: Cilium's network policy is based on cryptographic identities rather than IP addresses. An identity is derived from a pod's Kubernetes labels, and eBPF programs enforce that policy directly in the kernel. When a pod restarts its IP changes, but its labels stay the same — so identity-based policy is unaffected by IP changes. If labels change, the identity changes, and policy is reflected immediately.

Hubble — Seeing All Traffic Without Modifying Code

Hubble is the network observability layer built on top of Cilium. Because it captures flows at the eBPF level, no application code changes or sidecars are needed whatsoever.

Hubble Agent (each node)
    │  Collects eBPF flow events
    ↓
Hubble Relay (cluster-wide aggregation, gRPC streaming)
    │
    ↓
Hubble CLI / Hubble UI (visualization)

The Hubble Agent on each node collects eBPF events, and Hubble Relay aggregates them cluster-wide via gRPC-based streaming. The CLI or UI connects to this Relay to view the entire cluster's flow in a single pane.

What Hubble shows you: L4/L7 traffic flows between services, dropped packets and the reasons why (policy-denied, ct-map-insertion-failed, etc.), DNS queries and responses, and HTTP request methods, paths, and status codes — all without touching anything at the application level.

Hubble's data model has three main elements: flow (individual packet events), verdict (allow/deny decisions), and policy events (policy change detection). When debugging network issues, combining these three lets you immediately see which traffic was blocked, by which policy, and why — all as structured events.

Practical Application

Examples 1–3 below are not standalone snippets — they follow the sequence of cluster setup → Hubble activation → policy application. To apply Example 3, you need the environment from Examples 1 and 2 already in place.

Example 1: Setting Up a Cluster Without kube-proxy

The biggest transition is removing kube-proxy entirely. If you're migrating from an existing cluster, existing connections may be disrupted, so starting with a fresh cluster is recommended.

# Skip the kube-proxy addon phase when initializing the cluster with kubeadm
kubeadm init --skip-phases=addon/kube-proxy
 
# Add and install the Cilium Helm chart
helm repo add cilium https://helm.cilium.io/
helm repo update
 
# <API_SERVER_IP> can be found as the INTERNAL-IP of the control-plane node via `kubectl get nodes -o wide`
# In kind/minikube environments, you can also check with `docker inspect <control-plane-container> | grep IPAddress`
helm install cilium cilium/cilium \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=<API_SERVER_IP> \
  --set k8sServicePort=6443

After installation, running cilium status should produce output similar to the following.

    /¯¯\
 /¯¯\__/¯¯\    Cilium:             OK
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    OK
 \__/¯¯\__/    Hubble Relay:       disabled
    \__/
 
KubeProxyReplacement:    True
Masquerading:            BPF (masq)
NodePort:                ENABLED (Range: 30000-32767)

Key items to verify:

# Connectivity test that comprehensively validates internal networking, NodePort, and external connectivity
cilium connectivity test

Example 2: Enabling Hubble and Analyzing Live Traffic

Hubble can be enabled together when installing Cilium, or added afterward with helm upgrade.

# Enable Hubble + Relay + UI
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --reuse-values \
  --set hubble.enabled=true \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true
 
# Set up port-forwarding for the hubble CLI
cilium hubble port-forward &
 
# Monitor live flows
hubble observe --namespace production --follow
 
# Filter for dropped traffic only (most useful for policy debugging)
hubble observe --verdict DROPPED
 
# HTTP layer visibility
hubble observe --protocol http --namespace production
 
# More granular HTTP filtering — method and path can be combined
hubble observe --protocol http --http-method POST --namespace production
 
# Traffic heading to a specific service
hubble observe --to-pod production/backend-api

When I first applied network policies, --verdict DROPPED was the flag I used most. The first time I ran that command, I was genuinely surprised to see exactly which service's traffic to which service was dropped and why, printed right there. It's especially valuable in situations where a typo in a policy causes traffic to silently disappear.

Example 3: Applying L7 HTTP Policy Without Sidecars

The policy below allows only GET requests from frontend to the /api/ path on backend-api. In Istio, this would have required combining a VirtualService with an AuthorizationPolicy — here, a single CiliumNetworkPolicy handles it.

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-allow-get-only
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: backend-api
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/api/.*"   # RE2 regex syntax. Cilium uses RE2, not glob

Once this policy is applied, POST requests from frontend to backend-api:8080/api/* are automatically blocked. You can verify via Hubble and see the drop reason (policy-denied) as a structured event.

# Immediate validation with Hubble after applying the policy
hubble observe \
  --namespace production \
  --to-pod production/backend-api \
  --verdict DROPPED \
  --protocol http

When a POST request is actually blocked, Hubble output looks like this:

TIMESTAMP             SOURCE                        DESTINATION                   TYPE       VERDICT   SUMMARY
May 13 10:24:31.412   production/frontend           production/backend-api:8080   http       DROPPED   HTTP/1.1 POST /api/orders
  Reason: policy-denied

The source, the method, and the reason for blocking are all contained in a single line. That level of visibility appearing without any sidecars was the thing that impressed me most when I first encountered Cilium.

Pros and Cons Analysis

Advantages

Drawbacks and Caveats

bpftool: A kernel-native debugging tool for directly inspecting and dumping eBPF maps and programs. This is the tool you reach for when the "operational expertise" drawback above actually bites you — the bpftool map dump command lets you directly inspect the service endpoint maps managed by Cilium.

The Most Common Mistakes in Practice

These mistakes flow directly from the drawbacks above. Simple once you know them, but they can cost you hours if you don't.

1. Skipping the kernel version check: A common scenario is installing without first checking node kernel versions with uname -r, only to find certain features quietly disabled. Below 5.10, core features like eBPF Host Routing and Socket LB fall back to alternative implementations. If installation succeeds but performance doesn't match expectations, the kernel version is the first thing to check.

2. Abruptly removing kube-proxy from a running cluster: Deleting the kube-proxy DaemonSet from a live cluster leaves the existing iptables rules in place, causing conflicts. The order matters — remove kube-proxy only after Cilium is fully prepared with --set kubeProxyReplacement=true.

3. Applying L7 policy to all services at once: Enabling L7 policy routes that traffic through the node's Envoy instance. I apply only L4 policy to performance-sensitive internal services and enable L7 only for external-facing services or those that require authentication. I've seen quite a few cases where someone turned on L7 everywhere from the start, then had to roll it back after noticing increased latency.

Closing Thoughts

eBPF is the technology that lets you rewrite the rules of networking without recompiling the kernel, and Cilium + Hubble are the tools that put that power into a form Kubernetes operators can actually use.

There's an intimidation factor that comes with the phrase "kernel level" when you first encounter it. I felt the same way, and honestly, I was skeptical it was all marketing. But once you actually go through the experience of installing with a single cilium-cli and immediately seeing why traffic was dropped with a single hubble observe --verdict DROPPED command, that skepticism fades. It turned out not to be marketing copy — it was something that actually worked. If you're already using GKE or AKS, there's a good chance Cilium is already running beneath your cluster.

Here are 3 steps you can take right now to get started:

1. Experience the full flow locally first. Create a local cluster with kind or minikube, install with helm install cilium cilium/cilium --set kubeProxyReplacement=true, and verify normal operation with cilium connectivity test. You can experience the entire flow without needing access to a production environment.

2. See for yourself how different Hubble UI is from text logs. Add hubble.ui.enabled=true via helm upgrade, then open a service dependency graph in your browser with kubectl port-forward -n kube-system svc/hubble-ui 12000:80.

3. Apply a single L7 policy to a test namespace. Apply the CiliumNetworkPolicy from the example above, then compare allowed and blocked requests using hubble observe --verdict DROPPED — you can directly observe L7 policy being enforced without any sidecars.

If you run into issues, Cilium Slack and GitHub Discussions are both active communities. Response times tend to be fast, making it easy to get help when troubleshooting.

References

• Cilium Official Docs - Service Mesh — Full reference for Cilium service mesh features
• Cilium Official Docs - Kubernetes Without kube-proxy — Official installation guide for kube-proxy-free cluster setup
• Cilium kube-proxy Replacement Use Case — Source material for performance benchmark figures
• GitHub - cilium/cilium — Source code, release notes, and issue tracker
• GitHub - cilium/hubble — Hubble CLI source and releases
• How eBPF Streamlines the Service Mesh | The New Stack — An explanation of the relationship between eBPF and service mesh architecture
• Istio Ambient vs. Cilium Official Comparison | Istio Blog — Reference for comparing the differences between Istio Ambient Mode and the Cilium approach
• Linkerd vs Cilium: Five Key Differences | Buoyant — A comparison written from the Linkerd perspective; best read alongside counterarguments for a balanced view
• Introducing eBPF Host Routing: Azure CNI Powered by Cilium | Microsoft — Background on Cilium adoption in AKS and performance measurement data
• Configure Azure CNI Powered by Cilium in AKS | Microsoft Learn — Practical reference for applying Cilium in an AKS environment
• GKE Network Interface: from kubenet to eBPF/Cilium to DRANET | Google Cloud Blog — The story of how GKE adopted Cilium and what comes next
• eBPF-Based Network Observability with Cilium Hubble | CloudRaft — A good introductory resource for first-time encounters with the Hubble observability layer
• Monitor Cilium and Kubernetes with Hubble | Datadog — Reference for integrating Hubble with external monitoring tools
• Cilium Hubble Cheatsheet | Isovalent — A handy cheat sheet for quickly looking up hubble observe command options