Transparent Ingress Proxy with eBPF and Envoy

So far we learned how to transparently redirect egress traffic, where the requests are intercepted on the client side and proxied through Envoy. But if we really want to build a minimal service mesh, the interception also need to occur on the server side.

It might seem weird, that we actually intercept the traffic twice along the network path between services, but that's actually necessary for the service mesh to be fully functional.

Client-side interception, on one hand, allows the mesh to apply policies before the request leaves the workload—such as enforcing network policies, mTLS, and collecting L7 metrics.

While the Server-side interception ensures that incoming traffic is authenticated, authorized, and observable in a consistent way, regardless of how the client is implemented.

In this lab, we focus on transparent redirection on the receiving Pod—that is, the server side. You'll learn how eBPF can be used to make the server think as if nothing exists in the network path, even though the traffic is being transparently intercepted by Envoy.

eBPF Transparent Ingress Redirection

Compared to the eBPF transparent egress redirection, ingress redirection is slightly easier, but let's see how this was achieved with iptables - "the legacy way".

If we look at Istio Service Mesh as an example (again):

And this is only for handling incoming traffic to a Kubernetes Pod - the traffic passes through Pod-level iptables rules, the transparent proxy, back through iptables, and finally into the application.

Not to mention also that its performance relies on sequential rule processing—evaluating each rule one by one against observed traffic—so the more rules you have, the lower the performance.

So can we do better?

In fact we can - using eBPF.

Instead of jumping through multiple iptables rules, the kernel can make direct redirection decision in a "single" eBPF TC program, reducing latency and avoiding packet reinjection. This results in a simpler data path and has a more predictable performance under load.

The image already outlines the different eBPF concepts in play, but let's take a closer look how this works in the code.

eBPF Implementation Breakdown

At a high level, transparent ingress redirection requires the following steps to occur:

• Redirect ingress traffic targeting the application to the Envoy proxy (running alongside it)
• Either store the original destination before rewriting it and configure Envoy to retrieve and forward to it, or configure Envoy to load-balance the received traffic across a predefined set of backend endpoints
• Rewrite the source information on Envoy response ("on the way back") so the client is tricked into thinking it communicated directly with the application

To redirect ingress traffic to Envoy, we can simply just rewrite the destination port on the ingress traffic (including recalculating the TCP checksum):

SEC("tc")
int tc_both(struct __sk_buff* ctx) {
  ...
  if (ctx->ingress_ifindex) {
    ...
    // Store original destination so Envoy can retrieve it later on egress (as well as getsockopt):
    // * Key: client
    // * Value: original destination
    struct tuple3 client = {
      .ip4 = ip->saddr,
      .port = tcp->source,
      .proto = IPPROTO_TCP,
    };
    struct tuple3 orig_dst = {
      .ip4 = ip->daddr,
      .port = tcp->dest,
      .proto = IPPROTO_TCP,
    };
    int ret = bpf_map_update_elem(&conntrack, &client, &orig_dst, BPF_ANY);
    if (ret != 0) {
      return TC_ACT_OK;
    }

    // Store ports for TCP checksum recalculation
    __u16 old_dport = tcp->dest;
    __u16 new_dport = bpf_htons(ENVOY_PORT);

    // Redirect to envoy by rewriting destination port
    tcp->dest = bpf_htons(ENVOY_PORT);

    // Recalculate TCP checksum since we have modified the TCP destination port
    bpf_l4_csum_replace(ctx, csum_off, old_dport, new_dport, sizeof(__be16));
  }

While this redirects incoming traffic to the port where the Envoy proxy is listening, we now need to do the same for egress traffic by updating the source port when responding back to the client.

This is necessary, since the client originally queried the application port, it then also expects to receive the response from that same port:

SEC("tc")
int tc_both(struct __sk_buff* ctx) {
  ...
  } else {
    // Make sure traffic is coming from Envoy (source port)
    __u32 src_port = bpf_ntohs(tcp->source);
    if (src_port == ENVOY_PORT) {
      // Retrieve the original destination information so we can "emulate" a response from it
      struct tuple3 client = {
        .ip4 = ip->daddr,
        .port = tcp->dest,
        .proto = IPPROTO_TCP,
      };
      struct tuple3* orig_src = bpf_map_lookup_elem(&conntrack, &client);
      if (!orig_src) {
        return TC_ACT_OK;
      }

      // Store ports for TCP checksum recalculation
      __u16 old_sport = tcp->source;
      __u16 new_sport = orig_src->port;

      // Rewrite the source port with the original destination port so client doesn't know it talks to the envoy
      tcp->source = orig_src->port;

      // Recalculate TCP checksum since we have modified the TCP source port
      bpf_l4_csum_replace(ctx, csum_off, old_sport, new_sport, sizeof(__be16));
    }
  }

As you may have noticed, we also need to store and exchange the original destination information (through the conntrack eBPF map) because our TC eBPF program cannot know which (arbitrary) port the client queried unless we retain it before the rewrite on the ingress path.

  // TC Ingress Attachment
    tcin, err := link.AttachTCX(link.TCXOptions{
        Program:   objs.TcBoth,
        Attach:    ebpf.AttachTCXIngress,
        Interface: iface.Index,
    })
    if err != nil {
        log.Fatal("Attaching TC on Ingress:", err)
    }

  // TC Egress Attachment
    tcout, err := link.AttachTCX(link.TCXOptions{
        Program:   objs.TcBoth,
        Attach:    ebpf.AttachTCXEgress,
        Interface: iface.Index,
    })
    if err != nil {
        log.Fatal("Attaching TC on Egress:", err)
    }

But redirecting traffic to the proxy is only half the solution:

Once the traffic reaches Envoy, we must ensure it forwards that traffic to the original application.

There are two-ish ways to achieve this (actually more - but we reduce the scope here).

The simplest approach would be to just configure our application server as a static upstream cluster within Envoy.

...
static_resources:
  listeners:
  - name: listener_http # <- Listens for inbound HTTP traffic on port 8080
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 8080
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: main_route
            virtual_hosts:
            - name: backend # <- Redirect "everything" to the main_service cluster
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: main_service
  ...
  clusters:
  - name: main_service
    type: STATIC
    connect_timeout: 1s
    load_assignment:
      cluster_name: main_service
      endpoints: # <- Our application server is a single static endpoint in the main_service cluster
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 8000
...

In this scenario, Envoy is explicitly told that any traffic it receives on its inbound listener should be forwarded to our application server (127.0.0.1:8000).

This approach requires minimal changes to our setup, but static endpoints are often too limiting. It fails to scale when:

• Application ports are dynamic or assigned at runtime
• Services are ephemeral and frequently added or removed

To handle such dynamic changes, Envoy must be able to somehow "recover" the original destination of a connection during the interception.

This is where Envoy Original Destination Listener Filter comes in:

static_resources:
  listeners:
  - name: listener_http
    ...
    listener_filters:
    - name: envoy.filters.listener.original_dst
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
    ...
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          use_remote_address: true
          route_config:
            name: echo_route
            virtual_hosts:
            - name: backend
              domains: ["*"]
              routes:
              # Forward the request to the original-dst cluster
              - match:
                  prefix: "/"
                route:
                  cluster: original-dst
  ...
  clusters:
  # This "special" cluster type tells Envoy to ignore static host lists and instead 
  # open an upstream connection to the specific address recovered by the listener filter
  - name: original-dst
    type: ORIGINAL_DST
    connect_timeout: 1s
    lb_policy: CLUSTER_PROVIDED
...

Whenever packets are transparently redirected using iptables, this Envoy filter recovers the original destination by reading SO_ORIGINAL_DST socket option of the downstream connection using the getsockopt() system call.

But wait—our setup doesn’t use iptables. So how can this help us?

Even if Envoy would try (in our setup) to retrieve the original destination (using getsockopt()), it would fail since no such redirection has occured (with iptables).

Could eBPF assist us here, anyhow?

In fact, yes!

We can just intercept the getsockopt() system call Envoy makes using cgroup SockOpt eBPF program and rewrite the original destination it retrieves. Nonetheless, we have stored the original destination information in our conntrack eBPF map (used to perform connection redirection in TC ingress and egress hook):

SEC("cgroup/getsockopt")
int cg_getsockopt(struct bpf_sockopt* ctx) {
  ...
    // Key: client
  // Value: Original destination
    struct tuple3 client = {
        .ip4 = ctx->sk->dst_ip4,
        .port = ctx->sk->dst_port,
        .proto = IPPROTO_TCP,
    };
    struct tuple3* orig_dst = bpf_map_lookup_elem(&conntrack, &client);
  ...

  // Rewrite the original destination retrieved by the getsockopt syscall
    sa->sin_family = AF_INET;
    sa->sin_addr.s_addr = orig_dst->ip4;
    sa->sin_port = orig_dst->port;
  ...
}

This completes our setup, with eBPF handling connection redirection on the server side, while Envoy leverages SO_ORIGINAL_DST along with a cgroup SockOpt eBPF program to retrieve the original destination. This ensures that client requests are forwarded to the endpoints they were originally intended for.

But is this all there is? Not really.

Avoiding Re-Redirection to Envoy

The setup we just described actually only works for connections coming from outside the Pod—but what if the client runs in the same Pod and sends requests to Envoy via localhost?

Easy-peasy, we just attach the TC eBPF program to the localhost interface as well. This would also now redirect client requests made through localhost to Envoy, right?

Not really.

When the TC eBPF program is attached to localhost, client packets are correctly intercepted and redirected to Envoy. However, when Envoy forwards the request to the service over localhost, that traffic is intercepted again and redirected back to Envoy—resulting in an infinite loop, breaking the connection.

So what can we do about this?

The goal here is to prevent our eBPF TC program to redirect requests coming from Envoy, so couldn't we just in a way mark the connections from Envoy to the application server (upstream connections) in such a way that we can detect and make decision based upon this mark in our eBPF program.

This is where SO_MARK can help us.

Linux kernel allow us to configure the SO_MARK socket option on the sockets, and we can also do that in Envoy:

...
clusters:
# This "special" cluster type tells Envoy to ignore static host lists and instead 
# open an upstream connection to the specific address recovered by the listener filter
- name: original-dst
  type: ORIGINAL_DST
  connect_timeout: 1s
  lb_policy: CLUSTER_PROVIDED
  # Append the SO_MARK to the upstream connection to avoid re-redirection
  upstream_bind_config:
    socket_options:
    - level: 1        # SOL_SOCKET
      name: 36        # SO_MARK (Linux)
      int_value: 4242
      state: STATE_PREBIND

Once the SO_MARK is set on the upstream connections, we can then check against this mark in our eBPF TC program (before redirecting) using:

SEC("tc")
int tc_both(struct __sk_buff* ctx) {
  __u32 so_mark = ctx->mark;
  if (so_mark == ENVOY_MARK) {
    // Traffic from Envoy - don't re-redirect!
    return TC_ACT_OK;
  }
  ...

Although we are not gonna demonstrate this setup for both remote and local client below, this transparent eBPF redirection implementation will now in fact work, wherever the client is located.

    __u64 pid_tgid = bpf_get_current_pid_tgid();
    __u32 curr_tgid = pid_tgid >> 32;;

    // This prevents the proxy from proxying itself
    __u64 *pid = bpf_map_lookup_elem(&pid_map, &curr_tgid);
    if (pid) {
        return TC_ACT_OK;
    }

Great, we got through the theory - let's see it in action.

Running the Transparent Ingress Proxy

This simple playground has two machines on two different networks:

• client in network 10.0.0.20/16
• server in network 192.168.178.10/24
• gateway between networks 192.168.178.2/24 and 10.0.0.2/16

First, start an HTTP server on the server node (server tab):

python3 -m http.server

Open the second server tab and run an Envoy proxy (under lab directory):

sudo envoy -c envoy.yaml

Then, open another server tab and under lab directory, build and run our transparent proxy setup:

go generate
go build
sudo ./tproxy -i eth0 -ports 8000

Lastly, query the server from the client node (client tab) using:

curl http://192.168.178.10:8000

To verify the redirection actually worked, check the Envoy logs (server tab where you run envoy) to see that the request was indeed captured.

{"response_flags":"-","start_time":"2026-02-28T13:18:53.227Z","upstream_cluster":"original-dst","path":"/","route_name":null,"host_header":"192.168.178.10:8000","duration_ms":12,"protocol":"HTTP/1.1","downstream_remote":"10.0.0.20:47532","virtual_cluster":null,"response_code":200,"method":"GET","upstream_host":"192.168.178.10:8000","request_id":"da50e219-fd6d-43f7-92f2-30f316adc56a"}

Or view eBPF logs using (in one of the server tabs):

sudo bpftool prog trace

Connecting the dots from the previous tutorial, we now have a minimal service mesh in place, with traffic transparently redirected on both the client and server sides. In the upcoming tutorials, we’ll build on this foundation to introduce more advanced service mesh features.

While there's that, in high density environments like Kubernetes we often also need to think about performance - like throughput and latency.

Putting a middleman like Envoy inevitably hurts both, nonetheless it's pretty obvious the packets needs to traverse the kernel networking stack quite a few times even when using eBPF.

So is there something we can do about it and optimize this setup?

Of course we can, but we’ll talk more about this in the next lab.