I deployed the app in the kubernetes+istio cluster. I used the http probe for the readiness check. In the Graph section of Kiali, the kube-probe traffic is shown as a line from unkonwn to httpbin. I tried to add "x-b3-sampled" http header to avoid the record for this traffic. But it doesn't work. Is there any method to hide the traffic from kube-probe?
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: httpbin
spec:
replicas: 1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
containers:
- image: docker.io/citizenstig/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 8000
readinessProbe:
httpGet:
path: /get
port: 8000
httpHeaders:
- name: 'x-b3-sampled'
value: '0'
initialDelaySeconds: 5
timeoutSeconds: 1
livenessProbe:
tcpSocket:
port: 8000
initialDelaySeconds: 5
timeoutSeconds: 1UPDATE: This is actually going to be fixed in Istio 1.1, and the nice part is that you can easily apply the patch by yourself without waiting 1.1, as it's in the yaml configs:
Patch link: https://github.com/istio/istio/pull/10480
So for Istio 1.0.x, you basically have to edit the Custom Resource of type Rule, named promhttp, in namespace istio-system to set the following match expression :
match: (context.protocol == "http" || context.protocol == "grpc") && (match((request.useragent | "-"), "kube-probe*") == false)Initial response:
I'm not sure if there's a "clean" solution for that, but there's a workaround described at the bottom of this doc page : https://istio.io/docs/tasks/traffic-management/app-health-check/#liveness-and-readiness-probes-with-http-request-option
Because the Istio proxy only intercepts ports that are explicitly declared in the containerPort field, traffic to 8002 port bypasses the Istio proxy regardless of whether Istio mutual TLS is enabled.
So you can have your health endpoints using a different port that you would not declare as container ports, and that way the traffic is not intercepted by the envoy proxy, hence won't generate telemetry in Kiali.
This is not an ideal solution as it forces you to shape your app in a certain way for Istio... but still, it works.
[Edit, just found that: https://istio.io/help/faq/telemetry/#controlling-what-the-sidecar-reports . Looks like you can also filter out requests from telemetry based on source. Though I'm not sure if it's going to work in that case where source is "unknown"]