K
Q

Ingress configuration for k8s in different namespaces

January 21, 2020

I need to configure Ingress Nginx on azure k8s, and my question is if is possible to have ingress configured in one namespace et. ingress-nginx and some serivces in other namespace eg. resources? My files looks like so:

# ingress-nginx.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true' 
    spec:
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.12.0
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --annotations-prefix=nginx.ingress.kubernetes.io
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
# configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
---
# default-backend.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissible as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.4
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: ingress-nginx
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: https
        # app-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app-ingress
  namespace: ingress-nginx
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  tls:
    - hosts:
      - api-sand.fake.com
  rules:
  - host: api-sand.fake.com
    http:
      paths:
      - backend:
          serviceName: api-sand
          servicePort: 80
        path: /

And then I have some app running in the resources namespace, and problem is that I am getting the following error

error obtaining service endpoints: error getting service resources/api-sand from the cache: service resources/api-sand was not found

If I deploy api-sand in the same namespace where ingress is then this service works fine.

-- camel
kubernetes
kubernetes-ingress
nginx-ingress

Similar Questions

How to delete persistent volumes in Kubernetes
kubectl: get specific value from a secret in plaintext
Get total and free disk space using Prometheus
kubectl: Use custom-columns output with maps
Ingress controller to route TCP traffic

7 Answers

July 30, 2020

I would like to simplify the answer a bit for those who are relatively new to Kubernetes and its ingress options. There are 2 separate things that need to be present for ingress(es) to work:

  1. Ingress Controller: a separate 
    DaemonSet
    (a controller which runs on all nodes, including any future ones) along with a 
    Service
    that can be used to utilize routing and proxying. It's based for example on NGINX which acts as the old-school reverse proxy receiving incoming traffic and forwarding it to HTTP(S) routes defined in the 
    Ingress
    resources in point 2 below (distinguished by their different routes/URLs);
  2. Ingress rules: separate Kubernetes resources with 
    kind: Ingress
    . Will only take effect if Ingress Controller is already deployed on that node.

While Ingress Controller can be deployed in any namespace it is usually deployed in a namespace separate from your app services (e.g. 

ingress
or 
kube-system
). It can see 
Ingress
rules in all other namespaces and pick them up. However, each of the Ingress rules must reside in the namespace where the app that they configure reside.

There are some workarounds for that, but this is the most common approach.

-- yuranos
Source: StackOverflow
January 21, 2020

Instead of creating the ingress 

app-ingress
in 
ingress-nginx
namespace you should create it in the namespace where you have the service 
api-sand
and the pod.

Alternatively there is way to achieve ingress in one namespace and service in another namespace via 

externalName
.Checkout https://stackoverflow.com/questions/51878195/kubernetes-cross-namespace-ingress-network/51899301#51899301

Here is an example referred from here.

kind: Service

apiVersion: v1

metadata:

  name: my-service

spec:

  type: ExternalName

  externalName: test-service.namespacename.svc.cluster.local
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: my-service
          servicePort: 80
-- Arghya Sadhu
Source: StackOverflow
April 7, 2020

It's possible actually, you can define ingress and a service with ExternalName type in namespace A, while the ExternalName points to DNS of the service in namespace B. For further details, please refer to this answer: https://stackoverflow.com/a/51899301/2995449

-- Zhaoxing Lu
Source: StackOverflow
September 23, 2021

Outside traffic comes through 

ingress controller service
that is responsible for routing the traffic based on the defined routing rules or what we call 
ingress rules
in k8s world.

In other words, 

ingress
resources are just routing rules (think of it in away that's similar to DNS records) so when you define an 
ingress
resource you just defined a rule for 
ingress controller
to work on and route traffic based on such defined rules.

Solution:

  1. Since 

    Ingress
    are nothing but routing rules, you could define such rules anywhere in the cluster (in any 
    namespace
    ) and 
    controller
    should pick them up as it monitors creation of such resources and react accordingly.

    Here's how to create ingress easily using 

    kubectl

    kubectl create ingress <name> -n namespaceName --rule="host/prefix=serviceName:portNumber"

    Note: Add 

    --dry-run=client -oyaml
    to generate 
    yaml
    manifest file

  2. Or you may create a service of type 

    ExternalName
    in the same 
    namespace
    where you have defined your 
    ingress
    . such external service can point to any URL (a service that lives outside 
    namespace
    or even k8s cluster)

    Here's an example that shows how to create an 

    ExternalName
    service using kubectl:

    kubectl create service externalname ingress-ns -n namespaceName --external-name=serviceName.namespace.svc.cluster.local --tcp=80:80 --dry-run=client -oyaml

this should generate something similar to the following:

kind: Service
apiVersion: v1
metadata:
  name: nginx
  namespace: ingress-ns
spec:
  type: ExternalName
  externalName: serviceName.namespace.svc.cluster.local #or any external svc
  ports:
  - port: 80 #specify the port of service you want to expose 
    targetPort: 80 #port of external service

As described above, create an ingress as below: 

kubectl create ingress <name> -n namespaceName --rule="host/prefix=serviceName:portNumber"

Note: Add 

--dry-run=client -oyaml
to generate 
yaml
manifest file

-- Muhammad Soliman
Source: StackOverflow
August 28, 2024

you can use Mergeable Ingress Resources incase of nginx ingress.

please refer to the docs https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/cross-namespace-configuration/

and refer to the github repo for example : https://github.com/nginxinc/kubernetes-ingress/tree/v3.6.2/examples/ingress-resources/mergeable-ingress-types

-- Hari Kumar.G
Source: StackOverflow
May 16, 2021

There is a way to configure your default backend per ingress resource although the documentation says it is usually configured at the ingress controller level.

For example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myingress
  namespace: myns
spec: 
  defaultBackend:
    service:
      name: default-http-backend
      port: 
        number: 80
...

Here default-http-backend must be defined in the same namespace as the ingress resource.

-- user1707322
Source: StackOverflow
October 15, 2022

The way worked for me is creating ingress for each namespace


apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-service
  namespace: production
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: api.youtube.com
      http:
        paths:
          - pathType: Prefix
            path: "/api/users"
            backend:
              service:
                name: youtube-srv
                port:
                  number: 3000

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-service
  namespace: development
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: dev.youtube.com
      http:
        paths:
          - pathType: Prefix
            path: "/api/users"
            backend:
              service:
                name: youtube-srv
                port:
                  number: 3000
-- Rafiq
Source: StackOverflow