Not able to replace NGINX with NGINX Plus as reverse proxy for microservices on Google Cloud using Kubernetes

I am following this nice tutorial of How to create a scalable API with micro-services on Google Cloud using Kubernetes.

I have created 4 micro-services and to expose the services I am using NGINX Plus.

  • How to get user ip, if the requests go through hyberkube?
  • Kubernetes rc sent args to container app
  • How to routing http access to multiple docker container
  • Docker: Mongo exits on run
  • Deploy docker container to production preserving mounted volumes
  • How to run Caddy from Docker container? (443 bind: permission denied)
  • Note : The purpose of NGINX Plus / NGINX here is to work as reverse proxy.

    Below is the directory Structure :

    nginx

    –Dockerfile

    –deployment.yaml

    –index.html

    –nginx-repo.crt

    –nginx-repo.key

    –nginx.conf

    –svc.yaml

    Details of files can be seen here. I am pasting the Docker file & nginx.conf here :

    Dockerfile (Original with NGINX Plus):

    FROM debian:8.3
    
    RUN apt-get update && apt-get -y install wget
    
    RUN mkdir -p /etc/ssl/nginx && wget -P /etc/ssl/nginx https://cs.nginx.com/static/files/CA.crt
    
    COPY nginx-repo.key /etc/ssl/nginx/nginx-repo.key
    COPY nginx-repo.crt /etc/ssl/nginx/nginx-repo.crt
    
    RUN wget http://nginx.org/keys/nginx_signing.key && apt-key add nginx_signing.key
    
    RUN apt-get -y install apt-transport-https libcurl3-gnutls lsb-release
    RUN printf "deb https://plus-pkgs.nginx.com/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-plus.list
    RUN wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90nginx
    
    RUN apt-get update && apt-get -y install nginx-plus
    
    RUN mkdir /data
    COPY index.html /data/index.html
    
    COPY nginx.conf /etc/nginx/conf.d/backend.conf
    RUN rm /etc/nginx/conf.d/default.conf
    
    CMD ["nginx", "-g", "daemon off;"]
    

    nginx.conf (Original with NGINX Plus):

    resolver 10.11.240.10 valid=5s;
    
    upstream reverse-backend {
        zone reverse-backend 64k;
        server reverse.default.svc.cluster.local resolve;
    }
    
    upstream arrayify-backend {
        zone arrayify-backend 64k;
        server arrayify.default.svc.cluster.local resolve;
    }
    
    upstream lower-backend {
        zone lower-backend 64k;
        server lower.default.svc.cluster.local resolve;
    }
    
    upstream upper-backend {
        zone upper-backend 64k;
        server upper.default.svc.cluster.local resolve;
    }
    
    server {
        listen 80;
    
        root /data;
    
        location / {
            index  index.html index.htm;
        }
    
        status_zone backend-servers;
    
        location /reverse/ {
            proxy_pass http://reverse-backend/;
        }
    
        location /arrayify/ {
            proxy_pass http://arrayify-backend/;
        }
    
        location /lower/ {
            proxy_pass http://lower-backend/;
        }
    
        location /upper/ {
            proxy_pass http://upper-backend/;
        }
    
    }
    
    server {
        listen 8080;
    
        root /usr/share/nginx/html;
    
        location = /status.html { }
    
        location /status {
            status;
        }
    }
    

    Everything seem to be working fine with NGINX Plus and I am able to hit all 4 micro-services with with url eg. http://x.y.z.w/service[1|2|3|4]/?str=testnginx , where http://x.y.z.w is my external ip and NGINX is taking care of routing requests internally. Now I am willing to do the same work without NGINX Plus by using NGINX only.

    Below are the updated files for NGINX :

    Dockerfile (Updated for NGINX) :

        FROM debian:8.3
        RUN apt-get update && apt-get -y install wget
        RUN mkdir -p /etc/ssl/nginx && wget -P /etc/ssl/nginx https://cs.nginx.com/static/files/CA.crt
        #COPY nginx-repo.key /etc/ssl/nginx/nginx-repo.key
        #COPY nginx-repo.crt /etc/ssl/nginx/nginx-repo.crt
        #RUN wget http://nginx.org/keys/nginx_signing.key && apt-key add nginx_signing.key
        RUN apt-get -y install apt-transport-https libcurl3-gnutls lsb-release
        #RUN printf "deb https://plus-pkgs.nginx.com/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-plus.list
        RUN wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90nginx
        RUN apt-get update && apt-get -y install nginx
        RUN mkdir /data
        COPY index.html /data/index.html
        COPY nginx.conf /etc/nginx/conf.d/backend.conf
        #RUN rm /etc/nginx/conf.d/default.conf
        CMD ["nginx", "-g", "daemon off;"]
    

    nginx.conf (Updated for NGINX) :

    resolver 10.3.240.10 valid=5s;
    
    upstream reverse-backend {
        zone reverse-backend 64k;
        server reverse.default.svc.cluster.local;
    }
    
    upstream arrayify-backend {
        zone arrayify-backend 64k;
        server arrayify.default.svc.cluster.local;
    }
    upstream lower-backend {
        zone lower-backend 64k;
        server lower.default.svc.cluster.local;
    }
    upstream upper-backend {
        zone upper-backend 64k;
        server upper.default.svc.cluster.local;
    }
    server {
        listen 80;
        root /data;
        location / {
            index  index.html index.htm;
        }
    #    status_zone backend-servers;
        location /reverse/ {
            proxy_pass http://reverse-backend/;
        }
        location /arrayify/ {
            proxy_pass http://arrayify-backend/;
        }
        location /lower/ {
            proxy_pass http://lower-backend/;
        }
        location /upper/ {
            proxy_pass http://upper-backend/;
        }
    }
    #server {
    #    listen 8080;
    #
    #    root /usr/share/nginx/html;
    #
    #    location = /status.html { }
    #
    #    location /status {
    #        status;
    #    }
    #}
    

    Basically, I have removed resolve and server statements, which are the features of NGINX Plus and able to create docker image, uploading it on google container and creating my deployments and service but getting 404 not found.

    Am I missing something here or this is the limitation of NGINX ?

    Please suggest if anyone has any suggestion or prior experience working with NGINX, Docker and Kubernets on Google Cloud.

  • How to use a separate file system for the Docker daemon related files at start up on AWS EC2
  • Push access during gitlab ci process
  • Docker compose file memory, cpu limits
  • Why do I see the postgres process when I run top although it is running only inside Docker?
  • Kafka Docker - Difference between links and KAFKA_ADVERTISED_HOST_NAME
  • Running docker commands in bash script leads to segmentation fault
  • Docker will be the best open platform for developers and sysadmins to build, ship, and run distributed applications.