Traefik 2.0 as reverse proxy for services hosted at home

- traefik raspberrypi load-balancer reverse-proxy

Motivation

Recently containous released version 2 of traefik, see their blog. I wanted to try out traefik for quite a while and I take this release as a chance to replace my nginx based load-balancer at home with traefik.

Imagine you have some services running at home, some run as single instances, some run redundant. In my case I have a kubernetes cluster based on rancher running on two intel nucs. There is a rancher management interface listening on port 8443 on nuc 1 as well as the nginx-ingress-controller on both nucs listening on ports 80 and 443 each.

With my internet connection I have a single public ip address and I can forward each port to only one local ip address in my home network of course. So I could forward port 8443 to nuc 1 (rancher management interface) and I could fordward port 80 and 443 to one of the nucs. But which one? And what happens if one of it fails or I want to do some maintenance on it?

Goals

Out of scope

Versions used

Preparation

To assign a static local ip to your raspberry pi edit the file /etc/dhcpcd.conf. There is a block called Example static IP configuration

# Example static IP configuration:
interface eth0
static ip_address=192.168.3.7/24
static routers=192.168.3.1
static domain_name_servers=192.168.3.1

As last step for preparation you have to configure your router to forward port 80 and 443 to the ip of you raspberry pi. This is 192.168.3.7 in this case.

Traefik for the rescue

I strongly recommend just to take a look into traefik´s documentation at https://docs.traefik.io to get an overview over the basic concepts and architecture. What are entrypoints, routers, services etc.? Documentation is good and easy to understand.

Just download a proper release for your raspberry from https://github.com/containous/traefik/releases. In my case with raspberry pi 3 it´s the armv7 build.

wget https://github.com/containous/traefik/releases/download/v2.0.2/traefik_v2.0.2_linux_armv7.tar.gz
tar xfvz traefik_v2.0.2_linux_armv7.tar.gz
sudo mv traefik /usr/bin/
sudo useradd -r -s /bin/false -U -M traefik

See systemd-template and create a file called traefik.service in /etc/systemd/system/. Copy the content in there and adjust the file to your configuration. Mine looks like the following.

[Unit]
Description=Traefik
Documentation=https://docs.traefik.io
AssertFileIsExecutable=/usr/bin/traefik
AssertPathExists=/etc/traefik/traefik.yaml

[Service]
User=traefik
AmbientCapabilities=CAP_NET_BIND_SERVICE

Type=notify
ExecStart=/usr/bin/traefik --configFile=/etc/traefik/traefik.yaml
Restart=always
WatchdogSec=1s

[Install]
WantedBy=multi-user.target

Last step is to start and enable systemd service

sudo systemctl enable traefik.service
sudo systemctl start traefik.service

Basic traefik configuration

Of course traefik won´t start correctly because we don´t have any config files for it. Basically, I define the entrypoints and a provider. In other words there are ports where traefik is listening and a file-provider defines where the detail config is.

Create a file /etc/traefik/traefik.yaml and add the entryPoints and the providers configuration. I define two entrypoints web and websecure for port 80 and 443 as well as traefik entrypoint with port 8080. Traefik entrypoint will only be used for dashboard and metrics. For this we need to add api: {} to enable dashboard access.

The providers configuration defines a file provider with a config file /etc/traefik/dynamic_conf.yaml and the watch option to be true which enables updates on the file will automatically reload traefik configuration.

And at last we enable access log for traefik. But pay attention the access logs are only working for http routers, not for tcp routers. The complete file looks like the following.

entryPoints:
  web:
    address: :80
  websecure:
    address: :443
  traefik:
    address: :8080

providers:
  file:
    filename: /etc/traefik/dynamic_conf.yaml
    watch: true

api: {}

accessLog:
  filePath: /var/log/traefik/access.log

HTTP configuration e.g. non-ssl

The following configurations bascially apply on the chapter routing in traefik docs https://docs.traefik.io/routing/overview/.

I define the types of objects in the http configuration: services, middlewares and routers.

First I create a service named rancher-ingress pointing to my two nucs. This service does the load-balancing and because nginx-ingress-controller in kubernetes does a hostname based reverse-proxying to kubernetes services we activate passHostHeader: true.

With this I can create a router named ingress. This router is defined to listen on the entryPoints web and passes traffic to the service rancher-ingress if the configured rule matches. I chose a rule "HostRegexp(`{host:.*}.my-domain.com`)" because I want that all traffic for sub domains of my-domain.com should match. For traefik there is a need to define a named regex matcher {host:.*} to get this working.

The other route named dashboard point to the traffic dashboard. Because this dashboard should be protected, it is only accessible from localhost and local network and protected via basic auth. This is handled via middlewares. See the full config part for the http block below for all information.

/etc/traefik/dynamic_conf.yaml - part 1

http:
  routers:
    dashboard:
      entryPoints:
        - "traefik"
      rule: PathPrefix(`/api`) || PathPrefix(`/dashboard`)
      middlewares:
        - homelan-whitelist
        - auth
      service: api@internal
    ingress:
      entryPoints:
        - "web"
      rule: "HostRegexp(`{host:.*}.my-domain.com`)"
      service: rancher-ingress
  services:
    rancher-ingress:
      loadBalancer:
        passHostHeader: true
        servers:
          - url: "http://192.168.3.8/"
          - url: "http://192.168.3.9/"
  middlewares:
    auth:
      basicAuth:
        removeHeader: true
        users:
          - "admin:$apr1$UWr7Pkpy$iqE4S6so.dDj1kCQ0DMLq/" # admin:admin
    homelan-whitelist:
      ipWhiteList:
        sourceRange:
          - "127.0.0.1/32"
          - "192.168.3.0/24"

HTTPS configuration with tls passthrough

Traefik 2 introduces tcp routing which enables us to handle https traffic without managing any certificates.

For this we´ll create nearly the same config like above but in a tcp block. The rules use a different matcher HostSNI() which uses server name indication from the TLS protocol to identify which hostname is used for connection. After identification traffic is passed through directly to the backend service without de- and re-encryption communication.

In the following example there are two routers, one for rancher master and one for rancher-ingress. For a full configuration add the following tcp block as well as the http block above into /etc/traefik/dynamic_conf.yaml.

/etc/traefik/dynamic_conf.yaml - part 2

tcp:
  routers:
    rancher-server:
      entryPoints:
        - "websecure"
      rule: "HostSNI(`rancher.my-domain.com`)"
      service: "rancher-server"
      tls:
        passthrough: true
    rancher-ingress:
      entryPoints:
        - "websecure"
      rule: "HostSNI(`*`)"
      service: "rancher-ingresses"
      tls:
        passthrough: true
  services:
    rancher-server:
      loadBalancer:
        servers:
          - address: "192.168.3.9:8443"
    rancher-ingresses:
      loadBalancer:
        servers:
          - address: "192.168.3.8:443"
          - address: "192.168.3.9:443"

Some notes on tcp routers. Of course there is no accesslog for a passed through communication as well as no metrics on traefiks metrics endpoint.

Limiting to known domains

If we now want to limit the reverse proxy functionality to a known set of domains, we could create a default 404 service. Traefik itself is not able to serve 404 pages or something like this.

But we can just create a new router which listens on everything else and limit this router to to access via an ipWhiteList middleware. With this all requests from internet to domains different from *.my-domain.com will result in a 403 response.

A note on rules ordering in traefik. They are sorted descending by rule length. So traefik starts on the longest matching rule and ends on the shortest one.

This is just a small barrier but I don´t want to forward every single request to the ingress controllers on my kubernetes cluster.

http:
  routers:
    # ....
    all-rest:
      entryPoints:
        - "web"
      rule: "HostRegexp(`{host:.*}`)"
      middlewares:
        - homelan-whitelist
      service: rancher-ingress

Conclusion

It was real fun to get in touch with traefik. It was a bit tricky to get to know how exactly rules working, especially with wildcards and so on. But it was worth it. I´m looking forward to test traefik as kubernetes ingress controller in kubernetes, but I wait until helm chart will use version 2 of traefik 😄.