Ingress: Rate Limit

In production setup it's crucial to setup rate limit. These can be use to mitigate DDoS Attacks.

Setup Rate Limit

Nginx ingress have many annotations and few of them can be used for rate limit setup.

nginx.ingress.kubernetes.io/limit-rps
- Number of requests accepted from a given IP each second.
- Burst limit per second is limit-rps * limit-burst-multiplier.
nginx.ingress.kubernetes.io/limit-burst-multiplier
- Default: 5.

Lets update our ingress definition and add above annotations in metadata.annotations section.

annotations:
  nginx.ingress.kubernetes.io/limit-rps: "1"
  nginx.ingress.kubernetes.io/limit-burst-multiplier: "1"

This annotation will limit request per second to only 1.

Apply the changes and validate using kubectl describe. We should see annotations in the result.

➜ kubectl describe ingress simple-ingress          
Name:             simple-ingress
Labels:           <none>
Namespace:        default
Address:          192.168.49.2
Ingress Class:    nginx
Default backend:  <default>
TLS:
  simple-tls terminates simple-go.mine
Rules:
  Host            Path  Backends
  ----            ----  --------
  simple-go.mine  
                  /   simple-go:8080 (10.244.0.88:8080,10.244.0.89:8080,10.244.0.90:8080)
Annotations:      nginx.ingress.kubernetes.io/limit-burst-multiplier: 1
                  nginx.ingress.kubernetes.io/limit-rps: 1
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    71s (x7 over 27h)  nginx-ingress-controller  Scheduled for sync

Let's test it using curl with consecutive request like this.

➜ curl --resolve "simple-go.mine:443:127.0.0.1" -i -k \
    https://simple-go.mine \
    https://simple-go.mine \
    https://simple-go.mine

Output:

HTTP/2 200 
date: Sat, 01 Feb 2025 06:31:29 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 200 
date: Sat, 01 Feb 2025 06:31:29 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 503 
date: Sat, 01 Feb 2025 06:31:29 GMT
content-type: text/html
content-length: 190

<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>

We can see above output that the rate limiting work as expected. It will return response status 503 if the limit exceeded. The second request is allowed because it interpreted as single request in different second.

We might want to change the default response status to 429 Too Many Requests, it is preferred response status to clearly tell the user that rate limit is exceeded.

The 429 status code indicates that the user has sent too many requests in a given amount of time ("rate limiting").

To change the default response status code for exceeding rate limit we need to change the nginx ingress configmap named ingress-nginx-controller in namespace ingress-nginx. We can use kubectl edit command below to edit configmap, it will open a vim editor.

➜ kubectl -n ingress-nginx edit configmaps ingress-nginx-controller

Add this key value pair into data section.

limit-req-status-code: "429"

Save and wait few seconds for the ingress to sync with latest configmap. Then let's test it again using same curl command before. We should see the response is changed to 429 Too Many Requests.

HTTP/2 200 
date: Sat, 01 Feb 2025 07:03:03 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 200 
date: Sat, 01 Feb 2025 07:03:04 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 429 
date: Sat, 01 Feb 2025 07:03:04 GMT
content-type: text/html
content-length: 162

<html>
<head><title>429 Too Many Requests</title></head>
<body>
<center><h1>429 Too Many Requests</h1></center>
<hr><center>nginx</center>
</body>
</html>

References

PreviousIngress: TLS NextIngress: Basic Auth

Last updated 3 months ago

Ingress: Rate Limit

In production setup it's crucial to setup rate limit. These can be use to mitigate DDoS Attacks.

Setup Rate Limit

Nginx ingress have many annotations and few of them can be used for rate limit setup.

nginx.ingress.kubernetes.io/limit-rps
- Number of requests accepted from a given IP each second.
- Burst limit per second is limit-rps * limit-burst-multiplier.
nginx.ingress.kubernetes.io/limit-burst-multiplier
- Default: 5.

Lets update our ingress definition and add above annotations in metadata.annotations section.

annotations:
  nginx.ingress.kubernetes.io/limit-rps: "1"
  nginx.ingress.kubernetes.io/limit-burst-multiplier: "1"

This annotation will limit request per second to only 1.

Apply the changes and validate using kubectl describe. We should see annotations in the result.

➜ kubectl describe ingress simple-ingress          
Name:             simple-ingress
Labels:           <none>
Namespace:        default
Address:          192.168.49.2
Ingress Class:    nginx
Default backend:  <default>
TLS:
  simple-tls terminates simple-go.mine
Rules:
  Host            Path  Backends
  ----            ----  --------
  simple-go.mine  
                  /   simple-go:8080 (10.244.0.88:8080,10.244.0.89:8080,10.244.0.90:8080)
Annotations:      nginx.ingress.kubernetes.io/limit-burst-multiplier: 1
                  nginx.ingress.kubernetes.io/limit-rps: 1
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    71s (x7 over 27h)  nginx-ingress-controller  Scheduled for sync

Let's test it using curl with consecutive request like this.

➜ curl --resolve "simple-go.mine:443:127.0.0.1" -i -k \
    https://simple-go.mine \
    https://simple-go.mine \
    https://simple-go.mine

Output:

HTTP/2 200 
date: Sat, 01 Feb 2025 06:31:29 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 200 
date: Sat, 01 Feb 2025 06:31:29 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 503 
date: Sat, 01 Feb 2025 06:31:29 GMT
content-type: text/html
content-length: 190

<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>

We might want to change the default response status to 429 Too Many Requests, it is preferred response status to clearly tell the user that rate limit is exceeded.

The 429 status code indicates that the user has sent too many requests in a given amount of time ("rate limiting").

➜ kubectl -n ingress-nginx edit configmaps ingress-nginx-controller

Add this key value pair into data section.

limit-req-status-code: "429"

Save and wait few seconds for the ingress to sync with latest configmap. Then let's test it again using same curl command before. We should see the response is changed to 429 Too Many Requests.

HTTP/2 200 
date: Sat, 01 Feb 2025 07:03:03 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 200 
date: Sat, 01 Feb 2025 07:03:04 GMT
content-type: text/plain; charset=utf-8
content-length: 65

[{"id":1,"content":"Hello!"},{"id":2,"content":"Good Morning!"}]

HTTP/2 429 
date: Sat, 01 Feb 2025 07:03:04 GMT
content-type: text/html
content-length: 162

<html>
<head><title>429 Too Many Requests</title></head>
<body>
<center><h1>429 Too Many Requests</h1></center>
<hr><center>nginx</center>
</body>
</html>

We can also exclude certain IP addresses from rate limiting using nginx.ingress.kubernetes.io/limit-whitelist annotation. For list of annotations that supported by nginx ingress you can read further in .

References

PreviousIngress: TLS NextIngress: Basic Auth

Last updated 3 months ago