diff --git a/docs/content/reference/routing-configuration/http/middlewares/distributed-ratelimit.md b/docs/content/reference/routing-configuration/http/middlewares/distributed-ratelimit.md new file mode 100644 index 000000000..fb009464f --- /dev/null +++ b/docs/content/reference/routing-configuration/http/middlewares/distributed-ratelimit.md @@ -0,0 +1,183 @@ +--- +title: "Distributed RateLimit" +description: "Traefik Hub API Gateway - The Distributed RateLimit middleware ensures Services receive fair amounts of requests throughout your cluster and not only on an individual proxy." +--- + +!!! info "Traefik Hub Feature" + This middleware is available exclusively in [Traefik Hub](https://traefik.io/traefik-hub/). Learn more about [Traefik Hub's advanced features](https://doc.traefik.io/traefik-hub/api-gateway/intro). + +The Distributed RateLimit middleware ensures that requests are limited over time throughout your cluster and not only on an individual proxy. + +It is based on a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) implementation. + +--- + +## Configuration Example + +Below is an advanced configuration that enables the Distributed RateLimit middleware with Redis backend for cluster-wide rate limiting. + +```yaml tab="Middleware Distributed Rate Limit" +# Here, a limit of 100 requests per second is allowed. +# In addition, a burst of 200 requests is allowed. +apiVersion: traefik.io/v1alpha1 +kind: Middleware +metadata: + name: test-distributedratelimit + namespace: traefik +spec: + plugin: + distributedRateLimit: + burst: 200 + denyOnError: false + limit: 100 + period: 1s + responseHeaders: true + sourceCriterion: + ipStrategy: + excludedIPs: + - 172.20.176.201 + store: + redis: + endpoints: + - my-release-redis-master.default.svc.cluster.local:6379 + # Use the field password of the Secret redis in the same namespace + password: urn:k8s:secret:redis:password + timeout: 500ms +``` + +```yaml tab="Kubernetes Secret" +apiVersion: v1 +kind: Secret +metadata: + name: redis + namespace: traefik +stringData: + password: mysecret12345678 +``` + +## Rate and Burst + +The rate is defined by dividing `limit` by `period`. +For a rate below 1 req/s, define a `period` larger than a second + +The middleware is based on a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) implementation. +In this analogy, the `limit` and `period` parameters define the **rate** at which the bucket refills, and the `burst` is the size (volume) of the bucket. + +```yaml +apiVersion: traefik.io/v1alpha1 +kind: Middleware +metadata: + name: test-ratelimit +spec: + plugin: + distributedRateLimit: + burst: 100 + period: 1m + limit: 6 +``` + +In the example above, the middleware allows up to 100 connections in parallel (`burst`). +Each connection consume a token, once the 100 tokens are consumed, the other ones are blocked until at least one token is available in the bucket. + +When the bucket is not full, on token is generated every 10 seconds (6 every 1 minutes (`period` / `limit`)). + +## Configuration Options + +| Field | Description | Default | Required | +|:-----------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------|:---------| +| `limit` | Number of requests used to define the rate using the `period`.
0 means **no rate limiting**.
More information [here](#rate-and-burst).| 0 | No | +| `period` | Period of time used to define the rate.
More information [here](#rate-and-burst).| 1s | No | +| `burst` | Maximum number of requests allowed to go through at the very same moment.
More information [here](#rate-and-burst). | 1 | No | +| `denyOnError` | Forces to return a 429 error if the number of remaining requests accepted cannot be get.
Set to `false`, this option allows the request to reach the backend. | true | No | +| `responseHeaders` | Injects the following rate limiting headers in the response:
- X-Rate-Limit-Remaining
- X-Rate-Limit-Limit
- X-Rate-Limit-Period
- X-Rate-Limit-Reset
The added headers indicate how many tokens are left in the bucket (in the token bucket analogy) after the reservation for the request was made. | false | No | +| `store.redis.endpoints` | Endpoints of the Redis instances to connect to (example: `redis.traefik-hub.svc.cluster.local:6379`) | "" | Yes | +| `store.redis.username` | The username Traefik Hub will use to connect to Redis | "" | No | +| `store.redis.password` | The password Traefik Hub will use to connect to Redis | "" | No | +| `store.redis.database` | The database Traefik Hub will use to sore information (default: `0`) | "" | No | +| `store.redis.cluster` | Enable Redis Cluster | "" | No | +| `store.redis.tls.caBundle` | Custom CA bundle | "" | No | +| `store.redis.tls.cert` | TLS certificate | "" | No | +| `store.redis.tls.key` | TLS key | "" | No | +| `store.redis.tls.insecureSkipVerify` | Allow skipping the TLS verification | "" | No | +| `store.redis.sentinel.masterSet` | Name of the set of main nodes to use for main selection. Required when using Sentinel. | "" | No | +| `store.redis.sentinel.username` | Username to use for sentinel authentication (can be different from `username`) | "" | No | +| `store.redis.sentinel.password` | Password to use for sentinel authentication (can be different from `password`) | "" | No | +| `sourceCriterion.requestHost` | Whether to consider the request host as the source.
More information about `sourceCriterion`[here](#sourcecriterion). | false | No | +| `sourceCriterion.requestHeaderName` | Name of the header used to group incoming requests.
More information about `sourceCriterion`[here](#sourcecriterion). | "" | No | +| `sourceCriterion.ipStrategy.depth` | Depth position of the IP to select in the `X-Forwarded-For` header (starting from the right).
0 means no depth.
If greater than the total number of IPs in `X-Forwarded-For`, then the client IP is empty
If higher than 0, the `excludedIPs` options is not evaluated.
More information about [`sourceCriterion`](#sourcecriterion), [`ipStrategy`](#ipstrategy), and [`depth`](#sourcecriterionipstrategydepth) below. | 0 | No | +| `sourceCriterion.ipStrategy.excludedIPs` | Allows Traefik to scan the `X-Forwarded-For` header and select the first IP not in the list.
If `depth` is specified, `excludedIPs` is ignored.
More information about [`sourceCriterion`](#sourcecriterion), [`ipStrategy`](#ipstrategy), and [`excludedIPs`](#sourcecriterionipstrategyexcludedips) below. | | No | + +### sourceCriterion + +The `sourceCriterion` option defines what criterion is used to group requests as originating from a common source. +If several strategies are defined at the same time, an error will be raised. +If none are set, the default is to use the request's remote address field (as an `ipStrategy`). + +Check out the [OIDC + RateLimit & DistributedRateLimit guide](../../../../secure/middleware/drl-oidc.md) to see this option in action. + +### ipStrategy + +The `ipStrategy` option defines two parameters that configures how Traefik determines the client IP: `depth`, and `excludedIPs`. + +As a middleware, rate-limiting happens before the actual proxying to the backend takes place. +In addition, the previous network hop only gets appended to `X-Forwarded-For` during the last stages of proxying, that is after it has already passed through rate-limiting. +Therefore, during rate-limiting, as the previous network hop is not yet present in `X-Forwarded-For`, it cannot be found and/or relied upon. + +### sourceCriterion.ipStrategy.depth + +If `depth` is set to 2, and the request `X-Forwarded-For` header is `"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"` then the "real" client IP is `"10.0.0.1"` (at depth 4) but the IP used as the criterion is `"12.0.0.1"` (`depth=2`). + +| `X-Forwarded-For` | `depth` | clientIP | +|-----------------------------------------|---------|--------------| +| `"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"` | `1` | `"13.0.0.1"` | +| `"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"` | `3` | `"11.0.0.1"` | +| `"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"` | `5` | `""` | + +### sourceCriterion.ipStrategy.excludedIPs + +Contrary to what the name might suggest, this option is *not* about excluding an IP from the rate limiter, and therefore cannot be used to deactivate rate limiting for some IPs. + +`excludedIPs` is meant to address two classes of somewhat distinct use-cases: + +1. Distinguish IPs which are behind the same (set of) reverse-proxies so that each of them contributes, independently to the others, to its own rate-limit "bucket" (cf the [token bucket](https://en.wikipedia.org/wiki/Token_bucket)). +In this case, `excludedIPs` should be set to match the list of `X-Forwarded-For IPs` that are to be excluded, in order to find the actual clientIP. + +Example to use each IP as a distinct source: + +| X-Forwarded-For | excludedIPs | clientIP | +|--------------------------------|-----------------------|--------------| +| `"10.0.0.1,11.0.0.1,12.0.0.1"` | `"11.0.0.1,12.0.0.1"` | `"10.0.0.1"` | +| `"10.0.0.2,11.0.0.1,12.0.0.1"` | `"11.0.0.1,12.0.0.1"` | `"10.0.0.2"` | + +2. Group together a set of IPs (also behind a common set of reverse-proxies) so that they are considered the same source, and all contribute to the same rate-limit bucket. + +Example to group IPs together as same source: + +| X-Forwarded-For | excludedIPs | clientIP | +|--------------------------------|--------------|--------------| +| `"10.0.0.1,11.0.0.1,12.0.0.1"` | `"12.0.0.1"` | `"11.0.0.1"` | +| `"10.0.0.2,11.0.0.1,12.0.0.1"` | `"12.0.0.1"` | `"11.0.0.1"` | +| `"10.0.0.3,11.0.0.1,12.0.0.1"` | `"12.0.0.1"` | `"11.0.0.1"` | + +### store + +A Distributed Rate Limit middleware uses a persistent KV storage to store data. + +Refer to the [redis options](#configuration-options) to configure the Redis connection. + +Connection parameters to your [Redis](https://redis.io/ "Link to website of Redis") server are attached to your Middleware deployment. + +The following Redis modes are supported: + +- Single instance mode +- [Redis Cluster](https://redis.io/docs/management/scaling "Link to official Redis documentation about Redis Cluster mode") +- [Redis Sentinel](https://redis.io/docs/management/sentinel "Link to official Redis documentation about Redis Sentinel mode") + +For more information about Redis, we recommend the [official Redis documentation](https://redis.io/docs/ "Link to official Redis documentation"). + +!!! info + + If you use Redis in single instance mode or Redis Sentinel, you can configure the `database` field. + This value won't be taken into account if you use Redis Cluster (only database `0` is available). + + In this case, a warning is displayed, and the value is ignored. diff --git a/docs/content/reference/routing-configuration/http/middlewares/ratelimit.md b/docs/content/reference/routing-configuration/http/middlewares/ratelimit.md index 216d3d9f1..88a4d52fe 100644 --- a/docs/content/reference/routing-configuration/http/middlewares/ratelimit.md +++ b/docs/content/reference/routing-configuration/http/middlewares/ratelimit.md @@ -18,38 +18,108 @@ For a rate below 1 req/s, define a `period` larger than a second ```yaml tab="Structured (YAML)" # Here, an average of 100 requests per second is allowed. # In addition, a burst of 200 requests is allowed. +# Redis distributed rate limiting is configured with all available options. http: middlewares: test-ratelimit: rateLimit: average: 100 + period: 1s burst: 200 + redis: + endpoints: + - "redis-primary.example.com:6379" + - "redis-replica.example.com:6379" + username: "ratelimit-user" + password: "secure-password" + db: 2 + poolSize: 50 + minIdleConns: 10 + maxActiveConns: 200 + readTimeout: 3s + writeTimeout: 3s + dialTimeout: 5s + tls: + ca: "/etc/ssl/redis-ca.crt" + cert: "/etc/ssl/redis-client.crt" + key: "/etc/ssl/redis-client.key" + insecureSkipVerify: false ``` ```toml tab="Structured (TOML)" # Here, an average of 100 requests per second is allowed. # In addition, a burst of 200 requests is allowed. +# Redis distributed rate limiting is configured with all available options. [http.middlewares] [http.middlewares.test-ratelimit.rateLimit] average = 100 + period = "1s" burst = 200 + [http.middlewares.test-ratelimit.rateLimit.redis] + endpoints = ["redis-primary.example.com:6379", "redis-replica.example.com:6379"] + username = "ratelimit-user" + password = "secure-password" + db = 2 + poolSize = 50 + minIdleConns = 10 + maxActiveConns = 200 + readTimeout = "3s" + writeTimeout = "3s" + dialTimeout = "5s" + [http.middlewares.test-ratelimit.rateLimit.redis.tls] + ca = "/etc/ssl/redis-ca.crt" + cert = "/etc/ssl/redis-client.crt" + key = "/etc/ssl/redis-client.key" + insecureSkipVerify = false ``` ```yaml tab="Labels" # Here, an average of 100 requests per second is allowed. # In addition, a burst of 200 requests is allowed. +# Redis distributed rate limiting is configured with all available options. labels: - "traefik.http.middlewares.test-ratelimit.ratelimit.average=100" + - "traefik.http.middlewares.test-ratelimit.ratelimit.period=1s" - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=200" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.endpoints=redis-primary.example.com:6379,redis-replica.example.com:6379" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.username=ratelimit-user" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.password=secure-password" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.db=2" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.poolSize=50" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.minIdleConns=10" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.maxActiveConns=200" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.readTimeout=3s" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.writeTimeout=3s" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.dialTimeout=5s" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.ca=/etc/ssl/redis-ca.crt" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.cert=/etc/ssl/redis-client.crt" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.key=/etc/ssl/redis-client.key" + - "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.insecureSkipVerify=false" ``` ```json tab="Tags" // Here, an average of 100 requests per second is allowed. // In addition, a burst of 200 requests is allowed. +// Redis distributed rate limiting is configured with all available options. { "Tags": [ "traefik.http.middlewares.test-ratelimit.ratelimit.average=100", - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=50" + "traefik.http.middlewares.test-ratelimit.ratelimit.period=1s", + "traefik.http.middlewares.test-ratelimit.ratelimit.burst=200", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.endpoints=redis-primary.example.com:6379,redis-replica.example.com:6379", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.username=ratelimit-user", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.password=secure-password", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.db=2", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.poolSize=50", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.minIdleConns=10", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.maxActiveConns=200", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.readTimeout=3s", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.writeTimeout=3s", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.dialTimeout=5s", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.ca=/etc/ssl/redis-ca.crt", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.cert=/etc/ssl/redis-client.crt", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.key=/etc/ssl/redis-client.key", + "traefik.http.middlewares.test-ratelimit.ratelimit.redis.tls.insecureSkipVerify=false" ] } ``` @@ -57,6 +127,7 @@ labels: ```yaml tab="Kubernetes" # Here, an average of 100 requests per second is allowed. # In addition, a burst of 200 requests is allowed. +# Redis distributed rate limiting is configured with all available options. apiVersion: traefik.io/v1alpha1 kind: Middleware metadata: @@ -64,7 +135,53 @@ metadata: spec: rateLimit: average: 100 + period: 1s burst: 200 + redis: + endpoints: + - "redis-primary.example.com:6379" + - "redis-replica.example.com:6379" + secret: redis-credentials + db: 2 + poolSize: 50 + minIdleConns: 10 + maxActiveConns: 200 + readTimeout: 3s + writeTimeout: 3s + dialTimeout: 5s + tls: + caSecret: redis-ca + certSecret: redis-client-cert + insecureSkipVerify: false + +--- +apiVersion: v1 +kind: Secret +metadata: + name: redis-credentials + namespace: default +data: + username: cmF0ZWxpbWl0LXVzZXI= # base64 encoded "ratelimit-user" + password: c2VjdXJlLXBhc3N3b3Jk # base64 encoded "secure-password" + +--- +apiVersion: v1 +kind: Secret +metadata: + name: redis-ca + namespace: default +data: + tls.ca: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0t... + +--- +apiVersion: v1 +kind: Secret +metadata: + name: redis-client-cert + namespace: default +data: + tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0t... + tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0t... ``` ## Configuration Options @@ -79,6 +196,21 @@ spec: | `sourceCriterion.ipStrategy.depth` | Depth position of the IP to select in the `X-Forwarded-For` header (starting from the right).
0 means no depth.
If greater than the total number of IPs in `X-Forwarded-For`, then the client IP is empty
If higher than 0, the `excludedIPs` options is not evaluated.
More information about [`sourceCriterion`](#sourcecriterion), [`ipStrategy`](#ipstrategy), and [`depth`](#sourcecriterionipstrategydepth) below. | 0 | No | | `sourceCriterion.ipStrategy.excludedIPs` | Allows scanning the `X-Forwarded-For` header and select the first IP not in the list.
If `depth` is specified, `excludedIPs` is ignored.
More information about [`sourceCriterion`](#sourcecriterion), [`ipStrategy`](#ipstrategy), and [`excludedIPs`](#sourcecriterionipstrategyexcludedips) below. | | No | | `sourceCriterion.ipStrategy.ipv6Subnet` | If `ipv6Subnet` is provided and the selected IP is IPv6, the IP is transformed into the first IP of the subnet it belongs to.
More information about [`sourceCriterion`](#sourcecriterion), [`ipStrategy.ipv6Subnet`](#sourcecriterionipstrategyipv6subnet) below. | | No | +| `redis` | The `redis` configuration enables distributed rate limiting by using Redis to store rate limit tokens across multiple Traefik instances. This allows you to enforce consistent rate limits across a cluster of Traefik proxies.
When Redis is not configured, Traefik uses in-memory storage for rate limiting, which works only for the individual Traefik instance.| | No | +| `redis.endpoints` | List of Redis server endpoints for distributed rate limiting. You can specify multiple endpoints for Redis cluster or high availability setups. | "127.0.0.1:6379" | No | +| `redis.username` | Username for Redis authentication. | "" | No | +| `redis.password` | Password for Redis authentication. In Kubernetes, these can be provided via secrets. | "" | No | +| `redis.db` | Redis database number to select. | 0 | No | +| `redis.poolSize` | Defines the base number of socket connections in the pool. If set to 0, it defaults to 10 connections per CPU core as reported by `runtime.GOMAXPROCS`.
If there are not enough connections in the pool, new connections will be allocated beyond `poolSize`, up to `maxActiveConns`. | 0 | No | +| `redis.minIdleConns` | Minimum number of idle connections to maintain in the pool. This is useful when establishing new connections is slow. A value of 0 means idle connections are not automatically closed. | 0 | No | +| `redis.maxActiveConns` | Maximum number of connections the pool can allocate at any given time. A value of 0 means no limit. | 0 | No | +| `redis.readTimeout` | Timeout for socket reads. If reached, commands will fail with a timeout instead of blocking. Zero means no timeout. | 3s | No | +| `redis.writeTimeout` | Timeout for socket writes. If reached, commands will fail with a timeout instead of blocking. Zero means no timeout. | 3s | No | +| `redis.dialTimeout` | Timeout for establishing new connections. Zero means no timeout. | 5s | No | +| `redis.tls.ca` | Path to the certificate authority used for the secure connection to Redis, it defaults to the system bundle. | "" | No | +| `redis.tls.cert` | Path to the public certificate used for the secure connection to Redis. When this option is set, the `key` option is required. | "" | No | +| `redis.tls.key` | Path to the private key used for the secure connection to Redis. When this option is set, the `cert` option is required. | "" | No | +| `redis.tls.insecureSkipVerify` | If `insecureSkipVerify` is `true`, the TLS connection to Redis accepts any certificate presented by the server regardless of the hostnames it covers. | false | No | ### sourceCriterion diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index c00f103b0..e1ccb61ee 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -271,6 +271,7 @@ nav: - 'Compress': 'reference/routing-configuration/http/middlewares/compress.md' - 'ContentType': 'reference/routing-configuration/http/middlewares/contenttype.md' - 'DigestAuth': 'reference/routing-configuration/http/middlewares/digestauth.md' + - 'Distributed RateLimit Traefik Hub API Gateway' : 'reference/routing-configuration/http/middlewares/distributed-ratelimit.md' - 'Errors': 'reference/routing-configuration/http/middlewares/errorpages.md' - 'ForwardAuth': 'reference/routing-configuration/http/middlewares/forwardauth.md' - 'GrpcWeb': 'reference/routing-configuration/http/middlewares/grpcweb.md'