Commit Graph

1163 Commits

Author SHA1 Message Date
Michel Loiseleur
4a48a0a355
fix linter 2024-08-22 15:23:01 +02:00
Thibault Jamet
8d2f1c0aea
Fix invalid private variable reference 2024-08-14 10:21:01 +02:00
Thibault Jamet
6c5faafbfe
Dynamically register cache provider metrics 2024-08-14 10:21:00 +02:00
Thibault Jamet
9b759f0933
Update changes to match latest state of external-dns code 2024-08-14 10:20:59 +02:00
Thibault Jamet
b2d678f7d0
Run go imports -local sigs.k8s.io/external-dns 2024-08-14 10:20:59 +02:00
Thibault Jamet
b2ff1619f5
Add Domain filter interface 2024-08-14 10:20:59 +02:00
Thibault Jamet
82c6983fa3
Add a log line when no changes on cache provider
Change-Id: I13da2aa28eef3e2c8e81b502321c4dc137087b2d
2024-08-14 10:20:59 +02:00
Thibault Jamet
eb07eb9905
Add licence headers
Change-Id: I3f4646cabd66216fd028fbae3adf68129a8a2cbf
2024-08-14 10:20:59 +02:00
Thibault Jamet
29191e2362
Skip apply empty changes in the cache provider
Change-Id: Icaf1ffe34e75c320d4efbb428f831deb8784cd11
2024-08-14 10:20:58 +02:00
Thibault Jamet
089744c6ff
Add cache at provider level
**Description**

In the current implementation, DNS providers are called to list all
records on every loop. This is expensive in terms of number of requests
to the provider and may result in being rate limited, as reported in 1293
and 3397.

In our case, we have approximately 20,000 records in our AWS Hosted Zone.
The ListResourceRecordSets API call allows a maximum of 300 items per call.
That requires 67 API calls per external-dns deployment during every sync period

With this, we introduce an optional generic caching mechanism at the provider
level, that re-uses the latest known list of records for a given time.

This prevents from expensive Provider calls to list all records for each
object modification that does not change the actual record (annotations,
statuses, ingress routing, ...)

This introduces 2 trade-offs:

1. Any changes or corruption directly on the provider side will be
longer to detect and to resolve, up to the cache time

2. Any conflicting records in the DNS provider (such as a different
external-dns instance) injected during the cache validity will cause
the first iteration of the next reconcile loop to fail, and hence add a
delay until the next retry

**Checklist**

- [X] Unit tests updated
- [X] End user documentation updated

Change-Id: I0bdcfa994ac1b76acedb05d458a97c080284c5aa
2024-08-14 10:20:56 +02:00
Raffaele Di Fazio
b5e6817ba4 project id with context
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
2024-08-05 21:53:37 +02:00
Raffaele Di Fazio
015497e904 fix code build
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
2024-08-05 21:35:07 +02:00
Kubernetes Prow Robot
8245b89891
Merge pull request #4283 from angeloxx/feature/rfc-2136-ptr
feat(rfc2136): add PTR optional support
2024-06-20 00:44:17 -07:00
Kubernetes Prow Robot
618b8df220
Merge pull request #4515 from simonostendorf/fix/provider/cloudflare/clean-token-file
fix(cloudflare): trimSpace on token read from file
2024-06-10 23:33:40 -07:00
Kubernetes Prow Robot
0ba14d8873
Merge pull request #3973 from c445/roehrijn/aws-profiles
feat(aws): use AWS profiles using .credentials file
2024-06-10 13:59:23 -07:00
Jan Roehrich
604a93670e resolve SZUECS' findings 2024-06-10 21:09:18 +02:00
Jan Roehrich
7ff4b3e8fa resolve SZUECS' findings 2024-06-09 21:33:05 +02:00
Michel Loiseleur
84da6be1dd chore: Remove infoblox in-tree provider 2024-05-29 16:01:45 +02:00
Simon Ostendorf
ef066d3185
fix(provider/cloudflare): clean token from file 2024-05-29 09:27:44 +02:00
matthieugouel
57c351b66f feat(coredns): etcd authentication 2024-05-27 11:36:17 +02:00
Kubernetes Prow Robot
b4eea99efe
Merge pull request #4458 from Raffo/webhook-annotations
feat(webhooks): pass webhook-* annotations to webhook providers
2024-05-24 08:51:11 -07:00
angeloxx
1d93309e7f lint 2024-05-13 15:43:24 +02:00
angeloxx
a0040a4603 add PTR creation task 2024-05-13 15:31:18 +02:00
Raffaele Di Fazio
a44a44c111 fix logline mentioning plugin api
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
2024-05-11 15:08:10 +02:00
Raffaele Di Fazio
574f865741 consistency improvement
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
2024-05-11 14:55:33 +02:00
Raffaele Di Fazio
c375899f91 test
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
2024-05-10 20:11:38 +02:00
Kubernetes Prow Robot
49c6c26aa2
Merge pull request #4437 from ebachle/master
fix: soft error on cloudflare rate limits
2024-05-10 04:07:54 -07:00
Kubernetes Prow Robot
fac5b44cfc
Merge pull request #4324 from PseudoResonance/master
Add IPv6 AAAA record support to PiHole provider
2024-05-10 04:07:45 -07:00
Jeremy Albuixech
95ba9f335e fix: uses correct elb url 2024-05-07 11:17:26 -07:00
Jeremy Albuixech
8ef04ef30e aws: add ca-west-1 region 2024-05-07 11:08:23 -07:00
Eddie Bachle
c9e5f594cb add test for Records with rate limit 2024-05-07 12:04:10 -04:00
Eddie Bachle
ffa8f4a2c9 fmt 2024-05-07 11:56:26 -04:00
Eddie Bachle
4ee03cddbf fix: soft error on cloudflare rate limits 2024-05-06 16:28:03 -04:00
PseudoResonance
b4d76a9d94
Fix PiHole AAAA tests
Signed-off-by: PseudoResonance <kaio11604@gmail.com>
2024-04-26 06:20:14 -07:00
PseudoResonance
332f679cc0
Merge branch 'master' of https://github.com/kubernetes-sigs/external-dns 2024-04-26 05:05:26 -07:00
Khue Doan
b16d1b3a4b Merge branch 'master' into az-private-dns-zone-name-filter 2024-04-26 00:31:11 +07:00
Kubernetes Prow Robot
aee99e21f3
Merge pull request #4274 from IntegralProgrammer/dnsimple-user-api-token
feat(DNSimple): User API tokens
2024-04-25 08:17:56 -07:00
Jeremy-Boyle
fe0af65a14 Removed the env, and moved to config file changes for ADD endpoints, need to update docs next 2024-04-16 09:58:31 -07:00
Jeremy
c7a05610f4
Merge branch 'kubernetes-sigs:master' into feature/azure-add-endpoints-4209 2024-04-16 09:13:36 -07:00
Kubernetes Prow Robot
e1df503645
Merge pull request #4374 from SimonKienzler/fix/webhook-provider-wrong-gauge
Webhook provider: Use correct error gauge in `AdjustEndpoints()` func
2024-04-10 05:07:27 -07:00
PseudoResonance
a48287b5e0
Add PiHole IPv6/AAAA tests/documentation
Signed-off-by: PseudoResonance <kaio11604@gmail.com>
2024-04-09 21:58:14 -07:00
PseudoResonance
7b43890ec9
Merge branch 'master' of https://github.com/kubernetes-sigs/external-dns 2024-04-09 21:10:18 -07:00
Simon Kienzler
b020f7c956 Webhook provider: Use correct error gauge in AdjustEndpoints() func 2024-04-09 13:56:49 +02:00
Simon Kienzler
a88cae299a Extract check for retryable error into function 2024-04-09 13:51:42 +02:00
Simon Kienzler
e8e5d5c359 Let the WebhookProvider return SoftError on status codes >= 500 2024-04-09 13:51:42 +02:00
Khue Doan
2cad978659 test(azure): Azure Private DNS apply change with zone name filter 2024-03-28 20:38:22 +07:00
Khue Doan
aa396a9a2f test(azure): Azure Private DNS zone name filter 2024-03-28 20:38:13 +07:00
Khue Doan
425dea47f3 feat(azure): add zone name filter for Azure Private DNS 2024-03-28 19:56:27 +07:00
PseudoResonance
f8a9c07230
Add IPv6 AAAA record support to PiHole provider 2024-03-19 23:01:47 -07:00
angeloxx
cd2088dff3 Merged from master and tested. 2024-03-19 13:27:17 +01:00