mirror of
https://github.com/kubernetes-sigs/external-dns.git
synced 2025-12-14 08:11:48 +01:00
994f4dc24a
* docs(release): update release docs Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> * docs(release): update release docs Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> * docs(release): update release docs Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> * docs(release): update release docs Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> * docs(release): update release docs Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> * docs(release): update release docs Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> * docs(release): update release docs Co-authored-by: Michel Loiseleur <97035654+mloiseleur@users.noreply.github.com> * docs(release): update release docs Co-authored-by: Michel Loiseleur <97035654+mloiseleur@users.noreply.github.com> --------- Signed-off-by: ivan katliarchuk <ivan.katliarchuk@gmail.com> Co-authored-by: Michel Loiseleur <97035654+mloiseleur@users.noreply.github.com>
165 lines
4.9 KiB
Markdown
165 lines
4.9 KiB
Markdown
# 🧭 External-DNS Version Upgrade Playbook
|
|
|
|
## Overview
|
|
|
|
This playbook describes the best practices and steps to safely upgrade **External-DNS** in Kubernetes clusters.
|
|
|
|
Upgrading External-DNS involves validating configuration compatibility, testing changes, and ensuring no unintended DNS record modifications occur.
|
|
|
|
> Note; We strongly encourage the community to help the maintainers validate changes before they are merged or released.
|
|
> Early validation and feedback are key to ensuring stable upgrades for everyone.
|
|
|
|
---
|
|
|
|
## 1. Review Release Notes
|
|
|
|
- Visit the official [External-DNS Releases](https://github.com/kubernetes-sigs/external-dns/releases).
|
|
- Review all versions between your current and target release.
|
|
- Pay attention to:
|
|
- **Breaking changes** (flags, CRD fields, provider behaviors). Not all changes could be captured as breaking changes.
|
|
- **Deprecations**
|
|
- **Provider-specific updates**
|
|
- **Bug fixes**
|
|
|
|
> ⚠️ Breaking CLI flag or annotation changes are common in `0.x` releases.
|
|
|
|
---
|
|
|
|
## 2. Review Helm Chart and Configuration
|
|
|
|
If using Helm:
|
|
|
|
- Compare your Helm chart version to the version supporting the new app release.
|
|
- Check for:
|
|
- `values.yaml` structural changes
|
|
- Default arguments under `extraArgs`
|
|
- Updates to RBAC, ServiceAccounts, or Deployment templates
|
|
|
|
---
|
|
|
|
## 3. Check Compatibility
|
|
|
|
Before upgrading, confirm:
|
|
|
|
- The new version supports your **Kubernetes version** (e.g., 1.25+).
|
|
- The **DNS provider** integration you use is still supported.
|
|
|
|
> 💡 Watch out for deprecated Kubernetes API versions (e.g., `v1/endpoints` → `discovery.k8s.io/v1/endpointslices`).
|
|
|
|
---
|
|
|
|
## 4. Test in Non-Production or with Dry Run flag
|
|
|
|
Run the new External-DNS version in a **staging cluster**.
|
|
|
|
- Use `--dry-run` mode to preview intended changes:
|
|
- Validate logs for any unexpected record changes.
|
|
- Ensure `external-dns` correctly identifies and plans updates without actually applying them.
|
|
- **submit a feature request** if `dry-run` is not supported for a specific case
|
|
|
|
```yaml
|
|
args:
|
|
- --dry-run
|
|
```
|
|
|
|
---
|
|
|
|
5. Backup DNS State
|
|
|
|
Before applying the upgrade, take a snapshot of your DNS zone(s).
|
|
|
|
**Example (AWS Route53):**
|
|
|
|
```sh
|
|
aws route53 list-resource-record-sets --hosted-zone-id ZONE_ID > backup.json
|
|
```
|
|
|
|
Use equivalent tooling for your DNS provider (Cloudflare, Google Cloud DNS, etc.).
|
|
|
|
> Having a backup ensures you can restore records if External-DNS misconfigures entries and you have a solid DR solution.
|
|
|
|
6. Perform a Controlled Rollout
|
|
|
|
Instead of upgrading in-place, use a phased rollout across multiple environments or clusters.
|
|
|
|
Recommended Approaches
|
|
|
|
a. Multi-Cluster Rollout and Progression
|
|
|
|
1. Deploy the new `external-dns` version first in sandbox, then staging, and finally production.
|
|
2. Monitor each environment for correct record syncing and absence of unexpected deletions.
|
|
3. Promote the configuration only after validation in the lower environment.
|
|
|
|
b. Read-Only Parallel Deployment
|
|
|
|
1. Run a second External-DNS instance (e.g., external-dns-readonly) with:
|
|
|
|
```yaml
|
|
args:
|
|
- --dry-run
|
|
- ...other flags
|
|
```
|
|
|
|
1. Observe logs and planned record updates to confirm behavior.
|
|
2. Observe logs and planned record updates to confirm behavior.
|
|
|
|
7. Monitor and Validate
|
|
|
|
After deploying the new version, continuously observe both application logs and DNS synchronization metrics to ensure External-DNS behaves as expected.
|
|
|
|
**Logging**
|
|
|
|
Check logs for anomalies or unexpected record changes:
|
|
|
|
```yaml
|
|
kubectl logs -n external-dns deploy/external-dns --tail=100 -f
|
|
```
|
|
|
|
Look for:
|
|
|
|
- Creating record or Deleting record entries — validate these match expected changes.
|
|
- `WARN` or `ERROR` messages, particularly related to provider authentication or permissions.
|
|
- `TXT` registry conflicts (ownership issues between multiple instances).
|
|
|
|
If using a centralized logging stack (e.g., Loki, Elasticsearch, or CloudWatch Logs):
|
|
|
|
- Create a temporary dashboard or saved query filtering for "Creating record" OR "Deleting record".
|
|
- Correlate `external-dns` logs with DNS provider API logs to detect mismatches.
|
|
|
|
**Metrics and Observability**
|
|
|
|
Check metrics exposed by External-DNS (if Prometheus scraping is enabled):
|
|
|
|
Focus on:
|
|
|
|
- Error rate (*_errors_total)
|
|
- Number of syncs per interval (*_sync_duration_seconds)
|
|
- Provider API call spikes
|
|
|
|
Example PromQL checks:
|
|
|
|
```promql
|
|
rate(external_dns_registry_errors_total[5m]) > 0
|
|
rate(external_dns_provider_requests_total{operation="DELETE"}[5m])
|
|
```
|
|
|
|
## External Verification
|
|
|
|
Ideally, you should have a set of automated tests
|
|
|
|
Query key DNS records directly:
|
|
|
|
```sh
|
|
dig +short myapp.example.com
|
|
nslookup api.staging.example.com
|
|
```
|
|
|
|
Ensure that A, CNAME, and TXT records remain correct and point to expected endpoints.
|
|
|
|
Additional Tips
|
|
|
|
- Automate upgrade testing with CI/CD pipelines.
|
|
- Maintain clear CHANGELOGs and migration notes for internal users.
|
|
- Tag known good versions in Git or Helm values for rollback.
|
|
- Avoid skipping multiple minor versions when possible.
|