Currently, the minimum delay between 2 kubernetes events handling is
hard-coded to 5s.
This may cause higher synchronization rates and higher DNS provider API
calls when handling an important number of kubernetes events at once.
Give the opportunity to configure this delay so service owners can
define the acceptable thresholds on their side
* adds flag to opt in for NS records management
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* go fmt
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* goimports
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* fix more tests
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* go fmt again
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* fix test
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* more tests
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* make ordering of managed records consistent
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* fix flag
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
refactor: remove dns api logic and use dns api library
enhancement: add additional args for auth credential retieval
cleanup: simplify, organize processing logic
test: update automation and validate
When it syncs AWS DNS with k8s cluster content (at `--interval`), external-dns submits two distinct Route53 API calls:
* to fetch available zones (eg. for tag based zones discovery, or when zones are created after exernal-dns started),
* to fetch relevant zones' resource records.
Each call taxes the Route53 APIs calls budget (5 API calls per second per AWS account/region hard limit), increasing the probability of being throttled.
Changing synchronization interval would mitigate those calls' impact, but at the cost of keeping stale records for a longer time.
For most practical uses cases, zones list aren't expected to change frequently.
Even less so when external-dns is provided an explicit, static zones set (`--zone-id-filter` rather than `--aws-zone-tags`).
Using a zones list cache halves the number of Route53 read API calls.
Oracle Cloud Infrastructure (OCI) supports "instance princpal" authentication.
From
<https://docs.cloud.oracle.com/en-us/iaas/Content/Identity/Tasks/callingservicesfrominstances.htm>:
> After you set up the required resources and policies, an application running
> on an instance can call Oracle Cloud Infrastructure public services, removing
> the need to configure user credentials or a configuration file.
This change adds support to the OCI provider for instance principal
authentication when external-dns is run on an OCI instance (e.g. in OCI OKE).
Existing support for key/fingerprint-based authentication is unchanged.
This moves `domain_filter.go` to the `endpoint` package to make it
possible to filter and exclude record names in
`plan.filterRecordsForPlan()` so it does not have to be implemented in
every single provider.
Because some providers access `DomainFilter.filters` directly it had to
be exported.
The number of objects returned by the infoblox api is limited to 1000 objects
(see https://ipam.illinois.edu/wapidoc). If there are more then 1000 objects
the API returns an error. By setting max-results one can raise the limit.
number of retries that API calls will attempt before giving up.
This somewhat mitigates the issues discussed in #484 by allowing
the current sync attempt to complete vs. failing and starting anew.
Defaults to 3, which is what the aws-sdk-go defaults to where not
specified.
Signed-off-by: Joe Hohertz <joe@viafoura.com>
This is to change the way batching works when using the aws provider.
Originally, batching would take the first n records you want to update
and perform the desired actions on those records as part of a sync. It
would then wait for the configured sync period and take the first n
records again and sync them. The issue with this is that when you are
using the TXT registry with a custom prefix, the updates can sync a TXT
record and not the accompanying A/CNAME record. This causes external-dns
to get out of sync with what is created and what the current state
actually is. This update uses the same idea of batching, however, rather
than stopping after the first batch until the next run, batching will
now have a separate batch interval which controls the interval between
each batch in the same sync period. This allows external-dns to fully
sync with route53 as part of each sync and can then know that the state
is complete.
Fixes https://github.com/kubernetes-incubator/external-dns/issues/679
* add Istio Gateway Source
* add documentation for Istio Gateway Source
* make both istio namespace and ingress gateway service configurable
* prefix gateway types, constructors, and flags with 'istio-'
* fix: add missing sources to source flag docs
When running in a pod sometimes the request to get ingreses/services
stalls indefinitely. A simple pod restart fixes this. Hard to reproduce
but I got lucky and did thread dump which revealed a gorouting blocked
on call to k8s.
What's new is a `--request-timeout` flag that makes requests to k8s
bounded in time. The default is 30s - this may cause some deployments
with a slow api-server to timeout.
* Create `NewAWSProvider` with `AWSConfig` struct
Rather than calling `NewAWSProvider` with a list of objects, you will
now call it using a new `AWSConfig` struct. This allows for clearer
declarations of variables which becomes even more important as more
variables are added.
* Add `aws-max-change-count` flag
Adding a new `aws-max-change-count` flag to override the default max
change count on the aws provider.
Included updated tests with a new `defaultMaxChangeCount` constant and
tests for setting the value as a flag and as an environment variable.
* Update CHANGELOG.md
Updating CHANGELOG.md with 'Add aws max change count flag' PR.
Commit adds:
* Implementation of PowerDNS as a provider
* Tests for said implementation
* github.com/ffledgling/pdns-go, which provides go client bindings for
PowerDNS's HTTP API, as a dependency
* "pdns" as an additional option for the `--provider` flag
* `--pdns-server` and `--pdns-api-key` as additional flags for PowerDNS
specific configuration
* Add a flag that allows FQDN and Annotations to combine
Old behaviour is kept by default, a new flag is introduced to combine instead of overwrite
Fixes#218
* docs: add fqdn template combine to changelog
* Graceful handling of misconfigure password for dyn
If a bad password is given for provider "dyn" then the next
login attempt is at least 30minutes apart. This prevents an
account from being suspended.
Improve validation of flags for dyn provider. Add test for
ValidateConfig() and Config.String()
Also add --dyn-min-ttl option which sets the lower limit
of a record's TTL. Ignored if 0 (the default).
* docs: add graceful handling of misconfiguration to changelog
* Don't log passwords on start
The two passwords configurable as flags (for infoblox and dyn) are
masked now and not logged.
* docs: add masking sensitive data in logs to changelog
* add "dyn" provider
* add several --dyn-* args to configure Dyn login
* add github.com/nesv/go-dynect/dynect@0.6.0 to Gopkg and vender/ (the client
of choice by Terraform)
* make externdns.Version public so it can be stored when committing
zone changes
* add tutorial for Ingress resources and update root README.md file
Dyn REST API is documented here: https://help.dyn.com/dns-api-knowledge-base/
Example usage:
external-dns \
--provider=dyn \
--dyn-customer-name=acme \
--dyn-username=acme-api \
--dyn-password=t0pS3cr3t \
--domain-filter=portal.acme.com \
--zone-id-filter=acme.com \
--namespace=my-test-ns \
--log-level=debug \
--txt-prefix=_
* Add aws-zone-id flag
* Add Zone ID filter
* Update AWS provider and main
* Make ZoneIDFilter generic
* Implement ZoneIDFilter for all providers
* Update CHANGELOG
This commit adds ability to use TLS transport for etcd.
New logic is applied when the etcd URL has https:// scheme.
TLS parameters are passed in the environment variables:
ETCD_CA_FILE - path to CA certificate. If not specified, then
system-provided certificates are used.
ETCD_CERT_FILE - client certificate
ETCD_KEY_FILE - client key file
- either both of none of this two must be specified
ETCD_TLS_SERVER_NAME - expected CN of the certificate. Useful when
URL points to a different domain from that in server certificate
ETCD_TLS_INSECURE - if set to "1" (or "true" or "yes") makes client
bypass server certificate validation.
Also for unification with other providers and rest of connection
settings, etcd URL is no longer specified in the command line, but
rather in ETCD_URLS environment variable (defaults to
http://localhost:2379). More than one comma-separated URL can be
specified. All of the URLs must start with either http:// or https://
Also, now it possible to communicate with etcd through proxy specified
in standard environment variables
* allow filtering by ingress class
* generic source annotation filter as opposed to ingress class filter
* rename and fix argument ordering, switch to label selector semantics
* remove redundant parameters
This commit adds support for CoreDNS through its etcd middleware.
Because the middleware is backward compatible with SkyDNS this
commit adds support for SkyDNS as well. In fact, new provider
is available under two names in CLI (coredns and skydns).
All interactions with middleware happen through etcd cluster,
whose location (URIs) is specified via --etcd CLI parameter
by default http://localhost:2379).
The provider translates CoreDNS/DkyDNS SRV records to
A/CNAME + optional TXT endpoints, when reading from etcd and
combines A/CNAME with TXT endpoints back into single SRV record
when writing it back.
Also adds github.com/coreos/etcd package to glide.yaml and vendor folder
because it is used by the provider
- Add `--inmemory-zone flag`
- Implement `InMemoryZoneInit` function to setup initial zones for
inmemory provider
- Make "" the default zone for the inmemory provider instead of
nil/none when no initial zones are specified
- Update config/flag parsing tests to accept new flag
* Initial commit of Infoblox provider
* address @ideahitme's observations
* fail at addressing @szuecs comments
* fix(infoblox): avoid shadowing err variable
* fix flag descriptions
* default ssl verify to true. thanks @szuecs
* chore(infoblox): bump minium required version
This should ideally be a minor bump but let's do that when we have v0.5
* ClusterIP service support
- First pass at addresssing #187 by allowing services with type ClusterIP to be directly supported
* Getting existing tests to pass
* Adjusting formatting for gofmt/govet
* Adding in guard logic around publishing of ClusterIP sources
* Addressing PR feedback
* Adding in CHANGELOG entry
* Adding in Headless service test
* ref(sources): refactor source registration and lookup to be lazy.
* fix(fake): don't make changes to passed in config values
* rework without init, tests are missing
* make client provider public
* fix all tests
* change parameter list order, minor improvements
* clientprovider -> clientgenerator, switch naming for interface/struct
* Support for multiple domains within --domain-filter
The parameter accepts a comma separated list of domains with or without trailing dot. Example: --domain-filter="example.org, company.test.,staging.com". Closes#247 and #229
* Add boilerplate header
* Add documentation for methods and structs
* use StringsVar for the domain-filter flag
* go fmt
* Remove camel case from tests
* Revert changes in README.md
* Move DomainFilter to provider package
* Make a new slice and copy elements to it
* Update CHANGELOG.md
* docs: change minor spelling mistake
* CloudFlare Provider
* updating glide
* gofmt cloudflare_test.go
* Unset envs to test NewCloudFlareProvider
* More tests
* fix(cloudflare): fix compiler errors resulting from merge
* Typo
* Undo vendor changes
* decrease api calls, fix some nits
* Cloudflare iteration (#2)
* reduce the number of API calls
* match by type and name for record id
* improve coverage and fix the bug with suitable zone
* tests failed due to wrong formatting
* add cloudflare integration to the main
* vendor cloudflare deps
* fix cloudflare zone detection + tests
* fix conflicting test function names
* Initial Digital Ocean as a new provider
* chore: vendor dependencies
* Remove zone parameter from ApplyChanges and Records
* fix(digitalocean): fix correct imports, unshadow a variable
* fix(digitalocean): respect domain-filter if provided
* add changes to digital ocean provider from PR review (#3)
* fix tests and bugs in find suitable zones for digital ocean (#5)
* tests failed due to wrong formatting
* add digitalocean integration to the main and more tests
* fix suitable zone for digital ocean
* vendor Azure Go SDK (#210)
* vendor the Azure Go SDK and dependencies
* add initial Azure DNS provider implementation (#210)
* add 'azure' value to 'provider' command line option
* add 'azure-config-file' command line option
* add 'azure-resource-group' command line option
* implement initial Azure DNS provider
note: azure provider is not yet fully implemented (does not query for existing
records).
tests and documentation are forthcoming.
* add a tutorial for the Azure provider (#210)
* add tutorial for using ExternalDNS with Azure DNS
* finish implementation of Azure DNS provider (#210)
* implement the Records method for the Azure DNS provider
* refactor Azure API interface for future tests (#210)
* make Azure provider use an interface for future unit tests
* add unit tests for the Azure provider (#210)
* test retrieving Azure DNS records.
* test updating and deleting Azure DNS records.
* test dry run for the Azure provider (i.e. noop).
* Expose inmemory provider to cli
So we can test `--source fake` without needing to touch AWS/Google.
* Add FakeSource
`external-dns --provider inmemory --source fake --dry-run --once`
OR
`external-dns --provider aws --source fake --fqdn-template <hostname suffix> --dry-run --once`
NB: `--fqdn-template` because otherwise we'll default to creating, e.g.,
`abcd.example.com`, which `--provider aws` filters out because you
likely don't have a Zone for `example.com.` Could also be resolved by
removing the need to use a real provider; the inmemory provider,
perhaps, though it's not entirely hooked up.
Closeskubernetes-incubator/external-dns#22
* Style feedback from Travis CI
* Improve optionality of kubernetes client
* ref(sources): refactor source registration and lookup to be lazy.
* Revert "ref: refactor source registration/lookup to be lazily initialized"
