The number of objects returned by the infoblox api is limited to 1000 objects
(see https://ipam.illinois.edu/wapidoc). If there are more then 1000 objects
the API returns an error. By setting max-results one can raise the limit.
number of retries that API calls will attempt before giving up.
This somewhat mitigates the issues discussed in #484 by allowing
the current sync attempt to complete vs. failing and starting anew.
Defaults to 3, which is what the aws-sdk-go defaults to where not
specified.
Signed-off-by: Joe Hohertz <joe@viafoura.com>
This is to change the way batching works when using the aws provider.
Originally, batching would take the first n records you want to update
and perform the desired actions on those records as part of a sync. It
would then wait for the configured sync period and take the first n
records again and sync them. The issue with this is that when you are
using the TXT registry with a custom prefix, the updates can sync a TXT
record and not the accompanying A/CNAME record. This causes external-dns
to get out of sync with what is created and what the current state
actually is. This update uses the same idea of batching, however, rather
than stopping after the first batch until the next run, batching will
now have a separate batch interval which controls the interval between
each batch in the same sync period. This allows external-dns to fully
sync with route53 as part of each sync and can then know that the state
is complete.
Fixes https://github.com/kubernetes-incubator/external-dns/issues/679
* add Istio Gateway Source
* add documentation for Istio Gateway Source
* make both istio namespace and ingress gateway service configurable
* prefix gateway types, constructors, and flags with 'istio-'
* fix: add missing sources to source flag docs
When running in a pod sometimes the request to get ingreses/services
stalls indefinitely. A simple pod restart fixes this. Hard to reproduce
but I got lucky and did thread dump which revealed a gorouting blocked
on call to k8s.
What's new is a `--request-timeout` flag that makes requests to k8s
bounded in time. The default is 30s - this may cause some deployments
with a slow api-server to timeout.
* Create `NewAWSProvider` with `AWSConfig` struct
Rather than calling `NewAWSProvider` with a list of objects, you will
now call it using a new `AWSConfig` struct. This allows for clearer
declarations of variables which becomes even more important as more
variables are added.
* Add `aws-max-change-count` flag
Adding a new `aws-max-change-count` flag to override the default max
change count on the aws provider.
Included updated tests with a new `defaultMaxChangeCount` constant and
tests for setting the value as a flag and as an environment variable.
* Update CHANGELOG.md
Updating CHANGELOG.md with 'Add aws max change count flag' PR.
Commit adds:
* Implementation of PowerDNS as a provider
* Tests for said implementation
* github.com/ffledgling/pdns-go, which provides go client bindings for
PowerDNS's HTTP API, as a dependency
* "pdns" as an additional option for the `--provider` flag
* `--pdns-server` and `--pdns-api-key` as additional flags for PowerDNS
specific configuration
* Add a flag that allows FQDN and Annotations to combine
Old behaviour is kept by default, a new flag is introduced to combine instead of overwrite
Fixes#218
* docs: add fqdn template combine to changelog
* Graceful handling of misconfigure password for dyn
If a bad password is given for provider "dyn" then the next
login attempt is at least 30minutes apart. This prevents an
account from being suspended.
Improve validation of flags for dyn provider. Add test for
ValidateConfig() and Config.String()
Also add --dyn-min-ttl option which sets the lower limit
of a record's TTL. Ignored if 0 (the default).
* docs: add graceful handling of misconfiguration to changelog
* Don't log passwords on start
The two passwords configurable as flags (for infoblox and dyn) are
masked now and not logged.
* docs: add masking sensitive data in logs to changelog
* add "dyn" provider
* add several --dyn-* args to configure Dyn login
* add github.com/nesv/go-dynect/dynect@0.6.0 to Gopkg and vender/ (the client
of choice by Terraform)
* make externdns.Version public so it can be stored when committing
zone changes
* add tutorial for Ingress resources and update root README.md file
Dyn REST API is documented here: https://help.dyn.com/dns-api-knowledge-base/
Example usage:
external-dns \
--provider=dyn \
--dyn-customer-name=acme \
--dyn-username=acme-api \
--dyn-password=t0pS3cr3t \
--domain-filter=portal.acme.com \
--zone-id-filter=acme.com \
--namespace=my-test-ns \
--log-level=debug \
--txt-prefix=_
* Add aws-zone-id flag
* Add Zone ID filter
* Update AWS provider and main
* Make ZoneIDFilter generic
* Implement ZoneIDFilter for all providers
* Update CHANGELOG
This commit adds ability to use TLS transport for etcd.
New logic is applied when the etcd URL has https:// scheme.
TLS parameters are passed in the environment variables:
ETCD_CA_FILE - path to CA certificate. If not specified, then
system-provided certificates are used.
ETCD_CERT_FILE - client certificate
ETCD_KEY_FILE - client key file
- either both of none of this two must be specified
ETCD_TLS_SERVER_NAME - expected CN of the certificate. Useful when
URL points to a different domain from that in server certificate
ETCD_TLS_INSECURE - if set to "1" (or "true" or "yes") makes client
bypass server certificate validation.
Also for unification with other providers and rest of connection
settings, etcd URL is no longer specified in the command line, but
rather in ETCD_URLS environment variable (defaults to
http://localhost:2379). More than one comma-separated URL can be
specified. All of the URLs must start with either http:// or https://
Also, now it possible to communicate with etcd through proxy specified
in standard environment variables
* allow filtering by ingress class
* generic source annotation filter as opposed to ingress class filter
* rename and fix argument ordering, switch to label selector semantics
* remove redundant parameters
This commit adds support for CoreDNS through its etcd middleware.
Because the middleware is backward compatible with SkyDNS this
commit adds support for SkyDNS as well. In fact, new provider
is available under two names in CLI (coredns and skydns).
All interactions with middleware happen through etcd cluster,
whose location (URIs) is specified via --etcd CLI parameter
by default http://localhost:2379).
The provider translates CoreDNS/DkyDNS SRV records to
A/CNAME + optional TXT endpoints, when reading from etcd and
combines A/CNAME with TXT endpoints back into single SRV record
when writing it back.
Also adds github.com/coreos/etcd package to glide.yaml and vendor folder
because it is used by the provider
- Add `--inmemory-zone flag`
- Implement `InMemoryZoneInit` function to setup initial zones for
inmemory provider
- Make "" the default zone for the inmemory provider instead of
nil/none when no initial zones are specified
- Update config/flag parsing tests to accept new flag
* Initial commit of Infoblox provider
* address @ideahitme's observations
* fail at addressing @szuecs comments
* fix(infoblox): avoid shadowing err variable
* fix flag descriptions
* default ssl verify to true. thanks @szuecs
* chore(infoblox): bump minium required version
This should ideally be a minor bump but let's do that when we have v0.5
* ClusterIP service support
- First pass at addresssing #187 by allowing services with type ClusterIP to be directly supported
* Getting existing tests to pass
* Adjusting formatting for gofmt/govet
* Adding in guard logic around publishing of ClusterIP sources
* Addressing PR feedback
* Adding in CHANGELOG entry
* Adding in Headless service test
* ref(sources): refactor source registration and lookup to be lazy.
* fix(fake): don't make changes to passed in config values
* rework without init, tests are missing
* make client provider public
* fix all tests
* change parameter list order, minor improvements
* clientprovider -> clientgenerator, switch naming for interface/struct
* Support for multiple domains within --domain-filter
The parameter accepts a comma separated list of domains with or without trailing dot. Example: --domain-filter="example.org, company.test.,staging.com". Closes#247 and #229
* Add boilerplate header
* Add documentation for methods and structs
* use StringsVar for the domain-filter flag
* go fmt
* Remove camel case from tests
* Revert changes in README.md
* Move DomainFilter to provider package
* Make a new slice and copy elements to it
* Update CHANGELOG.md
* docs: change minor spelling mistake
* CloudFlare Provider
* updating glide
* gofmt cloudflare_test.go
* Unset envs to test NewCloudFlareProvider
* More tests
* fix(cloudflare): fix compiler errors resulting from merge
* Typo
* Undo vendor changes
* decrease api calls, fix some nits
* Cloudflare iteration (#2)
* reduce the number of API calls
* match by type and name for record id
* improve coverage and fix the bug with suitable zone
* tests failed due to wrong formatting
* add cloudflare integration to the main
* vendor cloudflare deps
* fix cloudflare zone detection + tests
* fix conflicting test function names
* Initial Digital Ocean as a new provider
* chore: vendor dependencies
* Remove zone parameter from ApplyChanges and Records
* fix(digitalocean): fix correct imports, unshadow a variable
* fix(digitalocean): respect domain-filter if provided
* add changes to digital ocean provider from PR review (#3)
* fix tests and bugs in find suitable zones for digital ocean (#5)
* tests failed due to wrong formatting
* add digitalocean integration to the main and more tests
* fix suitable zone for digital ocean
* vendor Azure Go SDK (#210)
* vendor the Azure Go SDK and dependencies
* add initial Azure DNS provider implementation (#210)
* add 'azure' value to 'provider' command line option
* add 'azure-config-file' command line option
* add 'azure-resource-group' command line option
* implement initial Azure DNS provider
note: azure provider is not yet fully implemented (does not query for existing
records).
tests and documentation are forthcoming.
* add a tutorial for the Azure provider (#210)
* add tutorial for using ExternalDNS with Azure DNS
* finish implementation of Azure DNS provider (#210)
* implement the Records method for the Azure DNS provider
* refactor Azure API interface for future tests (#210)
* make Azure provider use an interface for future unit tests
* add unit tests for the Azure provider (#210)
* test retrieving Azure DNS records.
* test updating and deleting Azure DNS records.
* test dry run for the Azure provider (i.e. noop).
* Expose inmemory provider to cli
So we can test `--source fake` without needing to touch AWS/Google.
* Add FakeSource
`external-dns --provider inmemory --source fake --dry-run --once`
OR
`external-dns --provider aws --source fake --fqdn-template <hostname suffix> --dry-run --once`
NB: `--fqdn-template` because otherwise we'll default to creating, e.g.,
`abcd.example.com`, which `--provider aws` filters out because you
likely don't have a Zone for `example.com.` Could also be resolved by
removing the need to use a real provider; the inmemory provider,
perhaps, though it's not entirely hooked up.
Closeskubernetes-incubator/external-dns#22
* Style feedback from Travis CI
* Improve optionality of kubernetes client
* ref(sources): refactor source registration and lookup to be lazy.
* Revert "ref: refactor source registration/lookup to be lazily initialized"
* feat(config): change defaults, switch flag processing to kingpin
* chore: vendor kingpin as a dependency
* feat(config): auto-detect cluster config from the environment
* chore: clean up definition of flags
* chore: sanitize flags even further
* chore: update changelog with latest flags changes
* fix(aws): fix messed up test name
* feat(google): auto-detect and multiple zone support
* chore: run gofmt with the simplified command
* fix: pass desired domain to google provider
* feat(google): correctly auto-detect records for sub-zones
* chore: update changelog with support for multiple zones in google
* fix(google): don't append traling dot to TXT records
* ref(provider): extract hostname sanitization to general provider
* add --fqdn-template
* add missing ,
* gofmt
* no endpoint creation on empty fqdntemplate
* improve test coverage
* gofmt simple on service_test.go and ingress_test.go
* import package order changed
* gofmt
* refactor to generate template in the source init
* refactor for err handling
* fix service tests
* fix wrong check, check for priorities, mate > template
* fix tests, check for controller annotation in the right place
* add to changelog
* add flag description, improve testing, reorganize imports
* review changes: log the error, use text/template, change func interface
* feat(aws): support multiple hosted zones and automatic lookup
* chore: run gofmt with the simplified command
* fix(aws): add missing method from google provider
* fix: remove superflous parameter from google provider
* feat: make domain configurable via flag
* fix(aws): remove unused constant
* fix(aws): don't log actions that were filtered out
* feat(aws): detect best possible zone to put dns entries in
* fix(aws): log error instead of failing if a change batch fails
* chore: update changelog with support for multiple zones
* feat(plan): allow plans to be parameterized via policies
* fix(controller): set a policy in the controller tests
* chore: allow go tip to fail
* chore: update changelog to include policies
* fix(plan): store a pointer to changes
* fix(controller): don't get address of a pointer
* kickoff txt registry
* fix inmemory dns provider to include recordtype info for validation
* Merge master
* fix ununsed variable in inmemory provider
* add tests for records
* add test for no prefix name formatter
* implement apply changes with tests
* add flag to enable txt registry
* add txt registry to main
* improve sort testing
* filter out non-owned records
* NewEndpoint(...) requires record type
* use newendpoint in aws_test, fix tests
* change suitable type implementation
* fix the test for compatibility component
* change inmemory provider to include recordtype and use suitable type
* fix comments, CNAME should target hostname
* name mapper do not use pointer on struct
* txt prefix - just concatenate, remove spew, fix txt record label
* allow TXT records as result from dns provider
* add changelog
* fix tests
* TXT records need to be enclosed in double quotes
* feat: expose basic go metrics via prometheus
* chore: vendor dependencies
* feat(metrics): combine healthz and metrics into single endpoint
* ref(metrics): rename metrics port to address
* docs(metrics): update changelog to include metrics feature
* ref(metrics): consistently rename port to address
* fix(dnsprovider): do not always initialize each DNS provider
* fix(dnsprovider): fix unnecessary error return value
* ref(dnsprovider): drop the store and use a simple switch for lookup
* feat(google): add ability to apply changes generated from a plan
* feat(controller): first implementation of controller
* feat: allow to configure in-cluster and kubeconfig
* fix(controller): call RunOnce at the right time and in a loop
* feat(google): add dryRun attribute to Google DNS provider
* fix: use hosted zone id instead of DNS name
* fix(google): stupidly filter by A records for now
* feat: allow specifying the google project and zone
* feat: provide a dry-run flag which defaults to false
* chore: vendor new dependencies
* fix(config): fix failing tests for config object
* ref(controller): return plain value of ApplyChanges
* ref: simplify how to get a valid kubernetes client
Types that we want to work using apimachinery typically go into
pkg/apis/X, and the validation goes into pkg/apis/X/validation. We then
add versions into e.g. pkg/apis/X/v1alpha1, but this feels premature at
the moment.
Changing this later is annoyingly difficult, especially in terms of
validation and dependencies.
We will want the apimachinery, so that we can configure from a
configuration file that is versioned. Hopefully dns-controller won't
end up so complicated that we will require it, but I think there is also
value in following the "standard" patterns for controllers that are
emerging from e.g. ingress.
For a fairly simple example of an API, please consult
https://github.com/kubernetes/kubernetes/tree/master/pkg/apis/certificates
