This change disables the CRD source's informer during tests. I made the mistake
of not running `make test` before the previous commit, and thus didn't realize
that leaving the informer enabled during the tests introduced a race condition:
WARNING: DATA RACE
Write at 0x00c0005aa130 by goroutine 59:
k8s.io/client-go/rest/fake.(*RESTClient).do()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/fake/fake.go:113 +0x69
k8s.io/client-go/rest/fake.(*RESTClient).do-fm()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/fake/fake.go:109 +0x64
k8s.io/client-go/rest/fake.roundTripperFunc.RoundTrip()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/fake/fake.go:43 +0x3d
net/http.send()
/usr/local/go/src/net/http/client.go:251 +0x6da
net/http.(*Client).send()
/usr/local/go/src/net/http/client.go:175 +0x1d5
net/http.(*Client).do()
/usr/local/go/src/net/http/client.go:717 +0x2cb
net/http.(*Client).Do()
/usr/local/go/src/net/http/client.go:585 +0x68b
k8s.io/client-go/rest.(*Request).request()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/request.go:855 +0x209
k8s.io/client-go/rest.(*Request).Do()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/request.go:928 +0xf0
sigs.k8s.io/external-dns/source.(*crdSource).List()
/Users/erath/go/src/github.com/ericrrath/external-dns/source/crd.go:250 +0x28c
sigs.k8s.io/external-dns/source.NewCRDSource.func1()
/Users/erath/go/src/github.com/ericrrath/external-dns/source/crd.go:125 +0x10a
k8s.io/client-go/tools/cache.(*ListWatch).List()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/listwatch.go:106 +0x94
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func1.1.2()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:233 +0xf4
k8s.io/client-go/tools/pager.SimplePageFunc.func1()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/pager/pager.go:40 +0x94
k8s.io/client-go/tools/pager.(*ListPager).List()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/pager/pager.go:91 +0x1f4
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func1.1()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:258 +0x2b7
Previous write at 0x00c0005aa130 by goroutine 37:
k8s.io/client-go/rest/fake.(*RESTClient).do()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/fake/fake.go:113 +0x69
k8s.io/client-go/rest/fake.(*RESTClient).do-fm()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/fake/fake.go:109 +0x64
k8s.io/client-go/rest/fake.roundTripperFunc.RoundTrip()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/fake/fake.go:43 +0x3d
net/http.send()
/usr/local/go/src/net/http/client.go:251 +0x6da
net/http.(*Client).send()
/usr/local/go/src/net/http/client.go:175 +0x1d5
net/http.(*Client).do()
/usr/local/go/src/net/http/client.go:717 +0x2cb
net/http.(*Client).Do()
/usr/local/go/src/net/http/client.go:585 +0x68b
k8s.io/client-go/rest.(*Request).request()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/request.go:855 +0x209
k8s.io/client-go/rest.(*Request).Do()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/rest/request.go:928 +0xf0
sigs.k8s.io/external-dns/source.(*crdSource).List()
/Users/erath/go/src/github.com/ericrrath/external-dns/source/crd.go:250 +0x28c
sigs.k8s.io/external-dns/source.(*crdSource).Endpoints()
/Users/erath/go/src/github.com/ericrrath/external-dns/source/crd.go:171 +0x13c4
sigs.k8s.io/external-dns/source.testCRDSourceEndpoints.func1()
/Users/erath/go/src/github.com/ericrrath/external-dns/source/crd_test.go:388 +0x4f6
testing.tRunner()
/usr/local/go/src/testing/testing.go:1193 +0x202
Goroutine 59 (running) created at:
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func1()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:224 +0x36f
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:316 +0x1ab
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:177 +0x4a
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1()
/Users/erath/go/pkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:155 +0x75
k8s.io/apimachinery/pkg/util/wait.BackoffUntil()
/Users/erath/go/pkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:156 +0xba
k8s.io/client-go/tools/cache.(*Reflector).Run()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:176 +0xee
k8s.io/client-go/tools/cache.(*Reflector).Run-fm()
/Users/erath/go/pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:174 +0x54
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
/Users/erath/go/pkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:56 +0x45
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1()
/Users/erath/go/pkg/mod/k8s.io/apimachinery@v0.18.8/pkg/util/wait/wait.go:73 +0x6d
Goroutine 37 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:1238 +0x5d7
sigs.k8s.io/external-dns/source.testCRDSourceEndpoints()
/Users/erath/go/src/github.com/ericrrath/external-dns/source/crd_test.go:376 +0x1fcf
testing.tRunner()
/usr/local/go/src/testing/testing.go:1193 +0x202
It looks like client-go's fake.RESTClient (used by crd_test.go) is known to
cause race conditions when used with informers:
<https://github.com/kubernetes/kubernetes/issues/95372>. None of the CRD tests
_depend_ on the informer yet, so disabling the informer at least allows the
existing tests to pass without race conditions. I'll look into further changes
that 1) test the new event-handler behavior, and 2) allow all tests to pass
without race conditions.
When the --events flag is passed at startup, Source.AddEventHandler() is called
on each configured source. Most sources provide AddEventHandler()
implementations that invoke the reconciliation loop when the configured source
changes, but the CRD source had a no-op implementation. I.e. when a custom
resource was created, updated, or deleted, external-dns remained unware, and the
reconciliation loop would not fire until the configured interval had passed.
This change adds an informer (on the CRD specified by --crd-source-apiversion
and --crd-source-kind=DNSEndpoint), and a Source.AddEventHandler()
implementation that calls Informer.AddEventHandler(). Now when a custom
resource is created, updated, or deleted, the reconciliation loop is invoked.
This feature adds a router field to the OCP Route Source so that a user can add an ingress controller name in flag --ocp-router-name which will be used to pick up the respective routerCanonicalHostname from Route's Status Ingress Object.
Signed-off-by: Miheer Salunke <miheer.salunke@gmail.com>
Currently the `--label-filter` flag can only be used to filter CRDs
which match the label selector passed through that flag. This change
extends the functionality to the Ingress, Service and Openshift Route
type objects. When the flag is not specified the default value is
`labels.Everything()` which is an empty string, the same as before.
Annotation based filter is inefficient because the filtering has to be
done in the controller instead of the API server like with label
filtering.
we don't want to get incompatible restrictions by letting someone set
"--ingress-class" and --annotation-filter="kubernetes.io/ingress.class"
at the same time
source/ocproute_test.go:
New file for source/ocproute.go unit tests.
Inspired by the service source unit tests,
this commit adds basic unit tests to verify
basic functionality of the OCP route source.
Signed-off-by: Stephen Greene <sgreene@redhat.com>
source/ocproute.go:
Add/clean up godocs and general in-line comments.
Change `routeapi` named import to `routev1`.
Update `routev1` calling sites accordingly.
Signed-off-by: Stephen Greene <sgreene@redhat.com>
Ambassador can be configured with `Host` resources (based on the
`Host` CRD) for defining the external DNS host name.
This code adds a new source, `ambassador-host`, that looks for the
`ambassador/ambassador` Service and and uses the `hostname` from the
`Host` resource.
Signed-off-by: Alvaro Saurin <alvaro.saurin@gmail.com>
Signed-off-by: Flynn <flynn@datawire.io>
* adds tests for shouldUpdateProviderSpecific
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* move AWS health to where it belongs
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* add test that breaks things
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* adds adjustendpoints method
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* fix controller
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* actually pass the provider where needed
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* OMG goland do your go fmt thing
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* use registry as proxy
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* make linter happy
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* change AdjustEndpoints signature
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* fix typo
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* actually use adjusted endpoints
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* revert cloudflare change
Signed-off-by: Raffaele Di Fazio <raffo@github.com>
* Update provider/cloudflare/cloudflare.go
Co-authored-by: Nick Jüttner <nick@juni.io>
Co-authored-by: Nick Jüttner <nick@juni.io>
Currently, the endpoints of headless services are retrieved
by querying pods using the pod selector of the service.
Instead, we now query for an Endpoints resource with the
same name as the Service object to get the endpoints for the
service. This is needed in order to support the
spec.publishNotReadyPods attribute of a service.
One feature that's supported for the Istio Gateway resource is the
ability to restrict what namespace may provide a VirtualService for
a given hostname/domain by specifying it in the form
`my-namespace/my-domain.com` which restricts the availability of
`my-domain.com` to the `my-namespace` namespace.
When creating the associated DNS records, the namespace should not
be included.
In its current state, ttlMaximum causes make to fail build with error:
source/source.go:71:39: constant 4294967295 overflows int
Because math.MaxUint32 is an untyped integer constant it is converted
to type int when used as an interface param, type int is signed,
MaxUint32 is larger than MaxInt32 and causes the type int to overflow.
Converting ttlMaximum to MaxInt32 stops this overflow.
Added warning when CR has a target with a "." in the end, which would make external-dns contantly add and remove the record
Signed-off-by: João Marçal <joao.marcal12@gmail.com>
* add Istio Gateway Source
* add documentation for Istio Gateway Source
* make both istio namespace and ingress gateway service configurable
* prefix gateway types, constructors, and flags with 'istio-'
* fix: add missing sources to source flag docs
When running in a pod sometimes the request to get ingreses/services
stalls indefinitely. A simple pod restart fixes this. Hard to reproduce
but I got lucky and did thread dump which revealed a gorouting blocked
on call to k8s.
What's new is a `--request-timeout` flag that makes requests to k8s
bounded in time. The default is 30s - this may cause some deployments
with a slow api-server to timeout.
Non statefulset pods associating to a headless service have different
IPs, but have a same hostname. In this case, external-dns registered
only one A record due to attempting to register multiple A records for
a same hostname for each IP.
This patch now registers one A record having multiple IPs.
* First stab at NodePort support. Testing incomplete
* Fix up the unit tests
* Remove some deadcode in the unittests
* gather node ips once and add support for srv records
* Make sure we match gofmt simple
* Move the nodes to the testcase and add a test for clusters that only have internal ip addresses
* Somehow forgot about the weight field in the records
* Add SRV as a supported record type
* Add a flag that allows FQDN and Annotations to combine
Old behaviour is kept by default, a new flag is introduced to combine instead of overwrite
Fixes#218
* docs: add fqdn template combine to changelog
* Support a comma separated list for the FQDN template
* Add documentation for comma separated FQDN templates
* chore: add multiple fqdn templates to changelog
* Endpoint.Target is now Endpoint.Targets. This is its own type representing mutliple targets for a single DNS name while adding some convenience for sorting and comparing
* Made everything compile and tests run through with the new Endpoint.Targets
* The ingress source can now properly handle multiple target ips per host
* Added custom conflict resolver, to better understand how conflict resolution has to work for me
* My custom conflict resolver behaves a bit different than the PerResource resolver, therefore I needed to modify the expected test result
Removed unnecessary FIXME
* The ingress source now creates CNAME endpoints with multiple targets to let the DNS provider decide how to handle multiple CNAME targets. This could be interesting for weighted targets etc.
* Adopted the expected results to the new way we create endpoints for CNAMEs
* Removed Add method from Targets since manipulating the slice through here is unnecessary complicated and doesn't deliver enough convenience
* Reverted ConflictResolver to the original one. There is some discussing to do what the best way is to handle conflicts
* Added missing documenting comment to IsLess of Targets
* Added documenting comments to Targets,Targets.Same and NewTargets to clarify their intention and usage
* Service source now also generates endpoints with multiple targets
* Service and Ingress source now sort all Targets for every Endpoint to make order of Targets predictable
* Endpoints generated by the Google Cloud DNS provider now also have sorted Targets to make order of Targets predictable
* Modified provider dyn to be able to compile with multi target changes
* Fixed small nitpicks, so my code is acceptable
* Fixed merge method after updating to new Targets. Replacing '!=' with .Same of course needs a boolean negation
* Tests for dyn provider now also use the new Targets instead of Target
* Simplified extractServiceIps as implied by linki to make it more readable
* ref: change service ClusterIP retrieval again
* Added entry to CHANGELOG.md describing the new features contained in this PR
* Make suitableType() be Endpoint method
With this change it becomes possible to work with endpoint
of empty type in packages other than "provider". Also
it seems logical for a smart property getter without side effects
to be a method rather than a function in different package
* Make plan computation work correctly with multi-target domains
* fix drawing
* drop comments
* fix boilerplate header
* fix comment
* fix the bug with empty map
* rework registry to support random lables
* serialize->serializeLabel function rename
* golint for err variable naming
* add additional test
* add tests for current case where one resource can generate multiple endpoints
* make labels have its own type, add serialization as a method
* add comment for exported error
* use greater rather than not equal zero
* update changelog
* Added initial support for Headless services
* service.go: Fixed some formatting
* forgot to run gometalinter
* service: implemented alternative proposal, to reuse existing annotation
* source: some refactoring
* allow filtering by ingress class
* generic source annotation filter as opposed to ingress class filter
* rename and fix argument ordering, switch to label selector semantics
* remove redundant parameters
* Add RecordTTL
* Route53: test for custom TTL
* Fix tests
* Fix remaining tests
* Add ttl when endpoint is created from ingress
* Missed a word
* Fix bad refactoring
* Add ingress custom TTL test
* gofmt
* Satisfy go-lint
* Unshadow `endpoint` in azure provider
* Fix and add an output test
* Add TTL for endpoints generated from service templates
* Take TTL into account when generating update plan
* Tests for TTL change impact on the plan
* Refactor factory method name
* Refactoring
* Run gofmt
* Make endpoint string format look like BIND config
* Update plan and plan_test
* Replace NewEndpointWithTTLValue with NewEndpointWithTTL in aws
* Remove NewEndpointWithTTLValue func
* Update references to TTL
* Remove getTTLValue func
* Handle merge conflict
* Update tests
* Update README, CHANGELOG and documentation
* Run gofmt
* Move getTTLFromAnnotations to a common file
* Refactor getTTLFromAnnotations
* Gofmt
* Add tests for getTTLFromAnnotations
* Trigger build
* Add boilerplate header
* Update README/CHANGELOG according to code review
* Add ttl.md and link it from README
* change CNAME string to endpoint.RecordTypeCNAME
* fix test cases with AWS ALIAS records, these do not behave different in these tests
* ClusterIP service support
- First pass at addresssing #187 by allowing services with type ClusterIP to be directly supported
* Getting existing tests to pass
* Adjusting formatting for gofmt/govet
* Adding in guard logic around publishing of ClusterIP sources
* Addressing PR feedback
* Adding in CHANGELOG entry
* Adding in Headless service test
If the user has specified a target for the ingress, treat that as
overriding any endpoints already set on the ingress, even if that list
is not empty. This allows overriding the IP address or hostname set
when using a service like kube-keepalived-vip.
* service source: support for multiple hostnames per annotation
* go fmt
* Make parseHostnameAnnontations inline
* Update CHANGELOG.md
* Update Changelog
* ref(sources): refactor source registration and lookup to be lazy.
* fix(fake): don't make changes to passed in config values
* rework without init, tests are missing
* make client provider public
* fix all tests
* change parameter list order, minor improvements
* clientprovider -> clientgenerator, switch naming for interface/struct
* ref(source): use testify with mocks in test code
* fix: re-introduce NewMockSource for convenience
* fix: avoid circular dependency
* ref: increase usage of testify
* chore: vendor testify as a dependency
* fix(*): cleanup testify expectations
* Expose inmemory provider to cli
So we can test `--source fake` without needing to touch AWS/Google.
* Add FakeSource
`external-dns --provider inmemory --source fake --dry-run --once`
OR
`external-dns --provider aws --source fake --fqdn-template <hostname suffix> --dry-run --once`
NB: `--fqdn-template` because otherwise we'll default to creating, e.g.,
`abcd.example.com`, which `--provider aws` filters out because you
likely don't have a Zone for `example.com.` Could also be resolved by
removing the need to use a real provider; the inmemory provider,
perhaps, though it's not entirely hooked up.
Closeskubernetes-incubator/external-dns#22
* Style feedback from Travis CI
* Improve optionality of kubernetes client
* ref(sources): refactor source registration and lookup to be lazy.
* Revert "ref: refactor source registration/lookup to be lazily initialized"
* continue on controller loop error
* add logging in source
* use formatter on logChanges for endpoints
* fix log messages, log skipped records
* add logging in aws, uppercase the rest
* respect google dry run policy
* add ing/svc namespace/name on logging
* fix error logging on template failure
* fix bugs, propagate template error
* log if nothing is being updated, debug log skipped endpoints
* change zone-not-found logging order
* add --fqdn-template
* add missing ,
* gofmt
* no endpoint creation on empty fqdntemplate
* improve test coverage
* gofmt simple on service_test.go and ingress_test.go
* import package order changed
* gofmt
* refactor to generate template in the source init
* refactor for err handling
* fix service tests
* fix wrong check, check for priorities, mate > template
* fix tests, check for controller annotation in the right place
* add to changelog
* add flag description, improve testing, reorganize imports
* review changes: log the error, use text/template, change func interface
* kickoff txt registry
* fix inmemory dns provider to include recordtype info for validation
* Merge master
* fix ununsed variable in inmemory provider
* add tests for records
* add test for no prefix name formatter
* implement apply changes with tests
* add flag to enable txt registry
* add txt registry to main
* improve sort testing
* filter out non-owned records
* NewEndpoint(...) requires record type
* use newendpoint in aws_test, fix tests
* change suitable type implementation
* fix the test for compatibility component
* change inmemory provider to include recordtype and use suitable type
* fix comments, CNAME should target hostname
* name mapper do not use pointer on struct
* txt prefix - just concatenate, remove spew, fix txt record label
* allow TXT records as result from dns provider
* add changelog
* fix tests
* TXT records need to be enclosed in double quotes
* feat(aws): support hostnames as endpoint targets
* docs: describe how to run ExternalDNS on AWS
* docs: update changelog with CNAME feature
* docs: update changelog to include AWS documentation
* fix(aws): test that updating records removes the old value
* feat(google): add CNAME support to Google provider
* fix(source): sanitize source and target hostnames
* docs: update changelog to include latest changes
* docs(aws): mention that ExternalDNS takes full ownership of a hosted zone
* fix(aws): switch route53 tests to use endpoint pointers
* docs: add TODO to remove record filtering once ownership is in place
* feat(source): promote MockSource to source package
* feat(source): add MultiSource to merge other Sources
* feat(source): make structs and members of multiSource private
