**Description**
In the current implementation, DNS providers are called to list all
records on every loop. This is expensive in terms of number of requests
to the provider and may result in being rate limited, as reported in 1293
and 3397.
In our case, we have approximately 20,000 records in our AWS Hosted Zone.
The ListResourceRecordSets API call allows a maximum of 300 items per call.
That requires 67 API calls per external-dns deployment during every sync period
With this, we introduce an optional generic caching mechanism at the provider
level, that re-uses the latest known list of records for a given time.
This prevents from expensive Provider calls to list all records for each
object modification that does not change the actual record (annotations,
statuses, ingress routing, ...)
This introduces 2 trade-offs:
1. Any changes or corruption directly on the provider side will be
longer to detect and to resolve, up to the cache time
2. Any conflicting records in the DNS provider (such as a different
external-dns instance) injected during the cache validity will cause
the first iteration of the next reconcile loop to fail, and hence add a
delay until the next retry
**Checklist**
- [X] Unit tests updated
- [X] End user documentation updated
Change-Id: I0bdcfa994ac1b76acedb05d458a97c080284c5aa
