If a single change fails during the retry, it will be added to a queue.
In the next iteration, changes from this queue will be submitted after
all other changes.
When submitting single changes, they are always submitted as batches of
changes with the same DNS name and ownership relation to avoid
inconsistency between the record created and the TXT records.
Currently, planning instructs to create all records even
those which does not match any zone.
Later, those records will be checked towards the existing
records and filtered whether they match or not a hosted zone.
This causes a problem, at least in the specific case of the Route53
implementation as it always calls the ApplyChanges method, which in its
turn always retrieves all records in all zones.
This causes high pressure on Route53 APIs, for non-necessary actions.
By being able to filter all unmanaged records from the plan, we can
prevent from calling ApplyChanges when nothing has to be done and hence
prevent an unnecessary listing of records.
By doing so, the rate of API calls to AWS Route53 is expected to be
reduced by 2
From measurements, AWS by default has pagination of 100 items per
page when listing hosted zone resources.
This increases the number of requests required to list all our zones,
and pushes a hard constraint on the rate limits.
From the experiments, it seems that on the server-side, there is a hard
limit of 300 elements per page, as per AWS documentation:
https://docs.aws.amazon.com/Route53/latest/APIReference/API_ListResourceRecordSets.html
> ListResourceRecordSets returns up to 300 resource record sets at a time in ASCII order,
> beginning at a position specified by the name and type elements
Hence raising the page size from 100 to 300 items would decrease by 3
the number of requests posted to Route53
We even set a higher limit so we can benefit from a lower number of
requests if ever AWS increases the hard limit of 300.
* adds flag to opt in for NS records management
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* go fmt
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* goimports
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* fix more tests
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* go fmt again
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* fix test
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* more tests
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* make ordering of managed records consistent
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
* fix flag
Signed-off-by: Raffaele Di Fazio <difazio.raffaele@gmail.com>
