mirrors/external-dns
1
0
Fork 0
mirror of https://github.com/kubernetes-sigs/external-dns.git synced 2025-08-07 01:56:57 +02:00
Code Issues Packages Projects Releases Wiki Activity

Refactor azure private dns auth

Browse Source 
Add common config to be shared by both azure and azure-private-dns providers
Update tests & docs
This commit is contained in:
Jonas-Taha El Sesiy 2021-01-08 12:01:10 -08:00
parent 6a7fb3a9a7
commit c851a7973e
No known key found for this signature in database
GPG Key ID: C55D457A823AEAB1
7 changed files with 299 additions and 228 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
docs/tutorials/azure-private-dns.md
View File

@ -1,12 +1,12 @@
# Set up ExternalDNS for Azure Private DNS # Set up ExternalDNS for Azure Private DNS
This tutorial describes how to set up ExternalDNS for managing records in Azure Private DNS. This tutorial describes how to set up ExternalDNS for managing records in Azure Private DNS.
It comprises of the following steps: It comprises of the following steps:
1) Install NGINX Ingress Controller 1) Install NGINX Ingress Controller
2) Provision Azure Private DNS 2) Provision Azure Private DNS
3) Configure service principal for managing the zone 3) Configure service principal for managing the zone
4) Deploy ExternalDNS 4) Deploy ExternalDNS
Everything will be deployed on Kubernetes. Everything will be deployed on Kubernetes.
Therefore, please see the subsequent prerequisites. Therefore, please see the subsequent prerequisites.
@ -26,25 +26,27 @@ $ helm install stable/nginx-ingress \
--name nginx-ingress \ --name nginx-ingress \
--set controller.publishService.enabled=true --set controller.publishService.enabled=true
``` ```
The parameter `controller.publishService.enabled` needs to be set to `true.` The parameter `controller.publishService.enabled` needs to be set to `true.`
It will make the ingress controller update the endpoint records of ingress-resources to contain the external-ip of the loadbalancer serving the ingress-controller. It will make the ingress controller update the endpoint records of ingress-resources to contain the external-ip of the loadbalancer serving the ingress-controller.
This is crucial as ExternalDNS reads those endpoints records when creating DNS-Records from ingress-resources. This is crucial as ExternalDNS reads those endpoints records when creating DNS-Records from ingress-resources.
In the subsequent parameter we will make use of this. If you don't want to work with ingress-resources in your later use, you can leave the parameter out. In the subsequent parameter we will make use of this. If you don't want to work with ingress-resources in your later use, you can leave the parameter out.
Verify the correct propagation of the loadbalancer's ip by listing the ingresses. Verify the correct propagation of the loadbalancer's ip by listing the ingresses.
``` ```
$ kubectl get ingress $ kubectl get ingress
``` ```
The address column should contain the ip for each ingress. ExternalDNS will pick up exactly this piece of information. The address column should contain the ip for each ingress. ExternalDNS will pick up exactly this piece of information.
``` ```
NAME HOSTS ADDRESS PORTS AGE NAME HOSTS ADDRESS PORTS AGE
nginx1 sample1.aks.com 52.167.195.110 80 6d22h nginx1 sample1.aks.com 52.167.195.110 80 6d22h
nginx2 sample2.aks.com 52.167.195.110 80 6d21h nginx2 sample2.aks.com 52.167.195.110 80 6d21h
``` ```
If you do not want to deploy the ingress controller with Helm, ensure to pass the following cmdline-flags to it through the mechanism of your choice: If you do not want to deploy the ingress controller with Helm, ensure to pass the following cmdline-flags to it through the mechanism of your choice:
``` ```
@ -144,6 +146,8 @@ This is per default done through the file `~/.kube/config`.
For general background information on this see [kubernetes-docs](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/). For general background information on this see [kubernetes-docs](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/).
Azure-CLI features functionality for automatically maintaining this file for AKS-Clusters. See [Azure-Docs](https://docs.microsoft.com/de-de/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials). Azure-CLI features functionality for automatically maintaining this file for AKS-Clusters. See [Azure-Docs](https://docs.microsoft.com/de-de/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials).
Follow the steps for [azure-dns provider](./azure.md#creating-configuration-file) to create a configuration file.
Then apply one of the following manifests depending on whether you use RBAC or not. Then apply one of the following manifests depending on whether you use RBAC or not.
The credentials of the service principal are provided to ExternalDNS as environment-variables. The credentials of the service principal are provided to ExternalDNS as environment-variables.
@ -175,13 +179,14 @@ spec:
- --provider=azure-private-dns - --provider=azure-private-dns
- --azure-resource-group=externaldns - --azure-resource-group=externaldns
- --azure-subscription-id=<use the id of your subscription> - --azure-subscription-id=<use the id of your subscription>
env: volumeMounts:
- name: AZURE_TENANT_ID - name: azure-config-file
value: "<use the tenantId discovered during creation of service principal>" mountPath: /etc/kubernetes
- name: AZURE_CLIENT_ID readOnly: true
value: "<use the aadClientId discovered during creation of service principal>" volumes:
- name: AZURE_CLIENT_SECRET - name: azure-config-file
value: "<use the aadClientSecret discovered during creation of service principal>" secret:
secretName: azure-config-file
``` ```
### Manifest (for clusters with RBAC enabled, cluster access) ### Manifest (for clusters with RBAC enabled, cluster access)
@ -200,7 +205,7 @@ rules:
resources: ["services","endpoints","pods"] resources: ["services","endpoints","pods"]
verbs: ["get","watch","list"] verbs: ["get","watch","list"]
- apiGroups: ["extensions","networking.k8s.io"] - apiGroups: ["extensions","networking.k8s.io"]
resources: ["ingresses"] resources: ["ingresses"]
verbs: ["get","watch","list"] verbs: ["get","watch","list"]
- apiGroups: [""] - apiGroups: [""]
resources: ["nodes"] resources: ["nodes"]
@ -245,13 +250,14 @@ spec:
- --provider=azure-private-dns - --provider=azure-private-dns
- --azure-resource-group=externaldns - --azure-resource-group=externaldns
- --azure-subscription-id=<use the id of your subscription> - --azure-subscription-id=<use the id of your subscription>
env: volumeMounts:
- name: AZURE_TENANT_ID - name: azure-config-file
value: "<use the tenantId discovered during creation of service principal>" mountPath: /etc/kubernetes
- name: AZURE_CLIENT_ID readOnly: true
value: "<use the aadClientId discovered during creation of service principal>" volumes:
- name: AZURE_CLIENT_SECRET - name: azure-config-file
value: "<use the aadClientSecret discovered during creation of service principal>" secret:
secretName: azure-config-file
``` ```
### Manifest (for clusters with RBAC enabled, namespace access) ### Manifest (for clusters with RBAC enabled, namespace access)
@ -315,13 +321,14 @@ spec:
- --provider=azure-private-dns - --provider=azure-private-dns
- --azure-resource-group=externaldns - --azure-resource-group=externaldns
- --azure-subscription-id=<use the id of your subscription> - --azure-subscription-id=<use the id of your subscription>
env: volumeMounts:
- name: AZURE_TENANT_ID - name: azure-config-file
value: "<use the tenantId discovered during creation of service principal>" mountPath: /etc/kubernetes
- name: AZURE_CLIENT_ID readOnly: true
value: "<use the aadClientId discovered during creation of service principal>" volumes:
- name: AZURE_CLIENT_SECRET - name: azure-config-file
value: "<use the aadClientSecret discovered during creation of service principal>" secret:
secretName: azure-config-file
``` ```
Create the deployment for ExternalDNS: Create the deployment for ExternalDNS:

2
main.go
View File

@ -191,7 +191,7 @@ func main() {
case "azure-dns", "azure": case "azure-dns", "azure":
p, err = azure.NewAzureProvider(cfg.AzureConfigFile, domainFilter, zoneNameFilter, zoneIDFilter, cfg.AzureResourceGroup, cfg.AzureUserAssignedIdentityClientID, cfg.DryRun) p, err = azure.NewAzureProvider(cfg.AzureConfigFile, domainFilter, zoneNameFilter, zoneIDFilter, cfg.AzureResourceGroup, cfg.AzureUserAssignedIdentityClientID, cfg.DryRun)
case "azure-private-dns": case "azure-private-dns":
p, err = azure.NewAzurePrivateDNSProvider(domainFilter, zoneIDFilter, cfg.AzureResourceGroup, cfg.AzureSubscriptionID, cfg.DryRun) p, err = azure.NewAzurePrivateDNSProvider(cfg.AzureConfigFile, domainFilter, zoneIDFilter, cfg.AzureResourceGroup, cfg.AzureUserAssignedIdentityClientID, cfg.DryRun)
case "vinyldns": case "vinyldns":
p, err = vinyldns.NewVinylDNSProvider(domainFilter, zoneIDFilter, cfg.DryRun) p, err = vinyldns.NewVinylDNSProvider(domainFilter, zoneIDFilter, cfg.DryRun)
case "vultr": case "vultr":

147
provider/azure/azure.go
View File

@ -19,17 +19,12 @@ package azure
import ( import (
"context" "context"
"fmt" "fmt"
"io/ioutil"
"strings" "strings"
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
yaml "gopkg.in/yaml.v2"
"github.com/Azure/azure-sdk-for-go/services/dns/mgmt/2018-05-01/dns" "github.com/Azure/azure-sdk-for-go/services/dns/mgmt/2018-05-01/dns"
"github.com/Azure/go-autorest/autorest" "github.com/Azure/go-autorest/autorest"
"github.com/Azure/go-autorest/autorest/adal"
"github.com/Azure/go-autorest/autorest/azure"
"github.com/Azure/go-autorest/autorest/to" "github.com/Azure/go-autorest/autorest/to"
"sigs.k8s.io/external-dns/endpoint" "sigs.k8s.io/external-dns/endpoint"
@ -41,18 +36,6 @@ const (
azureRecordTTL = 300 azureRecordTTL = 300
) )
type config struct {
Cloud string `json:"cloud" yaml:"cloud"`
TenantID string `json:"tenantId" yaml:"tenantId"`
SubscriptionID string `json:"subscriptionId" yaml:"subscriptionId"`
ResourceGroup string `json:"resourceGroup" yaml:"resourceGroup"`
Location string `json:"location" yaml:"location"`
ClientID string `json:"aadClientId" yaml:"aadClientId"`
ClientSecret string `json:"aadClientSecret" yaml:"aadClientSecret"`
UseManagedIdentityExtension bool `json:"useManagedIdentityExtension" yaml:"useManagedIdentityExtension"`
UserAssignedIdentityID string `json:"userAssignedIdentityID" yaml:"userAssignedIdentityID"`
}
// ZonesClient is an interface of dns.ZoneClient that can be stubbed for testing. // ZonesClient is an interface of dns.ZoneClient that can be stubbed for testing.
type ZonesClient interface { type ZonesClient interface {
ListByResourceGroupComplete(ctx context.Context, resourceGroupName string, top *int32) (result dns.ZoneListResultIterator, err error) ListByResourceGroupComplete(ctx context.Context, resourceGroupName string, top *int32) (result dns.ZoneListResultIterator, err error)
@ -82,46 +65,22 @@ type AzureProvider struct {
// //
// Returns the provider or an error if a provider could not be created. // Returns the provider or an error if a provider could not be created.
func NewAzureProvider(configFile string, domainFilter endpoint.DomainFilter, zoneNameFilter endpoint.DomainFilter, zoneIDFilter provider.ZoneIDFilter, resourceGroup string, userAssignedIdentityClientID string, dryRun bool) (*AzureProvider, error) { func NewAzureProvider(configFile string, domainFilter endpoint.DomainFilter, zoneNameFilter endpoint.DomainFilter, zoneIDFilter provider.ZoneIDFilter, resourceGroup string, userAssignedIdentityClientID string, dryRun bool) (*AzureProvider, error) {
contents, err := ioutil.ReadFile(configFile) cfg, err := getConfig(configFile, resourceGroup, userAssignedIdentityClientID)
if err != nil {
return nil, fmt.Errorf("failed to read Azure config file '%s': %v", configFile, err)
}
cfg := config{}
err = yaml.Unmarshal(contents, &cfg)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to read Azure config file '%s': %v", configFile, err) return nil, fmt.Errorf("failed to read Azure config file '%s': %v", configFile, err)
} }
// If a resource group was given, override what was present in the config file token, err := getAccessToken(*cfg, cfg.Environment)
if resourceGroup != "" {
cfg.ResourceGroup = resourceGroup
}
// If userAssignedIdentityClientID is provided explicitly, override existing one in config file
if userAssignedIdentityClientID != "" {
cfg.UserAssignedIdentityID = userAssignedIdentityClientID
}
var environment azure.Environment
if cfg.Cloud == "" {
environment = azure.PublicCloud
} else {
environment, err = azure.EnvironmentFromName(cfg.Cloud)
if err != nil {
return nil, fmt.Errorf("invalid cloud value '%s': %v", cfg.Cloud, err)
}
}
token, err := getAccessToken(cfg, environment)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to get token: %v", err) return nil, fmt.Errorf("failed to get token: %v", err)
} }
zonesClient := dns.NewZonesClientWithBaseURI(environment.ResourceManagerEndpoint, cfg.SubscriptionID) zonesClient := dns.NewZonesClientWithBaseURI(cfg.Environment.ResourceManagerEndpoint, cfg.SubscriptionID)
zonesClient.Authorizer = autorest.NewBearerAuthorizer(token) zonesClient.Authorizer = autorest.NewBearerAuthorizer(token)
recordSetsClient := dns.NewRecordSetsClientWithBaseURI(environment.ResourceManagerEndpoint, cfg.SubscriptionID) recordSetsClient := dns.NewRecordSetsClientWithBaseURI(cfg.Environment.ResourceManagerEndpoint, cfg.SubscriptionID)
recordSetsClient.Authorizer = autorest.NewBearerAuthorizer(token) recordSetsClient.Authorizer = autorest.NewBearerAuthorizer(token)
provider := &AzureProvider{ return &AzureProvider{
domainFilter: domainFilter, domainFilter: domainFilter,
zoneNameFilter: zoneNameFilter, zoneNameFilter: zoneNameFilter,
zoneIDFilter: zoneIDFilter, zoneIDFilter: zoneIDFilter,
@ -130,61 +89,7 @@ func NewAzureProvider(configFile string, domainFilter endpoint.DomainFilter, zon
userAssignedIdentityClientID: cfg.UserAssignedIdentityID, userAssignedIdentityClientID: cfg.UserAssignedIdentityID,
zonesClient: zonesClient, zonesClient: zonesClient,
recordSetsClient: recordSetsClient, recordSetsClient: recordSetsClient,
} }, nil
return provider, nil
}
// getAccessToken retrieves Azure API access token.
func getAccessToken(cfg config, environment azure.Environment) (*adal.ServicePrincipalToken, error) {
// Try to retrieve token with service principal credentials.
// Try to use service principal first, some AKS clusters are in an intermediate state that `UseManagedIdentityExtension` is `true`
// and service principal exists. In this case, we still want to use service principal to authenticate.
if len(cfg.ClientID) > 0 &&
len(cfg.ClientSecret) > 0 &&
// due to some historical reason, for pure MSI cluster,
// they will use "msi" as placeholder in azure.json.
// In this case, we shouldn't try to use SPN to authenticate.
!strings.EqualFold(cfg.ClientID, "msi") &&
!strings.EqualFold(cfg.ClientSecret, "msi") {
log.Info("Using client_id+client_secret to retrieve access token for Azure API.")
oauthConfig, err := adal.NewOAuthConfig(environment.ActiveDirectoryEndpoint, cfg.TenantID)
if err != nil {
return nil, fmt.Errorf("failed to retrieve OAuth config: %v", err)
}
token, err := adal.NewServicePrincipalToken(*oauthConfig, cfg.ClientID, cfg.ClientSecret, environment.ResourceManagerEndpoint)
if err != nil {
return nil, fmt.Errorf("failed to create service principal token: %v", err)
}
return token, nil
}
// Try to retrieve token with MSI.
if cfg.UseManagedIdentityExtension {
log.Info("Using managed identity extension to retrieve access token for Azure API.")
msiEndpoint, err := adal.GetMSIVMEndpoint()
if err != nil {
return nil, fmt.Errorf("failed to get the managed service identity endpoint: %v", err)
}
if cfg.UserAssignedIdentityID != "" {
log.Infof("Resolving to user assigned identity, client id is %s.", cfg.UserAssignedIdentityID)
token, err := adal.NewServicePrincipalTokenFromMSIWithUserAssignedID(msiEndpoint, environment.ServiceManagementEndpoint, cfg.UserAssignedIdentityID)
if err != nil {
return nil, fmt.Errorf("failed to create the managed service identity token: %v", err)
}
return token, nil
}
log.Info("Resolving to system assigned identity.")
token, err := adal.NewServicePrincipalTokenFromMSI(msiEndpoint, environment.ServiceManagementEndpoint)
if err != nil {
return nil, fmt.Errorf("failed to create the managed service identity token: %v", err)
}
return token, nil
}
return nil, fmt.Errorf("no credentials provided for Azure API")
} }
// Records gets the current records. // Records gets the current records.
@ -352,20 +257,20 @@ func (p *AzureProvider) mapChanges(zones []dns.Zone, changes *plan.Changes) (azu
func (p *AzureProvider) deleteRecords(ctx context.Context, deleted azureChangeMap) { func (p *AzureProvider) deleteRecords(ctx context.Context, deleted azureChangeMap) {
// Delete records first // Delete records first
for zone, endpoints := range deleted { for zone, endpoints := range deleted {
for _, endpoint := range endpoints { for _, ep := range endpoints {
name := p.recordSetNameForZone(zone, endpoint) name := p.recordSetNameForZone(zone, ep)
if !p.domainFilter.Match(endpoint.DNSName) { if !p.domainFilter.Match(ep.DNSName) {
log.Debugf("Skipping deletion of record %s because it was filtered out by the specified --domain-filter", endpoint.DNSName) log.Debugf("Skipping deletion of record %s because it was filtered out by the specified --domain-filter", ep.DNSName)
continue continue
} }
if p.dryRun { if p.dryRun {
log.Infof("Would delete %s record named '%s' for Azure DNS zone '%s'.", endpoint.RecordType, name, zone) log.Infof("Would delete %s record named '%s' for Azure DNS zone '%s'.", ep.RecordType, name, zone)
} else { } else {
log.Infof("Deleting %s record named '%s' for Azure DNS zone '%s'.", endpoint.RecordType, name, zone) log.Infof("Deleting %s record named '%s' for Azure DNS zone '%s'.", ep.RecordType, name, zone)
if _, err := p.recordSetsClient.Delete(ctx, p.resourceGroup, zone, name, dns.RecordType(endpoint.RecordType), ""); err != nil { if _, err := p.recordSetsClient.Delete(ctx, p.resourceGroup, zone, name, dns.RecordType(ep.RecordType), ""); err != nil {
log.Errorf( log.Errorf(
"Failed to delete %s record named '%s' for Azure DNS zone '%s': %v", "Failed to delete %s record named '%s' for Azure DNS zone '%s': %v",
endpoint.RecordType, ep.RecordType,
name, name,
zone, zone,
err, err,
@ -378,18 +283,18 @@ func (p *AzureProvider) deleteRecords(ctx context.Context, deleted azureChangeMa
func (p *AzureProvider) updateRecords(ctx context.Context, updated azureChangeMap) { func (p *AzureProvider) updateRecords(ctx context.Context, updated azureChangeMap) {
for zone, endpoints := range updated { for zone, endpoints := range updated {
for _, endpoint := range endpoints { for _, ep := range endpoints {
name := p.recordSetNameForZone(zone, endpoint) name := p.recordSetNameForZone(zone, ep)
if !p.domainFilter.Match(endpoint.DNSName) { if !p.domainFilter.Match(ep.DNSName) {
log.Debugf("Skipping update of record %s because it was filtered out by the specified --domain-filter", endpoint.DNSName) log.Debugf("Skipping update of record %s because it was filtered out by the specified --domain-filter", ep.DNSName)
continue continue
} }
if p.dryRun { if p.dryRun {
log.Infof( log.Infof(
"Would update %s record named '%s' to '%s' for Azure DNS zone '%s'.", "Would update %s record named '%s' to '%s' for Azure DNS zone '%s'.",
endpoint.RecordType, ep.RecordType,
name, name,
endpoint.Targets, ep.Targets,
zone, zone,
) )
continue continue
@ -397,20 +302,20 @@ func (p *AzureProvider) updateRecords(ctx context.Context, updated azureChangeMa
log.Infof( log.Infof(
"Updating %s record named '%s' to '%s' for Azure DNS zone '%s'.", "Updating %s record named '%s' to '%s' for Azure DNS zone '%s'.",
endpoint.RecordType, ep.RecordType,
name, name,
endpoint.Targets, ep.Targets,
zone, zone,
) )
recordSet, err := p.newRecordSet(endpoint) recordSet, err := p.newRecordSet(ep)
if err == nil { if err == nil {
_, err = p.recordSetsClient.CreateOrUpdate( _, err = p.recordSetsClient.CreateOrUpdate(
ctx, ctx,
p.resourceGroup, p.resourceGroup,
zone, zone,
name, name,
dns.RecordType(endpoint.RecordType), dns.RecordType(ep.RecordType),
recordSet, recordSet,
"", "",
"", "",
@ -419,9 +324,9 @@ func (p *AzureProvider) updateRecords(ctx context.Context, updated azureChangeMa
if err != nil { if err != nil {
log.Errorf( log.Errorf(
"Failed to update %s record named '%s' to '%s' for DNS zone '%s': %v", "Failed to update %s record named '%s' to '%s' for DNS zone '%s': %v",
endpoint.RecordType, ep.RecordType,
name, name,
endpoint.Targets, ep.Targets,
zone, zone,
err, err,
) )

86
provider/azure/azure_private_dns.go
View File

@ -21,9 +21,8 @@ import (
"fmt" "fmt"
"strings" "strings"
"github.com/Azure/azure-sdk-for-go/profiles/latest/privatedns/mgmt/privatedns" "github.com/Azure/azure-sdk-for-go/services/privatedns/mgmt/2018-09-01/privatedns"
"github.com/Azure/go-autorest/autorest" "github.com/Azure/go-autorest/autorest"
"github.com/Azure/go-autorest/autorest/azure/auth"
"github.com/Azure/go-autorest/autorest/to" "github.com/Azure/go-autorest/autorest/to"
log "github.com/sirupsen/logrus" log "github.com/sirupsen/logrus"
@ -47,44 +46,43 @@ type PrivateRecordSetsClient interface {
// AzurePrivateDNSProvider implements the DNS provider for Microsoft's Azure Private DNS service // AzurePrivateDNSProvider implements the DNS provider for Microsoft's Azure Private DNS service
type AzurePrivateDNSProvider struct { type AzurePrivateDNSProvider struct {
provider.BaseProvider provider.BaseProvider
domainFilter endpoint.DomainFilter domainFilter endpoint.DomainFilter
zoneIDFilter provider.ZoneIDFilter zoneIDFilter provider.ZoneIDFilter
dryRun bool dryRun bool
subscriptionID string resourceGroup string
resourceGroup string userAssignedIdentityClientID string
zonesClient PrivateZonesClient zonesClient PrivateZonesClient
recordSetsClient PrivateRecordSetsClient recordSetsClient PrivateRecordSetsClient
} }
// NewAzurePrivateDNSProvider creates a new Azure Private DNS provider. // NewAzurePrivateDNSProvider creates a new Azure Private DNS provider.
// //
// Returns the provider or an error if a provider could not be created. // Returns the provider or an error if a provider could not be created.
func NewAzurePrivateDNSProvider(domainFilter endpoint.DomainFilter, zoneIDFilter provider.ZoneIDFilter, resourceGroup string, subscriptionID string, dryRun bool) (*AzurePrivateDNSProvider, error) { func NewAzurePrivateDNSProvider(configFile string, domainFilter endpoint.DomainFilter, zoneIDFilter provider.ZoneIDFilter, resourceGroup, userAssignedIdentityClientID string, dryRun bool) (*AzurePrivateDNSProvider, error) {
authorizer, err := auth.NewAuthorizerFromEnvironment() cfg, err := getConfig(configFile, resourceGroup, userAssignedIdentityClientID)
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("failed to read Azure config file '%s': %v", configFile, err)
} }
settings, err := auth.GetSettingsFromEnvironment() token, err := getAccessToken(*cfg, cfg.Environment)
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("failed to get token: %v", err)
} }
zonesClient := privatedns.NewPrivateZonesClientWithBaseURI(settings.Environment.ResourceManagerEndpoint, subscriptionID) zonesClient := privatedns.NewPrivateZonesClientWithBaseURI(cfg.Environment.ResourceManagerEndpoint, cfg.SubscriptionID)
zonesClient.Authorizer = authorizer zonesClient.Authorizer = autorest.NewBearerAuthorizer(token)
recordSetsClient := privatedns.NewRecordSetsClientWithBaseURI(settings.Environment.ResourceManagerEndpoint, subscriptionID) recordSetsClient := privatedns.NewRecordSetsClientWithBaseURI(cfg.Environment.ResourceManagerEndpoint, cfg.SubscriptionID)
recordSetsClient.Authorizer = authorizer recordSetsClient.Authorizer = autorest.NewBearerAuthorizer(token)
provider := &AzurePrivateDNSProvider{ return &AzurePrivateDNSProvider{
domainFilter: domainFilter, domainFilter: domainFilter,
zoneIDFilter: zoneIDFilter, zoneIDFilter: zoneIDFilter,
dryRun: dryRun, dryRun: dryRun,
subscriptionID: subscriptionID, resourceGroup: cfg.ResourceGroup,
resourceGroup: resourceGroup, userAssignedIdentityClientID: cfg.UserAssignedIdentityID,
zonesClient: zonesClient, zonesClient: zonesClient,
recordSetsClient: recordSetsClient, recordSetsClient: recordSetsClient,
} }, nil
return provider, nil
} }
// Records gets the current records. // Records gets the current records.
@ -256,16 +254,16 @@ func (p *AzurePrivateDNSProvider) deleteRecords(ctx context.Context, deleted azu
log.Debugf("Records to be deleted: %d", len(deleted)) log.Debugf("Records to be deleted: %d", len(deleted))
// Delete records first // Delete records first
for zone, endpoints := range deleted { for zone, endpoints := range deleted {
for _, endpoint := range endpoints { for _, ep := range endpoints {
name := p.recordSetNameForZone(zone, endpoint) name := p.recordSetNameForZone(zone, ep)
if p.dryRun { if p.dryRun {
log.Infof("Would delete %s record named '%s' for Azure Private DNS zone '%s'.", endpoint.RecordType, name, zone) log.Infof("Would delete %s record named '%s' for Azure Private DNS zone '%s'.", ep.RecordType, name, zone)
} else { } else {
log.Infof("Deleting %s record named '%s' for Azure Private DNS zone '%s'.", endpoint.RecordType, name, zone) log.Infof("Deleting %s record named '%s' for Azure Private DNS zone '%s'.", ep.RecordType, name, zone)
if _, err := p.recordSetsClient.Delete(ctx, p.resourceGroup, zone, privatedns.RecordType(endpoint.RecordType), name, ""); err != nil { if _, err := p.recordSetsClient.Delete(ctx, p.resourceGroup, zone, privatedns.RecordType(ep.RecordType), name, ""); err != nil {
log.Errorf( log.Errorf(
"Failed to delete %s record named '%s' for Azure Private DNS zone '%s': %v", "Failed to delete %s record named '%s' for Azure Private DNS zone '%s': %v",
endpoint.RecordType, ep.RecordType,
name, name,
zone, zone,
err, err,
@ -279,14 +277,14 @@ func (p *AzurePrivateDNSProvider) deleteRecords(ctx context.Context, deleted azu
func (p *AzurePrivateDNSProvider) updateRecords(ctx context.Context, updated azurePrivateDNSChangeMap) { func (p *AzurePrivateDNSProvider) updateRecords(ctx context.Context, updated azurePrivateDNSChangeMap) {
log.Debugf("Records to be updated: %d", len(updated)) log.Debugf("Records to be updated: %d", len(updated))
for zone, endpoints := range updated { for zone, endpoints := range updated {
for _, endpoint := range endpoints { for _, ep := range endpoints {
name := p.recordSetNameForZone(zone, endpoint) name := p.recordSetNameForZone(zone, ep)
if p.dryRun { if p.dryRun {
log.Infof( log.Infof(
"Would update %s record named '%s' to '%s' for Azure Private DNS zone '%s'.", "Would update %s record named '%s' to '%s' for Azure Private DNS zone '%s'.",
endpoint.RecordType, ep.RecordType,
name, name,
endpoint.Targets, ep.Targets,
zone, zone,
) )
continue continue
@ -294,19 +292,19 @@ func (p *AzurePrivateDNSProvider) updateRecords(ctx context.Context, updated azu
log.Infof( log.Infof(
"Updating %s record named '%s' to '%s' for Azure Private DNS zone '%s'.", "Updating %s record named '%s' to '%s' for Azure Private DNS zone '%s'.",
endpoint.RecordType, ep.RecordType,
name, name,
endpoint.Targets, ep.Targets,
zone, zone,
) )
recordSet, err := p.newRecordSet(endpoint) recordSet, err := p.newRecordSet(ep)
if err == nil { if err == nil {
_, err = p.recordSetsClient.CreateOrUpdate( _, err = p.recordSetsClient.CreateOrUpdate(
ctx, ctx,
p.resourceGroup, p.resourceGroup,
zone, zone,
privatedns.RecordType(endpoint.RecordType), privatedns.RecordType(ep.RecordType),
name, name,
recordSet, recordSet,
"", "",
@ -316,9 +314,9 @@ func (p *AzurePrivateDNSProvider) updateRecords(ctx context.Context, updated azu
if err != nil { if err != nil {
log.Errorf( log.Errorf(
"Failed to update %s record named '%s' to '%s' for Azure Private DNS zone '%s': %v", "Failed to update %s record named '%s' to '%s' for Azure Private DNS zone '%s': %v",
endpoint.RecordType, ep.RecordType,
name, name,
endpoint.Targets, ep.Targets,
zone, zone,
err, err,
) )

35
provider/azure/azure_privatedns_test.go
View File

@ -18,16 +18,11 @@ package azure
import ( import (
"context" "context"
"os"
"testing" "testing"
"github.com/Azure/azure-sdk-for-go/services/privatedns/mgmt/2018-09-01/privatedns" "github.com/Azure/azure-sdk-for-go/services/privatedns/mgmt/2018-09-01/privatedns"
"github.com/Azure/go-autorest/autorest" "github.com/Azure/go-autorest/autorest"
"github.com/Azure/go-autorest/autorest/azure"
"github.com/Azure/go-autorest/autorest/azure/auth"
"github.com/Azure/go-autorest/autorest/to" "github.com/Azure/go-autorest/autorest/to"
"github.com/stretchr/testify/assert"
"sigs.k8s.io/external-dns/endpoint" "sigs.k8s.io/external-dns/endpoint"
"sigs.k8s.io/external-dns/plan" "sigs.k8s.io/external-dns/plan"
"sigs.k8s.io/external-dns/provider" "sigs.k8s.io/external-dns/provider"
@ -255,36 +250,6 @@ func newAzurePrivateDNSProvider(domainFilter endpoint.DomainFilter, zoneIDFilter
} }
} }
func validateAzurePrivateDNSClientsResourceManager(t *testing.T, environmentName string, expectedResourceManagerEndpoint string) {
err := os.Setenv(auth.EnvironmentName, environmentName)
if err != nil {
t.Fatal(err)
}
azurePrivateDNSProvider, err := NewAzurePrivateDNSProvider(endpoint.NewDomainFilter([]string{"example.com"}), provider.NewZoneIDFilter([]string{""}), "k8s", "sub", true)
if err != nil {
t.Fatal(err)
}
zonesClientBaseURI := azurePrivateDNSProvider.zonesClient.(privatedns.PrivateZonesClient).BaseURI
recordSetsClientBaseURI := azurePrivateDNSProvider.recordSetsClient.(privatedns.RecordSetsClient).BaseURI
assert.Equal(t, zonesClientBaseURI, expectedResourceManagerEndpoint, "expected and actual resource manager endpoints don't match. expected: %s, got: %s", expectedResourceManagerEndpoint, zonesClientBaseURI)
assert.Equal(t, recordSetsClientBaseURI, expectedResourceManagerEndpoint, "expected and actual resource manager endpoints don't match. expected: %s, got: %s", expectedResourceManagerEndpoint, recordSetsClientBaseURI)
}
func TestNewAzurePrivateDNSProvider(t *testing.T) {
// make sure to reset the environment variables at the end again
originalEnv := os.Getenv(auth.EnvironmentName)
defer os.Setenv(auth.EnvironmentName, originalEnv)
validateAzurePrivateDNSClientsResourceManager(t, "", azure.PublicCloud.ResourceManagerEndpoint)
validateAzurePrivateDNSClientsResourceManager(t, "AZURECHINACLOUD", azure.ChinaCloud.ResourceManagerEndpoint)
validateAzurePrivateDNSClientsResourceManager(t, "AZUREGERMANCLOUD", azure.GermanCloud.ResourceManagerEndpoint)
validateAzurePrivateDNSClientsResourceManager(t, "AZUREUSGOVERNMENTCLOUD", azure.USGovernmentCloud.ResourceManagerEndpoint)
}
func TestAzurePrivateDNSRecord(t *testing.T) { func TestAzurePrivateDNSRecord(t *testing.T) {
provider, err := newMockedAzurePrivateDNSProvider(endpoint.NewDomainFilter([]string{"example.com"}), provider.NewZoneIDFilter([]string{""}), true, "k8s", provider, err := newMockedAzurePrivateDNSProvider(endpoint.NewDomainFilter([]string{"example.com"}), provider.NewZoneIDFilter([]string{""}), true, "k8s",
&[]privatedns.PrivateZone{ &[]privatedns.PrivateZone{

129
provider/azure/common.go Normal file
View File

@ -0,0 +1,129 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package azure
import (
"fmt"
"io/ioutil"
"strings"
"github.com/Azure/go-autorest/autorest/adal"
"github.com/Azure/go-autorest/autorest/azure"
log "github.com/sirupsen/logrus"
"gopkg.in/yaml.v2"
)
// Config represents common config items for Azure DNS and Azure Private DNS
type config struct {
Cloud string `json:"cloud" yaml:"cloud"`
Environment azure.Environment `json:"-" yaml:"-"`
TenantID string `json:"tenantId" yaml:"tenantId"`
SubscriptionID string `json:"subscriptionId" yaml:"subscriptionId"`
ResourceGroup string `json:"resourceGroup" yaml:"resourceGroup"`
Location string `json:"location" yaml:"location"`
ClientID string `json:"aadClientId" yaml:"aadClientId"`
ClientSecret string `json:"aadClientSecret" yaml:"aadClientSecret"`
UseManagedIdentityExtension bool `json:"useManagedIdentityExtension" yaml:"useManagedIdentityExtension"`
UserAssignedIdentityID string `json:"userAssignedIdentityID" yaml:"userAssignedIdentityID"`
}
func getConfig(configFile, resourceGroup, userAssignedIdentityClientID string) (*config, error) {
contents, err := ioutil.ReadFile(configFile)
if err != nil {
return nil, fmt.Errorf("failed to read Azure config file '%s': %v", configFile, err)
}
cfg := &config{}
err = yaml.Unmarshal(contents, &cfg)
if err != nil {
return nil, fmt.Errorf("failed to read Azure config file '%s': %v", configFile, err)
}
// If a resource group was given, override what was present in the config file
if resourceGroup != "" {
cfg.ResourceGroup = resourceGroup
}
// If userAssignedIdentityClientID is provided explicitly, override existing one in config file
if userAssignedIdentityClientID != "" {
cfg.UserAssignedIdentityID = userAssignedIdentityClientID
}
var environment azure.Environment
if cfg.Cloud == "" {
environment = azure.PublicCloud
} else {
environment, err = azure.EnvironmentFromName(cfg.Cloud)
if err != nil {
return nil, fmt.Errorf("invalid cloud value '%s': %v", cfg.Cloud, err)
}
}
cfg.Environment = environment
return cfg, nil
}
// getAccessToken retrieves Azure API access token.
func getAccessToken(cfg config, environment azure.Environment) (*adal.ServicePrincipalToken, error) {
// Try to retrieve token with service principal credentials.
// Try to use service principal first, some AKS clusters are in an intermediate state that `UseManagedIdentityExtension` is `true`
// and service principal exists. In this case, we still want to use service principal to authenticate.
if len(cfg.ClientID) > 0 &&
len(cfg.ClientSecret) > 0 &&
// due to some historical reason, for pure MSI cluster,
// they will use "msi" as placeholder in azure.json.
// In this case, we shouldn't try to use SPN to authenticate.
!strings.EqualFold(cfg.ClientID, "msi") &&
!strings.EqualFold(cfg.ClientSecret, "msi") {
log.Info("Using client_id+client_secret to retrieve access token for Azure API.")
oauthConfig, err := adal.NewOAuthConfig(environment.ActiveDirectoryEndpoint, cfg.TenantID)
if err != nil {
return nil, fmt.Errorf("failed to retrieve OAuth config: %v", err)
}
token, err := adal.NewServicePrincipalToken(*oauthConfig, cfg.ClientID, cfg.ClientSecret, environment.ResourceManagerEndpoint)
if err != nil {
return nil, fmt.Errorf("failed to create service principal token: %v", err)
}
return token, nil
}
// Try to retrieve token with MSI.
if cfg.UseManagedIdentityExtension {
log.Info("Using managed identity extension to retrieve access token for Azure API.")
msiEndpoint, err := adal.GetMSIVMEndpoint()
if err != nil {
return nil, fmt.Errorf("failed to get the managed service identity endpoint: %v", err)
}
if cfg.UserAssignedIdentityID != "" {
log.Infof("Resolving to user assigned identity, client id is %s.", cfg.UserAssignedIdentityID)
token, err := adal.NewServicePrincipalTokenFromMSIWithUserAssignedID(msiEndpoint, environment.ServiceManagementEndpoint, cfg.UserAssignedIdentityID)
if err != nil {
return nil, fmt.Errorf("failed to create the managed service identity token: %v", err)
}
return token, nil
}
log.Info("Resolving to system assigned identity.")
token, err := adal.NewServicePrincipalTokenFromMSI(msiEndpoint, environment.ServiceManagementEndpoint)
if err != nil {
return nil, fmt.Errorf("failed to create the managed service identity token: %v", err)
}
return token, nil
}
return nil, fmt.Errorf("no credentials provided for Azure API")
}

67
provider/azure/common_test.go Normal file
View File

@ -0,0 +1,67 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package azure
import (
"fmt"
"github.com/Azure/go-autorest/autorest/azure"
"io/ioutil"
"os"
"reflect"
"testing"
)
func TestGetAzureEnvironmentConfig(t *testing.T) {
tmp, err := ioutil.TempFile("", "azureconf")
if err != nil {
t.Errorf("couldn't write temp file %v", err)
}
defer os.Remove(tmp.Name())
tests := map[string]struct {
cloud string
err error
}{
"AzureChinaCloud": {"AzureChinaCloud", nil},
"AzureGermanCloud": {"AzureGermanCloud", nil},
"AzurePublicCloud": {"", nil},
"AzureUSGovernment": {"AzureUSGovernmentCloud", nil},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
_, _ = tmp.Seek(0, 0)
_, _ = tmp.Write([]byte(fmt.Sprintf(`{"cloud": "%s"}`, test.cloud)))
got, err := getConfig(tmp.Name(), "", "")
if err != nil {
t.Errorf("got unexpected err %v", err)
}
if test.cloud == "" {
test.cloud = "AzurePublicCloud"
}
want, err := azure.EnvironmentFromName(test.cloud)
if err != nil {
t.Errorf("couldn't get azure environment from provided name %v", err)
}
if !reflect.DeepEqual(want, got.Environment) {
t.Errorf("got %v, want %v", got.Environment, want)
}
})
}
}