Adds support for spec.internalTrafficPolicy and fixes support for
spec.externalTrafficPolicy so that it only effects external traffic.
Keeps existing support for kube-router.io/service-local annotation which
overrides both to local when set to true. Any other value in this
annotation is ignored.
Differentiate headless services from ClusterIP being none, in
preparation for handling the service.kubernetes.io/headless label. One
might thing that handling these is similar, which it sort of is and sort
of isn't. ClusterIP is an immutable field, whereas labels are mutable.
This changes our handling of ClusterIP none-ness from the presence of
the headless label.
When we consider what to do with ClusterIP being none, that is
fundamentally different, because once it is None, the k8s API guarantees
that the service won't ever change.
Whereas the label can be added and removed.
With advertiseService set to false by default, it means that it won't
ever get re-evaluated if the service isn't a local host and will ALWAYS
result in withdrawing the VIPs which is incorrect. It needs to default
to true, and only override the boolean if serviceLocal is set to true.
Previously we used to do an idempotent sync all active VIPs any time we
got a service or endpoint update. However, this only worked when we
assumed a single-stack deployment model where IPs were never deleted
unless the whole service was deleted.
In a dual-stack model, we can add / remove LoadBalancer IPs and Cluster
IPs on updates. Given this, we need to take into account the finite
change that happens, and not just revert to sync-all because we'll never
stop advertising IPs that should be removed.
As a fall-back, we still have the outer Run loop that syncs all active
routes every X amount of seconds (configured by user CLI parameter). So
on that timer we'll still have something that syncs all active VIPs and
works as an outer control loop to ensure that desired state eventually
becomes active state if we accidentally remove a VIP that should have
been there.
Annotations were taken into account during startup, but after they were
advertised the affect of annotations was only additive because we
were only tracking current state of VIPs that should be advertised and
not taking into account VIPs that should be withdrawn for anything other
than service locality.
Fixes#1491
* doc(ecmp_vip.go): add info around extra withdraw
Rename getWithdraw to make it more explicit what its doing here. Also
add documentation as to why this is needed on Update and not
Create/Delete as well as why we only treat externalIPs.
* fix(ecmp_vip.go): remove superfluous AddPolicies
AddPolicies is already called downstream of nrc.OnEndpointsUpdate() so
there is no need to do it here as well, the only result is that this
expensive operation and idempotent operation is run twice.
* feat: better handling of headless services
Also introduces a consolidated Service utilities section for controller
functionality related to services that is shared.
* fix: add logging back to tryHandleServiceDelete
Properly consider the readiness of all services in the case where
multiple services share the same VIP. Don't withdraw a VIP just because
one service is not ready.
* Makefile: Add lint using golangci-lint
* build/travis-test.sh: Run lint step
* metrics_controller: Lint
pkg/metrics/metrics_controller.go:150:2: `mu` is unused (structcheck)
mu sync.Mutex
^
pkg/metrics/metrics_controller.go:151:2: `nodeIP` is unused (structcheck)
nodeIP net.IP
^
* network_service_graceful: Lint
pkg/controllers/proxy/network_service_graceful.go:21:6: `gracefulQueueItem` is unused (deadcode)
type gracefulQueueItem struct {
^
pkg/controllers/proxy/network_service_graceful.go:22:2: `added` is unused (structcheck)
added time.Time
^
pkg/controllers/proxy/network_service_graceful.go:23:2: `service` is unused (structcheck)
service *ipvs.Service
^
* network_services_controller_test: Lint
pkg/controllers/proxy/network_services_controller_test.go:80:6: func `logf` is unused (unused)
* ecmp_vip: Lint
pkg/controllers/routing/ecmp_vip.go:208:4: S1023: redundant `return` statement (gosimple)
return
^
* bgp_peers: Lint
pkg/controllers/routing/bgp_peers.go:331:4: S1023: redundant `return` statement (gosimple)
return
^
* bgp_policies: Lint
pkg/controllers/routing/bgp_policies.go:80:3: S1011: should replace loop with `externalBgpPeers = append(externalBgpPeers, nrc.nodePeerRouters...)` (gosimple)
for _, peer := range nrc.nodePeerRouters {
^
pkg/controllers/routing/bgp_policies.go:23:20: ineffectual assignment to `err` (ineffassign)
podCidrPrefixSet, err := table.NewPrefixSet(config.PrefixSet{
^
pkg/controllers/routing/bgp_policies.go:42:22: ineffectual assignment to `err` (ineffassign)
clusterIPPrefixSet, err := table.NewPrefixSet(config.PrefixSet{
^
pkg/controllers/routing/bgp_policies.go:33:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(podCidrPrefixSet)
^
pkg/controllers/routing/bgp_policies.go:48:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(clusterIPPrefixSet)
^
pkg/controllers/routing/bgp_policies.go:69:31: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(iBGPPeerNS)
^
pkg/controllers/routing/bgp_policies.go:108:31: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(ns)
^
pkg/controllers/routing/bgp_policies.go:120:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(ns)
^
^
* network_policy_controller: Lint
pkg/controllers/netpol/network_policy_controller.go:35:2: `networkPolicyAnnotation` is unused (deadcode)
networkPolicyAnnotation = "net.beta.kubernetes.io/network-policy"
^
pkg/controllers/netpol/network_policy_controller.go:1047:4: SA9003: empty branch (staticcheck)
if err != nil {
^
pkg/controllers/netpol/network_policy_controller.go:969:10: SA4006: this value of `err` is never used (staticcheck)
chains, err := iptablesCmdHandler.ListChains("filter")
^
pkg/controllers/netpol/network_policy_controller.go:1568:4: SA4006: this value of `err` is never used (staticcheck)
err = iptablesCmdHandler.Delete("filter", "FORWARD", strconv.Itoa(i-realRuleNo))
^
pkg/controllers/netpol/network_policy_controller.go:1584:4: SA4006: this value of `err` is never used (staticcheck)
err = iptablesCmdHandler.Delete("filter", "OUTPUT", strconv.Itoa(i-realRuleNo))
^
* network_services_controller: Lint
pkg/controllers/proxy/network_services_controller.go:66:2: `h` is unused (deadcode)
h *ipvs.Handle
^
pkg/controllers/proxy/network_services_controller.go:879:23: SA1019: client.NewEnvClient is deprecated: use NewClientWithOpts(FromEnv) (staticcheck)
dockerClient, err := client.NewEnvClient()
^
pkg/controllers/proxy/network_services_controller.go:944:5: unreachable: unreachable code (govet)
glog.V(3).Infof("Waiting for tunnel interface %s to come up in the pod, retrying", KUBE_TUNNEL_IF)
^
pkg/controllers/proxy/network_services_controller.go:1289:5: S1002: should omit comparison to bool constant, can be simplified to `!hasHairpinChain` (gosimple)
if hasHairpinChain != true {
^
pkg/controllers/proxy/network_services_controller.go:1237:43: S1019: should use make(map[string][]string) instead (gosimple)
rulesNeeded := make(map[string][]string, 0)
^
pkg/controllers/proxy/network_services_controller.go:1111:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:1114:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:1117:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:445:21: Error return value of `nsc.publishMetrics` is not checked (errcheck)
nsc.publishMetrics(nsc.serviceMap)
^
pkg/controllers/proxy/network_services_controller.go:1609:9: Error return value of `h.Write` is not checked (errcheck)
h.Write([]byte(ip + "-" + protocol + "-" + port))
^
pkg/controllers/proxy/network_services_controller.go:912:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:926:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:950:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:641:9: SA4006: this value of `err` is never used (staticcheck)
addrs, err := getAllLocalIPs()
^
* network_routes_controller: Lint
pkg/controllers/routing/network_routes_controller.go:340:2: S1000: should use for range instead of for { select {} } (gosimple)
for {
^
pkg/controllers/routing/network_routes_controller.go:757:22: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:770:22: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:782:23: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:717:12: Error return value of `g.Serve` is not checked (errcheck)
go g.Serve()
* ipset: Lint
pkg/utils/ipset.go:243:23: Error return value of `entry.Set.Parent.Save` is not checked (errcheck)
entry.Set.Parent.Save()
^
* pkg/cmd/kube-router: Lint
pkg/cmd/kube-router.go:214:26: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(os.Stderr, output)
^
pkg/cmd/kube-router.go:184:15: SA1017: the channel used with signal.Notify should be buffered (staticcheck)
signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)
^
pkg/cmd/kube-router.go:94:17: Error return value of `hc.RunServer` is not checked (errcheck)
go hc.RunServer(stopCh, &wg)
^
pkg/cmd/kube-router.go:112:16: Error return value of `hc.RunCheck` is not checked (errcheck)
go hc.RunCheck(healthChan, stopCh, &wg)
^
pkg/cmd/kube-router.go:121:12: Error return value of `mc.Run` is not checked (errcheck)
go mc.Run(healthChan, stopCh, &wg)
^
* cmd/kube-router/kube-router: Lint
cmd/kube-router/kube-router.go:31:24: Error return value of `flag.CommandLine.Parse` is not checked (errcheck)
flag.CommandLine.Parse([]string{})
^
cmd/kube-router/kube-router.go:33:10: Error return value of `flag.Set` is not checked (errcheck)
flag.Set("logtostderr", "true")
^
cmd/kube-router/kube-router.go:34:10: Error return value of `flag.Set` is not checked (errcheck)
flag.Set("v", config.VLevel)
^
cmd/kube-router/kube-router.go:62:27: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(os.Stdout, http.ListenAndServe("0.0.0.0:6060", nil).Error())
^
* kube-router_test: Lint
cmd/kube-router/kube-router_test.go:21:10: Error return value of `io.Copy` is not checked (errcheck)
io.Copy(stderrBuf, stderrR)
^
cmd/kube-router/kube-router_test.go:40:17: Error return value of `docBuf.ReadFrom` is not checked (errcheck)
docBuf.ReadFrom(docF)
^
* service_endpoints_sync: Lint
pkg/controllers/proxy/service_endpoints_sync.go:460:2: ineffectual assignment to `ipvsSvcs` (ineffassign)
ipvsSvcs, err := nsc.ln.ipvsGetServices()
^
pkg/controllers/proxy/service_endpoints_sync.go:311:5: SA4006: this value of `err` is never used (staticcheck)
err = nsc.ln.ipAddrDel(dummyVipInterface, externalIP)
^
* node: Lint
pkg/utils/node.go:19:16: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err := clientset.Core().Nodes().Get(nodeName, metav1.GetOptions{})
^
pkg/utils/node.go:27:15: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err := clientset.Core().Nodes().Get(hostName, metav1.GetOptions{})
^
pkg/utils/node.go:34:15: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err = clientset.Core().Nodes().Get(hostnameOverride, metav1.GetOptions{})
^
* aws: Lint
pkg/controllers/routing/aws.go:31:8: SA4006: this value of `err` is never used (staticcheck)
URL, err := url.Parse(providerID)
^
* health_controller: Lint
pkg/healthcheck/health_controller.go:54:10: Error return value of `w.Write` is not checked (errcheck)
w.Write([]byte("OK\n"))
^
pkg/healthcheck/health_controller.go:68:10: Error return value of `w.Write` is not checked (errcheck)
w.Write([]byte("Unhealthy"))
^
pkg/healthcheck/health_controller.go:159:2: S1000: should use a simple channel send/receive instead of `select` with a single case (gosimple)
select {
^
* network_routes_controller_test: Lint
pkg/controllers/routing/network_routes_controller_test.go:1113:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller_test.go:1314:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller_test.go:2327:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
* .golangci.yml: Increase timeout
Default is 1m, increase to 5m otherwise travis might fail
* Makefile: Update golangci-lint to 1.27.0
* kube-router_test.go: defer waitgroup
Co-authored-by: Aaron U'Ren <aauren@users.noreply.github.com>
* network_routes_controller: Incorporate review
* bgp_policies: Incorporate review
* network_routes_controller: Incorporate review
* bgp_policies: Log error instead
* network_services_controller: Incorporate review
Co-authored-by: Aaron U'Ren <aauren@users.noreply.github.com>
* in DeleteFunc handlers across the controllers handle the case where received object can be of
type DeletedFinalStateUnknown
fixes one of the symptoms (panic on receiving DeletedFinalStateUnknown objects) reported in #712
* address review comments
* Refactor: seperate fetching service VIPs from advertise/withdrawal decision
* Refactor: simplify advertise/withdrawal logic
* Pass svcDeleted param to getVIPsForService
* Don't advertise VIPs from deleted services
* Test for withdrawing VIPs from deleted service
* Refactor: use explicit handleServiceDelete functions
* rename export policies to make it direction independent
* split creating neighborsets and prefixsets from applying export policy
* add bgp import policy to deny service VIPs
* add tests for addition of import policy
* - refactor / clean up / extract code dupes into methods and reuse
* - fix 639
- get external IPs to withdraw as diff against previous generation
of service
* add unit tests for implementing #75
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
* integration tests for #75
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
* update docs for #75
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
* define new kube-router.io/service.advertise.* annotations
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
* Implement per service annotations for advertising IPs.
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
* more consistent annotation names
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
* remove redundant tests
Signed-off-by: Steven Armstrong <steven.armstrong@id.ethz.ch>
