Previously when a user selected to override the next-hop via GoBGP's
NextHopActions: Self functionality, we did it for all exported routes.
However, in a dual-stack use-case this causes problems for internal pod
IP routes that are spread via BGP advertisements.
Currently, kube-router only peers with an internal peer once over
whatever it's primary IP is according to it's Kubernetes node
information. This means that when overriding next-hop the IP is either
an IPv4 or IPv6 address depending on how the node has configured itself.
Therefore when it attempts to add a route for an IPv6 address and
override next-hop is configured, if the node's primary IP was an IPv4
address this will not succeed as a next-hop for an IPv6 address cannot
be an IPv4 gateway.
Rather than making the code base overly complicated with both an IPv4
and IPv6 peering for internal nodes, this change presents a bit of a
middle ground. By choosing not to override the next-hop for pod subnet
advertisements to internal (Kubernetes node) peers, we eliminate this
problem.
This does change the functionality of kube-router a bit, but one of the
foundational aspects to Kubernetes networking is that all nodes should
be able to contact each other. So I cannot currently think of a good
use-case where overriding the next-hop for pod subnets of internal peers
would be necessary, so I think that this is an ok concession to make.
The previous version of the bgp_policies code only allowed for creating
a policy when the policy didn't exist already. However, with the advent
of dual-stack we need to be able to add / remove statements if we add or
lose a specific IP family (e.g. IPv4 or IPv6) since they are handled in
different statements.
Given that the owner of GoBGP has let us know that policies are
idempotent, this now involves quite a bit of work. We need to follow the
following procedure:
add statements if missing -> add them to a policy -> if policy doesn't
equal the one already in GoBGP -> create the new policy and associate
it -> de-associate the old policy -> remove the old policy
When a single IP family's set looks to be equal, switch to continue
instead of return so that other families can still be evaluated as those
might have changes.
Fixes a problem where a user would end up with redundant external peers
in their BGP policies because getting peers is IP family agnostic and
yet is run twice on the same list.
This also ruined unit test consistency.
Without this logic, it appears that sometimes GoBGP is inclined to match
unintentional routes in policy because of the MATCHSET_ANY declaration
and the way that it interacts with empty sets.
In my testing, without this logic I found that it often resulted in
various routes not being advertised correctly and not even showing up in
GoBGP itself. My current guess is that policy keeps GoBGP from importing
the route into the RIB even from the Protobuf socket connection that
kube-router establishes directly.
We do a lot of getting defined sets for GoBGP and are planning to do
more of it in the future. This commit centralizes the logic for this and
reduces repetition.
Changes the custom import reject annotation support to not only block
the given subnet exactly, but also all subnets of the subnet given.
For example, this change blocks 10.100.100.0/24 when customimportreject
annotation has 10.100.0.0/16 in it.
* Makefile: Add lint using golangci-lint
* build/travis-test.sh: Run lint step
* metrics_controller: Lint
pkg/metrics/metrics_controller.go:150:2: `mu` is unused (structcheck)
mu sync.Mutex
^
pkg/metrics/metrics_controller.go:151:2: `nodeIP` is unused (structcheck)
nodeIP net.IP
^
* network_service_graceful: Lint
pkg/controllers/proxy/network_service_graceful.go:21:6: `gracefulQueueItem` is unused (deadcode)
type gracefulQueueItem struct {
^
pkg/controllers/proxy/network_service_graceful.go:22:2: `added` is unused (structcheck)
added time.Time
^
pkg/controllers/proxy/network_service_graceful.go:23:2: `service` is unused (structcheck)
service *ipvs.Service
^
* network_services_controller_test: Lint
pkg/controllers/proxy/network_services_controller_test.go:80:6: func `logf` is unused (unused)
* ecmp_vip: Lint
pkg/controllers/routing/ecmp_vip.go:208:4: S1023: redundant `return` statement (gosimple)
return
^
* bgp_peers: Lint
pkg/controllers/routing/bgp_peers.go:331:4: S1023: redundant `return` statement (gosimple)
return
^
* bgp_policies: Lint
pkg/controllers/routing/bgp_policies.go:80:3: S1011: should replace loop with `externalBgpPeers = append(externalBgpPeers, nrc.nodePeerRouters...)` (gosimple)
for _, peer := range nrc.nodePeerRouters {
^
pkg/controllers/routing/bgp_policies.go:23:20: ineffectual assignment to `err` (ineffassign)
podCidrPrefixSet, err := table.NewPrefixSet(config.PrefixSet{
^
pkg/controllers/routing/bgp_policies.go:42:22: ineffectual assignment to `err` (ineffassign)
clusterIPPrefixSet, err := table.NewPrefixSet(config.PrefixSet{
^
pkg/controllers/routing/bgp_policies.go:33:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(podCidrPrefixSet)
^
pkg/controllers/routing/bgp_policies.go:48:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(clusterIPPrefixSet)
^
pkg/controllers/routing/bgp_policies.go:69:31: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(iBGPPeerNS)
^
pkg/controllers/routing/bgp_policies.go:108:31: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(ns)
^
pkg/controllers/routing/bgp_policies.go:120:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(ns)
^
^
* network_policy_controller: Lint
pkg/controllers/netpol/network_policy_controller.go:35:2: `networkPolicyAnnotation` is unused (deadcode)
networkPolicyAnnotation = "net.beta.kubernetes.io/network-policy"
^
pkg/controllers/netpol/network_policy_controller.go:1047:4: SA9003: empty branch (staticcheck)
if err != nil {
^
pkg/controllers/netpol/network_policy_controller.go:969:10: SA4006: this value of `err` is never used (staticcheck)
chains, err := iptablesCmdHandler.ListChains("filter")
^
pkg/controllers/netpol/network_policy_controller.go:1568:4: SA4006: this value of `err` is never used (staticcheck)
err = iptablesCmdHandler.Delete("filter", "FORWARD", strconv.Itoa(i-realRuleNo))
^
pkg/controllers/netpol/network_policy_controller.go:1584:4: SA4006: this value of `err` is never used (staticcheck)
err = iptablesCmdHandler.Delete("filter", "OUTPUT", strconv.Itoa(i-realRuleNo))
^
* network_services_controller: Lint
pkg/controllers/proxy/network_services_controller.go:66:2: `h` is unused (deadcode)
h *ipvs.Handle
^
pkg/controllers/proxy/network_services_controller.go:879:23: SA1019: client.NewEnvClient is deprecated: use NewClientWithOpts(FromEnv) (staticcheck)
dockerClient, err := client.NewEnvClient()
^
pkg/controllers/proxy/network_services_controller.go:944:5: unreachable: unreachable code (govet)
glog.V(3).Infof("Waiting for tunnel interface %s to come up in the pod, retrying", KUBE_TUNNEL_IF)
^
pkg/controllers/proxy/network_services_controller.go:1289:5: S1002: should omit comparison to bool constant, can be simplified to `!hasHairpinChain` (gosimple)
if hasHairpinChain != true {
^
pkg/controllers/proxy/network_services_controller.go:1237:43: S1019: should use make(map[string][]string) instead (gosimple)
rulesNeeded := make(map[string][]string, 0)
^
pkg/controllers/proxy/network_services_controller.go:1111:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:1114:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:1117:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:445:21: Error return value of `nsc.publishMetrics` is not checked (errcheck)
nsc.publishMetrics(nsc.serviceMap)
^
pkg/controllers/proxy/network_services_controller.go:1609:9: Error return value of `h.Write` is not checked (errcheck)
h.Write([]byte(ip + "-" + protocol + "-" + port))
^
pkg/controllers/proxy/network_services_controller.go:912:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:926:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:950:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:641:9: SA4006: this value of `err` is never used (staticcheck)
addrs, err := getAllLocalIPs()
^
* network_routes_controller: Lint
pkg/controllers/routing/network_routes_controller.go:340:2: S1000: should use for range instead of for { select {} } (gosimple)
for {
^
pkg/controllers/routing/network_routes_controller.go:757:22: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:770:22: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:782:23: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:717:12: Error return value of `g.Serve` is not checked (errcheck)
go g.Serve()
* ipset: Lint
pkg/utils/ipset.go:243:23: Error return value of `entry.Set.Parent.Save` is not checked (errcheck)
entry.Set.Parent.Save()
^
* pkg/cmd/kube-router: Lint
pkg/cmd/kube-router.go:214:26: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(os.Stderr, output)
^
pkg/cmd/kube-router.go:184:15: SA1017: the channel used with signal.Notify should be buffered (staticcheck)
signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)
^
pkg/cmd/kube-router.go:94:17: Error return value of `hc.RunServer` is not checked (errcheck)
go hc.RunServer(stopCh, &wg)
^
pkg/cmd/kube-router.go:112:16: Error return value of `hc.RunCheck` is not checked (errcheck)
go hc.RunCheck(healthChan, stopCh, &wg)
^
pkg/cmd/kube-router.go:121:12: Error return value of `mc.Run` is not checked (errcheck)
go mc.Run(healthChan, stopCh, &wg)
^
* cmd/kube-router/kube-router: Lint
cmd/kube-router/kube-router.go:31:24: Error return value of `flag.CommandLine.Parse` is not checked (errcheck)
flag.CommandLine.Parse([]string{})
^
cmd/kube-router/kube-router.go:33:10: Error return value of `flag.Set` is not checked (errcheck)
flag.Set("logtostderr", "true")
^
cmd/kube-router/kube-router.go:34:10: Error return value of `flag.Set` is not checked (errcheck)
flag.Set("v", config.VLevel)
^
cmd/kube-router/kube-router.go:62:27: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(os.Stdout, http.ListenAndServe("0.0.0.0:6060", nil).Error())
^
* kube-router_test: Lint
cmd/kube-router/kube-router_test.go:21:10: Error return value of `io.Copy` is not checked (errcheck)
io.Copy(stderrBuf, stderrR)
^
cmd/kube-router/kube-router_test.go:40:17: Error return value of `docBuf.ReadFrom` is not checked (errcheck)
docBuf.ReadFrom(docF)
^
* service_endpoints_sync: Lint
pkg/controllers/proxy/service_endpoints_sync.go:460:2: ineffectual assignment to `ipvsSvcs` (ineffassign)
ipvsSvcs, err := nsc.ln.ipvsGetServices()
^
pkg/controllers/proxy/service_endpoints_sync.go:311:5: SA4006: this value of `err` is never used (staticcheck)
err = nsc.ln.ipAddrDel(dummyVipInterface, externalIP)
^
* node: Lint
pkg/utils/node.go:19:16: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err := clientset.Core().Nodes().Get(nodeName, metav1.GetOptions{})
^
pkg/utils/node.go:27:15: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err := clientset.Core().Nodes().Get(hostName, metav1.GetOptions{})
^
pkg/utils/node.go:34:15: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err = clientset.Core().Nodes().Get(hostnameOverride, metav1.GetOptions{})
^
* aws: Lint
pkg/controllers/routing/aws.go:31:8: SA4006: this value of `err` is never used (staticcheck)
URL, err := url.Parse(providerID)
^
* health_controller: Lint
pkg/healthcheck/health_controller.go:54:10: Error return value of `w.Write` is not checked (errcheck)
w.Write([]byte("OK\n"))
^
pkg/healthcheck/health_controller.go:68:10: Error return value of `w.Write` is not checked (errcheck)
w.Write([]byte("Unhealthy"))
^
pkg/healthcheck/health_controller.go:159:2: S1000: should use a simple channel send/receive instead of `select` with a single case (gosimple)
select {
^
* network_routes_controller_test: Lint
pkg/controllers/routing/network_routes_controller_test.go:1113:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller_test.go:1314:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller_test.go:2327:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
* .golangci.yml: Increase timeout
Default is 1m, increase to 5m otherwise travis might fail
* Makefile: Update golangci-lint to 1.27.0
* kube-router_test.go: defer waitgroup
Co-authored-by: Aaron U'Ren <aauren@users.noreply.github.com>
* network_routes_controller: Incorporate review
* bgp_policies: Incorporate review
* network_routes_controller: Incorporate review
* bgp_policies: Log error instead
* network_services_controller: Incorporate review
Co-authored-by: Aaron U'Ren <aauren@users.noreply.github.com>
Before we used to raise an error when a node was missing an IP, but it
turns out that this is not a required attribute of a node. And while it
is rare that a node would be missing an IP address, a node doesn't
require an IP address or a name or really much of anything in order to
exist.
This brings us to stronger conformance with the Kubernetes API and makes
it so that kube-router logs errors rather than changing it's health
status and potentially causing cascading failures across the fleet if a
user adds a node like this.
* rename export policies to make it direction independent
* split creating neighborsets and prefixsets from applying export policy
* add bgp import policy to deny service VIPs
* add tests for addition of import policy
