Moves all Service VIP range configurations into pkg/svcip this is where
validation and querying of ranges goes rather than passing each range to
each controller.
It also centralizes the validation logic since NRC and NSC need
basically equivalent logic. It additionally adds a RangeQuerier
interface for the NPC and LBC controllers which require knowing the
literal ranges.
Logic errors & regressions relating to traffic policies make up
approximately 8 or so preventable historical issues with the project.
Therefore prioritizing them as a unit testing surface.
This prepares the way for broader refactors in the way that we handle
nodes by:
* Separating frequently used node logic from the controller creation
steps
* Keeping reused code DRY-er
* Adding interface abstractions for key groups of node data and starting
to rely on those more rather than concrete types
* Separating node data from the rest of the controller data structure so
that it smaller definitions of data can be passed around to functions
that need it rather than always passing the entire controller which
contains more data / surface area than most functions need.
With the advent of IPv6 integrated into the NSC we no longer get all IPs
from endpoints, but rather just the primary IP of the pod (which is
often, but not always the IPv4 address).
In order to get all possible endpoint addresses for a given service we
need to switch to using EndpointSlice which also nicely groups addresses
into IPv4 and IPv6 by AddressType and also gives us more information
about the endpoint status by giving us attributes for serving and
terminating, instead of just ready or not ready.
This does mean that users will need to add another permission to their
RBAC in order for kube-router to access these objects.
* fact(NSC): consolidate constants to top
* fix(NSC): increase IPVS add service logging
* fix(NSC): improve logging for FWMark IPVS entries
* fix(NSC): add missing parameter to logging
* feat(NSC): generate unique FW marks
Because we trim the 32-bit FNV-1a hash to 16 bits there is the potential
for FW marks to collide with each other even for unique inputs of IP,
protocol, and port. This reduces that chance up to the 16-bit max by
keeping track of which FW marks we've already allocated and what IP,
protocol, port combo they've been allocated for.
Fixes#1045
* fact(NSC): move utility funcs to utils
* fix(NSC): reduce IPVS service shell outs
This also aligns it more with the almost identical function used for
non-FWmarked services ipvsAddService() which is also called from
setupExternalIPServices and passes in this same list of ipvsServices.
* fix(NSC): fix & consolidate DSR cleanup code
A lot of this is refactor work, but its important to know why the DSR
mangle tables were not being cleaned up in the first place. When we
transitioned to iptables-save to look over the mangle rules, we didn't
realize that iptables-save changes the format of the marks from integer
values (which is what the CLI works with) to hexadecimal.
This made it so that we were never actually matching on a mangle rule,
which left them all behind. When these mangle rules were left, it meant
that IPs that used to be part of a DSR service were essentially
black-holed on the system and were no longer route-able.
Fixes#1167
* doc(dsr): expand DSR documentation
fixes#1055
* ensure active service map is updated for non DSR services
Co-authored-by: Murali Reddy <muralimmreddy@gmail.com>
* Makefile: Add lint using golangci-lint
* build/travis-test.sh: Run lint step
* metrics_controller: Lint
pkg/metrics/metrics_controller.go:150:2: `mu` is unused (structcheck)
mu sync.Mutex
^
pkg/metrics/metrics_controller.go:151:2: `nodeIP` is unused (structcheck)
nodeIP net.IP
^
* network_service_graceful: Lint
pkg/controllers/proxy/network_service_graceful.go:21:6: `gracefulQueueItem` is unused (deadcode)
type gracefulQueueItem struct {
^
pkg/controllers/proxy/network_service_graceful.go:22:2: `added` is unused (structcheck)
added time.Time
^
pkg/controllers/proxy/network_service_graceful.go:23:2: `service` is unused (structcheck)
service *ipvs.Service
^
* network_services_controller_test: Lint
pkg/controllers/proxy/network_services_controller_test.go:80:6: func `logf` is unused (unused)
* ecmp_vip: Lint
pkg/controllers/routing/ecmp_vip.go:208:4: S1023: redundant `return` statement (gosimple)
return
^
* bgp_peers: Lint
pkg/controllers/routing/bgp_peers.go:331:4: S1023: redundant `return` statement (gosimple)
return
^
* bgp_policies: Lint
pkg/controllers/routing/bgp_policies.go:80:3: S1011: should replace loop with `externalBgpPeers = append(externalBgpPeers, nrc.nodePeerRouters...)` (gosimple)
for _, peer := range nrc.nodePeerRouters {
^
pkg/controllers/routing/bgp_policies.go:23:20: ineffectual assignment to `err` (ineffassign)
podCidrPrefixSet, err := table.NewPrefixSet(config.PrefixSet{
^
pkg/controllers/routing/bgp_policies.go:42:22: ineffectual assignment to `err` (ineffassign)
clusterIPPrefixSet, err := table.NewPrefixSet(config.PrefixSet{
^
pkg/controllers/routing/bgp_policies.go:33:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(podCidrPrefixSet)
^
pkg/controllers/routing/bgp_policies.go:48:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(clusterIPPrefixSet)
^
pkg/controllers/routing/bgp_policies.go:69:31: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(iBGPPeerNS)
^
pkg/controllers/routing/bgp_policies.go:108:31: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(ns)
^
pkg/controllers/routing/bgp_policies.go:120:30: Error return value of `nrc.bgpServer.AddDefinedSet` is not checked (errcheck)
nrc.bgpServer.AddDefinedSet(ns)
^
^
* network_policy_controller: Lint
pkg/controllers/netpol/network_policy_controller.go:35:2: `networkPolicyAnnotation` is unused (deadcode)
networkPolicyAnnotation = "net.beta.kubernetes.io/network-policy"
^
pkg/controllers/netpol/network_policy_controller.go:1047:4: SA9003: empty branch (staticcheck)
if err != nil {
^
pkg/controllers/netpol/network_policy_controller.go:969:10: SA4006: this value of `err` is never used (staticcheck)
chains, err := iptablesCmdHandler.ListChains("filter")
^
pkg/controllers/netpol/network_policy_controller.go:1568:4: SA4006: this value of `err` is never used (staticcheck)
err = iptablesCmdHandler.Delete("filter", "FORWARD", strconv.Itoa(i-realRuleNo))
^
pkg/controllers/netpol/network_policy_controller.go:1584:4: SA4006: this value of `err` is never used (staticcheck)
err = iptablesCmdHandler.Delete("filter", "OUTPUT", strconv.Itoa(i-realRuleNo))
^
* network_services_controller: Lint
pkg/controllers/proxy/network_services_controller.go:66:2: `h` is unused (deadcode)
h *ipvs.Handle
^
pkg/controllers/proxy/network_services_controller.go:879:23: SA1019: client.NewEnvClient is deprecated: use NewClientWithOpts(FromEnv) (staticcheck)
dockerClient, err := client.NewEnvClient()
^
pkg/controllers/proxy/network_services_controller.go:944:5: unreachable: unreachable code (govet)
glog.V(3).Infof("Waiting for tunnel interface %s to come up in the pod, retrying", KUBE_TUNNEL_IF)
^
pkg/controllers/proxy/network_services_controller.go:1289:5: S1002: should omit comparison to bool constant, can be simplified to `!hasHairpinChain` (gosimple)
if hasHairpinChain != true {
^
pkg/controllers/proxy/network_services_controller.go:1237:43: S1019: should use make(map[string][]string) instead (gosimple)
rulesNeeded := make(map[string][]string, 0)
^
pkg/controllers/proxy/network_services_controller.go:1111:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:1114:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:1117:4: S1023: redundant break statement (gosimple)
break
^
pkg/controllers/proxy/network_services_controller.go:445:21: Error return value of `nsc.publishMetrics` is not checked (errcheck)
nsc.publishMetrics(nsc.serviceMap)
^
pkg/controllers/proxy/network_services_controller.go:1609:9: Error return value of `h.Write` is not checked (errcheck)
h.Write([]byte(ip + "-" + protocol + "-" + port))
^
pkg/controllers/proxy/network_services_controller.go:912:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:926:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:950:13: Error return value of `netns.Set` is not checked (errcheck)
netns.Set(hostNetworkNamespaceHandle)
^
pkg/controllers/proxy/network_services_controller.go:641:9: SA4006: this value of `err` is never used (staticcheck)
addrs, err := getAllLocalIPs()
^
* network_routes_controller: Lint
pkg/controllers/routing/network_routes_controller.go:340:2: S1000: should use for range instead of for { select {} } (gosimple)
for {
^
pkg/controllers/routing/network_routes_controller.go:757:22: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:770:22: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:782:23: Error return value of `nrc.bgpServer.Stop` is not checked (errcheck)
nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller.go:717:12: Error return value of `g.Serve` is not checked (errcheck)
go g.Serve()
* ipset: Lint
pkg/utils/ipset.go:243:23: Error return value of `entry.Set.Parent.Save` is not checked (errcheck)
entry.Set.Parent.Save()
^
* pkg/cmd/kube-router: Lint
pkg/cmd/kube-router.go:214:26: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(os.Stderr, output)
^
pkg/cmd/kube-router.go:184:15: SA1017: the channel used with signal.Notify should be buffered (staticcheck)
signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)
^
pkg/cmd/kube-router.go:94:17: Error return value of `hc.RunServer` is not checked (errcheck)
go hc.RunServer(stopCh, &wg)
^
pkg/cmd/kube-router.go:112:16: Error return value of `hc.RunCheck` is not checked (errcheck)
go hc.RunCheck(healthChan, stopCh, &wg)
^
pkg/cmd/kube-router.go:121:12: Error return value of `mc.Run` is not checked (errcheck)
go mc.Run(healthChan, stopCh, &wg)
^
* cmd/kube-router/kube-router: Lint
cmd/kube-router/kube-router.go:31:24: Error return value of `flag.CommandLine.Parse` is not checked (errcheck)
flag.CommandLine.Parse([]string{})
^
cmd/kube-router/kube-router.go:33:10: Error return value of `flag.Set` is not checked (errcheck)
flag.Set("logtostderr", "true")
^
cmd/kube-router/kube-router.go:34:10: Error return value of `flag.Set` is not checked (errcheck)
flag.Set("v", config.VLevel)
^
cmd/kube-router/kube-router.go:62:27: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
fmt.Fprintf(os.Stdout, http.ListenAndServe("0.0.0.0:6060", nil).Error())
^
* kube-router_test: Lint
cmd/kube-router/kube-router_test.go:21:10: Error return value of `io.Copy` is not checked (errcheck)
io.Copy(stderrBuf, stderrR)
^
cmd/kube-router/kube-router_test.go:40:17: Error return value of `docBuf.ReadFrom` is not checked (errcheck)
docBuf.ReadFrom(docF)
^
* service_endpoints_sync: Lint
pkg/controllers/proxy/service_endpoints_sync.go:460:2: ineffectual assignment to `ipvsSvcs` (ineffassign)
ipvsSvcs, err := nsc.ln.ipvsGetServices()
^
pkg/controllers/proxy/service_endpoints_sync.go:311:5: SA4006: this value of `err` is never used (staticcheck)
err = nsc.ln.ipAddrDel(dummyVipInterface, externalIP)
^
* node: Lint
pkg/utils/node.go:19:16: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err := clientset.Core().Nodes().Get(nodeName, metav1.GetOptions{})
^
pkg/utils/node.go:27:15: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err := clientset.Core().Nodes().Get(hostName, metav1.GetOptions{})
^
pkg/utils/node.go:34:15: SA1019: clientset.Core is deprecated: please explicitly pick a version if possible. (staticcheck)
node, err = clientset.Core().Nodes().Get(hostnameOverride, metav1.GetOptions{})
^
* aws: Lint
pkg/controllers/routing/aws.go:31:8: SA4006: this value of `err` is never used (staticcheck)
URL, err := url.Parse(providerID)
^
* health_controller: Lint
pkg/healthcheck/health_controller.go:54:10: Error return value of `w.Write` is not checked (errcheck)
w.Write([]byte("OK\n"))
^
pkg/healthcheck/health_controller.go:68:10: Error return value of `w.Write` is not checked (errcheck)
w.Write([]byte("Unhealthy"))
^
pkg/healthcheck/health_controller.go:159:2: S1000: should use a simple channel send/receive instead of `select` with a single case (gosimple)
select {
^
* network_routes_controller_test: Lint
pkg/controllers/routing/network_routes_controller_test.go:1113:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller_test.go:1314:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
pkg/controllers/routing/network_routes_controller_test.go:2327:37: Error return value of `testcase.nrc.bgpServer.Stop` is not checked (errcheck)
defer testcase.nrc.bgpServer.Stop()
^
* .golangci.yml: Increase timeout
Default is 1m, increase to 5m otherwise travis might fail
* Makefile: Update golangci-lint to 1.27.0
* kube-router_test.go: defer waitgroup
Co-authored-by: Aaron U'Ren <aauren@users.noreply.github.com>
* network_routes_controller: Incorporate review
* bgp_policies: Incorporate review
* network_routes_controller: Incorporate review
* bgp_policies: Log error instead
* network_services_controller: Incorporate review
Co-authored-by: Aaron U'Ren <aauren@users.noreply.github.com>
* honor the ClientIP session affinity timeout
* update moq file
* Fix unit test failure due to adding a new arg to ipvsAddService
Co-authored-by: Bumyong Choi <bchoi@digitalocean.com>
* Add to set ip_vs_mh scheduler and flags
Signed-off-by: Inju Song <inju.song@navercorp.com>
* Use scheduler flags when adding or updating service
Signed-off-by: Inju Song <inju.song@navercorp.com>
* Refactor with gofmt, generate moq file and fix test source
Signed-off-by: Inju Song <inju.song@navercorp.com>
