mirrors/kube-router
1
0
Fork 0
mirror of https://github.com/cloudnativelabs/kube-router.git synced 2025-10-08 16:31:07 +02:00
Code Issues Packages Projects Releases Wiki Activity
kube-router/contrib/bootkube
History

README.md

Bootkube Integration

The following instructions and examples demonstrate how to create a new Bootkube provisioned Kubernetes cluster using kube-router in place of kube-proxy and flannel.

Asset Creation

Follow the documentation for your environment and setup arguments for bootkube render.

For example:

bootkube render --asset-dir=${PWD}/assets --api-servers=https://kube-api-dev.zbrbdl:443 --api-server-alt-names=DNS=kube-api-dev.zbrbdl --etcd-servers="http://127.0.0.1:2379"
Writing asset: /home/bzub/assets/manifests/kube-scheduler.yaml
Writing asset: /home/bzub/assets/manifests/kube-scheduler-disruption.yaml
Writing asset: /home/bzub/assets/manifests/kube-controller-manager-disruption.yaml
Writing asset: /home/bzub/assets/manifests/kube-dns-deployment.yaml
Writing asset: /home/bzub/assets/manifests/pod-checkpointer.yaml
Writing asset: /home/bzub/assets/manifests/kube-flannel.yaml
Writing asset: /home/bzub/assets/manifests/kube-system-rbac-role-binding.yaml
Writing asset: /home/bzub/assets/manifests/kube-controller-manager.yaml
Writing asset: /home/bzub/assets/manifests/kube-apiserver.yaml
Writing asset: /home/bzub/assets/manifests/kube-proxy.yaml
Writing asset: /home/bzub/assets/manifests/kube-flannel-cfg.yaml
Writing asset: /home/bzub/assets/manifests/kube-dns-svc.yaml
Writing asset: /home/bzub/assets/bootstrap-manifests/bootstrap-apiserver.yaml
Writing asset: /home/bzub/assets/bootstrap-manifests/bootstrap-controller-manager.yaml
Writing asset: /home/bzub/assets/bootstrap-manifests/bootstrap-scheduler.yaml
Writing asset: /home/bzub/assets/tls/ca.key
Writing asset: /home/bzub/assets/tls/ca.crt
Writing asset: /home/bzub/assets/tls/apiserver.key
Writing asset: /home/bzub/assets/tls/apiserver.crt
Writing asset: /home/bzub/assets/tls/service-account.key
Writing asset: /home/bzub/assets/tls/service-account.pub
Writing asset: /home/bzub/assets/tls/kubelet.key
Writing asset: /home/bzub/assets/tls/kubelet.crt
Writing asset: /home/bzub/assets/auth/kubeconfig
Writing asset: /home/bzub/assets/manifests/kube-apiserver-secret.yaml
Writing asset: /home/bzub/assets/manifests/kube-controller-manager-secret.yaml

Kube-router Installation

Next move/delete the manifests for kube-proxy and flannel from assets/manifests and replace them with the kube-router.yaml and kube-router-cfg.yaml files provided in this repo.

rm assets/manifests/kube-flannel{,-cfg}.yaml assets/manifests/kube-proxy.yaml
curl -L https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/contrib/bootkube/kube-router-cfg.yaml -o assets/manifests/kube-router-cfg.yaml
curl -L https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/contrib/bootkube/kube-router.yaml -o assets/manifests/kube-router.yaml

Additional steps when deploying on AWS

Since kube-router uses node routing rules to directly route pod-to-pod traffic to destination node, nodes will send and recieve IP traffic with source and destination IP's from pod CIDR. AWS by default prevents sending and recieving traffic from ip different from instance private ip. So we need to relax this restricition. Please run below commnad on each of the node in the cluster to send and recieve traffic from pod IP's.

aws ec2 modify-instance-attribute --instance-id <instance id>--no-source-dest-check

Cluster Startup

Finally, proceed by following the Bootkube documentation, which generally involves starting Kubelet and running bootkube start referring to your assets directory on a new Kubernetes node.

After starting multiple master nodes, our example cluster looks like this:

$ kubectl -n kube-system get pods,services
NAME                                          READY     STATUS    RESTARTS   AGE       IP          NODE
po/kube-apiserver-gztjp                       1/1       Running   0          15h       10.10.3.2   node2-dev.zbrbdl
po/kube-apiserver-h55t7                       1/1       Running   0          15h       10.10.3.3   node3-dev.zbrbdl
po/kube-apiserver-qn5xm                       1/1       Running   2          15h       10.10.3.1   node1-dev.zbrbdl
po/kube-controller-manager-3052101514-kp121   1/1       Running   1          15h       10.2.0.5    node1-dev.zbrbdl
po/kube-controller-manager-3052101514-n4q9p   1/1       Running   2          15h       10.2.0.6    node1-dev.zbrbdl
po/kube-dns-2431531914-pr9lg                  3/3       Running   0          15h       10.2.0.3    node1-dev.zbrbdl
po/kube-router-ckdj1                          1/1       Running   15         15h       10.10.3.3   node3-dev.zbrbdl
po/kube-router-dcgbr                          1/1       Running   15         15h       10.10.3.1   node1-dev.zbrbdl
po/kube-router-n0vcn                          1/1       Running   15         15h       10.10.3.2   node2-dev.zbrbdl
po/kube-scheduler-2172662190-g4q3w            1/1       Running   4          15h       10.2.0.2    node1-dev.zbrbdl
po/kube-scheduler-2172662190-hcq3t            1/1       Running   2          15h       10.2.0.4    node1-dev.zbrbdl
po/pod-checkpointer-jlfsv                     1/1       Running   0          15h       10.10.3.1   node1-dev.zbrbdl
po/pod-checkpointer-jlfsv-node1-dev.zbrbdl    1/1       Running   0          15h       10.10.3.1   node1-dev.zbrbdl
po/pod-checkpointer-lhckt                     1/1       Running   0          15h       10.10.3.3   node3-dev.zbrbdl
po/pod-checkpointer-lhckt-node3-dev.zbrbdl    1/1       Running   0          15h       10.10.3.3   node3-dev.zbrbdl
po/pod-checkpointer-tsbkh                     1/1       Running   0          15h       10.10.3.2   node2-dev.zbrbdl
po/pod-checkpointer-tsbkh-node2-dev.zbrbdl    1/1       Running   0          15h       10.10.3.2   node2-dev.zbrbdl

NAME                CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE       SELECTOR
svc/kube-dns        10.3.0.10    <none>        53/UDP,53/TCP   15h       k8s-app=kube-dns