It turns out that docker client only pays attention to the following
environment variables:
DOCKER_HOST to set the url to the docker server.
DOCKER_API_VERSION to set the version of the API to reach, leave empty for latest.
DOCKER_CERT_PATH to load the TLS certificates from.
DOCKER_TLS_VERIFY to enable or disable TLS verification, off by default.
A miss configured DOCKER_MACHINE_NAME won't affect docker client, so k3d
should just ignore the error.
Currently, when the 'docker-machine' command fails, we only logs
"exit status 1'
Which is not very helpful in root cause analysis.
With this patch, the 'docker-machine' command's stderr output is also
logged.
# To cause docker-machine command fail
$ export DOCKER_MACHINE_NAME=xx
$ k3d create
2019/06/13 16:45:31 Created cluster network with ID 6fc91e0e5e912443e6b847f113a6a0bb85ccd610e5232592296d4b199f0347cf
2019/06/13 16:45:31 Error executing 'docker-machine ip'
2019/06/13 16:45:31 Docker machine "xx" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.
exit status 1
We have been giving user warnings about --timeout going away for a
while. Now fully deprecate it.
Use -t for --wait option
Move -w from --wait to --workers. Many users of K3d generate multi-node
clusters, -w will make their life easier.
To allow some flexibilities in how user specifies the --api-port
argument. In case the 'host' is an string, We will try to convert
it into an IP address for port mapping. IP address is also allowed.
The APIs of createServer() and createWorker() takes too many arguments.
It is getting harder to maintain.
Use a new struct ClusterSpec to make API simpler. This also reduces
some code duplications.
When running on a docker machine, the default X598 certificate does not
allow access via docker machine's IP. This patch fixes this by adding
"--tls-san <docker machine IP>" to the K3S server argument list.
The kubeconfig.yaml generated by K3S uses local host as the host name by
default. It won't work when running with docker machine.
This patch detects if the docker environment is set up with docker
machine. If it is, then replace the host name to the IP address of
the docker machine.
In theory, we can execute 'k3d bash' again within an cluster shell. I
can't think of any practical value for allowing this capability.
On the contrary, this can lead to confusion to the user.
This patch adds a simple mechanism to detect and block recursive bash
invocation.
In addition to provide an interactive shell, this patch adds the
'--command' and '-c' options to allow user to issue a command in the
context of a cluster.
For example:
$ k3d bash -c 'kubectl cluster-info'
OS distribution and user may choose to install bash in different path.
This patch uses bash found by "$PATH" environment, rather than hard code
the bash path.
This patch also handle the case 'bash' are not found. Mostly likely due
to bash not being supported by the platform, or it is not installed.
Add the basic frame work for supporting spawning a bash shell by cli
command.
With this change, we can spawn a bash shell in the context of a cluster
$ k3d create -n my-cluster
$ k3d bash -n my-cluster
[my-cluster] $>
// execute commands with KUBECONFIG already set up
[my-cluster] $> kubectl get pods
Co-authored-by: Thorsten Klein <iwilltry42@gmail.com>
@iwilltry42 pointed out that currnet error path calls os.Exit(1), which
does follow the normal CLI framework. This patch implements this
suggestion.
Add a closure fucntion deleteCluster() to reduce code duplication.
When creating clusters with the --auto-restart flag, any running cluster
will remain "running" up on docker daemon restart.
By default, without this flag, a "running" cluster becomes "stopped"
after docker daemon restart.
Clusters stopped with 'k3d stop' command will remain stopped after
docker daemon restart regardless the settings of this flag.
Before this commit, when creating a cluster was not successful, some
resource may be lingering behind. This commit fixes those cases.
Now the cluster creation model is easier to understand. Either a cluster
is successfully created, or all resources created will be cleaned up.
The intention is to keep the "atomic create" model going forward for
k3d.
Thanks @zeerorg for the suggestion on possible container volume leak.
With out this fix the k3s container volumes are left in the reclaimable
state. This experiment confirms it:
$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 14 0 2.131GB 2.131GB (100%)
Containers 0 0 0B 0B
Local Volumes 0 0 0B 0B
Build Cache 0 0 0B 0B
$ bin/k3d create; sleep 5; bin/k3d delete
$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 14 0 2.131GB 2.131GB (100%)
Containers 0 0 0B 0B
Local Volumes 3 0 2.366MB 2.366MB (100%)
Build Cache 0 0 0B 0B
In this case, 2.36MB are left in the reclaimable state. This number can be
larger with a larger cluster.
With this fix, output of "docker system df" does not contain the
claimable volume
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 14 0 2.131GB 2.131GB (100%)
Containers 0 0 0B 0B
Local Volumes 0 0 0B 0B
Build Cache 0 0 0B 0B
This change improve the error message. It also avoids the tricky situation of
potentially destroying an existing clusters when the same name cluster
creation fails. For more details see issue #42.
Most k3d arguments are using in "stringSlice" style, allowing the
argument to supplied multiple times. Currently "volume" is an exception
to this style, require multiple arguments to be supplied in a single
argument, separated by comma.
This commit improve the k3d usability by improve the consistency of its
argument style.
Before this change, k3d list stopped cluster as "exited", This patch
change it to "stopped".
$ k3d create -n test
$ k3d stop -n test
$ k3d list --all
+------+------------------------------+--------+---------+
| NAME | IMAGE | STATUS | WORKERS |
+------+------------------------------+--------+---------+
| test | docker.io/rancher/k3s:v0.5.0 | exited | 0/0 |
+------+------------------------------+--------+---------+
Before this change, k3d report the server status as the cluster status.
This commit output the server status if the server and all worker
containers status agree. Otherwise, the cluster status is reported as "unhealthy".
$ k3d create --workers 2
$ docker ps -a
d14135f5929c rancher/k3s:v0.5.0 "/bin/k3s agent" 6 seconds ago Up 5 seconds k3d-k3s-default-worker-1
612d71f3ec23 rancher/k3s:v0.5.0 "/bin/k3s agent" 6 seconds ago Up 5 seconds k3d-k3s-default-worker-0
7f201731bf45 rancher/k3s:v0.5.0 "/bin/k3s server --h…" 7 seconds ago Up 6 seconds 0.0.0.0:6443->6443/tcp k3d-k3s-default-server
$ docker stop d14135f5929c
$ bin/k3d list --all
+-------------+------------------------------+-----------+---------+
| NAME | IMAGE | STATUS | WORKERS |
+-------------+------------------------------+-----------+---------+
| k3s-default | docker.io/rancher/k3s:v0.5.0 | unhealthy | 1/2 |
+-------------+------------------------------+-----------+---------+
Before this patch:
$ k3d create
$ k3d stop
$ k3d list
+------+-------+--------+---------+
| NAME | IMAGE | STATUS | WORKERS |
+------+-------+--------+---------+
+------+-------+--------+---------+
This commit omits such empty table.
Make list output center aligne. It is slightly eaier to read.
Before the commit:
$ bin/k3d list --all
+-------------+------------------------------+---------+---------+
| NAME | IMAGE | STATUS | WORKERS |
+-------------+------------------------------+---------+---------+
| k3s-default | docker.io/rancher/k3s:v0.5.0 | running | 2/2 |
+-------------+------------------------------+---------+---------+
After this commit:
$ bin/k3d list --all
+-------------+------------------------------+---------+---------+
| NAME | IMAGE | STATUS | WORKERS |
+-------------+------------------------------+---------+---------+
| k3s-default | docker.io/rancher/k3s:v0.5.0 | running | 2/2 |
+-------------+------------------------------+---------+---------+
In this output, only the last cell shows up differently.
MAke sure the cluster name is a RFC 1123 compliant host name, since the
cluster name is used as base for expanding into a auto generated
host names for k3s nodes.
All ports exposed by --publish will also be exported for all worker
nodes. The host port will be auto indexed based worker id.
For example: with the following command option:
k3d create --publish 80:80 --publish 90:90/udp --workers 1
The exposed ports will be:
host TCP port 80 -> k3s server TCP 80
host TCP port 90 -> k3s server TCP 90
host UDP port 81 -> k3s worker 0 UDP 80
host UDP port 91 -> k3s worker 0 UDP 90
