[Enhancement] docs: add a project overview (#680)
This commit is contained in:
Thorsten Klein
GitHub
parent
cb02f6bce1
commit
3b8c877b43
@ -7,6 +7,8 @@ We welcome everyone who likes to use and improve our software.
|
||||
|
||||
Before starting to work with and on k3d, please read and understand our [**Code of Conduct**](./CODE_OF_CONDUCT.md).
|
||||
|
||||
Get an Overview of the k3d project in the documentation: [k3d.io/internals/project](https://k3d.io/internals/project)
|
||||
|
||||
Before opening an issue or a Pull-Request, please use GitHub's search function to check whether something similar is already in process and hook in there instead.
|
||||
|
||||
## Get Recognized
|
||||
|
||||
@ -112,6 +112,8 @@ k3d is a community-driven project and so we welcome contributions of any form, b
|
||||
|
||||
Please read our [**Contributing Guidelines**](./CONTRIBUTING.md) and the related [**Code of Conduct**](./CODE_OF_CONDUCT.md).
|
||||
|
||||
You can find an overview of the k3d project (e.g. explanations and a repository guide) in the documentation: [k3d.io/internals/project](https://k3d.io/internals/project)
|
||||
|
||||
[](code_of_conduct.md)
|
||||
|
||||
## Contributors ✨
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
title: Internals
|
||||
nav:
|
||||
- defaults.md
|
||||
- networking.md
|
||||
- project.md
|
||||
- networking.md
|
||||
|
||||
109
docs/internals/project.md
Normal file
109
docs/internals/project.md
Normal file
@ -0,0 +1,109 @@
|
||||
# Project Overview
|
||||
|
||||
## About This Page
|
||||
|
||||
On this page we'll try to give an overview of all the moving bits and pieces in k3d to ease contributions to the project.
|
||||
|
||||
## Directory Overview
|
||||
|
||||
- [`.github/`](https://github.com/rancher/k3d/tree/main/.github)
|
||||
- templates for issues and pull requests
|
||||
- GitHub Action workflow definitions
|
||||
- [`cmd/`](https://github.com/rancher/k3d/tree/main/cmd)
|
||||
- everything related to the actual k3d CLI, like the whole command tree, config initialization, argument parsing, etc.
|
||||
- [`docgen/`](https://github.com/rancher/k3d/tree/main/docgen)
|
||||
- sub-module used to auto-generate the documentation for the CLI commands, which ends up in [`docs/usage/commands/`](https://github.com/rancher/k3d/tree/main/docs/usage/commands)
|
||||
- [`docs/`](https://github.com/rancher/k3d/tree/main/docs)
|
||||
- all the resources used to build [k3d.io](https://k3d.io) using mkdocs
|
||||
- [`pkg/`](<https://github.com/rancher/k3d/tree/main/pkg>)
|
||||
- the place where the magic happens.. here you find all the main logic of k3d
|
||||
- all function calls within [`cmd/`](https://github.com/rancher/k3d/tree/main/cmd) that do non-trivial things are imported from here
|
||||
- this (or rather sub-packages) is what other projects would import as a module to work with k3d without using the CLI
|
||||
- [`proxy/`](https://github.com/rancher/k3d/tree/main/proxy)
|
||||
- configuration to build the [`rancher/k3d-proxy`](https://hub.docker.com/r/rancher/k3d-proxy/) container image which is used as a loadbalancer/proxy in front of (almost) every k3d cluster
|
||||
- this is basically just a combination of NGINX with confd and some k3d-specific configuration details
|
||||
- [`tests/`](https://github.com/rancher/k3d/tree/main/tests)
|
||||
- a set of bash scripts used for end-to-end (E2E) tests of k3d
|
||||
- mostly used for all the functionality of the k3d CLI which cannot be tested using Go unit tests
|
||||
- [`tools/`](https://github.com/rancher/k3d/tree/main/tools)
|
||||
- sub-module used to build the [`rancher/k3d-tools`](https://hub.docker.com/r/rancher/k3d-tools) container image which supports some k3d functionality like `k3d image import`
|
||||
- [`vendor/`](https://github.com/rancher/k3d/tree/main/vendor)
|
||||
- result of `go mod vendor`, which contains all dependencies of k3d
|
||||
- [`version/`](https://github.com/rancher/k3d/tree/main/version)
|
||||
- package used to code k3d/k3s versions into releases
|
||||
- this is where `go build` injects the version tags when building k3d
|
||||
- that's the output you see when issuing `k3d version`
|
||||
|
||||
## Packages Overview
|
||||
|
||||
- [`pkg/`](https://github.com/rancher/k3d/tree/main/pkg)
|
||||
- [`actions/`](https://github.com/rancher/k3d/tree/main/pkg/actions)
|
||||
- hook actions describing actions (commands, etc.) that run at specific stages of the node/cluster lifecycle
|
||||
- e.g. writing configuration files to the container filesystem just before the node (container) starts
|
||||
- [`client/`](https://github.com/rancher/k3d/tree/main/pkg/client)
|
||||
- all the top level functionality to work with k3d primitives
|
||||
- create/retrieve/update/delete/start/stop clusters, nodes, registries, etc. managed by k3d
|
||||
- [`config/`](https://github.com/rancher/k3d/tree/main/pkg/config)
|
||||
- everything related to the k3d configuration (files), like `SimpleConfig` and `ClusterConfig`
|
||||
- [`runtimes/`](https://github.com/rancher/k3d/tree/main/pkg/runtimes)
|
||||
- interface and implementations of runtimes that power k3d (currently, that's only Docker)
|
||||
- functions in [`client/`](https://github.com/rancher/k3d/tree/main/pkg/client) eventually call runtime functions to "materialize" nodes and clusters
|
||||
- [`tools/`](https://github.com/rancher/k3d/tree/main/pkg/tools)
|
||||
- functions eventually calling the [`k3d-tools`](https://hub.docker.com/r/rancher/k3d-tools) container (see [`tools/`](https://github.com/rancher/k3d/tree/main/tools) in the repo root)
|
||||
- [`types/`](https://github.com/rancher/k3d/tree/main/pkg/types)
|
||||
- definition of all k3d primitives and many other details and defaults
|
||||
- e.g. contains the definition of a `Node` or a `Cluster` in k3d
|
||||
- [`util/`](https://github.com/rancher/k3d/tree/main/pkg/util)
|
||||
- some helper functions e.g. for string manipulation/generation, regexp or other re-usable usages
|
||||
|
||||
## Anatomy of a Cluster
|
||||
|
||||
By default, every k3d cluster consists of at least 2 containers (nodes):
|
||||
|
||||
1. (optional, but default and strongly recommended) loadbalancer
|
||||
|
||||
- image: [`rancher/k3d-proxy`](https://hub.docker.com/r/rancher/k3d-proxy/), built from [`proxy/`](https://github.com/rancher/k3d/tree/main/proxy)
|
||||
- purpose: proxy and load balance requests from the outside (i.e. most of the times your local host) to the cluster
|
||||
- by default, it e.g. proxies all the traffic for the Kubernetes API to port `6443` (default listening port of K3s) to all the server nodes in the cluster
|
||||
- can be used for multiple port-mappings to one or more nodes in your cluster
|
||||
- that way, port-mappings can also easily be added/removed after the cluster creation, as we can simply re-create the proxy without affecting cluster state
|
||||
|
||||
2. (required, always present) primary server node
|
||||
|
||||
- image: [`rancher/k3s`](https://hub.docker.com/r/rancher/k3s/), built from [`github.com/k3s-io/k3s`](https://github.com/k3s-io/k3s)
|
||||
- purpose: (initializing) server (formerly: master) node of the cluster
|
||||
- runs the K3s executable (which runs containerd, the Kubernetes API Server, etcd/sqlite, etc.): `k3s server`
|
||||
- in a multi-server setup, it initializes the cluster with an embedded etcd database (using the K3s `--cluster-init` flag)
|
||||
|
||||
3. (optional) secondary server node(s)
|
||||
|
||||
- image: [`rancher/k3s`](https://hub.docker.com/r/rancher/k3s/), built from [`github.com/k3s-io/k3s`](https://github.com/k3s-io/k3s)
|
||||
|
||||
4. (optional) agent node(s)
|
||||
|
||||
- image: [`rancher/k3s`](https://hub.docker.com/r/rancher/k3s/), built from [`github.com/k3s-io/k3s`](https://github.com/k3s-io/k3s)
|
||||
- purpose: running the K3s agent process (kubelet, etc.): `k3s agent`
|
||||
|
||||
## Automation (CI)
|
||||
|
||||
The k3d repository mainly leverages the following two CI systems:
|
||||
|
||||
- GitHub Actions
|
||||
- 2 workflows in <https://github.com/rancher/k3d/tree/main/.github/workflows> to push the artifact to AUR (Arch Linux User Repository)
|
||||
- logs/history can be seen in the Actions tab: <https://github.com/rancher/k3d/actions>
|
||||
- DroneCI
|
||||
- a set of pipelines in a single file: <https://github.com/rancher/k3d/blob/main/.drone.yml>
|
||||
- static code analysis
|
||||
- build
|
||||
- tests
|
||||
- docker builds + pushes
|
||||
- render + push docs
|
||||
- (pre-) release to GitHub
|
||||
- `push` events end up here (also does the releases, when a tag is pushed): <https://drone-publish.rancher.io/rancher/k3d>
|
||||
- `pr`s end up here: <https://drone-pr.rancher.io/rancher/k3d>
|
||||
|
||||
## Documentation
|
||||
|
||||
The website [k3d.io](https://k3d.io) containing all the documentation for k3d is built using [`mkdocs`](https://www.mkdocs.org/), configured via the [`mkdocs.yml`](https://github.com/rancher/k3d/blob/main/mkdocs.yml) config file with all the content residing in the [`docs/`](https://github.com/rancher/k3d/tree/main/docs) directory (Markdown).
|
||||
Use `mkdocs serve` in the repository root to build and serve the webpage locally.
|
||||
Some parts of the documentation are being auto-generated, like [`docs/usage/commands/`](https://github.com/rancher/k3d/tree/main/docs/usage/commands) is auto-generated using Cobra's command docs generation functionality in [`docgen/`](https://github.com/rancher/k3d/tree/main/docgen).
|
||||
153
thoughts.md
153
thoughts.md
@ -1,153 +0,0 @@
|
||||
# Thoughts
|
||||
|
||||
## Repository/Package Overview
|
||||
|
||||
- `cmd/`: everything around the CLI of k3d = human interface, printed output (e.g. list of clusters)
|
||||
- `pkg/`: everything else, can be used as a module from other Go projects
|
||||
- `cluster/`: everything around managing cluster components
|
||||
- `runtimes/`: translate k3d types (node, cluster, etc.) to container runtime specific types and manage them
|
||||
- `types/`: collection of types (structs) and constants used by k3d
|
||||
- `util/`: utilities, that could be used for everything, not directly related to the project
|
||||
|
||||
## k3d types <-> runtime translation
|
||||
|
||||
k3d _should_ work with more than one runtime, if we can implement the Runtime interface for it.
|
||||
Here's how k3d types should translate to a runtime type:
|
||||
|
||||
- `cluster` = set of _containers_ running in the same _network_, maybe mounting the same _volume(s)_
|
||||
- `node` = _container_ with _exposed ports_ and _volume mounts_
|
||||
|
||||
### Docker
|
||||
|
||||
#### Node to Container translation
|
||||
|
||||
`container = "github.com/docker/docker/api/types/container"`
|
||||
`network = "github.com/docker/docker/api/types/network"`
|
||||
|
||||
- Name -> container.Hostname = node.Name
|
||||
- Role -> container.Labels["k3d.role"] = node.Role
|
||||
- Image -> container.Image = node.Image
|
||||
- Volumes -> container.HostConfig.PortBindings
|
||||
- Env ->
|
||||
- Args ->
|
||||
- Ports ->
|
||||
- Restart ->
|
||||
- Labels -> container.Labels
|
||||
|
||||
## Node Configuration
|
||||
|
||||
- server node(s)
|
||||
- ENV
|
||||
- `K3S_CLUSTER_INIT`
|
||||
- if num_servers > 1 && no external datastore configured
|
||||
- `K3S_KUBECONFIG_OUTPUT`
|
||||
- k3d default -> `/output/kubeconfig.yaml`
|
||||
- CMD/ARGS
|
||||
- `--https-listen-port`
|
||||
- can/should be left default (unset = 6443), since we handle it via port mapping
|
||||
- `--tls-san=<some-ip-or-hostname>`
|
||||
- get from `--api-port` k3d flag and/or from docker machine
|
||||
- Runtime Configuration
|
||||
- nothing special
|
||||
- all nodes
|
||||
- ENV
|
||||
- `K3S_TOKEN` for node authentication
|
||||
- CMD/ARGS
|
||||
- nothing special
|
||||
- Runtime Configuration
|
||||
- Volumes
|
||||
- shared image volume
|
||||
- cluster-specific (create cluster) or inherit from existing (create node)
|
||||
- tmpfs for k3s to work properly
|
||||
- `/run`
|
||||
- `/var/run`
|
||||
- Capabilities/Security Context
|
||||
- `privileged`
|
||||
- Network
|
||||
- cluster network or external/inherited
|
||||
- agent nodes
|
||||
- ENV
|
||||
- `K3S_URL` to connect to server node
|
||||
- server hostname + port (6443)
|
||||
- cluster-specific or inherited
|
||||
- CMD/ARGS
|
||||
- nothing special
|
||||
- Runtime Configuration
|
||||
- nothing special
|
||||
|
||||
## Features
|
||||
|
||||
## [DONE] Node Filters
|
||||
|
||||
- `--port [host:]port[:containerPort][/protocol][@group_identifier[[index] | @node_identifier]`
|
||||
- Examples:
|
||||
- `--port 0.0.0.0:8080:8081/tcp@agents` -> whole group
|
||||
- `--port 80@agents[0]` -> single instance of group by list index
|
||||
- `--port 80@agents[0,2-3]` -> multiple instances of a group by index lists and ranges
|
||||
- `--port 80@k3d-test-agent-0` -> single instance by specific node identifier
|
||||
- `--port 80@k3d-test-server-0@agents[1-5]` -> multiple instances by combination of node and group identifiers
|
||||
|
||||
- analogous for volumes
|
||||
|
||||
## [WIP] Multi-Server Setup
|
||||
|
||||
- to make this possible, we always deploy a load-balancer (nginx) in front of the server nodes as an extra container
|
||||
- consider that in the kubeconfig file and `--tls-san`
|
||||
|
||||
### Variants
|
||||
|
||||
- [x] embedded datastore (dqlite)
|
||||
- if `--servers` > 1 deploy a load-balancer in front of them as an extra container
|
||||
- [ ] external datastore
|
||||
|
||||
## [DONE] Keep State in Docker Labels
|
||||
|
||||
- when creating a cluster, usually, you also create a new docker network (and maybe other resources)
|
||||
- store a reference to those in the container labels of cluster nodes
|
||||
- when deleting the cluster, parse the labels, deduplicate the results and delete the additional resources
|
||||
- DONE for network
|
||||
- new labels `k3d.cluster.network=<ID>` and `k3d.cluster.network.external=<true|false>` (determine whether to try to delete it when you delete a cluster, since network may have been created manually)
|
||||
|
||||
## Bonus Ideas
|
||||
|
||||
### Tools
|
||||
|
||||
- maybe rename `k3d load` to `k3d tools` and add tool cmds there?
|
||||
- e.g. `k3d tools import-images`
|
||||
- let's you set tools container version
|
||||
- `k3d tools --image k3d-tools:v2 import-images`
|
||||
- add `k3d create --image-vol NAME` flag to re-use existing image volume
|
||||
- will add `k3d.volumes.imageVolume.external: true` label to nodes
|
||||
- should not be deleted with cluster
|
||||
- possibly add `k3d create volume` and `k3d create network` to create external volumes/networks?
|
||||
|
||||
### Prune Command
|
||||
|
||||
- `k3d prune` to prune all dangling resources
|
||||
- nodes, volumes, networks
|
||||
|
||||
### Use Open Standards (OCI, CRI, ...)
|
||||
|
||||
- [https://github.com/opencontainers/runtime-spec/blob/master/specs-go/config.go](https://github.com/opencontainers/runtime-spec/blob/master/specs-go/config.go)
|
||||
- move node -> container translation out of runtime
|
||||
|
||||
### Private registry
|
||||
|
||||
- create a private registry to be used by k3d clusters
|
||||
- similar to [https://github.com/rancher/k3d/pull/161](https://github.com/rancher/k3d/pull/161)
|
||||
- add `k3d create registry` command to create external registry (maybe instead of flags as in PR #161?)
|
||||
|
||||
### Syntactical shortcuts for k3d v1 backwards compatibility
|
||||
|
||||
- e.g. `k3d create` -> `k3d create cluster k3s-default`
|
||||
|
||||
### Unsorted Ideas
|
||||
|
||||
- Integrate build tool (e.g. buildkit, buildah, ...)
|
||||
- use `tools.go` to keep tools (like `golangci-lint` and `gox`) dependencies
|
||||
- see e.g. [https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module](https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module)
|
||||
- see e.g. [https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module](https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module)
|
||||
|
||||
### Possible Enhancements
|
||||
|
||||
- [!] remove/add nodes -> needs to remove line in `/var/lib/rancher/k3s/server/cred/node-passwd` for the deleted node
|
||||
Loading…
Reference in New Issue
Block a user