Part 1: The Transition Guide from Docker to Orchestration

The Turning Point: From the Docker Box to a True System #

Set aside the tragicomic story about links and awkward ports from the intro. Today we go into the character quietly carrying the whole system: K3s.

If your daily work is SSH-ing into a VPS and running a few Docker containers, everything looks smooth. Push the app onto 3 or 4 servers and the illusion breaks: plain Docker is not enough. A cut link, a power failure, or a machine simply dying — who moves your application to a healthy machine automatically?

That is the job of orchestration, the layer sitting above containers that decides what runs where, how many copies exist, what gets rebuilt when something dies, and how a new version rolls out without downtime. Kubernetes is the standard for that layer.

But installing full Kubernetes makes you sweat, because it is bulky. Fortunately there is K3s. This post lays out the core concepts so you can walk in with confidence.

1. Why K3s? And Where Did Docker Go? #

Briefly: K3s is a lightweight Kubernetes distribution compressed into a single binary under 100MB. It strips out drivers for the major cloud providers and swaps the default etcd database for SQLite.

That last point deserves an explanation, because it is the main reason K3s is so light. Kubernetes needs somewhere to store the state of the whole cluster — what exists, where it runs, how it is configured. Upstream uses etcd, a distributed database designed for dozens of machines staying in sync; it is dependable but eats RAM and requires background processes. SQLite is a file on disk. For a cluster of a few nodes, that swap saves a great deal and costs almost nothing.

The result suits a personal homelab, hardware sitting at the edge, or any distributed setup that needs to be compact without giving up the Kubernetes core.

And here is the fact that surprises people arriving from Docker: K3s (and current Kubernetes) does not use Docker anymore.

• Docker’s job now is building images on your own machine.
• Under K3s, what actually runs containers is containerd — the same container engine Docker itself uses underneath, called directly instead of through Docker.
• You will not type container-start commands. Instead you declare the desired state in YAML, and K3s handles calling containerd to pull the image and run it. This is the biggest shift in thinking: you describe the destination, not the steps.

2. Naming the Resources: Don’t Get Overwhelmed #

To make this digestible, I split K3s concepts into two groups: the ones you write config for daily, and the ones the system handles where understanding the logic is enough.

Group 1: hands-on daily

• Pod: the smallest unit. Do not confuse one Pod with one container — a Pod can hold several containers running together and sharing one network space. Container A reaches container B in the same Pod over localhost, exactly like two processes on one machine.
• Deployment: the lifecycle manager. You declare “I need 3 copies of this app,” and the Deployment keeps 3 Pods running around the clock and handles rolling updates — replacing one Pod at a time, waiting for the new one to be healthy before killing the old one, so users see no interruption.
• Service: internal routing. Pods are short-lived and their IPs change every time they are rebuilt. A Service provides a fixed IP and a name inside the cluster, then forwards to whichever Pods are alive behind it — so everything else only needs to remember the name.
• Ingress: the outer gate, working at the application layer — meaning it can read HTTP content, see which hostname and path a request is asking for, and route accordingly. K3s uses Traefik for this by default.
• Gateway API: the newer routing standard, created to fix Ingress’s weaknesses — specifically by separating what a platform administrator configures from what each application team configures, instead of piling everything into one object. This series uses Envoy Gateway in place of Traefik.
• ConfigMap and Secret: keep configuration and passwords out of the image, so rotating a password does not require a rebuild. Pods read them at startup.
• PersistentVolumeClaim: a request for durable storage. By default everything a Pod writes evaporates when it dies — this keeps the data.

Group 2: infrastructure, understanding is enough

• ReplicaSet: sits between Deployment and Pod, holding the copy count. You rarely write one directly.
• LoadBalancer and ServiceLB: the part that opens a network-layer entry point — it looks at IP and port only, not at content. On cloud, the provider offers a load balancer service; on bare metal there is none, so K3s bundles ServiceLB (Klipper) as a stopgap.
• MetalLB: a real load balancer for bare metal. For better IP control, I will disable the default ServiceLB and use MetalLB instead. It gives the K3s cluster the ability to hand out real IPs on the local network, the same way big cloud providers do.
• Control plane and worker node: the master machine (control plane) is the deciding brain; agent machines (workers) are the limbs running Pods.

3. Data Flow: Tying Everything Together #

Follow one request from a browser into the system. The interesting part of this design is how packets dive into the Tailscale tunnel at the last step.

1. A visitor loads the site, traffic passes through Cloudflare and lands on the public IP of the master out in the cloud.
2. At the master, MetalLB accepts the packet at the network layer and hands it to Envoy Gateway.
3. Envoy reads the header carrying the hostname, sees the visitor wants vinhmdev.com, and forwards to the Service responsible for it.
4. Here is the interesting part. The Service finds the Pods living on a worker machine back at my house. The packet is encrypted and travels through the Tailscale tunnel — backed by the DERP server if the network configuration is too restrictive — hopping from the VPS to the home machine, where it is handled.

From the user’s side nothing looks unusual: they request a domain and receive a web page. The whole detour through a house sits hidden under the network layer.

4. The Network Hub: Tailscale and DERP Server #

In this hybrid cloud architecture the servers do not share a location. For them to “see” each other safely without opening ports to the internet, I bring in Tailscale.

• Tailscale builds a flat private network: every node, whether in Saigon or Singapore, talks as if plugged into the same switch. The hard part it handles for you is punching through NAT — the mechanism that lets a machine on your home network call out but stops anything from calling back in.
• DERP server is Tailscale’s relay. When two nodes cannot connect directly (usually because NAT is too strict), traffic must detour through an intermediary. By default that is a Tailscale machine somewhere in the world; self-hosting DERP means the detour runs on my own infrastructure, closer and faster.

5. Gearing Up for the Next Journey #

The next post starts the installation. Beyond clean Ubuntu LTS servers and SSH keys, note:

• Disable the defaults up front. When installing K3s I will add flags disabling Traefik and ServiceLB from the start, making room for Envoy Gateway and MetalLB.
• Turn off swap. Run sudo swapoff -a. The reason: Kubernetes budgets resources assuming RAM is RAM. With swap enabled, a Pod out of memory quietly grinds to a crawl instead of being killed cleanly, and the scheduler loses its ability to judge which machine actually has room.
• Editor skills. Be ready with vi or nano. SSH-ing in and dancing across YAML in vi is not only survival in a GUI-less environment, it is a genuinely engaging hands-on experience (even if it is mostly copy and paste).

[!NOTE]

What happens if the master machine catches fire?

In Kubernetes terms: losing the master means losing control of the brain — you cannot create new Pods, for instance. But Pods already scheduled onto workers do not die. They stay on their machines and keep running.

The harsh network truth in this architecture, though: I point Cloudflare straight at the master’s public IP, because there is no independent external load balancer. That makes the master double as the traffic gate. If the master stops breathing, the gate closes and nobody on the internet gets in, even though the applications on the workers are alive and well — they just have no route out. This is a single point of failure in the literal sense. Fixing it later means adding an external load balancer, or failing the IP over to another machine automatically.

[!WARNING]

Hard-won note: do NOT install Docker and K3s on the same master machine.

The two ecosystems fight: they compete for network ports, and both write into the system routing table and iptables — each adding rules for its own container network without knowing the other exists, so packets go astray in ways that are very hard to trace. Untangling that mess is exhausting and pointless.

Underneath, K3s already runs the very light containerd, so Docker adds nothing. Best to prepare a genuinely clean server (uninstall Docker) before installing.

See you in Part 2: setting up the VPN with Tailscale.