Compute

The compute layer provides isolated execution environments for workloads — virtual machines for full OS isolation, system containers for lightweight multi-process services, and orchestrated application containers for workloads that are packaged as OCI images. It manages CPU, memory, and local storage allocation, and coordinates workload recovery when hardware fails.

The platform's compute choices are confirmed: Proxmox VE runs virtual machines (KVM/QEMU) and system containers (LXC) and provides cluster high availability; Kubernetes runs on top of it to orchestrate application containers. Each choice is documented below with the reasoning behind it.

The compute layer is the foundation of the stack. Every other layer runs on top of it. Decisions made here — how much isolation workloads get, how resources are allocated, what happens when a node fails — constrain all of the layers above. Getting the compute layer right matters more than optimizing it.

Virtualization

Overview

Virtualization abstracts physical hardware into logical resources that can be allocated to isolated workloads independently. A hypervisor presents each virtual machine with a complete simulated hardware environment — CPU, memory, storage controllers, network interfaces — without the VMs being aware of each other or of the underlying physical host.

Type 1 hypervisors (bare-metal) run directly on hardware without an underlying general-purpose OS. They provide the lowest overhead and the strongest isolation. Type 2 hypervisors run inside a host OS — suitable for development but not for production infrastructure that requires predictable performance and security boundaries. Nyxever runs production workloads on a Type 1 hypervisor: KVM, managed through Proxmox VE.

⚠  Detailed virtualization concepts guide in progress.

Architecture

A virtual machine consists of virtual CPUs (vCPUs), a memory allocation, one or more virtual disks (backed by real storage), and virtual network interfaces. The hypervisor multiplexes these virtual resources onto physical hardware, scheduling vCPUs onto physical cores, mapping guest memory pages to host pages, and routing network traffic through virtual bridges to physical interfaces.

Storage for VMs can be local (on the node) or distributed (shared across nodes). Distributed storage is required for live migration and high availability — a VM backed by local-only storage cannot move to another node without downtime. The choice of storage backend directly affects what the compute layer can do.

⚠  VM architecture diagrams, CPU pinning, NUMA topology, and memory ballooning documentation in progress.

Technologies

The open-source hypervisor landscape for x86-64 server infrastructure centers on KVM (Kernel-based Virtual Machine) as the underlying virtualization engine, with various management layers on top. The management layer determines the operational experience — how VMs are created, configured, monitored, and migrated.

Proxmox VE

Proxmox VE is Nyxever's confirmed virtualization platform. It is an open-source (AGPLv3) hypervisor and container platform built on Debian Linux. It uses KVM for virtual machines and LXC for containers, and provides a web management interface, REST API, and CLI tooling. Cluster management, live migration, high availability, and Ceph and ZFS storage integration are built in.

Why it fits: one management plane for VMs and system containers, built-in clustering and HA, native integration with the storage layer's confirmed choices (Ceph and ZFS), and a cost model that suits a learning platform. The software is fully functional without a subscription — security updates are published to the no-subscription repository; the enterprise subscription adds a more conservatively tested repository and commercial support.

⚠  Proxmox VE installation, cluster setup, and configuration guide in progress.

KVM / QEMU

KVM (Kernel-based Virtual Machine) is a Linux kernel module that turns the kernel into a Type 1 hypervisor. QEMU provides the device emulation layer — virtual disks, network cards, USB controllers — that KVM VMs interact with. Together they form the engine that Proxmox VE (and most Linux-based hypervisors) are built on.

⚠  KVM/QEMU internals and direct management documentation in progress.

Containers

Overview

Containers provide process-level isolation using Linux kernel namespaces and cgroups. Unlike virtual machines, containers share the host kernel — there is no separate OS per container. This makes them lighter than VMs (milliseconds to start, megabytes of overhead) but with weaker isolation guarantees: a kernel vulnerability can potentially affect all containers on a host simultaneously.

System containers (LXC) provide a full Linux userland that shares the host kernel — suitable for services that expect a traditional Unix environment. Application containers (OCI images) package a single process and its dependencies — suitable for portable workloads that are deployed, scaled, and replaced as units.

Nyxever uses each where it fits: LXC system containers on Proxmox VE for lightweight infrastructure services, virtual machines where full kernel isolation is required, and Kubernetes — running on Proxmox VE virtual machines — to orchestrate application containers.

⚠  Container use-case guidance and selection criteria in progress.

Architecture

Container isolation uses three Linux kernel primitives: namespaces (separate views of the filesystem, network, process tree, and user IDs), cgroups (resource limits on CPU, memory, I/O), and capabilities (fine-grained privilege control below root). A well-configured container has only the capabilities it needs and nothing more.

⚠  Namespace isolation, cgroup v2 hierarchy, seccomp profiles, and capability hardening documentation in progress.

Technologies

Two container technologies are confirmed, at different levels of the stack. For infrastructure workloads on a Proxmox VE cluster, LXC system containers integrate directly with Proxmox's management layer — they appear alongside VMs in the same interface and are managed with the same tools. For application containers, Kubernetes provides scheduling, self-healing, and declarative deployment on top of the Proxmox VE cluster.

LXC

Linux Containers (LXC) is a userspace interface for the Linux kernel's container features. LXC system containers run a full init system (systemd or OpenRC) and behave like a lightweight VM from the perspective of the software running inside. Unprivileged LXC containers (where the container's root maps to an unprivileged host user) provide meaningfully better isolation than privileged containers — use unprivileged unless there is a specific reason not to.

⚠  LXC configuration, unprivileged containers, template management, and resource limits documentation in progress.

Kubernetes

Kubernetes is Nyxever's confirmed container orchestration layer. Its cluster nodes run as virtual machines on Proxmox VE, so the hypervisor handles hardware, VM placement, and host-level HA, while Kubernetes handles scheduling, restarting, and scaling application containers. Workloads are described declaratively through the Kubernetes API, which makes deployments reviewable and reproducible.

Kubernetes is built around open interfaces rather than a single implementation, and those interfaces are what we document first: OCI images and runtimes via the Container Runtime Interface (CRI), networking via the Container Network Interface (CNI), and persistent storage via the Container Storage Interface (CSI).

Under evaluation: the Kubernetes distribution and installation method, the container runtime, the CNI implementation (which must support NetworkPolicy), and how Ceph-backed block and file storage is exposed through CSI. Each will be documented with its selection criteria once decided.

⚠  Kubernetes cluster design, node sizing, and integration with Proxmox VE and Ceph documentation in progress.

High Availability

Overview

High availability in the compute layer means that when a node fails, its workloads are automatically restarted on surviving nodes — without manual intervention. HA does not mean zero downtime: there is always a recovery window while fencing completes and workloads restart. The goal is to reduce that window to seconds or minutes, not to eliminate it entirely.

HA requires distributed storage. A VM backed by local-only storage cannot be restarted on another node — its disk is physically unavailable. Shared or distributed storage (in Nyxever, Ceph) is the prerequisite for compute HA. ZFS replication between nodes can narrow the gap for local storage, at the cost of losing writes since the last replication. A cluster without shared storage can pool compute resources but cannot automatically recover from node failures without data loss.

HA works at two levels. Proxmox VE restarts failed VMs and containers on surviving nodes. Kubernetes, running on those VMs, reschedules pods away from failed Kubernetes nodes and relies on its own control-plane quorum (etcd), so Kubernetes control-plane VMs should be spread across separate physical hosts.

⚠  HA planning guide, failure mode analysis, and RPO/RTO documentation in progress.

Architecture

Cluster HA rests on two mechanisms: quorum and fencing. Quorum determines whether a node has the right to take action — a node that cannot communicate with a majority of the cluster must stop starting new workloads to prevent split-brain. Fencing ensures that a node suspected of failure is reliably stopped before its workloads are restarted elsewhere — without fencing, a VM could be running on two nodes simultaneously, causing data corruption.

Fencing comes in two broad forms. External fencing powers a node off through its out-of-band management interface (IPMI/BMC) or a switched power supply. Self-fencing relies on a watchdog: a node that loses quorum stops renewing its watchdog and is reset by it. Proxmox VE HA uses watchdog-based self-fencing; out-of-band management is still listed as a requirement for production deployments because it is how operators recover and inspect a node that has fenced itself.

⚠  Quorum configuration, fencing agent setup, and HA group documentation in progress.

Technologies

Cluster coordination in Linux-based HA stacks is handled by two complementary components: a cluster messaging layer that manages node membership and quorum, and a resource manager that decides which workloads run where and handles failover. In Proxmox VE, the messaging layer is Corosync and the resource manager is Proxmox VE's own built-in HA manager, which uses Corosync quorum and watchdog fencing.

Corosync

Corosync is the cluster communication layer. It maintains a consistent view of which nodes are members of the cluster, handles message passing between cluster nodes, and tracks quorum — the minimum number of nodes that must agree for the cluster to remain operational. In a three-node cluster, quorum requires two nodes to be reachable. A single node losing connectivity does not stop the cluster; a majority losing it does.

⚠  Corosync configuration, quorum settings, and multi-site cluster documentation in progress.