Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,7 @@ Launch a full Ubuntu VM with Docker pre-installed. Shows ~90ms boot time with sy
### Snapshot & Commit
[![asciicast](https://asciinema.org/a/1257525.svg)](https://asciinema.org/a/1257525)

Persist disk state between VM runs: install packages, create files, then commit to a new image with `nerdctl commit`. The next VM boots with all changes preserved - like Docker commits, but for entire VMs.

> **Note**: Snapshot support (for EROFS) requires a custom containerd build from [aledbf/containerd@aledbf/erofs-snapshot-narrow](https://github.com/aledbf/containerd/tree/aledbf/erofs-snapshot-narrow) until the changes are upstreamed.
Persist disk state between VM runs: install packages, create files, then commit to a new image - with the bundled `spinbox-commit` (no nerdctl required) or with `nerdctl commit`. `spinbox-commit` also supports **hot commit** (pause + freeze the running VM, commit, resume). The next VM boots with all changes preserved - like Docker commits, but for entire VMs. See [docs/hot-commit.md](docs/hot-commit.md).

---

Expand Down
44 changes: 44 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,6 +518,50 @@ memory_hotplug validation failed: increment_size_mb must be 128MB-aligned, got 1
```
**Solution**: Use 128MB-aligned value (128, 256, 512, 1024, etc.).

## Boot Profiling

Boot profiling is **opt-in** and adds no overhead to normal boots. It is not
part of the JSON config file - it is enabled per container via an OCI
annotation, or host-wide via an environment variable.

**Per container** (recommended) - set the annotation on the container spec:

```
io.spin.debug.boot = "true" # also accepts "1" / "yes"
```

**Host-wide** - set on the shim/containerd environment:

```
SPINBOX_DEBUG_BOOT=1
```

When enabled, the VM boots with:
- `initcall_debug` + verbose kernel output (per-initcall timings),
- the kernel console routed through **virtio-console (`hvc0`)** instead of the
emulated UART (`ttyS0`), so the verbose output does not backpressure the guest
and skew the timings,
- `log_buf_len=4M` so the early initcalls are not lost to ring-buffer overflow,
- a `spin.profile` cmdline marker that makes vminitd emit its own phase timings.

### Profiling output

All metrics are grep-able lines. The kernel ones land in the per-VM console log
(`<log_dir>/<container>/console.log`); the shim/vminitd ones in the shim journal
and console log.

| Line | Source | What it measures |
|------|--------|------------------|
| `BOOT_TIMELINE` | shim | wall-clock cold-start: `qemu_launch` (exec→QMP) vs `guest_boot` (→vsock ready) |
| `BOOT_METRIC` | test | `task.Start()` → entrypoint output (container start in a ready VM) |
| `VMINITD_READY` | vminitd | CLOCK_BOOTTIME milestones: `pid1-entry` (kernel boot), `system-init`, `vsock-listen`, `serve`, `first-accept` |
| `VMINITD_PROFILE` | vminitd | per-phase deltas of `system.Initialize` |
| `KERNEL_PROFILE` | test | slowest initcalls from the console (late initcalls only) |
| `KMSG_PROFILE` | vminitd | **complete** initcall set from `/dev/kmsg` (incl. early ACPI/PCI) |

The integration suite exercises these (`TestBootTimeline`, `TestVminitdReady`,
`TestKernelBootProfile`, `TestKernelBootProfileComplete`, `TestUserspaceBootProfile`).

## See Also

- [README.md](../README.md) - Main documentation
Expand Down
4 changes: 2 additions & 2 deletions internal/guest/vminit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The vminit package implements the guest-side init daemon (PID 1) that runs insid
│ │ (system/) │────▶│ ├─ Task Service (Create/Start/etc) │ │
│ │ • Mounts │ │ ├─ Streaming Service (I/O) │ │
│ │ • Cgroups v2 │ │ └─ Events Service │ │
│ │ • DNS │ └─────────────────────────────────────┘ │
│ │ • Network+DNS │ └─────────────────────────────────────┘ │
│ └─────────────────┘ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────┐ │
Expand All @@ -34,7 +34,7 @@ vminit/
├── task/ # Task service (containerd API implementation)
├── runc/ # Container abstraction over OCI runtime
├── process/ # Init and Exec process management
├── system/ # VM system initialization (mounts, cgroups)
├── system/ # VM system init (mounts, cgroups, networking via netlink, DNS)
├── streaming/ # vsock-based I/O streaming
├── events/ # Event publishing to containerd
├── devices/ # Block device handling
Expand Down