A VM Per Client: Isolating Against Supply Chain Attacks

Supply chain attacks aren’t new, but lately it feels a lot more likely that I’ll get caught by one. The thought of getting pwned by a malicious npm package, pulled in by some sub-sub-sub dependency I’ve never heard of, keeps me up at night sometimes.

There’s a lot of sensitive stuff on a developer machine. Source code for every project, SSH keys, cloud credentials, API tokens, a .env file or two. If I run npm install on one client’s project and that pulls in a compromised dependency, that code runs as me, on a machine that can reach everything. The problem isn’t just one project. It’s all of them at once.

That’s what pushed me to change how I work.

One VM Per Client

In my remote VM development setup, I described running a single development VM on Proxmox that I connect to from my Mac Studio and MacBook. It’s been great. But I didn’t have a process to provision it, so I ended up co-locating multiple client projects like one does on their typical development machine.

So I split it up: one VM per client.

Each client gets its own isolated environment. Their repos, their tooling, their credentials, all contained. If a dependency in Client A’s project turns out to be malicious, the worst it can reach is Client A’s VM. It can’t read Client B’s source, it can’t grab Client B’s tokens, and it can’t pivot to anything else.

It’s like a sandbox per client.

Provisioning with Ansible

The main thing I was lacking was a repeatable way to stand up each development VM. What I landed on splits into two phases:

A minimal base template. The template is deliberately bare: a stock Debian cloud image converted to a Proxmox template, with just enough cloud-init to move onto the next step. It sets up my user, my SSH key, the guest agent. It exists to be cloned and nothing else.

Provisioning does the real work. Almost everything that makes a box usable: dotfiles, tmux, Docker, mise for language runtimes, and the rest of my baseline toolchain lives in a single provisioning playbook that runs after the clone.

Keeping the template thin and the provisioning fat is deliberate. When I change my toolchain, I don’t rebuild a template and re-clone from it. I just re-run provisioning or blow the VM away and recreate it from scratch in one command.

There’s a second reason to keep the template minimal. Every base template I’ve hand-built in the past eventually rotted. The base OS moved on, packages went stale, and a few months later I’d start from scratch. Having a playbook that builds the template means the template itself is disposable too.

The One Rough Edge: Credentials

Right now I pull secrets at runtime with 1Password Service Account Tokens and fnox, rather than storing them on disk. It works, but the DX is annoying enough that I keep exploring other options.

The thing I want to try next is the 1Password Connect Server as a self-hosted secrets broker in my homelab. I haven’t set it up yet, but it looks like it could smooth out the per-VM friction. The main advantage over service account tokens is that the token requires access to the Connect server, and that server only listens on my LAN and tailnet. If a token leaks off a VM, an attacker on the public internet can’t do anything with it without access to my network. Unless it’s a targeted attack, it seems hackers are mostly interested in hoovering up access tokens.

Trade-offs

There are some downsides to my setup, but so far it’s been well worth it for me.

  • More VMs to manage: There are more environments now. Ansible keeps this from getting painful, but it’s still more moving parts.
  • Resource overhead: Each VM wants its own CPU, RAM, and disk on the Proxmox host. Thankfully I maxed out the RAM on my development box before it got expensive.
  • Credential friction: As above, the secrets story isn’t as smooth as I’d like yet.
  • Configuration drift: Tweaks I make on a live VM don’t make their way back into the playbook unless I remember to update it or my dotfiles repo.

For me, the containment is worth all of it.

Final Thoughts

Per-client VMs don’t solve the problem, but they lower the blast radius. Some clients prioritize security and maintenance more than others. When everything shares one machine, the whole setup effectively inherits the risk level of the weakest one.