The Tunnel Was the Easy Part: Site-to-Site WireGuard Between My House and a VPC

A.C. Jokela — Mon, 15 Jun 2026 21:30:00 GMT

The Tunnel Was the Easy Part: Site-to-Site WireGuard Between My House and a VPC

For about three years, a PostgreSQL database that runs dozens of projects — portfolio data, home-automation history, a dozen scraped feeds — had its port 5432 listening on the public internet. Not wide open: it sat behind a long password and TLS, the rest of the host was hardened, and nothing had ever gone wrong. But "open to the internet behind a password" is the kind of sentence that sounds fine right up until it doesn't, and I'd been meaning to fix it for months.

The fix I wanted wasn't a bastion host, and it wasn't a clutch of /32 firewall exceptions I'd have to update every time my home IP changed. I wanted the database to simply not be reachable from the public internet — and to reach it anyway, from my house, as if the AWS VPC it lives in were just another room in the building.

That's a site-to-site VPN. And the moment you say "VPN" in 2026 you mean WireGuard, because the alternatives — IPsec, OpenVPN — are heavier, slower, and far more painful to reason about. WireGuard is a few thousand lines of kernel code, a dozen lines of config per side, and a security model you can hold in your head: every peer is a public key, and a packet is either from a key you trust or it's dropped.

So I expected the WireGuard part to be the work. It wasn't. The WireGuard part took about fifteen minutes and came up on the first handshake. The other day and a half went to an IP-address collision I'd created years earlier, a load-balancing router that kept quietly breaking the tunnel's return path, and the slow realization that a VPN doesn't connect two networks so much as force you to confront every assumption each of them made while pretending the other didn't exist.

As an aside, my last dabbling with VPNs and such was during my FreeBSD phase in the early 2000s. To be more precise, February of 2005. I had an article published in Daemon News on using Racoon and X.509 certificates for IPSEC.

This is the story of the parts that weren't WireGuard.

The shape of the thing

Two networks. At home, a flat 10.1.1.0/24 LAN behind a MikroTik router — laptops, a large handful of Linux hosts, a few BSD hosts, the usual home-lab sprawl. In the cloud, an AWS VPC on 10.0.0.0/16 with the database sitting on a private address, 10.0.0.76, that I'd very much like to be the only way to reach it.

The naive instinct is to put WireGuard directly on the database host. Don't. The endpoint that terminates a site-to-site tunnel becomes a router — it forwards packets for an entire subnet, runs ip_forward, owns firewall and NAT rules — and you do not want that responsibility tangled up with your most important stateful service. Give it its own box.

So the tunnel terminates on a dedicated gateway instance: a t4g.nano (the smallest Graviton box AWS sells, a few dollars a month, comfortably inside the free tier) with an Elastic IP so the home side always has a fixed address to dial, and — critically — with its EC2 source/destination check disabled, because by default AWS drops any packet whose source or destination isn't the instance itself. A router's entire job is to handle packets addressed to other machines; leave that check on and the gateway silently swallows everything it's supposed to forward.

The home end of the tunnel lives on one of my always-on Linux boxes. It dials out — the house is behind NAT and a dynamic-ish IP, so it can't be dialed into — and the AWS gateway, with its stable Elastic IP, waits to be reached.

Between them I gave the tunnel its own little address space, a /30 carved out of a range that exists nowhere else in either network: 10.99.99.0/30. The gateway is 10.99.99.1, the house is 10.99.99.2. That deliberate, unused /30 turns out to matter enormously, for reasons that won't be obvious until the routing breaks. Hold onto it.

WireGuard, the easy twenty percent

Here is essentially the entire VPN. On the AWS gateway:

[Interface]
Address = 10.99.99.1/30
ListenPort = 51820
PrivateKey = <gateway private key>
PostUp   = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -D FORWARD -o wg0 -j ACCEPT

[Peer]
# the house
PublicKey  = <home public key>
AllowedIPs = 10.99.99.2/32

And on the home box:

[Interface]
Address = 10.99.99.2/30
PrivateKey = <home private key>
PostUp   = iptables -t nat -A POSTROUTING -o wg0 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o wg0 -j MASQUERADE

[Peer]
# the AWS gateway
PublicKey  = <gateway public key>
Endpoint   = 203.0.113.10:51820     # the gateway's Elastic IP
AllowedIPs = 10.0.0.0/16, 10.99.99.1/32
PersistentKeepalive = 25

wg-quick up wg0 on each end — or systemctl enable --now wg-quick@wg0 to make it survive reboots — and within a second or two wg show reports a handshake. Ping 10.99.99.1 from the house; it answers. The cryptographic part, the part everyone is nervous about, is done.

Two fields carry almost all the meaning. AllowedIPs is doing double duty — it's both an access-control list (packets arriving from this peer are only accepted if their source falls inside it) and a routing table (packets you send to those destinations get encrypted and shipped to this peer). On the home side I list 10.0.0.0/16 because that's the VPC I want to reach; that single line installs a route sending all VPC-bound traffic into the tunnel. PersistentKeepalive = 25 is on the home side only, and it's there because the house initiates from behind NAT: without a packet every 25 seconds, the NAT mapping that lets return traffic find its way home expires, and the tunnel goes deaf until the next outbound packet wakes it. The side with the stable public address doesn't need it; the side hiding behind NAT does.

That's the whole VPN. If your two networks were sensibly addressed and your path were clean, you would be done, and this would be a very short post.

The first wall: you can't route to your own house

The tunnel was up, but a tunnel is just a wire. For a machine inside the VPC to send a reply back to my house, the VPC has to know that house-bound packets go to the gateway instance. That's a route-table entry: destination 10.1.1.0/24 (my home LAN), target the gateway's network interface. I added it.

AWS rejected it:

Route destination doesn't match any subnet CIDR blocks

It took me an embarrassing minute to understand, and then a much longer one to forgive my past self. Years ago, for reasons that made sense at the time and that I have completely forgotten, I had given this VPC a secondary CIDR block of 10.1.0.0/16. My home LAN, 10.1.1.0/24, lives entirely inside that /16. As far as the VPC is concerned, 10.1.1.0/24 is local — it's part of the VPC's own address space — and AWS will not let you install a route that's more specific than a local one. You cannot route to a place the network already believes it is.

This is the oldest mistake in site-to-site networking and I'd walked straight into it: the two address spaces overlapped. RFC 1918 hands you 10.0.0.0/8 — sixteen million addresses — and the one discipline that makes joining networks painless is to carve home and cloud out of ranges that can never collide. I hadn't. And re-addressing a production VPC, or renumbering my entire house, to fix a self-inflicted overlap was not a Tuesday-afternoon project.

So I didn't fix the overlap. I routed around it — and this is where that deliberate, unused /30 earns its place.

One address to stand for a whole house

Look again at the home config: PostUp = iptables -t nat -A POSTROUTING -o wg0 -j MASQUERADE. That one line is doing the heavy lifting. It tells the home box to source-NAT every packet leaving the tunnel so it appears to originate from the tunnel interface's own address, 10.99.99.2. From the VPC's point of view, my entire house — every laptop, every Linux box, the whole 10.1.1.0/24 — collapses into a single client at 10.99.99.2.

And 10.99.99.0/30 lives outside every CIDR the VPC claims. So the route AWS refused to accept for 10.1.1.0/24 is perfectly legal for 10.99.99.0/30: destination 10.99.99.0/30, target the gateway interface. Accepted instantly. The VPC never has to route to the contested 10.1.1.0/24 at all, because it never sees 10.1.1.0/24 — every reply goes to 10.99.99.2, an address it's allowed to know about, and the gateway tunnels it home.

The cost of this trick is honesty about its asymmetry. Masquerade collapses a network into a point, and points don't have interior structure. Traffic the house initiates to the cloud works beautifully; the VPC sees one client and answers it. Traffic the cloud initiates toward an individual home machine has nowhere to land — to AWS, 10.99.99.2 is the whole house, with no way to address the laptop behind it. For my purpose — reaching cloud services from home — that's exactly the direction I needed, and the asymmetry is free. If you need the VPC to open connections into specific home hosts, you're back to real routes and real subnets, which means back to not overlapping your address spaces in the first place. Pick your ranges so that future-you never has to make this choice.

The second wall: a router doing exactly what I told it

With routing sorted, the tunnel worked — and then, intermittently, didn't. The handshake was solid. Small things — a ping, a quick psql connection — went through. But sessions would stall, and throughput was a coin flip. The classic shape of a working control path and a broken data path, which usually screams MTU (more on that later). It wasn't MTU. It was my own router being obedient.

My house has two internet connections, load-balanced on the MikroTik with per-connection-classifier (PCC) mangle rules — the standard dual-WAN recipe that pins each new connection to one uplink or the other so a single flow doesn't get split across two public IPs. Which is the entire problem. WireGuard is one long-lived UDP flow to a fixed Endpoint. The PCC rules looked at the tunnel's packets, classified them like any other connection, and cheerfully load-balanced them — sending some out WAN 1 and some out WAN 2. But the gateway had completed its handshake with whichever public IP came first, and a reply arriving from the other WAN's address didn't match the cryptographic session. WireGuard, correctly, dropped it.

The fix is a single exemption at the top of the mangle chain: traffic to or from the WireGuard endpoint is accepted before the PCC rules ever see it, pinning the entire tunnel to one WAN. In MikroTik terms, a mangle rule matching the WireGuard port and destination with an action=accept ahead of the connection-marking rules. The moment that landed, the flakiness vanished.

The lesson generalizes past MikroTik: a VPN tunnel over a multi-WAN uplink must be pinned to one path. And the deeper lesson is the one I keep relearning — this bug didn't come from the new thing. The router was doing precisely what I had configured it to do, years ago, for a completely unrelated reason. The tunnel just walked into a decision that had been sitting there, correct and forgotten, waiting for a single long-lived UDP flow to make it wrong.

Making it useful: split-horizon DNS

A tunnel that routes private addresses is only half a win. My home cron jobs and scripts didn't connect to 10.0.0.76; they connected to a hostname, and that hostname resolved — on the public internet, as it must — to the database's public address. Routing it privately is pointless if the name still points at the front door.

The answer is split-horizon DNS: the same name resolving differently depending on where you're standing. On the LAN I want db.example.net to mean 10.0.0.76; everywhere else it should keep meaning the public address. Two small overrides do it. A static DNS entry on the MikroTik (db.example.net → 10.0.0.76) catches anything using the router as its resolver, and a matching local-data line in the Unbound instance that actually serves the 10.1.1.0/24 segment catches the rest:

local-zone: "db.example.net." transparent
local-data: "db.example.net. IN A 10.0.0.76"

Now every home machine reaches the database over the tunnel, by name, with zero application changes — the code doesn't know or care that the address it gets back is private. The backup job that used to copy dumps across the public internet now runs entirely inside the tunnel.

One caveat worth stating because it bit me for ten confused minutes: split-horizon is per resolver. My main workstation sits on a different subnet — 192.168.0.x, behind a different router with a different upstream resolver — and it never sees the LAN's Unbound, so it kept resolving the public address and connecting the old way. A DNS horizon only covers the clients that actually ask the resolver you edited. Map your resolvers before you trust the trick.

The gotcha everyone eventually hits: MTU

I promised to come back to MTU, because it is the single most common way a WireGuard tunnel "works but doesn't," and the symptom is exactly the one I thought the dual-WAN bug was: the handshake succeeds, small requests fly, and then a large transfer hangs forever with no error anywhere.

WireGuard wraps every packet in roughly 60–80 bytes of encapsulation, which drops the usable MTU inside the tunnel to about 1420 bytes. On a clean path this is invisible. On a path that drops oversized packets without sending the ICMP "fragmentation needed" message back — and plenty of the internet does, because plenty of the internet firewalls ICMP into oblivion — full-size data packets vanish silently while the small control packets sail through. The cure is MSS clamping: an iptables rule on the forwarding chain that rewrites the TCP maximum-segment-size of passing connections down to fit the tunnel.

iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
  -j TCPMSS --clamp-mss-to-pmtu

In the interest of honesty: my home-to-AWS tunnel didn't need this. A 330 MB database dump rsync'd across it on the first attempt, full speed, no clamp. But a second tunnel I stood up later — a cloud-to-cloud hop over a different path, joining a second provider into the same private fabric — reproduced the "connects fine, transfers hang" symptom perfectly, and the clamp on both ends fixed it instantly. It's path-dependent, which is exactly why it's so maddening to diagnose. Reach for it the instant authentication succeeds but bulk data stalls; it is cheap insurance and an even cheaper first guess.

What it bought

With the tunnel carrying traffic and the name resolving privately, the actual goal became a one-line change. The database's security group dropped its 0.0.0.0/0 rule on 5432 and replaced it with the gateway and the VPC. The port that had been answering the entire internet for a year now answers exactly one place: the wire to my house. The home backup runs over the private path. My laptop talks to the database as if AWS were a closet down the hall.

The bill for all of this is a t4g.nano and an Elastic IP — call it a few dollars a month, much of it inside the free tier — plus a /30 of address space and a handful of config lines that now live in version control next to everything else I'd need to rebuild the setup from scratch. And the VPC stopped being a remote, walled-off place I reach through public endpoints. It became a routable extension of my own network, which changes what feels reasonable to build there next.

The tunnel was the easy part

Add it up and the proportions are absurd. The WireGuard configuration — the cryptography, the key exchange, the part that is genuinely hard to get right and that brilliant people spent years making bulletproof — was a dozen lines per side and worked on the first handshake. Every hour after that went to problems that had nothing to do with WireGuard: an address-space overlap I'd authored years earlier and forgotten, a load-balancing router executing old instructions faithfully into a new failure, a DNS horizon that stopped at a subnet boundary, an MTU gremlin lying in wait on the next path over.

None of those are VPN problems. They're the problems a VPN reveals. Two networks each grew up as the center of their own universe — chose their address ranges, their routing policies, their resolvers — with no expectation of ever meeting. The tunnel is just the introduction. Everything afterward is the negotiation between two designs that were each, locally, perfectly correct, and that turn out to disagree the moment you ask them to share a routing table.

So the practical advice compresses to almost nothing. Give the tunnel its own box and its own unused /30. Keep a keepalive on the side behind NAT. Pin the flow to one WAN if you have more than one. Clamp the MSS the moment large transfers stall. And, above all the rest: choose your private address ranges, on day one, so that home and cloud can never collide — because the fifteen minutes WireGuard costs you is real, and so is the day and a half that an overlapping /16 will cost you somewhere down the line, long after you've forgotten you created it.

The encryption is solved. The hard part was never the secret. The hard part was the addresses.

TinyComputers.io (Posts about Networking)

The Tunnel Was the Easy Part: Site-to-Site WireGuard Between My House and a VPC

The Tunnel Was the Easy Part: Site-to-Site WireGuard Between My House and a VPC

The shape of the thing

WireGuard, the easy twenty percent

The first wall: you can't route to your own house

One address to stand for a whole house

The second wall: a router doing exactly what I told it

Making it useful: split-horizon DNS

The gotcha everyone eventually hits: MTU

What it bought

The tunnel was the easy part