So I had this Mac Mini that I was originally using as a WiFi relay bridge (layer 2) but since then I was only using it as an rsync backup server. I decided to also turn it into a VPN tunnelling router (layer 3) for my home network setup by using OpenVPN on MacOS (connected to a Debian Linux server). I was testing out its performance and it was holding up pretty good and stable but I wanted to give WireGuard a try as well, just to see and compare. Upon initial configuration, I noticed that WireGuard was defaulting to a lower MTU (1420) compared to what I had set in OpenVPN (1450) and that some of my connections to websites were unstable/slow/hanging. Additionally, with OpenVPN, I was able to set some extra settings like MSS & fragmentation limits and I couldn’t find the equivalent of those with WireGuard. However, I do like the overall simplicity of the WG config and setup process!

I actually lowered the WG MTU (1410) but this could potentially cause larger size packets to fragment. I was searching around and found that it was possible for one to clamp the MSS values on forwarded/routed packets with an iptables forward/mangle rule. Since the Mac is the VPN client and has the BSD PF firewall, I also set a scrub rule as well:

# bsd pf
scrub all no-df random-id max-mss 1330
 
# nix nf
iptables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1330
 
# iptables-translate ftw!

I was also trying to read more about the “don’t fragment” bit that could get set on packets along the way (sometimes when the PMTUD setting is enabled) and the pf-scrub rule can help to clear that bit (no-df) where I couldn’t yet find the equivalent on the Linux side. I also tried setting these sysctl values as well:

# mac
sysctl net.inet.tcp.path_mtu_discovery=0
 
# nix
sysctl net.ipv4.ip_no_pmtu_disc=1

I’m still playing around with this setup but it’s been an interesting networking experiment so far in terms of seeing how your web connections handle and react to being encapsulated automatically by a network router…

Edit: I have written a Linux kernel module to do the equivalent of “scrub-all no-df“

Tunnelled Internet – 1G WAN – Speed Test

WireGuard:

~