Experiencing Connection Time-Out Troubles With Multi-Process NGINX

So I was trying to perform some basic redirect-proxying with load-balancing by having multiple nginx processes running for a given connection protocol and I was experiencing connection timeout troubles from time to time. Even when I tried playing with the different combinations of the proxy and connect timeout values (short-short, short-long, long-short, long-long, etc) it would still happen over time. I decided to switch up my solution here by having multiple single process nginx instances running on a different local host IP address instead to avoid port conflicts. My new setup is as follows:

Multi-Localhost Addresses

ifconfig lo0 127.0.0.2 netmask 255.0.0.0 alias
ifconfig lo0 127.0.0.3 netmask 255.0.0.0 alias
ifconfig lo0 127.0.0.4 netmask 255.0.0.0 alias

PF Load-balancing

# vpn
rdr on en0 inet proto udp from any to any port = 500 -> { 127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4 } port 3121 round-robin
rdr on en0 inet proto udp from any to any port = 4500 -> { 127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4 } port 3121 round-robin

# udp
rdr on en0 inet proto udp from any to any port 1:5352 -> { 127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4 } port 3125 round-robin
rdr on en0 inet proto udp from any to any port 5354:65535 -> { 127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4 } port 3125 round-robin

# tcp
rdr on en0 inet proto tcp from any to any port 1:21 -> { 127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4 } port 3129 round-robin
rdr on en0 inet proto tcp from any to any port 23:65535 -> { 127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4 } port 3129 round-robin

Leave a comment