So nginx has a stream proxy module that you can use for transparent SSL/TLS relaying/forwarding, however, it is only capable of reading the SNI hostname upon the initial handshake of the connection. In addition, the destination IP address is replaced because of the firewall redirect pointing to the proxy server. I wrote a small modification that can be compiled into nginx which allows you to run a script that can pull the missing destination IP address from a given state connection table in a firewall, for example pfctl or iptables.

Source: https://github.com/stoops/nginx

Reproduction Test:

echo 'test' | nc 8.8.4.4 443

Error Log:

[error]: no host in upstream ":443", client: 192.168.X.Y, server: 0.0.0.0:3129, …

C Hook Mod:

 

This code mod above will allow you to run a shell script of your choosing if nginx cannot get the hostname or address of a connection requesting to be proxied. You can then look up the destination IP address based on source IP + port combo from the connection state mapping table of the firewall. The result is a much more stable proxying experience for HTTPS connections without needing to wait for the SNI or hostname of the initial handshake!

Bonus: This mod also allows you to use nginx as a generic UDP proxy forwarder/server for the first time with the following configs below for the various connection states. This can be used for proxying protocols such as DNS and QUIC and some other VPN related ones. Note, you may need to adjust your own individual timeout values to match your needs.

VPN+MTU:

user root wheel;
worker_processes 1;
worker_rlimit_nofile 8192;
events {
	accept_mutex off;
	multi_accept off;
	worker_connections 1024;
	#use select;
}
stream {
	resolver 1.1.1.1 ipv6=off;
	server {
		listen 127.0.0.1:3121 udp;
		proxy_half_close off;
		proxy_socket_keepalive off;
		proxy_connect_timeout 30s;
		proxy_timeout 45s;
		proxy_pass $proxy_protocol_server_addr:1;
	}
}

rdr on en0 inet proto udp from any to any port 4500 -> 127.0.0.1 port 3121

DNS:

user root wheel;
worker_processes 1;
worker_rlimit_nofile 8192;
events {
	accept_mutex off;
	multi_accept off;
	worker_connections 1024;
	#use select;
}
stream {
	resolver 1.1.1.1 ipv6=off;
	server {
		listen 127.0.0.1:3125 udp;
		proxy_half_close off;
		proxy_socket_keepalive off;
		proxy_connect_timeout 15s;
		proxy_timeout 5s;
		proxy_pass $proxy_protocol_server_addr:1;
	}
}

rdr on en0 inet proto udp from any to any port 53 -> 127.0.0.1 port 3125

QUIC:

user root wheel;
worker_processes 1;
worker_rlimit_nofile 8192;
events {
	accept_mutex off;
	multi_accept off;
	worker_connections 1024;
	#use select;
}
stream {
	resolver 1.1.1.1 ipv6=off;
	server {
		listen 127.0.0.1:3123 udp;
		proxy_half_close off;
		proxy_socket_keepalive off;
		proxy_connect_timeout 15s;
		proxy_timeout 5s;
		proxy_pass $proxy_protocol_server_addr:1;
	}
}

rdr on en0 inet proto udp from any to any port 443 -> 127.0.0.1 port 3123

Update: Alternative Load Balancing With Single-Process NGINX

~