for some context, I recently I tried configuring nginx as a tcp proxy that
routes
connections based on sni to multiple upstream services
the server only exposes one tcp port, and receives all connections there,
for example
a connection to redis.example.com:1234 would be proxy_pass'ed to some port
in the
machine, a connection to www.example.com:1234 to another, etc.
i used nginx itself to terminate the tls for all services for convenience
the problem:
now here is the issue, 1: postgres does some weird custom ssl stuff, which
means I
cannot terminate the ssl from within nginx, and 2: doing a tcp pass through
without
the ssl termination, and attempting to use ssl_preread and
$ssl_preread_server_name
_does not_ work for postgres connections (the module fails to extract the
server name)
what I attempted:
what I first thought of was to expand on the ssl_preread module to support
postgres
connections, I went into the source code and found that the module inserts
a handler into
the `NGX_STREAM_PREREAD_PHASE`
I tried looking into the buffer in this phase and no useful data showed up,
I then tried to
insert a second handler into the `NGX_STREAM_CONTENT_PHASE` and first
noticed
it is never used or initialised to begin with, so I did that, but then it
looks like no buffer
is ever available in this phase
any input, pointers, or suggestions are really welcomed
thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230513/ec63e752/attachment.htm>
Hi Eduard,
On Sat, May 13, 2023 at 10:43:59PM -0600, Eduard Vercaemer wrote:
> for some context, I recently I tried configuring nginx as a tcp proxy that
> routes
> connections based on sni to multiple upstream services
>
> the server only exposes one tcp port, and receives all connections there,
> for example
> a connection to redis.example.com:1234 would be proxy_pass'ed to some port
> in the
> machine, a connection to www.example.com:1234 to another, etc.
>
> i used nginx itself to terminate the tls for all services for convenience
>
> the problem:
> now here is the issue, 1: postgres does some weird custom ssl stuff, which
> means I
> cannot terminate the ssl from within nginx
In this case there must be an SSL error logged in nginx error log.
Can you post it?
> 2: doing a tcp pass through
> without
> the ssl termination, and attempting to use ssl_preread and
> $ssl_preread_server_name
> _does not_ work for postgres connections (the module fails to extract the
> server name)
>
> what I attempted:
> what I first thought of was to expand on the ssl_preread module to support
> postgres
> connections, I went into the source code and found that the module inserts
> a handler into
> the `NGX_STREAM_PREREAD_PHASE`
> I tried looking into the buffer in this phase and no useful data showed up,
Incoming data is written to c->buffer as long as the handler returns NGX_AGAIN.
You just have to wait long enough and have large enough buffer (see directive
preread_buffer_size).
> I then tried to
> insert a second handler into the `NGX_STREAM_CONTENT_PHASE` and first
> noticed
> it is never used or initialised to begin with, so I did that, but then it
> looks like no buffer
> is ever available in this phase
>
> any input, pointers, or suggestions are really welcomed
If you want to register a content phase handler, assign it to cscf->handler.
A good example is ngx_stream_return() in src/stream/ngx_stream_return_module.c.
--
Roman Arutyunyan
Hello!
On Sun, May 14, 2023 at 09:55:54AM +0400, Roman Arutyunyan wrote:
> Hi Eduard,
>
> On Sat, May 13, 2023 at 10:43:59PM -0600, Eduard Vercaemer wrote:
> > for some context, I recently I tried configuring nginx as a tcp proxy that
> > routes
> > connections based on sni to multiple upstream services
> >
> > the server only exposes one tcp port, and receives all connections there,
> > for example
> > a connection to redis.example.com:1234 would be proxy_pass'ed to some port
> > in the
> > machine, a connection to www.example.com:1234 to another, etc.
> >
> > i used nginx itself to terminate the tls for all services for convenience
> >
> > the problem:
> > now here is the issue, 1: postgres does some weird custom ssl stuff, which
> > means I
> > cannot terminate the ssl from within nginx
>
> In this case there must be an SSL error logged in nginx error log.
> Can you post it?
Postgres uses their own protocol with STARTTLS-like interface to
initiate SSL handshake, see here:
https://www.postgresql.org/docs/current/protocol-flow.html#id-1.10.6.7.12
That is, it's not going to work with neither SSL termination, nor
SSL preread, and needs an implementation of the Postgres protocol.
[...]
--
Maxim Dounin
http://mdounin.ru/
Hello,
> On Sun, 14 May 2023 17:33:10 +0300
> Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Sun, May 14, 2023 at 09:55:54AM +0400, Roman Arutyunyan wrote:
>
> > Hi Eduard,
> >
> > On Sat, May 13, 2023 at 10:43:59PM -0600, Eduard Vercaemer wrote:
> > > for some context, I recently I tried configuring nginx as a tcp
> > > proxy that routes
> > > connections based on sni to multiple upstream services
> > >
> > > the server only exposes one tcp port, and receives all
> > > connections there, for example
> > > a connection to redis.example.com:1234 would be proxy_pass'ed to
> > > some port in the
> > > machine, a connection to www.example.com:1234 to another, etc.
> > >
> > > i used nginx itself to terminate the tls for all services for
> > > convenience
> > >
> > > the problem:
> > > now here is the issue, 1: postgres does some weird custom ssl
> > > stuff, which means I
> > > cannot terminate the ssl from within nginx
> >
> > In this case there must be an SSL error logged in nginx error log.
> > Can you post it?
>
> Postgres uses their own protocol with STARTTLS-like interface to
> initiate SSL handshake, see here:
>
> https://www.postgresql.org/docs/current/protocol-flow.html#id-1.10.6.7.12
>
> That is, it's not going to work with neither SSL termination, nor
> SSL preread, and needs an implementation of the Postgres protocol.
>
> [...]
>
Out of curiosity I looked into what 'others' had done for Postgres's
application level negotiation.
https://github.com/envoyproxy/envoy/issues/10942
OP, it might be possible for you to hack this into ssl_preread.c in
ngx_stream_ssl_preread_handler in a similar fashion to that workaround.
It seems you just need to listen / wait for the SSLRequest magic
message bytes, send the 'fake' response, then do the normal handshake
logic.
https://www.postgresql.org/docs/current/protocol-message-formats.html
The other issue is if you want TLS from NGINX -> Postgresql Upstream
you'd need another hack somewhere in ngx_stream_proxy_module.c
(or a custom content handler as mentioned above).