Reverse proxy to forward proxy to internet access

M
  • 25 May '23
Hi,

Can you guide to configuration to put in reverse proxy config file to use
forward internet proxy?

I have case to use reverse proxy to access intranet urls which is usual
configuration but for certain resources i guess its called tunneling or
proxying i need to use internet proxy.

Ex.  Https://myreverseproxy.com/internet-mypub/somepath
Should
Use forward proxy and request https://mypub/somepath.

I hope i was able to explain.  I read on internet its not possible and then
some examples of proxypass claiming its possible.

Regards,

Miten
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230525/a35d990c/attachment.htm>
F
  • 25 May '23
On Thu, May 25, 2023 at 05:12:26PM +0530, Miten Mehta wrote:

Hi there,

> Can you guide to configuration to put in reverse proxy config file to use
> forward internet proxy?

nginx does not talk to a proxy server.

If you need to talk to a proxy server, you need something other than
"stock" nginx.

Good luck with it,

    f
-- 
Francis Daly        francis at daoine.org
M
  • 26 May '23
Hi,

Thanks for guidance.  If i enable direct internet access from reverse proxy
then can i just use proxy_pass $request_uri and have user format his url as
https://myreverseproxy.com/https://mypub/somepath.

Regards,

Miten

On Fri 26 May, 2023, 4:39 AM Francis Daly, <francis at daoine.org> wrote:

> On Thu, May 25, 2023 at 05:12:26PM +0530, Miten Mehta wrote:
>
> Hi there,
>
> > Can you guide to configuration to put in reverse proxy config file to use
> > forward internet proxy?
>
> nginx does not talk to a proxy server.
>
> If you need to talk to a proxy server, you need something other than
> "stock" nginx.
>
> Good luck with it,
>
>         f
> --
> Francis Daly        francis at daoine.org
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230526/ca13232b/attachment.htm>
M
  • 27 May '23
I consider from your reply that niginx reverse proxy cannot provide
internet access through a forward proxy like squid, websense or alike.

I understand you mentioned that nginx cannot be used as forward proxy.

There are many blogs on net claiming to use nginx as forward proxy and also
using upstream forward proxy that is false? I don't see this in official
documentation of nginx.

Regards,

Miten

On Fri 26 May, 2023, 4:18 PM Miten Mehta, <indiamiten at gmail.com> wrote:

> Hi,
>
> Thanks for guidance.  If i enable direct internet access from reverse
> proxy then can i just use proxy_pass $request_uri and have user format his
> url as https://myreverseproxy.com/https://mypub/somepath.
>
> Regards,
>
> Miten
>
> On Fri 26 May, 2023, 4:39 AM Francis Daly, <francis at daoine.org> wrote:
>
>> On Thu, May 25, 2023 at 05:12:26PM +0530, Miten Mehta wrote:
>>
>> Hi there,
>>
>> > Can you guide to configuration to put in reverse proxy config file to
>> use
>> > forward internet proxy?
>>
>> nginx does not talk to a proxy server.
>>
>> If you need to talk to a proxy server, you need something other than
>> "stock" nginx.
>>
>> Good luck with it,
>>
>>         f
>> --
>> Francis Daly        francis at daoine.org
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> https://mailman.nginx.org/mailman/listinfo/nginx
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230527/9c46c77e/attachment.htm>
S
  • 27 May '23
Check
https://1eye.us
100% Nginx

On Sat, May 27, 2023 at 12:21 AM Miten Mehta <indiamiten at gmail.com> wrote:

> I consider from your reply that niginx reverse proxy cannot provide
> internet access through a forward proxy like squid, websense or alike.
>
> I understand you mentioned that nginx cannot be used as forward proxy.
>
> There are many blogs on net claiming to use nginx as forward proxy and
> also using upstream forward proxy that is false? I don't see this in
> official documentation of nginx.
>
> Regards,
>
> Miten
>
> On Fri 26 May, 2023, 4:18 PM Miten Mehta, <indiamiten at gmail.com> wrote:
>
>> Hi,
>>
>> Thanks for guidance.  If i enable direct internet access from reverse
>> proxy then can i just use proxy_pass $request_uri and have user format his
>> url as https://myreverseproxy.com/https://mypub/somepath.
>>
>> Regards,
>>
>> Miten
>>
>> On Fri 26 May, 2023, 4:39 AM Francis Daly, <francis at daoine.org> wrote:
>>
>>> On Thu, May 25, 2023 at 05:12:26PM +0530, Miten Mehta wrote:
>>>
>>> Hi there,
>>>
>>> > Can you guide to configuration to put in reverse proxy config file to
>>> use
>>> > forward internet proxy?
>>>
>>> nginx does not talk to a proxy server.
>>>
>>> If you need to talk to a proxy server, you need something other than
>>> "stock" nginx.
>>>
>>> Good luck with it,
>>>
>>>         f
>>> --
>>> Francis Daly        francis at daoine.org
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>
>> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230527/e4445bec/attachment.htm>
F
  • 27 May '23
On Fri, May 26, 2023 at 04:18:59PM +0530, Miten Mehta wrote:

Hi there,

> Thanks for guidance.  If i enable direct internet access from reverse proxy
> then can i just use proxy_pass $request_uri and have user format his url as
> https://myreverseproxy.com/https://mypub/somepath.

Here, $request_uri would start with /, so it would not Just Work as-is.

I'm not sure how https://myreverseproxy.com/https://mypub/somepath is
different from a "normal" https://myreverseproxy.com/mypub/somepath
with a "normal" nginx config based on

    location ^~ /mypub/ { proxy_pass https://mypub/; }

(plus the supporting configuration). So then you have a "normal" nginx
proxy_pass setup for specific remote web servers.

Which should Just Work like any other proxy_pass configuration.

Good luck with it,

    f
-- 
Francis Daly        francis at daoine.org
F
  • 27 May '23
On Sat, May 27, 2023 at 09:51:10AM +0530, Miten Mehta wrote:

Hi there,

> I consider from your reply that niginx reverse proxy cannot provide
> internet access through a forward proxy like squid, websense or alike.

"http through a proxy" uses a different form of requests from "http".

nginx as a client does not make the "http through a proxy" request when
it is talking to a configured upstream server.

The general "forward proxy" server will expect clients that talk to it,
to make "http through a proxy" requests.

Your specific "forward proxy" server might be configured to
"transparently" intercept "http" requests and make a best-guess effort at
interpreting them as if they had been "http through a proxy" requests. And
that might work in many cases.

If that works well enough in your specific case, great!

Only you can know whether it works well enough in your case, to be worth
investigating further for problems.

> I understand you mentioned that nginx cannot be used as forward proxy.

nginx as a server does not specially interpret any "http through a proxy"
requests that it receives, and it does not try to follow the "http proxy
server" rules for handling requests and responses.

If what it does do, works well enough for you, great!

> There are many blogs on net claiming to use nginx as forward proxy and also
> using upstream forward proxy that is false? I don't see this in official
> documentation of nginx.

Maybe those many blogs refer to cases where the combination of their
client, their upstream proxy server, and their configuration of nginx,
works well enough for them.

In which case -- great!

You should be able to build a test nginx configuration based on those
blogs, to see whether it works well enough for you, too.

Good luck with it,

    f
-- 
Francis Daly        francis at daoine.org
F
  • 27 May '23
On Sat, May 27, 2023 at 12:39:05AM -0400, Saint Michael wrote:

Hi there,

> 100% Nginx

That looks like an ad for a donation button; but it doesn't immediately
seem to say "here is how nginx is configured to access a remote web site
through a proxy server".

Or "here is how nginx is configured to be accessed as if it were a
proxy server".

(It does seem to indicate "this server acts as a reverse proxy
for some specific remote web sites"; but that's pretty much what
http://nginx.org/r/proxy_pass does. No doubt there is extra cleverness
to handle the "I don't control the upstream server" issues that usually
arise; but it does not seem to be relevant to this thread. Am I missing
something?)

Thanks,

    f
-- 
Francis Daly        francis at daoine.org
S
  • 27 May '23
Please look at the links.
All those links are a live digital tunnel to each website. For instance,
the client clicks on my
https://14ymedio.1eye.us/
and I send it forward to
https://14ymedio.com
except that he is still inside my own domain, secure.
It's live. So if the target domain is blocked via DNS, in Cuba, It still
works using my domain.

This is all using Nginx.
The question is: is there a better tool to do this?
I am trying to find out.
Some domains don't work, so I am still looking for a better solution.
Also, Cloudflare blocks me.

Federico

On Sat, May 27, 2023 at 4:24 AM Francis Daly <francis at daoine.org> wrote:

> On Sat, May 27, 2023 at 12:39:05AM -0400, Saint Michael wrote:
>
> Hi there,
>
> > 100% Nginx
>
> That looks like an ad for a donation button; but it doesn't immediately
> seem to say "here is how nginx is configured to access a remote web site
> through a proxy server".
>
> Or "here is how nginx is configured to be accessed as if it were a
> proxy server".
>
> (It does seem to indicate "this server acts as a reverse proxy
> for some specific remote web sites"; but that's pretty much what
> http://nginx.org/r/proxy_pass does. No doubt there is extra cleverness
> to handle the "I don't control the upstream server" issues that usually
> arise; but it does not seem to be relevant to this thread. Am I missing
> something?)
>
> Thanks,
>
>         f
> --
> Francis Daly        francis at daoine.org
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230527/967b9dc6/attachment.htm>
D
  • 28 May '23
What does the project do for you and what would you like it do for you.
Share ideas

On Sat, May 27, 2023, 12:21 AM Miten Mehta <indiamiten at gmail.com> wrote:

> I consider from your reply that niginx reverse proxy cannot provide
> internet access through a forward proxy like squid, websense or alike.
>
> I understand you mentioned that nginx cannot be used as forward proxy.
>
> There are many blogs on net claiming to use nginx as forward proxy and
> also using upstream forward proxy that is false? I don't see this in
> official documentation of nginx.
>
> Regards,
>
> Miten
>
> On Fri 26 May, 2023, 4:18 PM Miten Mehta, <indiamiten at gmail.com> wrote:
>
>> Hi,
>>
>> Thanks for guidance.  If i enable direct internet access from reverse
>> proxy then can i just use proxy_pass $request_uri and have user format his
>> url as https://myreverseproxy.com/https://mypub/somepath.
>>
>> Regards,
>>
>> Miten
>>
>> On Fri 26 May, 2023, 4:39 AM Francis Daly, <francis at daoine.org> wrote:
>>
>>> On Thu, May 25, 2023 at 05:12:26PM +0530, Miten Mehta wrote:
>>>
>>> Hi there,
>>>
>>> > Can you guide to configuration to put in reverse proxy config file to
>>> use
>>> > forward internet proxy?
>>>
>>> nginx does not talk to a proxy server.
>>>
>>> If you need to talk to a proxy server, you need something other than
>>> "stock" nginx.
>>>
>>> Good luck with it,
>>>
>>>         f
>>> --
>>> Francis Daly        francis at daoine.org
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>
>> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230527/bdf20f7b/attachment.htm>
S
  • 28 May '23
I would like to find a way to do what I do now at https://1eye.us but
better.
Many websites fail to work. For example https://bing.com/new

On Sun, May 28, 2023 at 12:00 AM David Siembab <siembabdavid at gmail.com>
wrote:

> What does the project do for you and what would you like it do for you.
> Share ideas
>
> On Sat, May 27, 2023, 12:21 AM Miten Mehta <indiamiten at gmail.com> wrote:
>
>> I consider from your reply that niginx reverse proxy cannot provide
>> internet access through a forward proxy like squid, websense or alike.
>>
>> I understand you mentioned that nginx cannot be used as forward proxy.
>>
>> There are many blogs on net claiming to use nginx as forward proxy and
>> also using upstream forward proxy that is false? I don't see this in
>> official documentation of nginx.
>>
>> Regards,
>>
>> Miten
>>
>> On Fri 26 May, 2023, 4:18 PM Miten Mehta, <indiamiten at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Thanks for guidance.  If i enable direct internet access from reverse
>>> proxy then can i just use proxy_pass $request_uri and have user format his
>>> url as https://myreverseproxy.com/https://mypub/somepath.
>>>
>>> Regards,
>>>
>>> Miten
>>>
>>> On Fri 26 May, 2023, 4:39 AM Francis Daly, <francis at daoine.org> wrote:
>>>
>>>> On Thu, May 25, 2023 at 05:12:26PM +0530, Miten Mehta wrote:
>>>>
>>>> Hi there,
>>>>
>>>> > Can you guide to configuration to put in reverse proxy config file to
>>>> use
>>>> > forward internet proxy?
>>>>
>>>> nginx does not talk to a proxy server.
>>>>
>>>> If you need to talk to a proxy server, you need something other than
>>>> "stock" nginx.
>>>>
>>>> Good luck with it,
>>>>
>>>>         f
>>>> --
>>>> Francis Daly        francis at daoine.org
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>
>>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> https://mailman.nginx.org/mailman/listinfo/nginx
>>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230528/7245f1e4/attachment.htm>
F
  • 28 May '23
On Sat, May 27, 2023 at 10:42:01AM -0400, Saint Michael wrote:

Hi there,

> Please look at the links.
> All those links are a live digital tunnel to each website.

Yes; it looks like you are making "normal" use of nginx's proxy_pass
directive, to provide indirect access for clients, to some content on the
web that you have access to, that someone on either the upstream-server
or client-network side had attempted to block direct access to.

That looks like a convenient service for a client who wants to avoid
those attempted blocks.

I'm just not sure how what you wrote relates to what the original poster
asked, or to anything else in the thread.

And your original mail that I responded to, could just as well have
been written in response to pretty much any message on the mailing list,
and it would have had the same looks-like-spam appearance.

For your follow-up questions about your service: I would have imagined
that there would be a bigger readership for a new thread, rather than
hiding things in an unrelated thread; but whatever works for you is good.

Cheers,

    f
-- 
Francis Daly        francis at daoine.org
M
  • 29 May '23
Hi Francis Daly,

client -> reverse proxy: https://myreverseproxy.com/https://mypub/somepath
reverse proxy to forward proxy: I'm not sure how https://forwardproxy.com/
https://mypub/somepath <https://myreverseproxy.com/https://mypub/somepath>

I was trying to append the actual url into the reverse proxy so in its
config we can easily user $request_uri and pass it along and may be
thinking that is what you meant by "http through proxy" and forward proxy
will be able to interpret and do needful.  like you said some forward proxy
will be able to do it and if so then above looks good to me.

On Sat, May 27, 2023 at 1:18 PM Francis Daly <francis at daoine.org> wrote:

> On Fri, May 26, 2023 at 04:18:59PM +0530, Miten Mehta wrote:
>
> Hi there,
>
> > Thanks for guidance.  If i enable direct internet access from reverse
> proxy
> > then can i just use proxy_pass $request_uri and have user format his url
> as
> > https://myreverseproxy.com/https://mypub/somepath.
>
> Here, $request_uri would start with /, so it would not Just Work as-is.
>
> I'm not sure how https://myreverseproxy.com/https://mypub/somepath is
> different from a "normal" https://myreverseproxy.com/mypub/somepath
> with a "normal" nginx config based on
>
>         location ^~ /mypub/ { proxy_pass https://mypub/; }
>
> (plus the supporting configuration). So then you have a "normal" nginx
> proxy_pass setup for specific remote web servers.
>
> Which should Just Work like any other proxy_pass configuration.
>
> Good luck with it,
>
>         f
> --
> Francis Daly        francis at daoine.org
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230529/c84cceb4/attachment.htm>
Y
  • 30 May '23
Hi

When I have 2 servers in nginx.conf  with same  listen port   if the server have name, nginx issue warning  ignore …  but nit failed to load.
What happens is only the first server in conf binds the socket.
And worse, If no server names I did not get a warning.

Only if the keyword “default server” used nginx enforces one server to use the port.

Why????

How come it warning and error?

It misleading option.

Thanks

Yuval Abadi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230530/f33bad5a/attachment.htm>
F
  • 30 May '23
On Tue, May 30, 2023 at 06:50:19AM +0000, Yuval Abadi via nginx wrote:

Hi there,

> When I have 2 servers in nginx.conf  with same  listen port   if the server have name, nginx issue warning  ignore …  but nit failed to load.
> What happens is only the first server in conf binds the socket.
> And worse, If no server names I did not get a warning.

Does https://nginx.org/en/docs/http/request_processing.html explain what
you are seeing?

If not, can you show one small but complete configuration that shows
the problem that you are reporting?

"name-based virtual servers" are based around listening on the same
port, and having the http server responding differently based on the
Host: in the incoming request. It would be surprising if that feature
became broken.

The documentation for "listen" at https://nginx.org/r/listen does note
that some parameters only make sense when set once (or set the same each
time, if they are set more than once); I don't know if you are hitting
one of those cases?

Cheers,

    f
-- 
Francis Daly        francis at daoine.org
Y
  • 31 May '23
Hi
Thanks for the response.

The link explains the problem, I think we could do better work for users.

If the configuration has 2 servers sharing the same name and the same port
I got this warning:
"nginx: [warn] conflicting server name "http://www.mut.com/" on 0.0.0.0:80, ignored"

Why not block this mistake?
I assume the second server ignored, but why let it possible?

If the servers do not have name
I got this warning:
nginx: [warn] conflicting server name "" on 0.0.0.0:9002, ignored
nginx: [warn] conflicting server name "" on 0.0.0.0:80, ignored

 both warning:
first no way for NGINX gives good warning, both server looks the same.
if user did such mistake, better to block.

Why not enforce using at list one server have "listen   default_server  port"?
Why not enforce server names , and not let more than one server with same name?

Is NGINX  set the bit default_server ,on the first  "ngx_http_conf_addr_t", of the first server,  that read from conf file?  (if no default_server was defined)?

Thanks

-----Original Message-----
From: nginx <nginx-bounces at nginx.org> On Behalf Of Francis Daly
Sent: Tuesday, 30 May 2023 19:57
To: Yuval Abadi via nginx <nginx at nginx.org>
Subject: Re: duplicate ports across servers in nginx.conf

EXTERNAL MAIL: nginx-bounces at nginx.org

On Tue, May 30, 2023 at 06:50:19AM +0000, Yuval Abadi via nginx wrote:

Hi there,

> When I have 2 servers in nginx.conf  with same  listen port   if the server have name, nginx issue warning  ignore …  but nit failed to load.
> What happens is only the first server in conf binds the socket.
> And worse, If no server names I did not get a warning.

Does https://nginx.org/en/docs/http/request_processing.html explain what you are seeing?

If not, can you show one small but complete configuration that shows the problem that you are reporting?

"name-based virtual servers" are based around listening on the same port, and having the http server responding differently based on the
Host: in the incoming request. It would be surprising if that feature became broken.

The documentation for "listen" at https://nginx.org/r/listen does note that some parameters only make sense when set once (or set the same each time, if they are set more than once); I don't know if you are hitting one of those cases?

Cheers,

        f
--
Francis Daly        francis at daoine.org
_______________________________________________
nginx mailing list
nginx at nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx.conf
Type: application/octet-stream
Size: 4588 bytes
Desc: nginx.conf
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230531/b31a53e1/attachment.obj>
F
  • 31 May '23
On Wed, May 31, 2023 at 06:14:41AM +0000, Yuval Abadi via nginx wrote:

Hi there,

I don't speak for the project, but my guess is:

> If the configuration has 2 servers sharing the same name and the same port
> I got this warning:
> "nginx: [warn] conflicting server name "http://www.mut.com/" on 0.0.0.0:80, ignored"
> 
> Why not block this mistake?

If you have 20 server{}s, and 2 share the name and port, should the
entire system fail to start (or reload config)?

It seems friendlier to me to use the config as-provided, and alert on
things that are not used as the administrator apparently expected.

Some configuration issues are considered more important than some
others. This particular one is currently not considered "fatal".

> I assume the second server ignored, but why let it possible?

nginx does not control what the administrator types.

> If the servers do not have name
> I got this warning:
> nginx: [warn] conflicting server name "" on 0.0.0.0:9002, ignored
> nginx: [warn] conflicting server name "" on 0.0.0.0:80, ignored

Yes; it's the same message, showing the listen ip:port and server_name
values that are unexpected.

>  both warning:
> first no way for NGINX gives good warning, both server looks the same.

I agree that it would be even friendlier if the error message indicated
the filename and line number that the unexpected configuration came from;
I suspect that a patch to change that would be thoughtfully considered.

Maybe someone will be interested in providing that patch, now that the
issue has been mentioned.

(Maybe the only reason the log omits the filename is that no-one thought
to add it here, where it is added in other places. Or maybe it is harder
than that to implement.)

> if user did such mistake, better to block.

I disagree.

It appears that the current code disagrees too; maybe that will change
in the future.

> Why not enforce using at list one server have "listen   default_server  port"?

I think that is enforced already -- if you have more than one
"default_server", you get an "emerg" failure. If you have none explicitly,
then the implicit config applies -- and I would rather not lose the
implicit config.

> Why not enforce server names , and not let more than one server with same name?

I think that is what it is doing already; it considers it a "warning"
rather than an "emergency" configuration issue.

> Is NGINX  set the bit default_server ,on the first  "ngx_http_conf_addr_t", of the first server,  that read from conf file?  (if no default_server was defined)?
>

I'm not quite sure what you are asking: if it is about the code, it
is not hidden and is quite readable; if it is about which server is
default_server if none is explicit, then the documentation also describes
that -- the "implicit" default_server for a specific ip:port is the first
server{} that was read with that (possibly implicit) "listen" config.

Cheers,

    f
-- 
Francis Daly        francis at daoine.org