resolver does not work

M
Hello,
I have a nginx proxy in front of systemd-nspawn containers.
The IP's of the containers are dynamic.

When I start nginx *after* the  containers it works.
When the IP of the container is changed while nginx is running i get a 
"Bad Gateway" error.

The config looks like:

server {
     server_name foobar.example.com
...
    location / {
       resolver 127.0.0.53 valid=10s;
       ...
       proxy_pass http://container;
    }
}

nginx is 1.1.18 so it should work as documented in 
http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
The workaround there 
https://stackoverflow.com/questions/42720618/docker-nginx-stopped-emerg-11-host-not-found-in-upstream/52319161#52319161 
doesn't work.

I have also try to config a upstream backend and the resolver in the 
server part or in the http part.
The errors are: "upstream timed out" or "container could not be resolved 
(3: Host not found)"

Whats wrong there?
Best Regards
A
  • 18 Apr '23
Am 18.04.23 um 14:58 schrieb mailinglist at unix-solution.de:
> nginx is 1.1.18
> Whats wrong there?
released in March 2012

couldn't you use a "not so historic" version?

Andreas
B
  • 18 Apr '23
Sorry, my mistake.
1.18.0 debian11 is installed.

Am 18.04.23 um 19:21 schrieb A. Schulze via nginx:
> 
> 
> Am 18.04.23 um 14:58 schrieb mailinglist at unix-solution.de:
>> nginx is 1.1.18
>> Whats wrong there?
> released in March 2012
> 
> couldn't you use a "not so historic" version?
> 
> Andreas
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
M
  • 18 Apr '23
Hello!

On Tue, Apr 18, 2023 at 02:58:08PM +0200, mailinglist at unix-solution.de wrote:

> Hello,
> I have a nginx proxy in front of systemd-nspawn containers.
> The IP's of the containers are dynamic.
> 
> When I start nginx *after* the  containers it works.
> When the IP of the container is changed while nginx is running i get a 
> "Bad Gateway" error.
> 
> The config looks like:
> 
> server {
>      server_name foobar.example.com
> ...
>     location / {
>        resolver 127.0.0.53 valid=10s;
>        ...
>        proxy_pass http://container;
>     }
> }
> 
> nginx is 1.1.18 so it should work as documented in 
> http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
> The workaround there 
> https://stackoverflow.com/questions/42720618/docker-nginx-stopped-emerg-11-host-not-found-in-upstream/52319161#52319161 
> doesn't work.
> 
> I have also try to config a upstream backend and the resolver in the 
> server part or in the http part.
> The errors are: "upstream timed out" or "container could not be resolved 
> (3: Host not found)"
> 
> Whats wrong there?

The "resolver" directive is only used when the "proxy_pass" 
directive contain variables, and therefore nginx isn't able to 
determine upstream addresses when parsing configuration.  Quoting 
http://nginx.org/r/proxy_pass:

: Parameter value can contain variables. In this case, if an 
: address is specified as a domain name, the name is searched among 
: the described server groups, and, if not found, is determined 
: using a resolver.

The "workaround", however, should work fine.  Something like this 
should be enough:

        resolver 127.0.0.53 valid=10s;
        set $upstream "http://container";
        proxy_pass $upstream;

Note though that there should be an actual DNS name which can be 
resolved with the DNS server configured with the "resolver" 
directive.  The "container could not be resolved (3: Host not 
found)" error suggests that your DNS server does not return 
anything for the name - you may want to dig further into systemd 
to find out why it doesn't return anything.

-- 
Maxim Dounin
http://mdounin.ru/
J
  • 18 Apr '23
Hi,

On 18/04/2023 13:58, mailinglist at unix-solution.de wrote:
> Hello,
> I have a nginx proxy in front of systemd-nspawn containers.
> The IP's of the containers are dynamic.
> 
> When I start nginx *after* the  containers it works.
> When the IP of the container is changed while nginx is running i get a 
> "Bad Gateway" erro
> 
> The config looks like:
> 
> server {
>      server_name foobar.example.com
> ...
>     location / {
>        resolver 127.0.0.53 valid=10s;
>        ...
>        proxy_pass http://container;

You need to use a variable in the proxy_pass here instead to do dynamic 
proxy pass. This force nginx to re-resolve the hostname. The variable's 
value should be set the target hostname (looks like it's 'containers' 
from your config).

something like
set $container_hostname 'containers';
proxy_pass http://$containers_hostname;

Note, you can't target an upstream server group with this technique - it 
has to be a hostname that the dns resolver returns.

If you need more advanced re-resolving, such as the ability to use 
upstream server groups and resolve servers within it, NGINX Plus has 
this feature.

In fact the patches from NGINX Plus that do the dynamic re-resolving are 
already on the devel mailing list - just not integrated.

https://mailman.nginx.org/pipermail/nginx-devel/2023-February/4MCLSVRK7EX6DNKHFZN6CA4SKZUSA3GA.html

So it can also be obtained by comping from source with that set of 
patches applied.

This is another alternative upstream resolver - although not as good, as 
it requires requests to initiate a re-resolve.

https://www.nginx.com/resources/wiki/modules/domain_resolve/

>     }
> }
> 
> nginx is 1.1.18 so it should work as documented in 
> http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
> The workaround there 
> https://stackoverflow.com/questions/42720618/docker-nginx-stopped-emerg-11-host-not-found-in-upstream/52319161#52319161 doesn't work.
> 
> I have also try to config a upstream backend and the resolver in the 
> server part or in the http part.
> The errors are: "upstream timed out" or "container could not be resolved 
> (3: Host not found)"
> 
> Whats wrong there?
> Best Regards
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
B
  • 18 Apr '23
Hello Maxim,

that does not work for me.

root at tower:~# resolvectl query icinga2
icinga2: 192.168.145.217                       -- link: ve-icinga2

-- Information acquired via protocol LLMNR/IPv4 in 105.7ms.
-- Data is authenticated: no

root at tower:~# getent hosts icinga2
192.168.145.217 icinga2

root at tower:~# cat /etc/resolv.conf
nameserver 127.0.0.53
options edns0
root at tower:~#

    location / {

         proxy_redirect off;
         proxy_set_header Host $remote_addr;
         proxy_set_header X-Forwarded-For  $remote_addr;
         proxy_set_header X-Forwarded-Host $remote_addr;
         proxy_set_header X-Real-IP $remote_addr;
#        proxy_pass https://icinga2;
         resolver 127.0.0.53 valid=10s;
         set $upstream "https://icinga2";
         proxy_pass $upstream;

         # WebSocket support
         proxy_http_version 1.1;
         proxy_set_header Upgrade $http_upgrade;
         proxy_set_header Connection "upgrade";
}

2023/04/18 22:16:28 [error] 779163#779163: *26641 icinga2 could not be 
resolved (3: Host not found),

503 bad gateway

Am 18.04.23 um 21:29 schrieb Maxim Dounin:
> Hello!
> 
> On Tue, Apr 18, 2023 at 02:58:08PM +0200, mailinglist at unix-solution.de wrote:
> 
>> Hello,
>> I have a nginx proxy in front of systemd-nspawn containers.
>> The IP's of the containers are dynamic.
>>
>> When I start nginx *after* the  containers it works.
>> When the IP of the container is changed while nginx is running i get a
>> "Bad Gateway" error.
>>
>> The config looks like:
>>
>> server {
>>       server_name foobar.example.com
>> ...
>>      location / {
>>         resolver 127.0.0.53 valid=10s;
>>         ...
>>         proxy_pass http://container;
>>      }
>> }
>>
>> nginx is 1.1.18 so it should work as documented in
>> http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver
>> The workaround there
>> https://stackoverflow.com/questions/42720618/docker-nginx-stopped-emerg-11-host-not-found-in-upstream/52319161#52319161
>> doesn't work.
>>
>> I have also try to config a upstream backend and the resolver in the
>> server part or in the http part.
>> The errors are: "upstream timed out" or "container could not be resolved
>> (3: Host not found)"
>>
>> Whats wrong there?
> 
> The "resolver" directive is only used when the "proxy_pass"
> directive contain variables, and therefore nginx isn't able to
> determine upstream addresses when parsing configuration.  Quoting
> http://nginx.org/r/proxy_pass:
> 
> : Parameter value can contain variables. In this case, if an
> : address is specified as a domain name, the name is searched among
> : the described server groups, and, if not found, is determined
> : using a resolver.
> 
> The "workaround", however, should work fine.  Something like this
> should be enough:
> 
>          resolver 127.0.0.53 valid=10s;
>          set $upstream "http://container";
>          proxy_pass $upstream;
> 
> Note though that there should be an actual DNS name which can be
> resolved with the DNS server configured with the "resolver"
> directive.  The "container could not be resolved (3: Host not
> found)" error suggests that your DNS server does not return
> anything for the name - you may want to dig further into systemd
> to find out why it doesn't return anything.
>
M
  • 19 Apr '23
Hello!

On Tue, Apr 18, 2023 at 10:25:09PM +0200, basti wrote:

> Hello Maxim,
> 
> that does not work for me.
> 
> root at tower:~# resolvectl query icinga2
> icinga2: 192.168.145.217                       -- link: ve-icinga2
> 
> -- Information acquired via protocol LLMNR/IPv4 in 105.7ms.
> -- Data is authenticated: no
> 
> root at tower:~# getent hosts icinga2
> 192.168.145.217 icinga2
> 
> root at tower:~# cat /etc/resolv.conf
> nameserver 127.0.0.53
> options edns0
> root at tower:~#
> 

And what "nslookup icinga2 127.0.0.53" shows?

>     location / {
> 
>          proxy_redirect off;
>          proxy_set_header Host $remote_addr;
>          proxy_set_header X-Forwarded-For  $remote_addr;
>          proxy_set_header X-Forwarded-Host $remote_addr;
>          proxy_set_header X-Real-IP $remote_addr;
> #        proxy_pass https://icinga2;
>          resolver 127.0.0.53 valid=10s;
>          set $upstream "https://icinga2";
>          proxy_pass $upstream;
> 
>          # WebSocket support
>          proxy_http_version 1.1;
>          proxy_set_header Upgrade $http_upgrade;
>          proxy_set_header Connection "upgrade";
> }
> 
> 2023/04/18 22:16:28 [error] 779163#779163: *26641 icinga2 could not be 
> resolved (3: Host not found),

As previously mentioned, the message suggests that the name is not 
available via DNS.  This is to be resolved on the systemd side.

Unfortunately, I'm not familiar with systemd-nspawn and have no 
idea how to do it properly and if it's at all possible.

-- 
Maxim Dounin
http://mdounin.ru/
B
  • 19 Apr '23
Hello Maxim,

I have found.

nginx query ipv4 and ipv6 as expected.
systemd-resolved return servfail when there is no ipv6. I think it 
should return nodata.

When I set ipv6=off in nginx all is fine.

Best Regards,

On 19.04.23 02:51, Maxim Dounin wrote:
> Hello!
> 
> On Tue, Apr 18, 2023 at 10:25:09PM +0200, basti wrote:
> 
>> Hello Maxim,
>>
>> that does not work for me.
>>
>> root at tower:~# resolvectl query icinga2
>> icinga2: 192.168.145.217                       -- link: ve-icinga2
>>
>> -- Information acquired via protocol LLMNR/IPv4 in 105.7ms.
>> -- Data is authenticated: no
>>
>> root at tower:~# getent hosts icinga2
>> 192.168.145.217 icinga2
>>
>> root at tower:~# cat /etc/resolv.conf
>> nameserver 127.0.0.53
>> options edns0
>> root at tower:~#
>>
> 
> And what "nslookup icinga2 127.0.0.53" shows?
> 
>>      location / {
>>
>>           proxy_redirect off;
>>           proxy_set_header Host $remote_addr;
>>           proxy_set_header X-Forwarded-For  $remote_addr;
>>           proxy_set_header X-Forwarded-Host $remote_addr;
>>           proxy_set_header X-Real-IP $remote_addr;
>> #        proxy_pass https://icinga2;
>>           resolver 127.0.0.53 valid=10s;
>>           set $upstream "https://icinga2";
>>           proxy_pass $upstream;
>>
>>           # WebSocket support
>>           proxy_http_version 1.1;
>>           proxy_set_header Upgrade $http_upgrade;
>>           proxy_set_header Connection "upgrade";
>> }
>>
>> 2023/04/18 22:16:28 [error] 779163#779163: *26641 icinga2 could not be
>> resolved (3: Host not found),
> 
> As previously mentioned, the message suggests that the name is not
> available via DNS.  This is to be resolved on the systemd side.
> 
> Unfortunately, I'm not familiar with systemd-nspawn and have no
> idea how to do it properly and if it's at all possible.
>