I am developing an AuthZ module.
While testing using the rate limiting module. I can see rate limiting kick
in for GET requests fine (it's tuned extra low to demonstrate this case):
curl -s -I http://localhost/login?{1..3}
HTTP/1.1 200 OK
Server: nginx/1.21.6
Date: Sun, 04 Dec 2022 16:43:17 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 1651
Connection: keep-alive
HTTP/1.1 429 Too Many Requests
Server: nginx/1.21.6
Date: Sun, 04 Dec 2022 16:43:17 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
HTTP/1.1 429 Too Many Requests
Server: nginx/1.21.6
Date: Sun, 04 Dec 2022 16:43:17 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
However, doing the same for POST requests, this does not work:
curl -s -w "\nStatus: %{http_code}\n\n" http://localhost/login?{1..3}
--data-raw 'username=user&password=user'
login success: user
Status: 200
login success: user
Status: 200
login success: user
Status: 200
Setting my module to run in the `precontent` phase allows this to work, so
it's all happening in rewrite (where the rate limiting module would be
kicking in).
I obviously don't want to run in precontent and my module gets its
advice from an external "agent" as to what to set the status. So I'm
assuming it is overwriting the nginx rate limiting module's status and
setting it back to a 200, when I'd rather respect the rate limiting modules
429.
What would be the best approach here to avoid this from happening? I have
read about module ordering, but that would require a recompile of my end,
however, I am more intrigued about how to handle this in code.
Thanks
Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20221204/778c5165/attachment.htm>
Actually analysing the log files of this, it seems the rate limiting module
never kicks in for POST requests, my module just sets the status and bails.
Assuming this is because POST actually needs to write content?
On Sun, 4 Dec 2022 at 16:57, Jeremy Cocks <jeremy at jeremy.cx> wrote:
> I am developing an AuthZ module.
>
> While testing using the rate limiting module. I can see rate limiting kick
> in for GET requests fine (it's tuned extra low to demonstrate this case):
>
> curl -s -I http://localhost/login?{1..3}
> HTTP/1.1 200 OK
> Server: nginx/1.21.6
> Date: Sun, 04 Dec 2022 16:43:17 GMT
> Content-Type: text/html; charset=utf-8
> Content-Length: 1651
> Connection: keep-alive
>
> HTTP/1.1 429 Too Many Requests
> Server: nginx/1.21.6
> Date: Sun, 04 Dec 2022 16:43:17 GMT
> Content-Type: text/html
> Content-Length: 169
> Connection: keep-alive
>
> HTTP/1.1 429 Too Many Requests
> Server: nginx/1.21.6
> Date: Sun, 04 Dec 2022 16:43:17 GMT
> Content-Type: text/html
> Content-Length: 169
> Connection: keep-alive
>
>
> However, doing the same for POST requests, this does not work:
>
> curl -s -w "\nStatus: %{http_code}\n\n" http://localhost/login?{1..3}
> --data-raw 'username=user&password=user'
> login success: user
> Status: 200
>
> login success: user
> Status: 200
>
> login success: user
> Status: 200
>
> Setting my module to run in the `precontent` phase allows this to work, so
> it's all happening in rewrite (where the rate limiting module would be
> kicking in).
>
> I obviously don't want to run in precontent and my module gets its
> advice from an external "agent" as to what to set the status. So I'm
> assuming it is overwriting the nginx rate limiting module's status and
> setting it back to a 200, when I'd rather respect the rate limiting modules
> 429.
>
> What would be the best approach here to avoid this from happening? I have
> read about module ordering, but that would require a recompile of my end,
> however, I am more intrigued about how to handle this in code.
>
> Thanks
> Jeremy
>
>
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20221204/66e8ab2e/attachment.htm>
Hello!
On Sun, Dec 04, 2022 at 04:57:17PM +0000, Jeremy Cocks via nginx wrote:
> I am developing an AuthZ module.
[...]
> Setting my module to run in the `precontent` phase allows this to work, so
> it's all happening in rewrite (where the rate limiting module would be
> kicking in).
>
> I obviously don't want to run in precontent and my module gets its
> advice from an external "agent" as to what to set the status. So I'm
> assuming it is overwriting the nginx rate limiting module's status and
> setting it back to a 200, when I'd rather respect the rate limiting modules
> 429.
>
> What would be the best approach here to avoid this from happening? I have
> read about module ordering, but that would require a recompile of my end,
> however, I am more intrigued about how to handle this in code.
Auth modules are expected to work at the access phase
(NGX_HTTP_ACCESS_PHASE), so these can be combined by using the
"satisfy" directive (http://nginx.org/r/satisfy), and won't
interfere with request limiting, which happens just before the
access phase, at the preaccess phase (NGX_HTTP_PREACCESS_PHASE).
In particular, such order ensures that rate limiting is able to
protect auth modules from bruteforce attacks.
It also ensures that you don't need to think about any overwriting
and or anything like this - requests which do not satisfy rate
limits configured will be rejected before the control reaches the
access phase.
--
Maxim Dounin
http://mdounin.ru/
Hello!
On Sun, Dec 04, 2022 at 05:29:29PM +0000, Jeremy Cocks via nginx wrote:
> Actually analysing the log files of this, it seems the rate limiting module
> never kicks in for POST requests, my module just sets the status and bails.
> Assuming this is because POST actually needs to write content?
Your observation is wrong. Rate limiting as implemented in the
limit_req module does not distinguish between different request
methods and always works after reading the request headers.
--
Maxim Dounin
http://mdounin.ru/
> Your observation is wrong. Rate limiting as implemented in the
> limit_req module does not distinguish between different request
> methods and always works after reading the request header
Sorry, I wasn't clear there. It kicks in for POST requests without my
module ;)
> does not distinguish between different request methods and always works
after reading the request headers.
I am assuming, given the request I am testing, is on a proxy_pass which is
a content handler, that has something to do with why rate limiting is not
working on POST and not GET here?
If I remove the location block and just have my module and rate limiting
going without a proxy_pass, it seems to be working fine for all requests.
Thanks!
J
On Sun, 4 Dec 2022 at 19:52, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Sun, Dec 04, 2022 at 05:29:29PM +0000, Jeremy Cocks via nginx wrote:
>
> > Actually analysing the log files of this, it seems the rate limiting
> module
> > never kicks in for POST requests, my module just sets the status and
> bails.
> > Assuming this is because POST actually needs to write content?
>
> Your observation is wrong. Rate limiting as implemented in the
> limit_req module does not distinguish between different request
> methods and always works after reading the request headers.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx mailing list -- nginx at nginx.org
> To unsubscribe send an email to nginx-leave at nginx.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20221204/478afde7/attachment.htm>
Hello!
On Sun, Dec 04, 2022 at 08:00:04PM +0000, Jeremy Cocks via nginx wrote:
> > Your observation is wrong. Rate limiting as implemented in the
> > limit_req module does not distinguish between different request
> > methods and always works after reading the request header
>
> Sorry, I wasn't clear there. It kicks in for POST requests without my
> module ;)
So, it looks like your module breaks something.
> > does not distinguish between different request methods and always works
> after reading the request headers.
>
> I am assuming, given the request I am testing, is on a proxy_pass which is
> a content handler, that has something to do with why rate limiting is not
> working on POST and not GET here?
> If I remove the location block and just have my module and rate limiting
> going without a proxy_pass, it seems to be working fine for all requests.
It's hard to say anything beyond that your module breaks something
without seeing your module's code and the configuration you are
trying to use.
A properly implemented auth module, as already suggested, should
work in the access phase, and wouldn't interfere with any rate
limits, since rate limiting happens before the auth module ever
sees a request.
--
Maxim Dounin
http://mdounin.ru/
At the moment, no authz is really happening. During this testing I was
statically setting the advice from the policy server to be 200.
Setting this in the access phase works fine for me with no random hitches,
so that's fixed. Thanks.
For future reference, is there anything to go by which dictates what phase
a module should be in and its impact?
Obviously, access is quite self-explanatory and not sure how i missed it ;')
On Sun, 4 Dec 2022 at 20:19, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Sun, Dec 04, 2022 at 08:00:04PM +0000, Jeremy Cocks via nginx wrote:
>
> > > Your observation is wrong. Rate limiting as implemented in the
> > > limit_req module does not distinguish between different request
> > > methods and always works after reading the request header
> >
> > Sorry, I wasn't clear there. It kicks in for POST requests without my
> > module ;)
>
> So, it looks like your module breaks something.
>
> > > does not distinguish between different request methods and always works
> > after reading the request headers.
> >
> > I am assuming, given the request I am testing, is on a proxy_pass which
> is
> > a content handler, that has something to do with why rate limiting is
> not
> > working on POST and not GET here?
> > If I remove the location block and just have my module and rate limiting
> > going without a proxy_pass, it seems to be working fine for all requests.
>
> It's hard to say anything beyond that your module breaks something
> without seeing your module's code and the configuration you are
> trying to use.
>
> A properly implemented auth module, as already suggested, should
> work in the access phase, and wouldn't interfere with any rate
> limits, since rate limiting happens before the auth module ever
> sees a request.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx mailing list -- nginx at nginx.org
> To unsubscribe send an email to nginx-leave at nginx.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20221204/306a7b62/attachment.htm>
Hello!
On Sun, Dec 04, 2022 at 08:32:13PM +0000, Jeremy Cocks via nginx wrote:
> At the moment, no authz is really happening. During this testing I was
> statically setting the advice from the policy server to be 200.
> Setting this in the access phase works fine for me with no random hitches,
> so that's fixed. Thanks.
Note that in an auth module you are not expected to return or set
200 anywhere. Instead, you should return NGX_DECLINED, NGX_OK,
NGX_HTTP_FORBIDDEN, or NGX_HTTP_UNAUTHORIZED.
> For future reference, is there anything to go by which dictates what phase
> a module should be in and its impact?
> Obviously, access is quite self-explanatory and not sure how i missed it ;')
First of all, you may want to read the relevant chapter of the
development guide:
http://nginx.org/en/docs/dev/development_guide.html#http_phases
It explains what different phases are expected to do, and lists
various standard modules which work at the relevant phases.
Reading the relevant core code and corresponding standard modules
might also help.
--
Maxim Dounin
http://mdounin.ru/