Nginx and dynamic X-Forwarded-For blocklist

I am running Nginx behind a proxy. I am getting visitors' real IP addresses through X-Forwarded-For HTTP request header. I'd like to block individual IPs based on a dynamic block list, up to 10000+ IPs.

I was looking into ModSecurity to do this, but it looks too complex with very arcane and brittle configuration language just for doing IP blocking. What other alternatives there are?

Also, it is viable to generate Nginx rules configuration (ip-blocklist.conf) for the inclusion in the main Nginx config e.g. for every 5 minutes and then gracefully restart Nginx? How scaleable are Nginx rules e.g. up to 10k entries?