I have a multithreaded application that listens to events and needs to respond to them instantly.
With the application running, my server is under 5% CPU load in a 1 core server.
I wonder though, would increasing the number of threads my server has lower response time?
Seems intuitive but just to make sure.
Depends on how the web application is written. In general, no, there is no benefit for one single request.
Now let's detail a bit. Problems that can be parallelized have a portion that can't be parallelized and you are stuck with that, no mater how much threads you are adding.
After a certain number of threads, you might have worse performance because of overhead or managing the threads.
You can not execute more threads at the same time than physical CPU cores.
You might have a performance gain if you enable hyper-threading.
Most web server application are executing each request in one single thread. You might have a separate thread for DB requests or other jobs, but in general each request is executed by one single thread.
If you use other services and your goal is to lower the latency for your application, lowering the number of parallel threads could actually lead to increase of performance as you avoid bottlenecks caused because too many requests are running in parallel.
TL;DR benchmark. Micro benchmarks, and real, full stack benchmarks.
Let that sink in: 9 women cannot deliver a baby in one month.