A production server has a constant stream of incoming data which is delivered over HTTP. To test a new revision of the system, I deploy the new version to a staging server, run tests on it, and when I'm satisfied, I want to try it on the real data, but without affecting the production system.
To achieve that, I would like to copy the incoming HTTP traffic and direct the copy to the staging system. I had previously baked that into the application itself, but I don't want to do that anymore, for a number of reasons. What are my options?
- Are there proxies that do this?
- Are there web servers that do this? The closest I found was Apache's forensic module, but I'm not sure if that's suitable, it clearly has a slightly different goal.
- Or should I do it at the Internet layer?
Of course, the staging server wants to send replies somewhere, just like the production system. And naturally, these replies shouldn't go back to the production system's clients. Simply discarding them at some stage is OK, but it would be great if I could direct them to some logging facility for inspection.
I'd be inclined to capture the TCP traffic, rip the request data out of that, and use a few lines of script to feed the requests back into the staging server. For requests that you don't expect to change between staging and production, you could even have the responses from the TCP stream compared to those produced by the staging server, to validate correctness.