What would cause log entries to be out of order?
For the most part, the log entries are in chronological order, but every once in a while there will be a request logged with a time stamp that's much older than the surrounding requests - sometimes, it can be a minute or so older.
It seems that most of these are POST requests, and the back-end server has been experiencing some problems - is the request only logged at the end of the pipeline?
Apache does indeed only log events at the end of the request (when the page or error is served). This is how Apache knows how many bytes were transferred, what response to log, etc.