When AWS documentation and pricing refer to "usage" does this simply mean "if the instance is on" instead of "if the instance is on and doing work".
E.g., if I had an EC2 instance running but it was idle (CPU=0%), I assume I still get charged for that hour's usage.
In this case, if I had an EC2 instance which hosted a website (which should be accessible 24/7), it would make sense to purchase a Reserved Instance. Then, if I had to bring other instance online to share the load, those would (likely) best be served as On-Demand Instances.
Is my understanding correct?
Yes, your understanding is correct. There's no AWS charging based on CPU usage -- you pay the same for an instance whether its CPU usage is 0% or 100%.
Yes, that's correct.
You might also look at spot instances for load sharing as they're much cheaper than on-demand instances with the drawback that they can disappear at any time. As long as you can manage that, they're a good option.