As the question states, what are the drawbacks of making every capable Windows 2003 or 2008 Server in my organization a domain controller for the domain? Is it just overkill? Will many 3rd party applications explode? Something else I'm not thinking of?
Are there any advantages?
Best advice I can give you is to run domain controllers as very discrete entities wherever possible i.e. load no services onto a domain controller that is not essential to the operation of the domain controller. This is commonly overlooked with very small shops and especially Small Business Server for practical/cost reasons, but once you scale beyond that you ideally want to be heading towards a point where DCs are JUST DCs, and you only run as many DCs as you realistically need for adequate replication and fault tolerance.
To add to the very good list Chris Thorpe posted, here are some more reasons why doing that is a bad idea:
Really, the "replication overhead" argument is a really strong one. If you have a small number of servers local to each other, say under 10, it isn't as bad. Once you get to large numbers, especially if they're remote from each other, problems start magnifying. Replication inside an AD Site is one-to-many, and between sites are generally configured with bridge-head hosts funneling the updates. Not just AD information needs to get replicated, so does all the Group Policy information (that's what's stored in "SYSVOL") needs to get replicated to every DC. It's a very complex replication mesh when you get many DCs in an environment, and it's a lot easier for things to go wrong.
From a security stand-point you really do not want potential attackers getting local access to a DC. It is a lot easier to extract the entire domain's password hashes when you're local to a DC, and with Rainbow Tables that's pretty much game-over unless your password policies are much more strict than are commonly used today.
It's a very bad idea. I can't really think of a reason to do this. Many MS services will not run or are not supported on DCs. Also any bug or hole in any software you install on a DC risks your entire domain's security.