I want to set up a site-to-site VPN using multiple ISPs and WAN links, such that all traffic to/from site A goes via site B, and the implementation details are completely transparent to the LAN at each end.
I've had a good look at the Peplink devices that have been widely recommended in response to similar questions, but site B will probably be a cloud-hosted environment. That makes a hardware solution less than ideal. I've also checked out a couple of service providers who offer link aggregation as a service, but for a fairly high price.
Can this be done with a combination of open protocols, to avoid sticker shock and vendor lock-in?
Take a look at MLPPP - Multi-Link PPP. Its capabilities should meet what you described with probably even non-appliance (a.k.a. pure software) implementations as well.