Windows Azure now offers Virtual Machine hosting similar to IaaS solutions. One of the associated services is something called Azure Connect so that your VM can access on-premise resources (in your own office). What is unclear to me is that AC does not appear to be a complete VPN connection, but you can install the AC connector onto each local resource you wish your hosted VM to have access to. That doesn't sound so fun because to access our SQL server it seems like I'd need to install it on my SQL server AND on my AD-controller. There's no way our admin guys will like that.
Is setting up a more traditional IPSEC VPN a better option for full network integration?
Today, Azure Connect is the solution, and it's as you describe: Install the agent on each machine to be added to your virtual network. Keep this in mind, as you discuss this with your admins:
One more thing to keep in mind: Azure Connect, while connecting over IPv4, creates an IPv6 tunnel. SQL Server and AD should both work fine with IPv6 (I've had no issues with connecting with an on-premises SQL Server box).
Azure now supports site to site IPSEC tunnels to your on premises hosts if you are using a supported Cisco or Juniper firewall/vpn device.
http://www.windowsazure.com/en-us/manage/services/networking/cross-premises-connectivity/