My company (an inward-looking institution without customer or external interface) has moved all operations from a local data centre to AWS a few years back, mostly to a handful of large AWS accounts (network, management, billing, sandbox, dev, test, prod). Developers have been having a hard time innovating in this initial setup, because IAM policies and service whitelisting had to be quite restrictive due to the shared nature of those accounts.
So, the latest thing we're busy with now, is introducing more accounts, e.g. one triplet of dev/test/prod accounts per product or development team. With a few key accounts creating guardrails around those product accounts (network VPCs providing, security scanning, log/reporting, etc...). Separate account triplets for each dev team will mean the splash radius of what they do will more easily be contained via inter-account firewalls, security groups, etc... Once developers get their accounts, they also get an admin account to more freely explore, and if they do something suspicious (e.g. a machine opening up all ports to 0.0.0.0/0), we've got Turbot.com nuking the rogue resources/account in no time.
Seems easy enough, but for some reason our 3rd party account/billing provider who I will not mention in here, has been having challenges providing us new accounts on the fly (we're talking dozens of account triplets as we start up new projects/apps internally).
I am puzzled by a few things:
- why would our internal cloud team not buy accounts straight from AWS ?
- what can possibly be the issue in them not being to provision new accounts swiftly ?
- what's complex about that service offering ?
- How are they possibly architected ?
- How do they provide security, IAM, and care for our root credentials ? I understand we're restricted from creating roles and policies with certain names that they reserve for themselves
- What does it take to automate 3rd party AWS account provisioning the way they do ? I understand they're kind of owning each of our accounts, and billing gets aggregated to a specific account they've large control over.
I'm trying to understand the complexity of what they do and whether there are alternative routes. As a developer, I've been waiting weeks for my accounts internally and I'm searching for answers. To me it looks like red tape that was decided upon arbitrarily in the past.
They probably get discounts. These large "billing consolidation providers" invoice their customers millions of dollars per month and for that they get decent discounts from AWS, Azure, etc, and pass on some of that to the customers. Typically you get around 5% discount.
The big downside is that they technically own your accounts and depending on the billing provider and the configuration you may be quite restricted in what you can and can't do. Typically you can't create sub-accounts as you have seen yourself, often you can't view billing data directly (e.g. AWS Cost Explorer) and will be forced to use the billing provider's portal for that, you may not be able to buy reserved instances or spending plans or even subscribe to support plans directly, etc.
I personally hate these billing arrangements - the little discount is not worth the limitations and the loss in productivity it brings. But the management often only sees the 5% discount on the bill and not the frustration and the extra hours their engineers spend fighting and working around the restrictions. It probably costs them way more than those 5% in the lost productivity, but hey, that's not as visible as a discount item on an invoice.
That'd be my explanation ;)