I have been looking at an Active Directory that has several thousand groups, where pairs of groups are members of each other.
GroupA has GroupB as a Member. GroupB has GroupA as a Member.
Oy. I am trying to think through possible consequences of this circular nesting of groups.
Well first of all, be careful that you don't have users that are members of too many groups - this can cause their token to be too large and you end up with things like this:
And also GPOs will stop being processed, startup scripts, etc.
This isn't directly answering your question, but a bunch of nested groups can definitely exacerbate this problem. There's nothing inherently terrible about groups being members of one another. i.e. the space-time continuum will not tear open... the only things I can think of is that you might confuse some applications that make extensive use of LDAP queries... things like Exchange, etc.
There are no consequences - at least not as far as Active Directory is concerned.
I've seen deployments with this condition several times; the only thing it breaks is poorly written code that recursively enumerates groups. And in those cases, it's a simple thing to check for this kind of loop in code and ignore the groups that you've already enumerated, or else just limit recursion depth.
So, I wouldn't say it's bad, but it can be. There's a few reasons, one of them has to do with scripting. Circular nesting is essentially an "infinite loop" because scripts use a lot of recursive functions. This would obviously cause a script to error out, etc.
Then there is the idea of 'simplification' in AD that circular nesting inherently goes against.
There is a powershell script on technet gallery that helps locate circular nested groups, you can find it here and it'll help in the locating of circular groups: Find Circular Nested Groups
Two other PowerShell scripts enabling to draw nested groups and so helping to find quickly circular nesting :