In my experience the number of trusts doesn't really matter, its the amount of users and traffic that comes into the box. Like anything IIS related you really have to scale the IIS portion like you normally would as well as scale the ADFS service. By default it relies on the Windows Internal DB, my first step is usually to replace that with a MSSQL cluster or highly redundant server to store and hold config data. This also lets you get past the node limit imposed by the Windows internal DB setup.
In my experience the number of trusts doesn't really matter, its the amount of users and traffic that comes into the box. Like anything IIS related you really have to scale the IIS portion like you normally would as well as scale the ADFS service. By default it relies on the Windows Internal DB, my first step is usually to replace that with a MSSQL cluster or highly redundant server to store and hold config data. This also lets you get past the node limit imposed by the Windows internal DB setup.