ErikE's questions

We have a huge directory tree, many levels deep and with a (very) large number of small files at each level.

From time to time we have large file data changes where parts of the tree are replaced and the permissions for these parts will need to be reset. It is not possible to foresee which parts will change next time.

The files currently reside on a Windows NTFS partition.

Resetting permissions needs to be done recursively from the root. This takes the better part of a working day, where the actual requirement is near instant change (or the business suffers).

I have tried GUI. I have tried robocopy. I have tried powershell. I have tried a Go library (Wrapper for the Windows API), as Go has a reputation for being fast, but it turned out little was gained.

I have contemplated letting applications work through symlinks, where the symlinks have the restrictive permissions and the data has (very) permissive permissions. But the root problem would remain: after a data replacement we would still need to trawl through the tree and set these permissive permissions.

Setting group acl:s is not a solution, we use that already. The replacement data has a different set of permissions and these need to be replaced.

Windows is not a requirement, we also run Linux. If yet other OS platforms could solve the task of setting massive file permissions and making the files accessible through common file sharing protocols (http, smb et c), such an OS could be considered.

So my question is: Using Windows NTFS as a baseline, are there file system/OS combinations which provide significantly faster recursive file system operations (setting permissions), whilst still serving the files through a common file sharing protocol?

Procedural suggestions are also welcome - have I and my colleagues just missed some obviously more simple solution than replacing the file system or OS?

EDIT based on constructive comments: yes we have a dev team inhouse and can potentially leverage anything between sysadmin hacks (me) to well designed code (devs).

EDIT2 Answering questions from @GregAskew (approximations as on holiday until monday)

1. How many ACE's are in the ACL?

   • About 8 ACE’s per ACL.

2. Is the file system optimized for performance (short file names disabled, last access time disabled)?

   • No, I was not aware of these optimizations and will try them out.

3. What are the maximum number of files in a directory>

   • Will have to measure.

4. What is the state of file system fragmentation and directory index fragmentation before setting the ACL?

   • Unknown, will investigate.

5. What is the allocation unit size of the volume? What operating system version is hosting the volume?

   • Currently a VMware vSAN hosting a Windowws 2008 R2 Std, upgrade imminent to Win2016 Std.

6. Are you setting the permissions locally (based on another comment, it seems like you are doing this over the network)?

   • We are setting the permissions locally in the VM, but then let them DFS-replicate for redundancy (excruciating and this will be redesigned). We have full control over design, it is just the implementation of the initial file delivery which is out of our control (but based on a comment we may try to change this). The question is just about changing the local file permissions (I realize the underlying SAN is network connected but will gladly take constructive suggestions along those lines all the same).

As is well known, to avoid the Powershell Second Hop problem when Powershell Remoting, one needs to set the CredSSP Client and Server settings.

e.g.

# On the client, a.k.a the First Hop Server
Enable-WSManCredSSP -Role "Client" -DelegateComputer "secondhop.example.com"

# On the server, a.k.a the Second Hop Server
Enable-WSManCredSSP -Role "Server"

And now I can PSRemote to the "client" server and from there access resources on the "server" server. All good and well.

However, the environment is largeish, and I want to know the "Client DelegateComputers" in effect on a good number of machines. How do I achieve this?

I know I can (Get-WSManCredSSP)[0] to get a list of Client DelegateComputers. But this is presented as an ugly, unwieldy string which I would very much like to not parse, given that this is Powershell and that I am in love with the object oriented approach.

I would like to think that I have been all over wsman:/ both using Get-Item and Get-​WS​Man​Instance, as well as googled through what reasonably must be all of Google.

It seems unthinkable that this is so elusive, I must have missed something embarrassingly simple.

So ServerFault, how do I Powershell natively or using a direct .NET call get a nicely presented array or similar of CredSSP Client DelegateComputers on a given host?

I can run any version of .NET Framework and Powershell available on Win2008r2 and Win2012r2.