Security concern for LDAP Query

That is correct. That information is available as part of LDAP. You could lock down AD using delegation and modifying security rights, however I wouldn't recommend it.

Yes, the default security permissions in Active Directory give all users read access to the majority of attributes on objects in the directory including other users.

If removing that ability is necessary to satisfy your business's security requirements, unfortunately it's not as easy as modifying the permissions on the OU/container where your sensitive users are located. The permissions that grant read access to those attributes are aren't actually inherited from their container. They're set directly on the object at creation time.

In order to change that, you need to edit the AD schema and modify the default security ACL on the user class to whatever your security requirements demand. It's a sensitive operation to be sure. But unlike other schema changes, it's entirely reversible (just change the permissions back).

It also won't retroactively affect users that already exist. You'll need to go back after the fact and use a tool like dsacls to reset the users to their default security permissions from the schema.

Keep in mind that a lot of applications that access Active Directory will assume the default security permissions exist and may fail in odd ways if they can't read those user attributes. So make sure any applications that need access are running with credentials that have been given explicit access to read the attributes they care about.