There's a related question with an answer at apache auth: combination of LDAP and htpasswd but I'd like to expand on that.
I want to allow users if they are in the htpasswd file or if they are in LDAP and members of a valid group.
So Require valid-user
from the other question is not strict enough as it would allow anyone from LDAP.