Home / server / Questions / 661577
In Process
Mircea Vutcovici
Asked: 2015-01-23 07:20:40 +0800 CST

Debugging a chdir(2) syscall

  • 772

I have often faced various file system permission problems on Linux. And now with SELinux a bit complicated. I am searching for a generic way to debug this.

In my particular case I have a chdir(2) syscall that fails with:

chdir("/var/log/httpd")                 = -1 EACCES (Permission denied)

With ftrace I have a more detailed graph, but I am missing the arguments and the return values.

# trace-cmd record -p function_graph -g sys_enter_chdir -P 26510
# trace-cmd report
[...]
smbd-26510 [001] 10861828.204161: funcgraph_entry:                   |  sys_chdir() {
smbd-26510 [001] 10861828.204162: funcgraph_entry:                   |    user_path_at() {
smbd-26510 [001] 10861828.204162: funcgraph_entry:                   |      getname() {
smbd-26510 [001] 10861828.204163: funcgraph_entry:                   |        kmem_cache_alloc() {
smbd-26510 [001] 10861828.204163: funcgraph_entry:        0.447 us   |          _cond_resched();
smbd-26510 [001] 10861828.204165: funcgraph_exit:         1.784 us   |        }
smbd-26510 [001] 10861828.204165: funcgraph_entry:                   |        strncpy_from_user() {
smbd-26510 [001] 10861828.204166: funcgraph_entry:        0.454 us   |          _cond_resched();
smbd-26510 [001] 10861828.204167: funcgraph_exit:         1.473 us   |        }
smbd-26510 [001] 10861828.204167: funcgraph_exit:         4.926 us   |      }
smbd-26510 [001] 10861828.204168: funcgraph_entry:                   |      filename_lookup() {
smbd-26510 [001] 10861828.204168: funcgraph_entry:                   |        path_init() {
smbd-26510 [001] 10861828.204169: funcgraph_entry:        0.725 us   |          _read_lock();
smbd-26510 [001] 10861828.204170: funcgraph_exit:         1.928 us   |        }
smbd-26510 [001] 10861828.204171: funcgraph_entry:                   |        path_walk() {
smbd-26510 [001] 10861828.204171: funcgraph_entry:                   |          __link_path_walk() {
smbd-26510 [001] 10861828.204172: funcgraph_entry:                   |            acl_permission_check() {
smbd-26510 [001] 10861828.204173: funcgraph_entry:                   |              ext4_check_acl() {
smbd-26510 [001] 10861828.204174: funcgraph_entry:        0.612 us   |                ext4_get_acl();
smbd-26510 [001] 10861828.204175: funcgraph_exit:         1.807 us   |              }
smbd-26510 [001] 10861828.204175: funcgraph_entry:                   |              in_group_p() {
smbd-26510 [001] 10861828.204176: funcgraph_entry:        0.770 us   |                groups_search();
smbd-26510 [001] 10861828.204177: funcgraph_exit:         1.826 us   |              }
smbd-26510 [001] 10861828.204177: funcgraph_exit:         5.527 us   |            }
smbd-26510 [001] 10861828.204178: funcgraph_entry:                   |            security_inode_permission() {
smbd-26510 [001] 10861828.204179: funcgraph_entry:                   |              selinux_inode_permission() {
smbd-26510 [001] 10861828.204180: funcgraph_entry:                   |                inode_has_perm() {
smbd-26510 [001] 10861828.204180: funcgraph_entry:                   |                  avc_has_perm() {
smbd-26510 [001] 10861828.204181: funcgraph_entry:        1.443 us   |                    avc_has_perm_noaudit();
smbd-26510 [001] 10861828.204183: funcgraph_entry:        0.593 us   |                    avc_audit();
smbd-26510 [001] 10861828.204184: funcgraph_exit:         3.735 us   |                  }
smbd-26510 [001] 10861828.204184: funcgraph_exit:         4.892 us   |                }
smbd-26510 [001] 10861828.204185: funcgraph_exit:         5.910 us   |              }
smbd-26510 [001] 10861828.204185: funcgraph_exit:         7.219 us   |            }
smbd-26510 [001] 10861828.204186: funcgraph_entry:                   |            do_lookup() {
smbd-26510 [001] 10861828.204187: funcgraph_entry:                   |              __d_lookup() {
smbd-26510 [001] 10861828.204208: funcgraph_entry:      + 19.857 us  |                _spin_lock();
smbd-26510 [001] 10861828.204210: funcgraph_exit:       + 23.168 us  |              }
smbd-26510 [001] 10861828.204210: funcgraph_entry:                   |              follow_managed() {
smbd-26510 [001] 10861828.204211: funcgraph_entry:                   |                lookup_mnt() {
smbd-26510 [001] 10861828.204211: funcgraph_entry:        0.496 us   |                  _spin_lock();
smbd-26510 [001] 10861828.204212: funcgraph_entry:        1.007 us   |                  __lookup_mnt();
smbd-26510 [001] 10861828.204214: funcgraph_exit:         3.066 us   |                }
smbd-26510 [001] 10861828.204215: funcgraph_entry:        0.601 us   |                dput();
smbd-26510 [001] 10861828.204216: funcgraph_exit:         5.659 us   |              }
smbd-26510 [001] 10861828.204217: funcgraph_exit:       + 30.543 us  |            }
smbd-26510 [001] 10861828.204218: funcgraph_entry:        0.519 us   |            dput();
smbd-26510 [001] 10861828.204219: funcgraph_entry:        0.485 us   |            mntput_no_expire();
smbd-26510 [001] 10861828.204220: funcgraph_entry:                   |            acl_permission_check() {
smbd-26510 [001] 10861828.204221: funcgraph_entry:                   |              ext4_check_acl() {
smbd-26510 [001] 10861828.204221: funcgraph_entry:        0.751 us   |                ext4_get_acl();
smbd-26510 [001] 10861828.204222: funcgraph_exit:         1.660 us   |              }
smbd-26510 [001] 10861828.204223: funcgraph_entry:                   |              in_group_p() {
smbd-26510 [001] 10861828.204223: funcgraph_entry:        0.530 us   |                groups_search();
smbd-26510 [001] 10861828.204224: funcgraph_exit:         1.455 us   |              }
smbd-26510 [001] 10861828.204225: funcgraph_exit:         4.949 us   |            }
smbd-26510 [001] 10861828.204225: funcgraph_entry:                   |            security_inode_permission() {
smbd-26510 [001] 10861828.204226: funcgraph_entry:                   |              selinux_inode_permission() {
smbd-26510 [001] 10861828.204226: funcgraph_entry:                   |                inode_has_perm() {
smbd-26510 [001] 10861828.204227: funcgraph_entry:                   |                  avc_has_perm() {
smbd-26510 [001] 10861828.204228: funcgraph_entry:        0.849 us   |                    avc_has_perm_noaudit();
smbd-26510 [001] 10861828.204229: funcgraph_entry:        0.496 us   |                    avc_audit();
smbd-26510 [001] 10861828.204230: funcgraph_exit:         2.754 us   |                  }
smbd-26510 [001] 10861828.204231: funcgraph_exit:         4.186 us   |                }
smbd-26510 [001] 10861828.204231: funcgraph_exit:         5.159 us   |              }
smbd-26510 [001] 10861828.204231: funcgraph_exit:         6.132 us   |            }
smbd-26510 [001] 10861828.204232: funcgraph_entry:                   |            do_lookup() {
smbd-26510 [001] 10861828.204232: funcgraph_entry:                   |              __d_lookup() {
smbd-26510 [001] 10861828.204234: funcgraph_entry:        0.451 us   |                _spin_lock();
smbd-26510 [001] 10861828.204235: funcgraph_exit:         2.446 us   |              }
smbd-26510 [001] 10861828.204235: funcgraph_entry:        0.503 us   |              follow_managed();
smbd-26510 [001] 10861828.204236: funcgraph_exit:         4.343 us   |            }
smbd-26510 [001] 10861828.204237: funcgraph_entry:        0.462 us   |            dput();
smbd-26510 [001] 10861828.204238: funcgraph_entry:                   |            acl_permission_check() {
smbd-26510 [001] 10861828.204239: funcgraph_entry:                   |              ext4_check_acl() {
smbd-26510 [001] 10861828.204239: funcgraph_entry:        0.579 us   |                ext4_get_acl();
smbd-26510 [001] 10861828.204240: funcgraph_exit:         1.484 us   |              }
smbd-26510 [001] 10861828.204241: funcgraph_entry:                   |              in_group_p() {
smbd-26510 [001] 10861828.204241: funcgraph_entry:        0.481 us   |                groups_search();
smbd-26510 [001] 10861828.204242: funcgraph_exit:         1.383 us   |              }
smbd-26510 [001] 10861828.204243: funcgraph_exit:         4.479 us   |            }
smbd-26510 [001] 10861828.204243: funcgraph_entry:                   |            security_inode_permission() {
smbd-26510 [001] 10861828.204244: funcgraph_entry:                   |              selinux_inode_permission() {
smbd-26510 [001] 10861828.204244: funcgraph_entry:                   |                inode_has_perm() {
smbd-26510 [001] 10861828.204245: funcgraph_entry:                   |                  avc_has_perm() {
smbd-26510 [001] 10861828.204245: funcgraph_entry:        0.786 us   |                    avc_has_perm_noaudit();
smbd-26510 [001] 10861828.204246: funcgraph_entry:        0.461 us   |                    avc_audit();
smbd-26510 [001] 10861828.204247: funcgraph_exit:         2.638 us   |                  }
smbd-26510 [001] 10861828.204248: funcgraph_exit:         3.814 us   |                }
smbd-26510 [001] 10861828.204248: funcgraph_exit:         4.727 us   |              }
smbd-26510 [001] 10861828.204249: funcgraph_exit:         5.647 us   |            }
smbd-26510 [001] 10861828.204249: funcgraph_entry:                   |            do_lookup() {
smbd-26510 [001] 10861828.204250: funcgraph_entry:                   |              __d_lookup() {
smbd-26510 [001] 10861828.204253: funcgraph_entry:        0.451 us   |                _spin_lock();
smbd-26510 [001] 10861828.204254: funcgraph_exit:         4.645 us   |              }
smbd-26510 [001] 10861828.204255: funcgraph_entry:        0.466 us   |              follow_managed();
smbd-26510 [001] 10861828.204256: funcgraph_exit:         6.492 us   |            }
smbd-26510 [001] 10861828.204257: funcgraph_entry:                   |            path_to_nameidata() {
smbd-26510 [001] 10861828.204257: funcgraph_entry:        0.466 us   |              dput();
smbd-26510 [001] 10861828.204258: funcgraph_exit:         1.402 us   |            }
smbd-26510 [001] 10861828.204272: funcgraph_exit:       + 87.715 us  |          }
smbd-26510 [001] 10861828.204273: funcgraph_entry:                   |          path_put() {
smbd-26510 [001] 10861828.204273: funcgraph_entry:        0.508 us   |            dput();
smbd-26510 [001] 10861828.204274: funcgraph_entry:        0.481 us   |            mntput_no_expire();
smbd-26510 [001] 10861828.204275: funcgraph_exit:         2.544 us   |          }
smbd-26510 [001] 10861828.204276: funcgraph_exit:       ! 105.077 us |        }
smbd-26510 [001] 10861828.204276: funcgraph_entry:                   |        path_put() {
smbd-26510 [001] 10861828.204277: funcgraph_entry:        0.462 us   |          dput();
smbd-26510 [001] 10861828.204277: funcgraph_entry:        0.459 us   |          mntput_no_expire();
smbd-26510 [001] 10861828.204278: funcgraph_exit:         2.270 us   |        }
smbd-26510 [001] 10861828.204279: funcgraph_exit:       ! 111.221 us |      }
smbd-26510 [001] 10861828.204279: funcgraph_entry:                   |      putname() {
smbd-26510 [001] 10861828.204280: funcgraph_entry:                   |        final_putname() {
smbd-26510 [001] 10861828.204280: funcgraph_entry:        0.567 us   |          kmem_cache_free();
smbd-26510 [001] 10861828.204281: funcgraph_exit:         1.499 us   |        }
smbd-26510 [001] 10861828.204282: funcgraph_exit:         2.431 us   |      }
smbd-26510 [001] 10861828.204282: funcgraph_exit:       ! 120.501 us |    }
smbd-26510 [001] 10861828.204283: funcgraph_entry:                   |    inode_permission() {
smbd-26510 [001] 10861828.204283: funcgraph_entry:                   |      generic_permission() {
smbd-26510 [001] 10861828.204284: funcgraph_entry:                   |        acl_permission_check() {
smbd-26510 [001] 10861828.204285: funcgraph_entry:                   |          in_group_p() {
smbd-26510 [001] 10861828.204285: funcgraph_entry:        0.533 us   |            groups_search();
smbd-26510 [001] 10861828.204286: funcgraph_exit:         1.416 us   |          }
smbd-26510 [001] 10861828.204286: funcgraph_exit:         2.570 us   |        }
smbd-26510 [001] 10861828.204287: funcgraph_entry:                   |        capable() {
smbd-26510 [001] 10861828.204287: funcgraph_entry:                   |          security_capable() {
smbd-26510 [001] 10861828.204288: funcgraph_entry:                   |            selinux_capable() {
smbd-26510 [001] 10861828.204288: funcgraph_entry:        0.463 us   |              cap_capable();
smbd-26510 [001] 10861828.204289: funcgraph_exit:         1.390 us   |            }
smbd-26510 [001] 10861828.204290: funcgraph_exit:         2.409 us   |          }
smbd-26510 [001] 10861828.204290: funcgraph_exit:         3.351 us   |        }
smbd-26510 [001] 10861828.204291: funcgraph_entry:                   |        capable() {
smbd-26510 [001] 10861828.204291: funcgraph_entry:                   |          security_capable() {
smbd-26510 [001] 10861828.204292: funcgraph_entry:                   |            selinux_capable() {
smbd-26510 [001] 10861828.204292: funcgraph_entry:        0.451 us   |              cap_capable();
smbd-26510 [001] 10861828.204293: funcgraph_exit:         1.349 us   |            }
smbd-26510 [001] 10861828.204293: funcgraph_exit:         2.232 us   |          }
smbd-26510 [001] 10861828.204294: funcgraph_exit:         3.114 us   |        }
smbd-26510 [001] 10861828.204294: funcgraph_exit:       + 10.922 us  |      }
smbd-26510 [001] 10861828.204295: funcgraph_exit:       + 12.024 us  |    }
smbd-26510 [001] 10861828.204295: funcgraph_entry:                   |    path_put() {
smbd-26510 [001] 10861828.204296: funcgraph_entry:        0.455 us   |      dput();
smbd-26510 [001] 10861828.204297: funcgraph_entry:        0.455 us   |      mntput_no_expire();
smbd-26510 [001] 10861828.204297: funcgraph_exit:         2.243 us   |    }
smbd-26510 [001] 10861828.204298: funcgraph_exit:       ! 136.819 us |  }
[...]

Now the real problem down to earth problem. I have a RHEL 6.5 64bit with SELinux enabled and enforced. I want to share the httpd logs via a read-only samba share. I have tried to give proper access to both apache and samba with:

setsebool -P allow_httpd_anon_write=1
semanage fcontext -a -t public_content_rw_t '/var/log/httpd(/.*)?'
restorecon -R /var/log/httpd

The samba share configuration for this particular share is:

[apache-logs]
    path = /var/log/httpd/
    writable = no

Nothing is logged in SELinux audit file /var/log/audit/audit.log for this chdir(2). Any help would appreciated.

file-permissions

1 Answers

  1. Some denials in SELinux (sometimes frustratingly) are not audited in the audit log.

    I would be interested to know if you also experience this issue of no data occuring if you run semanage dontaudit off.

    This will change the rules of SELinux to even audit dontaudit rules. Note it can be very chatty keeping this on.

    Also note that DAC (traditional filesystem permissions) are checked before MAC (SELinux, in this case), so if the problem is the path is not permitted, or a directory in the path tree does not permit it because of file ownerships or modes, then you will not get a logged report about it.

    Additionally, when searching the audit log, use ausearch. A rule that should encompass all results attributed to SELinux would be:

    ausearch -m avc -m user_avc -m selinux_err

    The avc reports standard SELinux permission issues, such as when the policy does not allow it, user_avc reports userspace AVC errors, such as dbus or systemd and selinux_err reports errors with 'superpolicy', such as where the type is normally allowed, but the role is not allowed for the type -- or the role is allowed but the user is not allowed in that role.

    Roles and users are generally glossed over in SELinux as they are often not utilized, but there is a faint possibility that an issue could crop up because of them.

    So, if you feel the problem is SELinux related - this will help identify the problem.

    For the total avoidance of doubt (and where the system is insubstantial enough to warrant it) you can run setenforce 0 to disable SELinux and retry to know for certain if the problem is SELinux related. You can setenforce 1 when you have finished. But note, 'superpolicy' problems will still fail in this case as it is caused by runtime errors of the policy attempting to set labels that do not exist.

    • 2