Home / server / Questions / 737546
In Process
mick
Asked: 2015-11-20 06:41:56 +0800 CST

Permission denied with Openshift PersistentStorage on GlusterFS

  • 772

I'm trying to use OpenShift with PersistentStorage on a GusterFS cluster.

I'm starting one of the default templates : mysql-persistent

I've installed a GlusterFS cluster and create a volume gv_mysql_01

I've added the glusterfs endpoints in openshift :

oc get endpoints
NAME                ENDPOINTS                                 AGE
glusterfs-cluster   10.100.134.26:24007,10.100.134.28:24007   1h

I've the create the PersistentVolume on openshift :

cat gluster-mysql_01-storage.yaml 
apiVersion: "v1"
kind: "PersistentVolume"
metadata:
  name: "mysql" 
spec:
  capacity:
    storage: "512Mi" 
  accessModes:
    - "ReadWriteOnce"
  glusterfs: 
    endpoints: "glusterfs-cluster" 
    path: "gv_mysql_01" 
    readOnly: false
  persistentVolumeReclaimPolicy: "Recycle"

oc create -f gluster-mysql_01-storage.yaml

The PersistentStorage is bind to the container :

oc get pv
NAME      LABELS    CAPACITY   ACCESSMODES   STATUS    CLAIM        REASON    AGE
mysql     <none>    512Mi      RWO           Bound     test/mysql             53m

oc get pvc
NAME      LABELS                               STATUS    VOLUME    CAPACITY   ACCESSMODES   AGE
mysql     template=mysql-persistent-template   Bound     mysql     512Mi      RWO           1h

On on host of the cluster the volume is mount :

10.100.134.26:gv_mysql_01 on /var/lib/origin/openshift.local.volumes/pods/c111c480-8ec7-11e5-8405-0a57f8bdd6b3/volumes/kubernetes.io~glusterfs/mysql type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

but in the container logs :

docker logs b8cd5bb3b0be
Running mysql_install_db ...
mkdir: cannot create directory '/var/lib/mysql/data': Permission denied
chmod: cannot access '/var/lib/mysql/data/mysql': Permission denied
mkdir: cannot create directory '/var/lib/mysql/data': Permission denied
chmod: cannot access '/var/lib/mysql/data/test': Permission denied
151119 14:30:20 [Warning] Can't create test file /var/lib/mysql/data/mysql-1-q2yxh.lower-test
151119 14:30:20 [Warning] Can't create test file /var/lib/mysql/data/mysql-1-q2yxh.lower-test
/opt/rh/mysql55/root/usr/libexec/mysqld: Can't change dir to '/var/lib/mysql/data/' (Errcode: 13)
151119 14:30:20 [ERROR] Aborting

I've try to change owner,group,rights of /var/lib/origin/openshift.local.volumes/pods/c111c480-8ec7-11e5-8405-0a57f8bdd6b3/volumes/kubernetes.io~glusterfs/mysql with the uid:gid of the mysql user in the container, but it still doesn't work.

Am I missing something ?

docker

2 Answers

  1. Try these SELinux settings

    setsebool -P virt_use_fusefs 1
setsebool -P virt_sandbox_use_fusefs 1
    • 1

  2. I was having the same error and I was able to get it working by changing the default scc restricted policy strategy. Since you are running the openshift mysql image that runs as a specific user (27), you need to change the restricted scc to allow the id to run

    [root@ose1 ceph]# oc edit scc restricted

    then edit the file (with vi) and change the fsGroup value from RunAsAny to MustRunAs

        fsGroup:
        type: MustRunAs

    You can see this:

    [root@ose1 ceph]# oc get scc restricted
NAME         PRIV      CAPS      HOSTDIR   SELINUX     RUNASUSER        FSGROUP     SUPGROUP   PRIORITY
restricted   false     []        false     MustRunAs   MustRunAsRange   MustRunAs   RunAsAny   <none>

    You also need to change the default UID range in the namespace you are using:

    [root@ose1 ceph]# oc edit ns default  (sub your namespace for 'default')

openshift.io/sa.scc.uid-range: 25/10000

    I made start at 25 as an example, but this will allow the openshift mysql image to run it's user of 27

    • 1