I have a cluster with 3 nodes.
First I create 3 Persistent Disks in Google Cloud.
Now I start a DaemonSet, which would create a POD on each of the node.
The POD mounts a persistent disk to store state. The POD is not stateless. In another word, Each node contains its own state stored on its own persistent disk. An individual persistent disk can not be attached by more than one POD -- otherwise it is read-only.
How to make the POD automatically pick one free Persistent Disk not used by others and mount?
You can manually mount the Persistent Disks (PDs) at a fixed path on each node. Then, your DaemonSet pods can use a hostPath volume to mount the node path of the PD into the pod.