Home / server / Questions / 959664
In Process
user3273814
Asked: 2019-03-24 14:29:51 +0800 CST

Slurm node daemon error: Can't open PID file

  • 772

I run systemctl start slurmd.service, and it times out:

Job for slurmd.service failed because a timeout was exceeded.

The relevant lines from running systemctl status slurmd.service:

Mar 23 17:13:42 fedora1 systemd[1]: Starting Slurm node daemon...
Mar 23 17:13:43 fedora1 systemd[1]: slurmd.service: Can't open PID file /var/run/slurm/slurmd.pid (yet?) after start: No such file or directory
Mar 23 17:15:11 fedora1 systemd[1]: slurmd.service: Start operation timed out. Terminating.
Mar 23 17:15:11 fedora1 systemd[1]: slurmd.service: Failed with result 'timeout'.
Mar 23 17:15:11 fedora1 systemd[1]: Failed to start Slurm node daemon.

The contents of the slurm.conf file:

# Put this file on all nodes of your cluster.
# See the slurm.conf man page for more information.
#
#SlurmctldHost=fedora1
#
ControlMachine=fedora1
ControlAddr=192.168.1.4
#MailProg=/bin/mail 
MpiDefault=none
#MpiParams=ports=#-# 
ProctrackType=proctrack/cgroup
ReturnToService=1
SlurmctldPidFile=/var/run/slurmctld.pid
#SlurmctldPort=6817 
SlurmdPidFile=/var/run/slurmd.pid
#SlurmdPort=6818 
SlurmdSpoolDir=/var/spool/slurmd
SlurmUser=slurm
#SlurmdUser=root 
StateSaveLocation=/var/spool/slurmctld
SwitchType=switch/none
TaskPlugin=task/affinity
# 
# 
# TIMERS 
#KillWait=30 
#MinJobAge=300 
#SlurmctldTimeout=120 
#SlurmdTimeout=300 
# 
# 
# SCHEDULING 
FastSchedule=1
SchedulerType=sched/backfill
SelectType=select/cons_res
SelectTypeParameters=CR_Core
# 
# 
# LOGGING AND ACCOUNTING 
AccountingStorageType=accounting_storage/none
ClusterName=fedora
#JobAcctGatherFrequency=30 
JobAcctGatherType=jobacct_gather/none
#SlurmctldDebug=info 
SlurmctldLogFile=/var/log/slurmctld.log
#SlurmdDebug=info 
SlurmdLogFile=/var/log/slurmd.log
# 
# 
# COMPUTE NODES 
NodeName=fedora1 NodeAddr=192.168.1.4 CPUs=1 State=UNKNOWN
NodeName=fedora2 NodeAddr=192.168.1.5 CPUs=1 State=UNKNOWN
PartitionName=debug Nodes=fedora[1-2] Default=YES MaxTime=INFINITE State=UP
slurm

1 Answers

  1. Your slurmd.service file is specifying /var/run/slurm/slurmd.pid whereas your slurm.conf file is specifying /var/run/slurmd.pid.

    In the slurm.conf file, change this line:

    SlurmdPidFile=/var/run/slurmd.pid

    to this:

    SlurmdPidFile=/var/run/slurm/slurmd.pid

    And then start the service.

    • 5