Home / server / Questions / 412114
Accepted
quanta
Asked: 2012-07-20 19:28:25 +0800 CST

Cannot switch, ssh to specific user: su: cannot set user id: Resource temporarily unavailable?

  • 772

/var/log/secure:

su: pam_keyinit(su-l:session): Unable to change UID to 500 temporarily
su: pam_keyinit(su-l:session): Unable to change UID to 500 temporarily
su: pam_unix(su-l:session): session opened for user adtech by root(uid=0)
su: pam_unix(su-l:session): session closed for user adtech

I guess this is caused by per-user limit, but there is no different when comparing with another user.

Here're ulimit -n for adtech:

[adtech@hmaster87 root]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 192025
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

and this one for quanta:

[quanta@hmaster87 ~]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 192025
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

the number of processes running by adtech:

[root@hmaster87 ~]# ps -U adtech | wc -l
25

Any other thing to check?

UPDATE Sat Jul 21 09:21:26 ICT 2012:

# getent passwd adtech
adtech:x:500:502::/home/adtech:/bin/bash

As I said in the below comment, my co-worker had found out the process which maybe the culprit:

adtech 12901 1 0 08:58 ? 00:00:00 /home/adtech/nexus/bin/../bin/jsw/linux-x86-64/wrapper /home/adtech/nexus/bin/../bin/jsw/conf/wrapper.conf wrapper.syslog.ident=nexus wrapper.pidfile=/home/adtech/nexus/bin/../bin/jsw/linux-x86-64/nexus.pid wrapper.daemonize=TRUE

adtech 12903 12901 1 08:58 ? 00:00:24 java -Dsun.net.inetaddr.ttl=3600 -DbundleBasedir=. -Djava.io.tmpdir=./tmp -DjettyContext=nexus.properties -DjettyContextIncludeKeys=bundleBasedir -DjettyPlexusCompatibility=true -Djava.library.path=bin/jsw/lib -classpath bin/jsw/lib/wrapper-3.2.3.jar:./lib/plexus-classworlds-2.4.jar:./conf/ -Dwrapper.key=ejxHaBJASiFkAB8w -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.pid=12901 -Dwrapper.version=3.2.3 -Dwrapper.native_library=wrapper -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 org.codehaus.plexus.classworlds.launcher.Launcher ./conf/jetty.xml

By killing this process the problem go away but we still don't know which limit was exceeded.

UPDATE Sat Dec 15 00:56:13 ICT 2012:

The @favadi's answer is right, but I update here in case someone google's this thread.

The log file said that:

jvm 1    | Server daemon died!
jvm 1    | java.lang.OutOfMemoryError: unable to create new native thread
jvm 1    |      at java.lang.Thread.start0(Native Method)
jvm 1    |      at java.lang.Thread.start(Thread.java:640)
jvm 1    |      at org.tanukisoftware.wrapper.WrapperManager.privilegedStopInner(WrapperManager.java:3152)
jvm 1    |      at org.tanukisoftware.wrapper.WrapperManager.handleSocket(WrapperManager.java:3797)
jvm 1    |      at org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:4084)
jvm 1    |      at java.lang.Thread.run(Thread.java:662)
ulimit

4 Answers

  1. Best Answer

    It could be possible that the max user processes (-u) 1024 is too low.

    Remember that processes and threads are counting together. You can use ps -eLF | grep adtech | wc -l to show your current value.

    • 12

  2. Look in the jvm log for evidence it is hitting resource limits. Stack size might be the problem, depending how many java threads the killed process was running.

    Searching on your error message finds bug reports for pam_keyinit: check with your vendor's repository whether an updated version is available.

    • 2

  3. The error was reported by pam_keyinit. Since I am not familiar with this module, I searched for documentation and found this manpage. Based on the description, I wonder if perhaps the process that you killed prevented the necessary access to some files that pam_keyinit needs to modify? Hopefully this gives you some direction.

    • 0

  4. This issue can happen if the user's process-run-limit is reached. The process limit can be increased by editing: /etc/security/limits.conf file with a user having root permission. The entry to check will be similar to:

    *          hard     nproc         100

    No need to restart any services.

    • 0