Ping a Specific Port

Question

SvennD

Asked: 2016-07-28 00:06:09 +0800 CST2016-07-28 00:06:09 +0800 CST 2016-07-28 00:06:09 +0800 CST

Centos 7 freezing on large workload

772

I recently upgraded my server to Centos 7; I asked the users to backup there data for the upgrade. After the upgrade some user tried to untar a backup tar (~300Gb) and this caused the server to hang / freeze. (not responding to keyboard/ssh input) (reproducable) This happened as well with a other binary that was resource intensive.

I monitored memory and CPU and both are reasonable until the freeze, so the conclusion on the Centos Board was that it is I/0 limitation. This seemed correct as the workaround nice finished the untar succesfully.

My question is, how can I be certain that this does not happen again in the future ? (Can I limit the I/0 ?) This is a "offline" machine so users expect a large freedom to experiment. (but obviously we don't want to reset the machine every day.) Can I nice processes by default ? Or limit the I/0 so that the machine does not "freeze" ? The system is running of a raidcontroller that has raid 6 on 6 x Western Digital RE4

Specs for reference : (while workload was niced to 19)

iotop : (max about 60 M/s)

Total DISK READ :      51.72 M/s | Total DISK WRITE :      42.40 M/s
Actual DISK READ:     786.07 K/s | Actual DISK WRITE:     131.94 K/s

top while :

top - 09:58:27 up 1 day, 26 min,  6 users,  load average: 142.30, 144.85, 148.19
Tasks: 785 total,   8 running, 777 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  6.5 sy, 93.0 ni,  0.0 id,  0.1 wa,  0.0 hi,  0.3 si,  0.0 st
KiB Mem : 52838147+total,   707112 free, 51992025+used,  7754104 buff/cache
KiB Swap:  4194300 total,        0 free,  4194300 used.  7746868 avail Mem

free -m

              total        used        free      shared  buff/cache   available
Mem:         515997      507679         719           3        7598        7631
Swap:          4095        4095           0

cat /proc/cpuinfo

processor       : 63
vendor_id       : GenuineIntel
cpu family      : 6
model           : 47
model name      : Intel(R) Xeon(R) CPU E7- 4830  @ 2.13GHz
stepping        : 2
microcode       : 0x37
cpu MHz         : 2134.000
cache size      : 24576 KB
physical id     : 3
siblings        : 16
core id         : 25
cpu cores       : 8
apicid          : 243
initial apicid  : 243
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt aes lahf_lm ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 4266.05
clflush size    : 64
cache_alignment : 64
address sizes   : 44 bits physical, 48 bits virtual
power management:

lspci (raid controller)

02:00.0 RAID bus controller: 3ware Inc 9750 SAS2/SATA-II RAID PCIe (rev 05)

system :

cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
uname -r
3.10.0-327.el7.x86_64

 lspci
00:00.0 Host bridge: Intel Corporation 5520/5500/X58 I/O Hub to ESI Port (rev 22)
00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 22)
00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 22)
00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 22)
00:13.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub I/OxAPIC Interrupt Controller (rev 22)
00:14.0 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers (rev 22)
00:14.1 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 22)
00:14.2 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 22)
00:14.3 PIC: Intel Corporation 7500/5520/5500/X58 I/O Hub Throttle Registers (rev 22)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2
02:00.0 RAID bus controller: 3ware Inc 9750 SAS2/SATA-II RAID PCIe (rev 05)
04:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
05:01.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)
fc:00.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 1 (rev 0c)
fc:00.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 2 (rev 0c)
fc:00.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 3 (rev 0c)
fc:00.6 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 4 (rev 0c)
fc:01.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Power Controller (rev 0c)
fc:02.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 0 (rev 0c)
fc:03.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 1 (rev 0c)
fc:04.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 0 (rev 0c)
fc:05.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0a (rev 0c)
fc:05.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0b (rev 0c)
fc:05.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0c (rev 0c)
fc:05.6 Host bridge: Intel Corporation Device 2b53 (rev 0c)
fc:06.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 1 (rev 0c)
fc:07.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1a (rev 0c)
fc:07.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1b (rev 0c)
fc:07.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1c (rev 0c)
fc:07.6 Host bridge: Intel Corporation Device 2b5b (rev 0c)
fc:08.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 0 (rev 0c)
fc:09.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 1 (rev 0c)
fc:0a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 2 (rev 0c)
fc:0b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 3 (rev 0c)
fc:0c.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 4 (rev 0c)
fc:0d.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 5 (rev 0c)
fc:0e.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 6 (rev 0c)
fc:0f.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 7 (rev 0c)
fc:10.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 0-1 (rev 0c)
fc:10.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 2-3 (rev 0c)
fc:10.4 Host bridge: Intel Corporation Device 2b32 (rev 0c)
fc:10.6 Host bridge: Intel Corporation Device 2b3a (rev 0c)
fc:11.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 4-5 (rev 0c)
fc:11.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 6-7 (rev 0c)
fc:11.4 Host bridge: Intel Corporation Device 2b36 (rev 0c)
fc:11.6 Host bridge: Intel Corporation Device 2b3e (rev 0c)
fc:12.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 0 (rev 0c)
fc:13.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 1 (rev 0c)
fc:14.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: REUT control/status (rev 0c)
fc:14.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: Misc. control/status (rev 0c)
fc:15.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: REUT control/status (rev 0c)
fc:15.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: Misc. control/status (rev 0c)
fc:17.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: REUT control/status (rev 0c)
fc:17.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: Misc. control/status (rev 0c)
fc:18.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: REUT control/status (rev 0c)
fc:18.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: Misc control/status (rev 0c)
fc:19.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: REUT control/status (rev 0c)
fc:19.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: Misc control/status (rev 0c)
fc:1a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 8 (rev 0c)
fc:1b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 9 (rev 0c)
fd:00.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 1 (rev 0c)
fd:00.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 2 (rev 0c)
fd:00.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 3 (rev 0c)
fd:00.6 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 4 (rev 0c)
fd:01.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Power Controller (rev 0c)
fd:02.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 0 (rev 0c)
fd:03.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 1 (rev 0c)
fd:04.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 0 (rev 0c)
fd:05.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0a (rev 0c)
fd:05.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0b (rev 0c)
fd:05.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0c (rev 0c)
fd:05.6 Host bridge: Intel Corporation Device 2b53 (rev 0c)
fd:06.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 1 (rev 0c)
fd:07.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1a (rev 0c)
fd:07.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1b (rev 0c)
fd:07.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1c (rev 0c)
fd:07.6 Host bridge: Intel Corporation Device 2b5b (rev 0c)
fd:08.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 0 (rev 0c)
fd:09.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 1 (rev 0c)
fd:0a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 2 (rev 0c)
fd:0b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 3 (rev 0c)
fd:0c.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 4 (rev 0c)
fd:0d.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 5 (rev 0c)
fd:0e.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 6 (rev 0c)
fd:0f.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 7 (rev 0c)
fd:10.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 0-1 (rev 0c)
fd:10.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 2-3 (rev 0c)
fd:10.4 Host bridge: Intel Corporation Device 2b32 (rev 0c)
fd:10.6 Host bridge: Intel Corporation Device 2b3a (rev 0c)
fd:11.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 4-5 (rev 0c)
fd:11.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 6-7 (rev 0c)
fd:11.4 Host bridge: Intel Corporation Device 2b36 (rev 0c)
fd:11.6 Host bridge: Intel Corporation Device 2b3e (rev 0c)
fd:12.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 0 (rev 0c)
fd:13.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 1 (rev 0c)
fd:14.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: REUT control/status (rev 0c)
fd:14.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: Misc. control/status (rev 0c)
fd:15.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: REUT control/status (rev 0c)
fd:15.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: Misc. control/status (rev 0c)
fd:17.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: REUT control/status (rev 0c)
fd:17.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: Misc. control/status (rev 0c)
fd:18.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: REUT control/status (rev 0c)
fd:18.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: Misc control/status (rev 0c)
fd:19.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: REUT control/status (rev 0c)
fd:19.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: Misc control/status (rev 0c)
fd:1a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 8 (rev 0c)
fd:1b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 9 (rev 0c)
fe:00.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 1 (rev 0c)
fe:00.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 2 (rev 0c)
fe:00.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 3 (rev 0c)
fe:00.6 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 4 (rev 0c)
fe:01.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Power Controller (rev 0c)
fe:02.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 0 (rev 0c)
fe:03.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 1 (rev 0c)
fe:04.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 0 (rev 0c)
fe:05.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0a (rev 0c)
fe:05.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0b (rev 0c)
fe:05.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0c (rev 0c)
fe:05.6 Host bridge: Intel Corporation Device 2b53 (rev 0c)
fe:06.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 1 (rev 0c)
fe:07.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1a (rev 0c)
fe:07.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1b (rev 0c)
fe:07.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1c (rev 0c)
fe:07.6 Host bridge: Intel Corporation Device 2b5b (rev 0c)
fe:08.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 0 (rev 0c)
fe:09.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 1 (rev 0c)
fe:0a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 2 (rev 0c)
fe:0b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 3 (rev 0c)
fe:0c.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 4 (rev 0c)
fe:0d.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 5 (rev 0c)
fe:0e.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 6 (rev 0c)
fe:0f.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 7 (rev 0c)
fe:10.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 0-1 (rev 0c)
fe:10.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 2-3 (rev 0c)
fe:10.4 Host bridge: Intel Corporation Device 2b32 (rev 0c)
fe:10.6 Host bridge: Intel Corporation Device 2b3a (rev 0c)
fe:11.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 4-5 (rev 0c)
fe:11.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 6-7 (rev 0c)
fe:11.4 Host bridge: Intel Corporation Device 2b36 (rev 0c)
fe:11.6 Host bridge: Intel Corporation Device 2b3e (rev 0c)
fe:12.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 0 (rev 0c)
fe:13.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 1 (rev 0c)
fe:14.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: REUT control/status (rev 0c)
fe:14.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: Misc. control/status (rev 0c)
fe:15.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: REUT control/status (rev 0c)
fe:15.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: Misc. control/status (rev 0c)
fe:16.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 2: REUT control/status (rev 0c)
fe:16.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 2: Misc. control/status (rev 0c)
fe:17.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: REUT control/status (rev 0c)
fe:17.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: Misc. control/status (rev 0c)
fe:18.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: REUT control/status (rev 0c)
fe:18.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: Misc control/status (rev 0c)
fe:19.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: REUT control/status (rev 0c)
fe:19.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: Misc control/status (rev 0c)
fe:1a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 8 (rev 0c)
fe:1b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 9 (rev 0c)
ff:00.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 1 (rev 0c)
ff:00.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 2 (rev 0c)
ff:00.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 3 (rev 0c)
ff:00.6 Host bridge: Intel Corporation Xeon Processor E7 Product Family System Configuration Controller 4 (rev 0c)
ff:01.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Power Controller (rev 0c)
ff:02.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 0 (rev 0c)
ff:03.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Caching Agent 1 (rev 0c)
ff:04.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 0 (rev 0c)
ff:05.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0a (rev 0c)
ff:05.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0b (rev 0c)
ff:05.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 0c (rev 0c)
ff:05.6 Host bridge: Intel Corporation Device 2b53 (rev 0c)
ff:06.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Home Agent 1 (rev 0c)
ff:07.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1a (rev 0c)
ff:07.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1b (rev 0c)
ff:07.4 Host bridge: Intel Corporation Xeon Processor E7 Product Family Memory Controller 1c (rev 0c)
ff:07.6 Host bridge: Intel Corporation Device 2b5b (rev 0c)
ff:08.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 0 (rev 0c)
ff:09.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 1 (rev 0c)
ff:0a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 2 (rev 0c)
ff:0b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 3 (rev 0c)
ff:0c.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 4 (rev 0c)
ff:0d.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 5 (rev 0c)
ff:0e.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 6 (rev 0c)
ff:0f.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 7 (rev 0c)
ff:10.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 0-1 (rev 0c)
ff:10.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 2-3 (rev 0c)
ff:10.4 Host bridge: Intel Corporation Device 2b32 (rev 0c)
ff:10.6 Host bridge: Intel Corporation Device 2b3a (rev 0c)
ff:11.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 4-5 (rev 0c)
ff:11.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Router Port 6-7 (rev 0c)
ff:11.4 Host bridge: Intel Corporation Device 2b36 (rev 0c)
ff:11.6 Host bridge: Intel Corporation Device 2b3e (rev 0c)
ff:12.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 0 (rev 0c)
ff:13.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Test and Debug 1 (rev 0c)
ff:14.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: REUT control/status (rev 0c)
ff:14.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 0: Misc. control/status (rev 0c)
ff:15.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: REUT control/status (rev 0c)
ff:15.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 1: Misc. control/status (rev 0c)
ff:16.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 2: REUT control/status (rev 0c)
ff:16.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 2: Misc. control/status (rev 0c)
ff:17.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: REUT control/status (rev 0c)
ff:17.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family QPI Physical Port 3: Misc. control/status (rev 0c)
ff:18.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: REUT control/status (rev 0c)
ff:18.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 0: Misc control/status (rev 0c)
ff:19.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: REUT control/status (rev 0c)
ff:19.2 Host bridge: Intel Corporation Xeon Processor E7 Product Family SMI Physical Port 1: Misc control/status (rev 0c)
ff:1a.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 8 (rev 0c)
ff:1b.0 Host bridge: Intel Corporation Xeon Processor E7 Product Family Last Level Cache Coherence Engine 9 (rev 0c)

// update

this also happened with allot java jobs running, the filesystem felt "slow" before the freeze. (ls -l, cat, ...)
changed the title to reflect workload, and not only tar.

2 Answers

Voted

Andrew Meredith · Answer 1 · 2016-08-02T05:27:13+08:00

Andrew Meredith

2016-08-02T05:27:13+08:002016-08-02T05:27:13+08:00

Was any hardware changed as part of the upgrade?

Also please post "lspci" output.

This does rather sound like a hard issue as I have several EL7 machines that are pretty heavily loaded and there is no issue.

0

SvennD · Answer 2 · 2016-08-12T00:39:04+08:00

Best Answer

SvennD

2016-08-12T00:39:04+08:002016-08-12T00:39:04+08:00

In the end it seemed some sort of new kernel "feature". With help of the community this seems to work : /etc/sysctl.d/01-tune.conf (new file) # /proc/sys/vm/swappiness # 60 # /proc/sys/vm/vfs_cache_pressure # 100

# don't swap, our disks are slow
# and we have plenty of memory
vm.swappiness=5

# lesser reclaim of "VFS caches, versus pagecache and swap"
vm.vfs_cache_pressure=50

0

Centos 7 freezing on large workload

Can you pass user/pass for HTTP Basic Authentication in URL parameters?

Ping a Specific Port

Check if port is open or closed on a Linux server?

How to automate SSH login with password?

How do I tell Git for Windows where to find my private RSA key?

What's the default superuser username/password for postgres after a new install?

What port does SFTP use?

Command line to list users in a Windows Active Directory group?

What is a Pem file and how does it differ from other OpenSSL Generated Key File Formats?

How to determine if a bash variable is empty?