Ping a Specific Port

Question

Omid Kosari

Asked: 2014-08-28 23:50:47 +0800 CST2014-08-28 23:50:47 +0800 CST 2014-08-28 23:50:47 +0800 CST

5 ESXi same hardware . 2 of them crash

772

I even could not find where to start research ! There is no pattern between crashes .

2014-08-27T13:48:01.033Z cpu0:32957)MCE: 1118: cpu0: MCA error detected via CMCI (Gbl status=0x0): Restart IP: invalid, Error IP: invalid, MCE in progress: no.
2014-08-27T13:48:01.033Z cpu0:32957)MCE: 222: cpu0: bank0: status=0x9000004000010005: (VAL=1, OVFLW=0, UC=0, EN=1, PCC=0, S=0, AR=0), ECC=no, Addr:0x0 (invalid), Misc:0x0 (invalid)
2014-08-27T13:48:01.033Z cpu0:32957)MCE: 231: cpu0: bank0: MCA recoverable error (CE): "Internal Parity Error."
2014-08-27T13:48:01.033Z cpu1:33242)World: 8773: PRDA 0x418040400000 ss 0x0 ds 0x10b es 0x10b fs 0x0 gs 0x13b
2014-08-27T13:48:01.033Z cpu1:33242)World: 8775: TR 0x4020 GDT 0x4123876a1000 (0x402f) IDT 0x418014cf3000 (0xfff)
2014-08-27T13:48:01.033Z cpu1:33242)World: 8776: CR0 0x80010031 CR3 0x11e3e4000 CR4 0x42768
2014-08-27T13:48:01.039Z cpu1:33242)Backtrace for current CPU #1, worldID=33242, ebp=0x4119c0013dd0
2014-08-27T13:48:01.039Z cpu1:33242)0x4119c0013dd0:[0x418014c8cf99]PanicvPanicInt@vmkernel#nover+0x575 stack: 0x8, 0x4119c0013e40, 0x41
2014-08-27T13:48:01.039Z cpu1:33242)0x4119c0013e30:[0x418014c8d1dd]Panic_NoSave@vmkernel#nover+0x49 stack: 0x800000001, 0x5, 0xbe200000
2014-08-27T13:48:01.039Z cpu1:33242)0x4119c0013e90:[0x418014c63e75]IDTReturnPrepare@vmkernel#nover+0x2c5 stack: 0x67374e, 0x6738ce, 0xf
2014-08-27T13:48:01.039Z cpu1:33242)0x4119c0013f20:[0x418014c6475f]Int18_MachineCheck@vmkernel#nover+0x163 stack: 0xff8f3bb8, 0x1196300
2014-08-27T13:48:01.039Z cpu1:33242)0x4119c0013f30:[0x418014cf1064]gate_entry@vmkernel#nover+0x64 stack: 0x0, 0x13b, 0x0, 0x12a7495c, 0
2014-08-27T13:48:01.040Z cpu1:33242)[45m[33;1mVMware ESXi 5.5.0 [Releasebuild-1892794 x86_64][0m
NOT_REACHED bora/vmkernel/main/idt.c:1165
2014-08-27T13:48:01.040Z cpu1:33242)cr0=0x80010031 cr2=0x12a18be0 cr3=0x11e3e4000 cr4=0x42768
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:0 world:32949 name:"memMapKernel-0" (S)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:1 world:33242 name:"vobd" (U)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:2 world:32791 name:"CmdCompl-2" (S)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:3 world:33210 name:"vmsyslogd" (U)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:4 world:32783 name:"coalesceWorld-0" (S)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:5 world:53216 name:"vmm1:Analytics_VM" (V)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:6 world:33212 name:"vmsyslogd" (U)
2014-08-27T13:48:01.040Z cpu1:33242)pcpu:7 world:32784 name:"netCoalesce2World" (S)
2014-08-27T13:48:01.040Z cpu1:33242)@BlueScreen: NOT_REACHED bora/vmkernel/main/idt.c:1165
2014-08-27T13:48:01.040Z cpu1:33242)Code start: 0x418014c00000 VMK uptime: 2:02:57:31.656
2014-08-27T13:48:01.040Z cpu1:33242)0x4119c0013dd0:[0x418014c8cf99]PanicvPanicInt@vmkernel#nover+0x575 stack: 0x8
2014-08-27T13:48:01.040Z cpu1:33242)0x4119c0013e30:[0x418014c8d1dd]Panic_NoSave@vmkernel#nover+0x49 stack: 0x800000001
2014-08-27T13:48:01.040Z cpu1:33242)0x4119c0013e90:[0x418014c63e75]IDTReturnPrepare@vmkernel#nover+0x2c5 stack: 0x67374e
2014-08-27T13:48:01.040Z cpu1:33242)0x4119c0013f20:[0x418014c6475f]Int18_MachineCheck@vmkernel#nover+0x163 stack: 0xff8f3bb8
2014-08-27T13:48:01.040Z cpu1:33242)0x4119c0013f30:[0x418014cf1064]gate_entry@vmkernel#nover+0x64 stack: 0x0
2014-08-27T13:48:01.042Z cpu1:33242)base fs=0x0 gs=0x418040400000 Kgs=0x0
2014-08-27T13:48:01.042Z cpu1:33242)MC:PCPU0 B:0 S:0x9000004000010005 M:0x0 A:0x0 0
2014-08-27T13:48:01.043Z cpu1:33242)MC:PCPU1 B:8 S:0xbe2000000005110a M:0x9080000086 A:0x118be2600 5
MC:PCPU0: 1 hardware errors seen since boot (1 corrected by hardware)
MC:PCPU1: 1 hardware errors seen since boot (0 corrected by hardware)
2014-08-27T13:48:01.043Z cpu1:33242)PCPU fam:6 model:58 step:9 type:2 name:Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
2014-08-27T13:48:01.043Z cpu1:33242)vmkernel             0x0 .data 0x0 .bss 0x0
2014-08-27T13:48:01.043Z cpu1:33242)chardevs             0x418015171000 .data 0x417fc0000000 .bss 0x417fc0000400
2014-08-27T13:48:01.043Z cpu1:33242)user                 0x418015178000 .data 0x417fc0400000 .bss 0x417fc040e180
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_mgmt          0x41801522b000 .data 0x417fc0800000 .bss 0x417fc0800140
2014-08-27T13:48:01.043Z cpu1:33242)vprobe               0x418015231000 .data 0x417fc0c00000 .bss 0x417fc0c0b7c0
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_socket        0x41801526f000 .data 0x417fc1000000 .bss 0x417fc10005c0
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_0_0_0_vmkernel_shim 0x418015274000 .data 0x417fc1400000 .bss 0x417fc14080c0
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_1_0_0_vmkernel_shim 0x418015279000 .data 0x417fc1800000 .bss 0x417fc1808840
2014-08-27T13:48:01.043Z cpu1:33242)procfs               0x41801527e000 .data 0x417fc1c00000 .bss 0x417fc1c00240
2014-08-27T13:48:01.043Z cpu1:33242)vfat                 0x418015281000 .data 0x417fc2000000 .bss 0x417fc2002600
2014-08-27T13:48:01.043Z cpu1:33242)procMisc             0x41801528b000 .data 0x417fc2400000 .bss 0x417fc2400000
2014-08-27T13:48:01.043Z cpu1:33242)vmci                 0x41801528c000 .data 0x417fc2800000 .bss 0x417fc28057c0
2014-08-27T13:48:01.043Z cpu1:33242)iodm                 0x4180152af000 .data 0x417fc2c00000 .bss 0x417fc2c00138
2014-08-27T13:48:01.043Z cpu1:33242)vmkplexer            0x4180152b3000 .data 0x417fc3000000 .bss 0x417fc3000260
2014-08-27T13:48:01.043Z cpu1:33242)vmklinux_9           0x4180152b7000 .data 0x417fc3400000 .bss 0x417fc3408e80
2014-08-27T13:48:01.043Z cpu1:33242)vmklinux_9_2_0_0     0x41801533e000 .data 0x417fc3800000 .bss 0x417fc3807e84
2014-08-27T13:48:01.043Z cpu1:33242)vmklinux_9_2_1_0     0x418015341000 .data 0x417fc3c00000 .bss 0x417fc3c07f98
2014-08-27T13:48:01.043Z cpu1:33242)vmklinux_9_2_2_0     0x418015344000 .data 0x417fc4000000 .bss 0x417fc4008838
2014-08-27T13:48:01.043Z cpu1:33242)iscsi_trans          0x418015347000 .data 0x417fc4400000 .bss 0x417fc4401800
2014-08-27T13:48:01.043Z cpu1:33242)iscsi_trans_compat_shim 0x418015352000 .data 0x417fc4800000 .bss 0x417fc480096c
2014-08-27T13:48:01.043Z cpu1:33242)iscsi_trans_incompat_shim 0x418015353000 .data 0x417fc4c00000 .bss 0x417fc4c007e4
2014-08-27T13:48:01.043Z cpu1:33242)etherswitch          0x418015354000 .data 0x417fc5000000 .bss 0x417fc5013a00
2014-08-27T13:48:01.043Z cpu1:33242)netsched             0x418015389000 .data 0x417fc5400000 .bss 0x417fc5404800
2014-08-27T13:48:01.043Z cpu1:33242)cnic_register        0x41801539b000 .data 0x417fc5800000 .bss 0x417fc58001e0
2014-08-27T13:48:01.043Z cpu1:33242)e1000                0x41801539d000 .data 0x417fc5c00000 .bss 0x417fc5c01240
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_1_0_0_iscsi_shim 0x4180153c3000 .data 0x417fc6000000 .bss 0x417fc6000970
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_0_0_0_iscsi_shim 0x4180153c4000 .data 0x417fc6400000 .bss 0x417fc6400970
2014-08-27T13:48:01.043Z cpu1:33242)random               0x4180153c5000 .data 0x417fc6800000 .bss 0x417fc6800600
2014-08-27T13:48:01.043Z cpu1:33242)usb                  0x4180153c9000 .data 0x417fc6c00000 .bss 0x417fc6c01660
2014-08-27T13:48:01.043Z cpu1:33242)ehci-hcd             0x4180153eb000 .data 0x417fc7000000 .bss 0x417fc70002a0
2014-08-27T13:48:01.043Z cpu1:33242)hid                  0x4180153f6000 .data 0x417fc7400000 .bss 0x417fc74004e0
2014-08-27T13:48:01.043Z cpu1:33242)healthchk            0x4180153fb000 .data 0x417fc7800000 .bss 0x417fc7811e00
2014-08-27T13:48:01.043Z cpu1:33242)teamcheck            0x418015411000 .data 0x417fc7c00000 .bss 0x417fc7c12240
2014-08-27T13:48:01.043Z cpu1:33242)vlanmtucheck         0x418015424000 .data 0x417fc8000000 .bss 0x417fc8012000
2014-08-27T13:48:01.043Z cpu1:33242)heartbeat            0x418015439000 .data 0x417fc8400000 .bss 0x417fc8411f00
2014-08-27T13:48:01.043Z cpu1:33242)shaper               0x41801544a000 .data 0x417fc8800000 .bss 0x417fc8813e80
2014-08-27T13:48:01.043Z cpu1:33242)lldp                 0x41801545d000 .data 0x417fc8c00000 .bss 0x417fc8c00040
2014-08-27T13:48:01.043Z cpu1:33242)cdp                  0x418015462000 .data 0x417fc9000000 .bss 0x417fc9013400
2014-08-27T13:48:01.043Z cpu1:33242)ipfix                0x41801547e000 .data 0x417fc9400000 .bss 0x417fc9412540
2014-08-27T13:48:01.043Z cpu1:33242)tcpip4               0x418015492000 .data 0x417fc9800000 .bss 0x417fc9818180
2014-08-27T13:48:01.043Z cpu1:33242)dvsdev               0x418015598000 .data 0x417fc9c00000 .bss 0x417fc9c00030
2014-08-27T13:48:01.043Z cpu1:33242)dvfilter             0x41801559b000 .data 0x417fca000000 .bss 0x417fca000b00
2014-08-27T13:48:01.043Z cpu1:33242)lacp                 0x4180155bd000 .data 0x417fca400000 .bss 0x417fca400160
2014-08-27T13:48:01.043Z cpu1:33242)hbr_filter           0x4180155c7000 .data 0x417fca800000 .bss 0x417fca800300
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_1_0_0_dvfilter_shim 0x4180155f3000 .data 0x417fcac00000 .bss 0x417fcac009b0
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_0_0_0_dvfilter_shim 0x4180155f4000 .data 0x417fcb000000 .bss 0x417fcb000930
2014-08-27T13:48:01.043Z cpu1:33242)dvfilter-generic-fastpath 0x4180155f5000 .data 0x417fcb400000 .bss 0x417fcb412380
2014-08-27T13:48:01.043Z cpu1:33242)vmkstatelogger       0x41801560f000 .data 0x417fcb800000 .bss 0x417fcb803a00
2014-08-27T13:48:01.043Z cpu1:33242)esxfw                0x418015633000 .data 0x417fcbc00000 .bss 0x417fcbc12d00
2014-08-27T13:48:01.043Z cpu1:33242)dm                   0x418015648000 .data 0x417fcc000000 .bss 0x417fcc000000
2014-08-27T13:48:01.043Z cpu1:33242)nmp                  0x41801564a000 .data 0x417fcc400000 .bss 0x417fcc403e50
2014-08-27T13:48:01.043Z cpu1:33242)vmw_satp_local       0x41801566d000 .data 0x417fcc800000 .bss 0x417fcc800028
2014-08-27T13:48:01.043Z cpu1:33242)vmw_satp_default_aa  0x41801566f000 .data 0x417fccc00000 .bss 0x417fccc00000
2014-08-27T13:48:01.043Z cpu1:33242)vmw_psp_lib          0x418015670000 .data 0x417fcd000000 .bss 0x417fcd000290
2014-08-27T13:48:01.043Z cpu1:33242)vmw_psp_fixed        0x418015672000 .data 0x417fcd400000 .bss 0x417fcd400000
2014-08-27T13:48:01.043Z cpu1:33242)vmw_psp_rr           0x418015674000 .data 0x417fcd800000 .bss 0x417fcd800068
2014-08-27T13:48:01.043Z cpu1:33242)vmw_psp_mru          0x418015677000 .data 0x417fcdc00000 .bss 0x417fcdc00000
2014-08-27T13:48:01.043Z cpu1:33242)libata_92            0x418015679000 .data 0x417fce000000 .bss 0x417fce002660
2014-08-27T13:48:01.043Z cpu1:33242)libata_9_2_0_0       0x41801569b000 .data 0x417fce400000 .bss 0x417fce401750
2014-08-27T13:48:01.043Z cpu1:33242)libata_9_2_1_0       0x41801569c000 .data 0x417fce800000 .bss 0x417fce801750
2014-08-27T13:48:01.043Z cpu1:33242)usb-storage          0x41801569d000 .data 0x417fcec00000 .bss 0x417fcec04780
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_1_0_0_nmp_shim 0x4180156a9000 .data 0x417fcf000000 .bss 0x417fcf000ca8
2014-08-27T13:48:01.043Z cpu1:33242)vmkapi_v2_0_0_0_nmp_shim 0x4180156aa000 .data 0x417fcf400000 .bss 0x417fcf400ca8
2014-08-27T13:48:01.043Z cpu1:33242)svmmirror            0x4180156ab000 .data 0x417fcf800000 .bss 0x417fcf8000c0
2014-08-27T13:48:01.043Z cpu1:33242)cbt                  0x4180156b7000 .data 0x417fcfc00000 .bss 0x417fcfc00080
2014-08-27T13:48:01.043Z cpu1:33242)migrate              0x4180156bb000 .data 0x417fd0000000 .bss 0x417fd0004d40
2014-08-27T13:48:01.043Z cpu1:33242)libfc_92             0x41801571a000 .data 0x417fd0400000 .bss 0x417fd0400b80
2014-08-27T13:48:01.043Z cpu1:33242)libfcoe_92           0x418015733000 .data 0x417fd0800000 .bss 0x417fd08001c0
2014-08-27T13:48:01.043Z cpu1:33242)libfc_9_2_0_0        0x418015739000 .data 0x417fd0c00000 .bss 0x417fd0c00868
2014-08-27T13:48:01.043Z cpu1:33242)libfcoe_9_2_0_0      0x41801573a000 .data 0x417fd1000000 .bss 0x417fd10001f4
2014-08-27T13:48:01.043Z cpu1:33242)libfc_9_2_1_0        0x41801573b000 .data 0x417fd1400000 .bss 0x417fd1400868
2014-08-27T13:48:01.043Z cpu1:33242)libfcoe_9_2_1_0      0x41801573c000 .data 0x417fd1800000 .bss 0x417fd18001f4
2014-08-27T13:48:01.043Z cpu1:33242)ahci                 0x41801573d000 .data 0x417fd1c00000 .bss 0x417fd1c00420
2014-08-27T13:48:01.043Z cpu1:33242)sunrpc               0x418015744000 .data 0x417fd2000000 .bss 0x417fd2002b80
2014-08-27T13:48:01.043Z cpu1:33242)nfsclient            0x418015753000 .data 0x417fd2400000 .bss 0x417fd2403940
2014-08-27T13:48:01.043Z cpu1:33242)vmkibft              0x41801576c000 .data 0x417fd2800000 .bss 0x417fd28037c0
2014-08-27T13:48:01.043Z cpu1:33242)lvmdriver            0x41801576f000 .data 0x417fd2c00000 .bss 0x417fd2c03380
2014-08-27T13:48:01.043Z cpu1:33242)deltadisk            0x418015783000 .data 0x417fd3000000 .bss 0x417fd3005c00
2014-08-27T13:48:01.043Z cpu1:33242)tracing              0x4180157ae000 .data 0x417fd3400000 .bss 0x417fd3405b40
2014-08-27T13:48:01.043Z cpu1:33242)rdt                  0x4180157b5000 .data 0x417fd3800000 .bss 0x417fd3804e00
2014-08-27T13:48:01.043Z cpu1:33242)vsanutil             0x4180157db000 .data 0x417fd3c00000 .bss 0x417fd3c069c0
2014-08-27T13:48:01.043Z cpu1:33242)lsomcommon           0x4180157fa000 .data 0x417fd4000000 .bss 0x417fd4001680
2014-08-27T13:48:01.043Z cpu1:33242)plog                 0x41801582e000 .data 0x417fd4400000 .bss 0x417fd44056c0
2014-08-27T13:48:01.043Z cpu1:33242)vmfs3                0x418015871000 .data 0x417fd4800000 .bss 0x417fd4803840
2014-08-27T13:48:01.043Z cpu1:33242)dvfg-igmp            0x4180158d9000 .data 0x417fd5a00000 .bss 0x417fd5a00208
2014-08-27T13:48:01.043Z cpu1:33242)cmmds_net            0x4180158df000 .data 0x417fd5e00000 .bss 0x417fd5e02f40
2014-08-27T13:48:01.043Z cpu1:33242)cmmds                0x4180158ec000 .data 0x417fd6200000 .bss 0x417fd6204d80
2014-08-27T13:48:01.043Z cpu1:33242)cmmds_resolver       0x418015921000 .data 0x417fd6600000 .bss 0x417fd6600140
2014-08-27T13:48:01.043Z cpu1:33242)vsan                 0x41801592d000 .data 0x417fd6a00000 .bss 0x417fd6a1c200
2014-08-27T13:48:01.043Z cpu1:33242)vmklink_mpi          0x418015a48000 .data 0x417fd6e00000 .bss 0x417fd6e02400
2014-08-27T13:48:01.043Z cpu1:33242)swapobj              0x418015a4d000 .data 0x417fd7200000 .bss 0x417fd7203010
2014-08-27T13:48:01.043Z cpu1:33242)osfs                 0x418015a55000 .data 0x417fd7600000 .bss 0x417fd7603380
2014-08-27T13:48:01.043Z cpu1:33242)vflash               0x418015a63000 .data 0x417fd7a00000 .bss 0x417fd7a03540
2014-08-27T13:48:01.043Z cpu1:33242)vfc                  0x418015a6e000 .data 0x417fd7e00000 .bss 0x417fd7e02ac0
Coredump to disk. 
2014-08-27T13:48:01.093Z cpu1:33242)Slot 1 of 1.
2014-08-27T13:48:01.093Z cpu1:33242)Dump: 2212: Using dump slot size 2684354560.

2 Answers

Voted

Janne Pikkarainen · Answer 1 · 2014-08-29T00:02:53+08:00

Janne Pikkarainen

2014-08-29T00:02:53+08:002014-08-29T00:02:53+08:00

Error message looks like you have a hardware problem with CPU or RAM. Have you tried running memtest or if available, a built-in hardware diagnostic tool which you can choose during the server boot?

2

Prashant Lakhera · Answer 2 · 2014-08-29T00:14:09+08:00

Prashant Lakhera

2014-08-29T00:14:09+08:002014-08-29T00:14:09+08:00

This is definitely look like a hardware error.You can try tool like mcelog which just act as a translator for any machine check exception(mce).You haven't menrioned which version of OS you are using but in RedHat/Centos you can install it via yum

               yum install mcelog

and the message are logged in /var/log/mcelog

For more info you can refer to there main website http://www.mcelog.org/

-1

5 ESXi same hardware . 2 of them crash

Can you pass user/pass for HTTP Basic Authentication in URL parameters?

Ping a Specific Port

Check if port is open or closed on a Linux server?

How to automate SSH login with password?

How do I tell Git for Windows where to find my private RSA key?

What's the default superuser username/password for postgres after a new install?

What port does SFTP use?

Command line to list users in a Windows Active Directory group?

What is a Pem file and how does it differ from other OpenSSL Generated Key File Formats?

How to determine if a bash variable is empty?