Greg Bray's questions

One of our Ubuntu 14.04 production servers stopped accepting SSH connections. When we try to login we get the SSH Banner text, but then it just hangs. If we login using the management console, we can see the following error messages in /var/log/auth.log

Oct  4 17:37:20 servername sshd[10975]: error: Could not load host key: /etc/ssh/ssh_host_ed25519_key
Oct  4 17:37:21 servername sshd[10975]: Accepted publickey for username from 10.0.0.1 port 57230 ssh2: RSA xx:xx:xx:xx
Oct  4 17:37:21 servername sshd[10975]: pam_unix(sshd:session): session opened for user username by (uid=0)
Oct  4 17:37:25 servername sshd[10975]: error: openpty: No such file or directory
Oct  4 17:37:25 servername sshd[6869]: error: session_pty_req: session 0 alloc failed

Using cat /proc/mounts| grep devpts; ls -hal /dev/{pts,ptmx} I can verify it exists and has the correct permissions, and that there aren't any disk/inode issues:

devpts /dev/pts devpts rw,nosuid,noexec,relatime,mode=600,ptmxmode=000 0 0

crw-rw-rw- 1 root tty  5, 2 Oct  4 17:01 /dev/ptmx

/dev/pts:
total 0
drwxr-xr-x  2 root root       0 Aug 14 00:52 .
drwxr-xr-x 17 root root    4.3K Oct  4 17:01 ..
crw--w----  1 root tty  136, 18 Oct  4 17:41 18
crw--w----  1 root tty  136, 24 Oct  1 13:57 24
crw--w----  1 root tty  136,  3 Oct  4 17:39 3
crw--w----  1 root tty  136, 30 Oct  4 11:29 30
c---------  1 root root   5,  2 Aug 14 00:52 ptmx

df -h
    Filesystem      Size  Used Avail Use% Mounted on
    udev            252G  4.0K  252G   1% /dev
    tmpfs            51G   53M   51G   1% /run
    /dev/sdi2       220G   13G  197G   6% /
    none            4.0K     0  4.0K   0% /sys/fs/cgroup
    none            5.0M     0  5.0M   0% /run/lock
    none            252G   12K  252G   1% /run/shm
    none            100M     0  100M   0% /run/user
    /dev/sdi1        75M   512   75M   1% /boot/efi
    /dev/md1        3.5T  282G  3.0T   9% /ssd

df -hi
    Filesystem     Inodes IUsed IFree IUse% Mounted on
    udev              63M   526   63M    1% /dev
    tmpfs             63M   725   63M    1% /run
    /dev/sdi2         14M  171K   14M    2% /
    none              63M     2   63M    1% /sys/fs/cgroup
    none              63M     1   63M    1% /run/lock
    none              63M     4   63M    1% /run/shm
    none              63M     4   63M    1% /run/user
    /dev/sdi1           0     0     0     - /boot/efi
    /dev/md1         224M    46  224M    1% /ssd

I also verified the sshd_config matches another server and have restarted the ssh service. I believe the devpty system is mounted on startup, but is there any way to resolve the issue without restarting the server?

I see https://access.redhat.com/solutions/67972 has a unverified solution for this issue on RedHat, but I don't have access to a RedHat Subscription.

After updating firmware on a Dell PowerEdge R730xd we received the following error message:

UEFI0023: BIOS is unable to send the End of POST message to Management Engine because the Management Engine is not responding.

Description: The End of POST message could not be sent to the Management Engine because the Management Engine is not responding. Recommended Action: Remove input power from the system, wait for 30 seconds, and re-apply input power. Turn on the system.

System has halted due to critical issue(s)!

This server is in a remote datacenter without remote power management. Is there anyway we can fix this issue without physically pulling the power cables? We tried multiple power cycles and using racadm -r dracname -u username -p password racreset soft to power cycle the DRAC but it still has the same error when booting.

We have a script that declines KB890830 updates for our on-premise Windows Update Server, but we recently found someone approved one of the monthly updates before the script could run and the Malicious Software Removal Tool (MRT) was installed on all of our servers.

We've had issues with MRT in the past and want to remove it, but now the script has declined the update and we cannot find anything under the View installed updates section to remove it. We also tried running wusa.exe /uninstall /KB:890830 but it returned the error:

The update KB890830 is not installed on this computer.

According to the C:\Windows\debug\mrt.log, C:\Windows\System32\MRT.exe is being run daily during the "Automatic Maintenance" window defined in the Action Center section of the control panel. So it is definitely installed and being run daily.

I tried using SysInternals AutoRuns and looking at the Scheduled Tasks but was not able to find where it was being started.

How can we disable or uninstall the Malicious Software Removal Tool on our Windows Servers to prevent it from running?

Recently we noticed the following errors were occurring daily in our Event Logs for servers in our DMZ:

CertificateServicesClient-CertEnroll EventID 82
Certificate enrollment for Local system failed in authentication to all urls for enrollment
server associated with policy id: {00B9F3A7-...-50628BC5AE7E} (The RPC server is
unavailable. 0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE)). Failed to enroll for
template: Machine

CertificateServicesClient-CertEnroll EventID 13
Certificate enrollment for Local system failed to enroll for a Machine certificate with 
request ID N/A from NY-CA01.company.com\Company Internal Root CA (d0 7a ... f3 e4 70).

CertificateServicesClient-AutoEnrollment EventID 6
Automatic certificate enrollment for local system failed (0x800706ba) The RPC server is
unavailable.

I suspect it is a firewall issue, and tried to use the certutil.exe tool to verify connectivity to the certificate authorities, but when running the -TCAInfo command I received the following error message:

PS C:\windows\system32> certutil -tcainfo
================================================================
CA Name: Company Internal Root CA

Machine Name: NY-CA01.Company.com

DS Location: CN=Company Internal Root CA,CN=Enrollment Services,CN=Public Key Services,CN=Services,CN=Configuration,DC=Company,DC=com

Cert DN: CN=CompanyInternal Root CA, DC=Company, DC=com
RegConnectRegistry/RegOpenKeyEx: The network path was not found. 0x80070035 (WIN32: 53 ERROR_BAD_NETPATH)

CA Registry Validity Period: ? ???
 NotAfter: 10/11/2031 7:05 PM

Connecting to NY-CA01.Company.com\Company Internal Root CA ...
Server "Company Internal Root CA" ICertRequest2 interface is alive (47ms)
...
================================================================
NY-CA01.Company.com\Company Internal Root CA:
  Enterprise Root CA
  A certification chain processed correctly, but one of the CA certificates is not trusted by the policy provider. 0x800
b0112 (-2146762478 CERT_E_UNTRUSTEDCA)
  Online

It appears to think that the CA is Online and alive, but there is an The network path was not found. error and the CA Registry Validity Period: ? ??? is unknown.

I confirmed that https://ny-ca01.company.com/certsrv/ is accessible from the DMZ servers, so what other ports are needed for renewing certificates?

I use cygwin at work to get quick access to unix tools and as an ssh terminal. I recently installed some new Cygwin packages and now when I try and open a terminal it is very slow and gives the following error:

/usr/bin/id: cannot find name for group ID 10519

I'm not sure what the error means, but it takes much longer to open a cygwin terminal now than it use to.

We recently noticed a change in the login process on one of our Windows 2012 Servers. When we go to login it now prompts us for "Other User"

After which it then prompts for a User Name, Password, and Passcode

I have never seen the request for a Passcode when logging to a windows server before. I am able to login using my standard username and password (leaving the Passcode blank) but I am curious why it is asking for a passcode. Also it prevents my Remote Desktop Manager software from using automatic login.

Is there any way to disable the Passcode request at login? We aren't seeing this on a 2012 R2 server in the same datacenter, and nobody I have talked to knows anything about it. I don't see any recent updates that look like they could have changed the authentication process, and an online search doesn't show any results.

We have a QA team that needs to test both Citrix and Microsoft RemoteApp publishing and we are wondering if the same server could be used for both systems. We already have a Citrix environment but need to add a few more servers to the test farm. Would we be able to also setup Terminal Services and use the same server to publish RemoteApp packages or are Citrix and Terminal Services not able to be installed at the same time?

We use System Center 2012 Virtual Machine Manager to deploy VMs to our Hyper-V servers. We are starting to use the VM and Service templates to automate our system deployments, but one step we need to automate is adding the server to our Active Directory domain under a specific Organizational Unit. I am able to create a Guest OS Profile that uses a Run As account to join the server to the domain, but there is no option for specifying the OU:

How can I specify the OU to use when joining the domain?

I would like to batch reset all the iLO passwords on our HP Blade chassis without having to login to each one via the HTTPS website. How can I reset the root and administrator iLO passwords via the SSH command line?

We recently setup SCOM 2012 and deployed agents to all of our servers. We also have noticed that a number of these servers do not have hyper threading enabled, which they should. Is there any way we can use SCOM or WMI to find which servers do not have Hyper Threading turned on. My thought is that I should be able to find all systems where total cores = total threads, as this is an indication that Hyper Threading is disabled (or not available).

Example from Server 2012 Task Manager (12 cores and 12 "logical processors"/threads, should be 24 threads):

I have a Windows Server 2008 R2 Enterprise SP1 machine that I log into and use to manage virtual machines running on multiple Hyper-V servers on our domain. Sometimes, when I right-click on a local or remote VM, the Hyper-V Manager will crash and display the following error message:

If I use the Actions menu on the lower right, it works just fine, but for some reason right-clicking causes MMC to stop working. Is there any way to fix this issue? Here are the full details of the error message.

Description:
  Stopped working

Problem signature:
  Problem Event Name:   CLR20r3
  Problem Signature 01: mmc.exe
  Problem Signature 02: 6.1.7600.16385
  Problem Signature 03: 4a5bc808
  Problem Signature 04: Microsoft.Virtualization.Client
  Problem Signature 05: 6.1.0.0
  Problem Signature 06: 4ce7c9e3
  Problem Signature 07: 342
  Problem Signature 08: 1f
  Problem Signature 09: System.OverflowException
  OS Version:   6.1.7601.2.1.0.274.10
  Locale ID:    1033

Read our privacy statement online:
  http://go.microsoft.com/fwlink/?linkid=104288&clcid=0x0409

If the online privacy statement is not available, please read our privacy statement offline:
  C:\Windows\system32\en-US\erofflps.txt

Also this stack trace:

FX:{922180d7-b74e-45f6-8c74-4b560cc100a5}

Exception has been thrown by the target of an invocation.

  at Microsoft.ManagementConsole.Internal.SnapInMessagePumpProxy.OnThreadException(Object sender, ThreadExceptionEventArgs e)
   at System.Windows.Forms.Application.ThreadContext.OnThreadException(Exception t)
   at System.Windows.Forms.Control.WndProcException(Exception e)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Forms.UnsafeNativeMethods.CallWindowProc(IntPtr wndProc, IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
   at System.Windows.Forms.NativeWindow.DefWndProc(Message& m)
   at System.Windows.Forms.ListView.WndProc(Message& m)
   at Microsoft.Virtualization.Client.Controls.EnhancedListView.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

We tried installing a 3rd party software product on a new Windows Server 2008 R2 machine and found that everything works except for accessing local services through loopback addresses such as localhost or the computer name (ex: VPS-Web which resolves to localhost). We are not using IPv6 and would like to disable it until the software is compatible.

I tried using these instructions for disabling IPv6 on Windows 2008 R2 but it did not disable the protocol for localhost. Pinging localhost or VPS-Web will still return ::1: instead of 127.0.0.1. I can use ping localhost -4 to get the correct address, but IPv6 takes precedence over IPv4 so the 3rd party software only gets the IPv6 address.