Arseni Mourzenko's questions

I don't understand how to solve chicken or the egg problem when automating the installation of the servers.

I have a series of servers which can be rebuilt through PXE. When a machine is being rebuilt, it loads all the settings it needs—including its private certificate it will use to authenticate itself when using different services later on—from an Apache server. This Apache server identifies the clients by their IP addresses in order to either serve them the configuration or the certificate intended for a given server, or refuse to serve it.

However, the IP address of a client could be spoofed. Same for MAC address, if at some point I add this sort of verification too.

In order to grab its configuration and its private certificate safely, the machine which boots through PXE should therefore already have a certificate that it could use when communicating with Apache server. This, however, doesn't look possible, as a machine which boots from PXE is either fresh new, or will format its disk anyway during the installation.

Am I missing something? How can I identify a fresh new machine without the risk of spoofing?

Should I use an always connected USB key which contains the private key? Or are there other options?

I'm (still) trying to properly configure VLANs on a level 3 switch (Netgear GS516TP).

Basic scenario: there are three VLANs: VLAN 10, 11, and 12, with respectively three ports, and three machines.

• 10.0.10.5, connected to port g10 and belonging to VLAN 10.
• 10.0.11.5, connected to port g11 and belonging to VLAN 11.
• 10.0.12.5, connected to port g12 and belonging to VLAN 12.

VLAN 10 has untagged ports g10, g11, and g12.

VLAN 11 has untagged ports g10 and g11. Similarly, VLAN 12 has untagged ports g10 and g12.

The goal is to be able for machines belonging to VLAN 11 and VLAN 12 to communicate with the machines in VLAN 10. However, a machine from VLAN 11 should know nothing about machines in VLAN 12 (and the other way around).

While all three machines are using the netmask 255.255.0.0, the routing configuration is set like this:

Now, the problem. When I send a TCP or UDP packet from 10.0.10.5 to 10.0.11.5 (for instance by doing nc -n 10.0.11.5 100), I can see this packet in Wireshark running on the machine which belongs to VLAN 12. It doesn't work the other way around, though, i.e. a packet sent from 10.0.11.5 is not visible in VLAN 12.

What should I do in order for the packets targeting machines from VLAN 11 to never reach the ports belonging to VLAN 12?

There are plenty of videos and descriptions about how to use an RJ45 crimping tool. I've seen a few, and I have even tested a crimping tool, but I still don't understand how it does what it does.

What exactly happens to the eight wires inside the crimping tool, and why isn't it necessary to remove the insulation from the eight wires before putting the cable into the crimping tool?

I have a CNAME RR which points to the same domain as defined in the $ORIGIN:

$TTL 604800
$ORIGIN example.com.
...
    IN A     10.0.0.5
www IN CNAME example.com.

Is it mandatory to specify the FQDN in the canonical name here, or can I use instead the following forms?

www IN CNAME @

or:

www IN CNAME

I checked Pro DNS and BIND 10 by Ron Aitchison, as well as the CNAME record article on Wikipedia, and none explain if this is allowed or not (and both sources put a FQDN in the examples).

I'm inspecting a problem with a MongoDB replica set reelecting a primary multiple times per minute. It seems that there is something wrong with the network configuration, but I can't nail it down yet.

When running ifconfig on the virtual machine of one of the nodes, I see a high number of TX collisions on eth0:

RX packets 850479  bytes 253445128 (241.7 MiB)
RX errors 0  dropped 12070  overruns 0  frame 0
TX packets 512851  bytes 210486965 (200.7 MiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 2648104

I don't understand what it means.

When I search what exactly are TX collisions, all I find is that those are related to the actual, physical network cables. However, the hardware server hosting this virtual machine reports zero collisions.

So what exactly is a TX collision in a context of a virtual machine (a Debian 9 running on QEMU/KVM, hosted on Ubuntu 18, if it matters)?

I am using a Nginx server as a reverse proxy. It serves both HTTP and HTTPS traffic and dispatches the calls to multiple HTTP-only websites.

Recently, I wanted to be able to encrypt the traffic between Nginx and the underlying server. Nginx still decrypts the HTTPS traffic it receives from the Internet just like before, but then should use HTTPS with a different certificate to access the underlying server.

I started to change the configuration, following the official documentation. However, proxy_ssl on can be set only within a stream level, and not http. This is confirmed by nginx -t which shows for the line containing proxy_ssl on; the following error:

"proxy_ssl" directive is not allowed here [...]

My difficulty is that I'm using Nginx to log traffic, add headers, etc., so I have to use http. If my understanding is correct, stream level is needed when Nginx is used to transmit HTTPS traffic directly, without the ability to decrypt it (i.e. SSL Pass-thru), which is not my case.

Can I make Nginx connect through HTTPS to the underlying server while still using http level and all its benefits such as logging and changing headers?

Current configuration:

http {
    proxy_set_header ... # Logging and custom headers.

    ...

    upstream demo-failover {
        server demo1.example.com:443 weight=1000 max_fails=0;
        server demo2.example.com:443 backup;
    }

    server {
        listen 80 proxy_protocol;
        listen 443 ssl http2 proxy_protocol;
        server_name demo.example.com;

        # HTTPS configuration for public exchange (Internet <-> Nginx).
        ssl_certificate /.../fullchain.pem;
        ssl_certificate_key /.../privkey.pem;

        ssl_stapling on;
        ssl_stapling_verify on;

        # HTTPS configuration for private exchange (Nginx <-> underlying server).
        proxy_ssl on; # This is the line which is invalid.
        proxy_ssl_trusted_certificate /.../ca.pem;
        proxy_ssl_certificate /.../demo.pem;
        proxy_ssl_certificate_key /.../demo.key;
        proxy_ssl_verify on;
        proxy_ssl_verify_depth 2;
        proxy_ssl_session_reuse on;

        if ($scheme = http) {
            return 301 https://$server_name$request_uri;
        }

        location / {
            proxy_pass http://demo-failover;
            proxy_next_upstream error timeout invalid_header;
            proxy_connect_timeout 2;
            proxy_redirect off;
            proxy_http_version 1.1;
        }
    }
}

I'm trying to configure HAProxy to be used for both HTTPS traffic and OpenVPN connections through port 443. The configuration is straightforward (first-level domain replaced by example.com for the sake of anonymousity):

frontend www_ssl
    mode tcp
    bind *:443

    acl host_vpn hdr(host) -i vpn.example.com

    use_backend vpn_backend if host_vpn
    default_backend nginx_pool_ssl

backend nginx_pool_ssl
    balance first
    mode tcp
    server web1 192.168.1.2:443 send-proxy check
    server web2 192.168.1.3:443 send-proxy check

backend vpn_backend
    mode tcp
    server vpn1 192.168.1.4:443

When establishing an OpenVPN connection, it fails with the following message:

WARNING: Bad encapsulated packet length from peer (18516), which must be > 0 and <= 1547 -- please ensure that --tun-mtu or --link-mtu is equal on both peers -- this condition could also indicate a possible active attack on the TCP link -- [Attempting restart...]

What happens is that instead of redirecting the traffic to 192.168.1.4, HAProxy goes to 192.168.1.2 which attempts to treat it as an HTTPS request. The same goes when I open vpn.example.com in the browser: I just get the website served by 192.168.1.2.

If I replace the above configuration by this:

frontend www_ssl
    mode tcp
    bind *:443
    default_backend host_vpn

backend vpn_backend
    mode tcp
    server vpn1 192.168.1.4:443

then OpenVPN connection is established correctly and the browser is unable to show anything, meaning that the issue comes from HAProxy configuration.

What is happening here? Is there a error in the original configuration?

I noticed a strange behavior on one machine using Debian that I can't reproduce on another machine running Ubuntu. When listing virsh networks as an ordinary user, it shows an empty list:

~$ virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------

When running the same command with sudo, it shows the default connection:

~$ sudo virsh net-list --all
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     no            yes

The permissions on the files themselves seem to be set correctly:

~$ ls -l /etc/libvirt/qemu/networks
total 8
drwxr-xr-x 2 root root 4096 Jul  1 18:19 autostart
-rw-r--r-- 1 root root  228 Jul  1 18:19 default.xml

The user belongs to kvm and libvirtd groups.

What is happening? Why can't I list the networks as an ordinary user?

I have a DHCP server used to do unattended installations of Debian: boot from PXE, and then install the operating system with a preseed.

The DHCP server (Debian's isc-dhcp-server package) is configured to do an operation (do an HTTP POST) when handing out a new lease:

subnet 192.168.0.0 netmask 255.255.252.0 {
    [...]
    filename "pxelinux.0";

    on commit {
        set client_ip = binary-to-ascii(10, 8, ".", leased-address);
        execute("curl", "-X", "POST", [...])
    }
}

[...]

host vmhost2 {
    hardware ethernet 00:19:66:60:c3:61;
    fixed-address 192.168.1.13;
}

I noticed that during the unattended installation, the HTTP POST is done twice: the first time a few seconds after the machine starts (this is expected), and approximately thirty seconds later when Debian installer is configuring the network.

I wasn't expecting the second request. In fact, this is the corresponding preseed configuration:

d-i netcfg/get_hostname string vmhost2
d-i netcfg/get_domain string pelicandd.com

d-i netcfg/choose_interface select auto
d-i netcfg/disable_autoconfig boolean true
d-i netcfg/disable_dhcp boolean true
d-i netcfg/dhcp_failed note
d-i netcfg/dhcp_options select Configure network manually

d-i netcfg/get_ipaddress string 192.168.1.13
d-i netcfg/get_netmask string 255.255.252.0
d-i netcfg/get_gateway string 192.168.1.1
d-i netcfg/get_nameservers string 192.168.1.3 192.168.1.4 8.8.8.8 8.8.4.4
d-i netcfg/confirm_static boolean true

I thought that the netcfg/disable_dhcp option indicates that the installer should not need to contact the DHCP server, but still, it does.

Questions:

• Why is the installer contacting the DHCP server the second time, despite the preseed options?

• Is there a way to stop it doing it, either through a preseed option, or by modifying the configuration of ISC DHCP server to ignore the second lease?

I tried to use on the same motherboard at the same time:

• Three LR-DIMM memory sticks,

• Four RDIMM memory sticks.

The server refused to start; when I use either three LR-DIMM sticks or four RDIMM sticks, the server starts successfully. However, I haven't tested different configurations and different slots to be perfectly sure that it's the mix of two types which prevents the server from starting.

According to the manual of this motherboard (Asus Z9PR-12):

You may install 1GB, 2GB, 4GB, 8GB, 16GB and 32GB* RDIMMs or 1GB, 2GB, 4GB and 8GB* UDIMMs or 8GB, 16GB and 32GB* LR-DIMMs into the DIMM sockets using the memory configurations in this section

I'm not sure whether the usage of “or” in the manual shows that one cannot and shouldn't mix memory of different types, such as, in my case, LR-DIMM with RDIMM.

Questions:

• In general, is it common for server motherboards to support only one type at the same time? Could it be possible that in my case, the server wasn't starting for some other reason, such as wrong slots being used?

• Technically speaking, what is preventing the motherboard from using memory of different types at the same time? If two CPUs are used, can I put LR-DIMM memory to the slots associated with CPU 1, and RDIMM memory to the slots associated with CPU 2?

• Was I at risk of damaging either the motherboard or the memory by trying to use both types at the same time?

I have a router configured to send all the incoming traffic to a single reverse proxy machine.

This machine hosts Nginx which forwards the traffic to different servers, based on the domain name. For instance, http://a.example.com/ is forwarded to server A, and http://b.example.com/ is redirected to server B.

How do I do the same thing for Remote Desktop connections? In other words, if I have servers C and D hosting Windows, how do I connect, outside the network, to server C through c.example.com:3389 and to D through d.example.com:3389?

On several large-scale websites, I've seen being redirected from http://www.example.com/ to http://www2.example.com/

One of such examples is Netflix.

My hypothesis is that this is used to:

1. Either distribute the load across data centers before even reaching the load balancers (clients from some IPs are redirected to www2. during a session, while others still use www.),

2. Or, less probably, switch users to another data center during a maintenance operation. I say "less probably" since those switches are quite frequent—I hardly doubt maintenance operations on front load balancers are done that often.

In the first case, what are the benefits of this technique?

In the second case, wouldn't a change in DNS records be enough?

As I understand it, bonding brings among other benefits the ability to increase the network speed between two machines in a LAN.

Bonding [...] means combining several network interfaces (NICs) to a single link, providing either high-availability, load-balancing, maximum throughput, or a combination of these.

^{Source: Ubuntu documentation, emphasis mine.}

I have bonding configured on two servers; both have two 1Gbps NIC adapters. When testing speed between those servers using iperf, the report indicates:

• 930 to 945 Mbits/sec when using balance-rr bonding mode.
• 520 to 530 Mbits/sec from machine A to B when using 802.3ad,
• 930 to 945 Mbits/sec from machine B to A when using 802.3ad.

An interesting thing is that when using 802.3ad, ifconfig indicates that practically all RX is on eth0 (2.5 GB vs. a few KB/MB) and all TX on eth1 on machine A, and the inverse on machine B.

When asking iperf to use multiple connections (iperf -c 192.168.1.2 -P 10), the obtained sum is very close to the results displayed when using a single connection.

Two machines are connected to a Netgear GS728TS which has LACP configured properly (I hope), with two LAGs covering two ports each. IEEE 802.3x mode is enabled.

Is iperf suited well for this sort of tests? If yes, is there something I'm missing?

I have a bunch of virtual machines (Ubuntu Server 14.04) hosted on a single real server (Ubuntu Server 14.04 with KVM). The number of virtual machines grow, and I'll soon have more than 254 of them.

To test,

1. I changed in /etc/network/interfaces of the real server the netmask from 255.255.255.0 to 255.255.254.0. The IP address of the real server is 192.168.1.30.

2. I did the similar operation with virsh net-edit default, which changed the netmask of virbr0 to 255.255.254.0.

3. I changed the netmask of the switch to 255.255.254.0 as well.

4. I changed the configuration of the test virtual machine like this:

   auto eth0
   iface eth0 inet static
   address 192.168.2.35
   netmask 255.255.254.0
   network 192.168.1.0
   broadcast 192.168.2.255
   gateway 192.168.1.1
   dns-nameservers 192.168.1.13
   dns-search example.com
   

This is the current state:

1. I can't ping 192.168.2.35 from the real server: it is just stuck at "PING 192.168.2.35 (192.168.2.35) 56(84) bytes of data." When stopped (Ctrl+C), it complains: "143 packets transmitted, 0 received, 100% packet loss, time 142134ms".

2. The virtual machine 192.168.2.35 can't ping anything (neither the switch, nor the real server, nor google.com), throwing "connect: Network is unreachable" error for the first two, or "ping: unknown host google.com" for the last one.

3. The virtual machine can ping itself.

What's wrong with my configuration?

I have a server with two network adapters. I configured bonding and it works. Here's the working configuration:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
bond-master bond0
bond-primary eth0

auto eth1
iface eth1 inet manual
bond-master bond0

auto bond0
iface bond0 inet static
bond-mode balance-rr
bond-miimon 100
bond-slaves none
address 192.168.1.2
gateway 192.168.1.1
netmask 255.255.255.0
dns-nameservers 192.168.1.13

I try to add a bridge, but then, the machine loses connectivity.

I tried:

• To specify the same IP address, gateway, netmask and nameserver for both bond0 and br0, or to specify ones for br0 only,

• To add post-up/pre-down to bond0,

• To add pre-up/post-down to br0,

• Clone the solution by Benjamin Franz (adjusting the IP addresses accordingly),

• Clone the solution by Kendall Gifford (adjusting the IP addresses accordingly).

For example, the following configuration doesn't work:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
bond-master bond0

auto eth1
iface eth1 inet manual
bond-master bond0

auto bond0
iface bond0 inet static
bond-mode balance-rr
bond-miimon 100
bond-slaves none

auto br0
iface br0 inet static
bridge_ports bond0
bridge_maxwait 0
bridge_fd 0
post-up ifup bond0
post-down ifdown bond0
address 192.168.1.2
gateway 192.168.1.1
netmask 255.255.255.0
dns-nameservers 192.168.1.13

What else can I try to make it work?

I've blown up the firewall rules on a domain controller (DC). Since there are no restore points in Windows 2008 and restoring everything from a full backup is not a solution in this case, how do I:

• Either get the rules back from a full backup, without doing a full restore,

• Or get a document explaining most firewall rules used by Windows servers (including DC, servers with DNS role, or acting as web servers, etc.), letting me to spend more time on this, but knowing precisely what's opened in the firewall. Where can I find such documentation?

I am aware of the fact that I can just install a fresh copy of a DC on a virtual machine, export the policy, and import it to the existent DC. I want to avoid that, since the rules were changed a lot since the original installation of the DC.

Note that Restore Default option results in a "Could not restore the default settings (error: 3)" error, so it is not an option.

I have a following configuration: two Windows Server 2008 non-virtual machines (a database server and an application server) are connected to the same network through two network adapters, a fast one (1 Gbps), and a slow one (100 Mbps). They have a different IP, but share the same configuration.

The application server requests data from another server either from the database or from file shares. It connects to the shares using the machine name: \\DataServer01\<FileName>. The first IP associated with \\DataServer01 in DNS server is 192.168.1.19 (used by 1 Gbps adapter). I want it to be used every time, and the slow one to be used only if the fast one fails.

Sometimes, the application server downloads the files from the share at the maximum speed, but sometimes, the transfer still uses the fast 192.168.1.22 at application server side, but the slow 192.168.1.18 at database server side, limiting the speed to ≈11 MB/s.

I don't have precise metrics, but from what I've seen, I imagine that it fails to use the default connection half of the time, randomly.

If I specify \\192.168.1.19\<FileName> instead of \\DataServer01\<FileName>, everything works well and at maximum speed.

How to diagnose what's happening? Is there a policy which forces Windows to choose random network adapters when sending files from a share? Are there settings to check in DNS server (a role of Windows Server 2008)?

I'm trying to linking the SVN hosted on Apache on a Windows Server 2008 to the Active Directory.

I understood what do I need to do to manage the groups of users on repository level:

<Location "/SampleRepository1">
  DAV svn
  SVNPath H:/Repositories/SampleDirectory1
  AuthBasicProvider ldap
  AuthzLDAPAuthoritative Off
  AuthLDAPURL "ldap://.../DC=...,DC=com?sAMAccountName?sub?(objectClass=*)" none
  AuthLDAPBindDN "CN=Subversion,OU=Subversion,DC=...,DC=com"
  AuthLDAPBindPassword "..."
  AuthType Basic
  AuthName "Use your sAMAccountName to connect. If you're unsure, write to [email protected]."
  require ldap-group CN=Subversion OpenSource Contributors,OU=Subversion,DC=...,DC=com
  require ldap-group CN=Subversion Administrators,OU=Subversion,DC=...,DC=com
</location>

<Location "/SampleRepository2">
  DAV svn
  SVNPath H:/Repositories/SampleDirectory2
  AuthBasicProvider ldap
  AuthzLDAPAuthoritative Off
  AuthLDAPURL "ldap://.../DC=...,DC=com?sAMAccountName?sub?(objectClass=*)" none
  AuthLDAPBindDN "CN=Subversion,OU=Subversion,DC=...,DC=com"
  AuthLDAPBindPassword "..."
  AuthType Basic
  AuthName "Use your sAMAccountName to connect. If you're unsure, write to [email protected]."
  require ldap-group CN=Subversion Administrators,OU=Subversion,DC=...,DC=com
</location>

What bothers me is that there is too mach duplication: if SVNPath and ldap-groups change from repository to repository, everything else remains the same.

How to avoid duplicating code while being able to authorize some repositories to different groups?

From what I read and hear about datacenters, there are not too many server rooms which use water cooling, and none of the largerst datacenters use water cooling (correct me if I'm wrong). Also, it's relatively easy to buy an ordinary PC components using water cooling, while water cooled rack servers are nearly nonexistent.

On the other hand, using water can possibly (IMO):

1. Reduce the power consumption of large datacenters, especially if it is possible to create direct cooled facilities (i.e. the facility is located near a river or the sea).

2. Reduce noise, making it less painful for humans to work in datacenters.

3. Reduce space needed for the servers:

   • On server level, I imagine that in both rack and blade servers, it's easier to pass the water cooling tubes than to waste space to allow the air to pass inside,
   • On datacenter level, if it's still required to keep the alleys between servers for maintenance access to servers, the empty space under the floor and at the ceiling level used for the air can be removed.

So why water cooling systems are not widespread, neither on datacenter level, nor on rack/blade servers level?

Is it because:

• The water cooling is hardly redundant on server level?

• The direct cost of water cooled facility is too high compared to an ordinary datacenter?

• It is difficult to maintain such system (regularly cleaning the water cooling system which uses water from a river is of course much more complicated and expensive than just vacuum cleaning the fans)?

I heard that using RAID5 or RAID6 can be problematic: if RAID controller stops working, there is too much pain to recover the data when you do not have another controller of the same type.

Now, is it the same thing with RAID1? Or in RAID1, everything is pretty easy, and if something breaks, I just remove one of n hard drives, and recreate a new RAID from this disk?