Niels Basjes's questions

While trying to configure an experimental Kubernetes cluster (in a few VMs on my laptop) as "high available" I found the advise to do this using the combination of keepalived and haproxy ( https://github.com/kubernetes/kubeadm/blob/master/docs/ha-considerations.md#options-for-software-load-balancing ).

Looking at the configuration settings I read

${STATE} is MASTER for one and BACKUP for all other hosts, hence the virtual IP will initially be assigned to the MASTER.

${PRIORITY} should be higher on the master than on the backups. Hence 101 and 100 respectively will suffice.

and these settings surprise me. Seemingly I have to choose which of those systems is to be the initial master and I have to "hard" configure this in the nodes themselves.

To me this "high available" setup deviates from the "pet"/"cattle" analogy I find in Kubernetes.

Other systems like for example HBase have a similar setup (one active and multiple standby leaders) and all are configured "identically" (election is done via ZooKeeper).

Is there a way that I can configure Keepalived (for use in Kubernetes) in such a way that all nodes have the same config and it still works correctly?

Because I like to have things separate (security concern) I’m currently experimenting to see if I can have completely separate networks in Kubernetes for the “hosts” network and the “external” network where I want MetalLB to manage the traffic.

As an experimentation setup I now have two separate vlans and a VMware virtual machine which simply sees two normal network interfaces.

On that VM I have installed CentOS 7, Kubernetes 1.17, Calico 3.13.1 and MetalLB 0.8.2

I expect that I do not need to give the node an IP address on the “external” interface because in the MetalLB manual I read

Layer 2 mode does not require the IPs to be bound to the network interfaces of your worker nodes. It works by responding to ARP requests on your local network directly, to give the machine’s MAC address to clients.

This is nice because that way the node is effectively unreachable from the "external" network.

For my testing I have configured metallb to manage a range of IP on both networks (just for testing)

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 10.123.123.100-10.123.123.199
      - 10.123.45.100-10.123.45.199

On the VM I give the host interface an ip address (DHCP) and that all works fine. I do not give the “external” interface an IP/routing/… yet I have made sure it is “up”.

So I see this (ens192 = hosts, ens224 = external)

[root@node1 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0c:29:ca:06:13 brd ff:ff:ff:ff:ff:ff
    inet 10.123.123.6/24 brd 10.123.123.255 scope global dynamic ens192
       valid_lft 75786sec preferred_lft 75786sec
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0c:29:ca:06:1d brd ff:ff:ff:ff:ff:ff
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:dc:25:87:ec brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

(and a lot of cali* interfaces)

[root@node1 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.123.123.1    0.0.0.0         UG    0      0        0 ens192
10.123.123.0    0.0.0.0         255.255.255.0   U     0      0        0 ens192
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.135.192 0.0.0.0         255.255.255.192 U     0      0        0 *
192.168.135.242 0.0.0.0         255.255.255.255 UH    0      0        0 calia47abf08c66
…

For testing I have a simple web app and I’ve attached two K8s services to it: One with a loadBalancerIP on the “host” network and one on the “external” network.

[root@node1 ~]# kubectl -n yauaa get all
NAME                         READY   STATUS    RESTARTS   AGE
pod/yauaa-7d7bffd9fc-tdhg9   1/1     Running   5          2d6h

NAME                TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
service/yauaa       LoadBalancer   10.104.33.179   10.123.123.100   80:31908/TCP   2d3h
service/yauaa-pub   LoadBalancer   10.105.4.161    10.123.45.100    80:30695/TCP   2d7h

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/yauaa   1/1     1            1           2d7h

NAME                               DESIRED   CURRENT   READY   AGE
replicaset.apps/yauaa-7d7bffd9fc   1         1         1       2d6h

Running locally on the k8s node itself I can do a curl to either IP and both respond immediately. So the binding provided by metallb to the services seems to work.

Now on my firewall (a linux device) I can do a curl to either IP, the “host” one responds, the “external” one does not. I do see that the correct IP-MAC mapping is present in the arp table of this firewall. Yet I do not get a response.

# arp -a | fgrep 10.123
node1.k8s.basjes.nl (10.123.123.6) at 00:0c:29:ca:06:13 [ether] on eth1.2222
? (10.123.123.100) at 00:0c:29:ca:06:13 [ether] on eth1.2222
? (10.123.45.100) at 00:0c:29:ca:06:1d [ether] on eth1.2345

I have tried setting the ens224 in promiscuous mode, I have tried adding a routing entry to the gateway for that interface but that did not help.

I did manage to get it running by actually giving this ens224 interface an IP address but that is what I prefer not to have and which (if I understand the documentation correctly) is not needed.

Apparently I’m doing something wrong.

What is the correct way to configure this?

I have trouble understanding a specific effect in DNS name resolution for subdomain names that came to light when enabling lets encrypt.

The context of my question:

I have a domain (basjes.nl) and I have set all hostnames in that domain to resolve to the same IP address.

In the UI of my DNS provider (Transip) I see something like this:

@                      A      x.x.x.x
*                      CNAME  @

Now I can do this to get the specified IP address.

dig blurp.basjes.nl

Recently I deployed the ACME-DNS tool and setup the appropriate DNS entries (i.e. delegate auth.basjes.nl to my own ACME-DNS server) to allow Lets Encrypt to verify I own the specified hostname via DNS.

For this specific hostname I want to have a 'full name' HTTPS certificate (i.e. not the wild card) so I added something like this:

_acme-challenge.blurp  CNAME  something.auth.basjes.nl.

After adding this entry I find that doing dig blurp.basjes.nl no longer works. The strange thing is that I do not get an error (so no NXDOMAIN), I either get nothing or just

;; AUTHORITY SECTION:
basjes.nl.      300 IN  SOA ns0.transip.net. hostmaster.transip.nl. 2018041107 14400 1800 2419200 300

So at this moment I have something like this:

@                      A      x.x.x.x
*                      CNAME  @
_acme-challenge.blurp  CNAME  something.auth.basjes.nl.

I have found that if I add an explicit DNS record for this hostname then suddenly it DOES work again:

@                      A      x.x.x.x
*                      CNAME  @
blurp                  CNAME  @
_acme-challenge.blurp  CNAME  something.auth.basjes.nl.

The effects I would like to understand:

• Why DOESN'T it resolve anymore when asking a normal DNS server ?

  The '*' record is still present!

  dig blurp.basjes.nl @1.1.1.1

• Why DOES it resolve when asking directly the SOA DNS server of Transip?

  dig blurp.basjes.nl @ns0.transip.net

• Why DOES it resolve when adding the explicit record?

  This is what I currently use as a workaround to make it work for me.

P.S. I have contacted Transip support before posting this question. They didn't know what went wrong here and/or why this happens.

I have a few CentOS 6/7 systems (non-production, experimental purposes) that have been configured to use DHCP for their IP address. Last week there was a big network disruption and I found that those systems had lost their IP address and the DHCP client had terminated. I guess after too many/too long retries.

What is the proper way to make it try to recover forever? Is there a dhcp client setting that can do this? Or should I add a cron entry that does something like 'ifup eth0' every hour? Or is there a much better way to do this?

I know CentOS 6 and CentOS 7 do these things differently and I'm looking for answers for both these cases.

Update:

For now I have created this script (which I put in /etc/cron.hourly/ ) that seems to work in the specific situation of CentOS 6. This is probably not the best solution but it "Works on my machine".

#!/bin/bash

IF=eth0

ifconfig ${IF} | fgrep 'inet addr' > /dev/null

if [ $? -ne 0 ];
then
    echo "Network is dead, trying restart"
    ifup ${IF}
fi

Still looking for the proper way to do this so I do not think this is the right answer to my question.

I have CentOS 7.0 VM with a volume mounted on a Windows 2012-R2 file server. The volume contains some big files related to an internal Wordpress site.

About a week ago the site became 'broken' and what I found is that from Linux the files look like this:

The readlink tool on Linux shows the same 'chineese' letters as shown in the screenshot.

Under Windows the same files look like this:

I talked to the sysadmin of the Windows server and he told me they turned on file deduplication on the Windows server about a week ago.

Looking at the properties on one of this files reveals that the 'size on disk' is really small, in addition the 'problem' files have the 'L' attribute which seems to support the relationship to deduplication.

The way I currently look at this is that what is apparently some kind of 'hardlink' on the Windows server is interpreted as a softlink by my CentOS system.

I have been experimenting with some mount settings but I have been unable to fix this problem (other than turning off the deduplication).

My current (Client) setup:

• CentOS 7.0
• samba-client-4.1.1-38.el7_0.x86_64
• samba-common-4.1.1-38.el7_0.x86_64
• samba-libs-4.1.1-38.el7_0.x86_64
• cifs-utils-6.2-7.el7.x86_64

The command in my /etc/fstab

\\xxxxxxxx\file\video /var/www/html/wp-content/uploads cifs nouser_xattr,nounix,iocharset=utf8,credentials=/etc/fstab.cifs.sa_video.credentials,_netdev,uid=apache,gid=apache,rw,auto 0 0

How do I get the Linux to see these files as files again?

Update 2015-06-15:

I updated the system to CentOS 7.1. Although this still runs Kernel "3.10" (to be exact: 3.10.0-229.4.2.el7.x86_64) this problem no longer occurs. So the solution is really to 'simply update' to CentOS 7.1.

I've created a kickstart script to install CentOS 5.5 (32bit) in a fully automated way.

The DNS/DHCP setup correctly gives the system the right hostname in both the forward and reverse lookups.

dig node4.mydomain.com. +short

10.10.10.64

dig -x 10.10.10.64 +short

node4.mydomain.com.

In the state the installed system is right after the installation completed is as follows:

cat /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=yes
GATEWAY=10.10.10.1
HOSTNAME=node4.mydomain.com

echo ${HOSTNAME}

node4.mydomain.com

cat /etc/hosts

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6
10.10.10.64            node4

My problem is that this automatically generated hosts file is slightly different from the way I want it (or better: the way Hadoop wants it).

The last line should look like this:

10.10.10.64            node4.mydomain.com node4

What do I modify where to fix this?

Thanks.

Edit: I've tried to find the code that actually creates this file. I've had a look inside the acaconda sources and the setup src rpm but it wasn't done from one of those. Does anyone here know where to code is located that creates the /etc/hosts file?

I added the following to the %post of my kickstart file which works for my situation:

# =========================
# Force the right hosts file for Hadoop and such
cat > /etc/hosts <<End-of-file
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1     localhost.localdomain localhost
::1           localhost6.localdomain6 localhost6
\$(dig +short \$(hostname))      \$(hostname -f) \$(hostname -s)
End-of-file

I have some systems (Mostly CentOS 5 and some XP all running under ESXi) running behind my firewall. When I'm at friends, family or on vacation I sometimes want to connect to those systems over either SSH or RDP (which I can easily tunnel over SSH).

In the past (years ago) I had the SSH port fully open, but that caused me to be the target of a dictionary attack over the internet, so that's not an option.

The best option would be to install a VPN like OpenVPN but that means that I need to install a client on the computers of the family member I'm at. I don't want to do that and in case I'm on vacation it's not even possible. Also most internet cafes only allow http and https traffic, so any port other than that is not an option for that case.

I asked around and someone was suggested to use an HTTPS (SSL) based VPN which basically works by tunneling over SSL instead of SSH.

So I did some googleing and only found OpenVPN-ALS (aka Adito) which sounds good. I installed it, tried it and never got it to work very well under Linux (I probably did something wrong, gonna look into that anyway). I also noticed that this tool seems pretty 'dead' in terms of releases and commits to their sourceforge SVN.

My question is: Is OpenVPN-ALS (aka Adito) the way to go? Or is there a better solution for my requirements (that I can run for free for a single user (=me)) that I can run under CentOS 5 (32bit) or Windows XP (32bit)? Perhaps there is an "fully functional single user" version of a commercial product I can use?

Thanks,

Niels Basjes

My wife is a graphic designer who is starting her own business in the field of designing templates for wordpress other php based systems. I'm going to setup the servers (VMware with CentOS) so she has a good spot for development and showing the customers the progress.

At large companies you would set up a DTAP (Develop, Test, Acceptance, Production) environment with automated builds, testing, deployment, etc. I'm aiming much lower than that but I still want to have some of the benefits of a DTAP setup.

What I have in mind is a Develop (her editing corner), Acceptance (publicly available behind password for customers) and Production (simply export to ZIP file or upload to external production site). I want version control (SVN/GIT/HG) and preferably automated deployment of a specific tag she creates to the acceptance environment so the customer can see the progress.

I'm lazy so I'm looking for a ready-to-run tool or toolchain that I can download from somewhere that does (most of) this at a minimal effort from me? Or is this something that no one has created before and I have to code this from scratch using existing tools?

Any suggestions are welcome.

I have a system that is a few years old that is running Fedora 7 (I know, it's old too). Now I have PS3 Media Server running (and some other things, like Vmware and Plone) and I noticed that I was getting unexpected interruptions in the streamed movies.

Normally this system reads and writes at speeds in the 25-30 MiB/sec. I found out (see below) that at some moments everything stops for 15-30 seconds. I looked over the other questions and answers here but I was unable to find any hints what could be wrong (software wise). What I did here was open a movie and seek to a later point. My PS3 uses the media server to get thumbnails for every minute to allow me to seek to the desired scene.

What I see below is that this simple operation grinds the entire system to a halt. The whole system halts waiting for IO. Also the number of context switches drops dramatically to about 650. I checked to see if the system was swapping or otherwise reporting any errors or warnings in the logfiles. I have not been able to spot anything that seems related.

I used iotop to get more info and I just disccovered that all the running applications that were doing a handfull of KiB/sec of IO were suddenly 99% IOWAIT instead of the 0.01% that they were doing normally.

At this moment my best guess is failing hardware (harddisk).

My question to you guys: How do I find out what the real problem is?

System info: Intel Pentium D CPU 2.80GHz Linux storage 2.6.22.1-27.fc7 #1 SMP Tue Jul 17 17:13:26 EDT 2007 i686 i686 i386 GNU/Linux

Sample of the output from dstat -f on my system.

-------cpu0-usage--------------cpu1-usage------ --dsk/sda-----dsk/sdb-----dsk/sdc-----dsk/sdd-- ---paging-- ---system-->
usr sys idl wai hiq siq:usr sys idl wai hiq siq| read  writ: read  writ: read  writ: read  writ|  in   out | int   csw >
  3  12  84   1   0   0:  2  14  83   1   0   0|4096B   68k:  32k    0 :   0     0 :   0     0 |   0     0 | 125  4903 >
  0   6  94   0   0   0:  1   8  91   0   0   0|   0  8192B:  16k    0 :   0     0 :   0     0 |   0     0 | 109  3734 >
  0   2  96   2   0   0:  0   1  96   0   1   2|4096B   28k:1040k    0 :   0     0 :   0     0 |   0     0 | 998  3969 >
  0   2  98   0   0   0:  1   0  97   2   0   0|   0     0 :  32k    0 :   0     0 :   0     0 |   0     0 | 115  3805 >
  1   1  98   0   0   0:  0   4  96   0   0   0|   0     0 :  32k    0 :   0     0 :   0     0 |   0     0 | 121  3850 >
  1   3  96   0   0   0:  2   4  80   0   2  12|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |8110  5076 >
  1   3  40  55   0   1:  1   2  32  57   1   7|   0   716k:2656k    0 :   0     0 :   0     0 |   0     0 |4213  4562 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0    56k:   0     0 :   0     0 :   0     0 |   0     0 |  78   833 >
  0   0  85  15   0   0:  0   0   0 100   0   0|   0    16k:   0     0 :   0     0 :   0     0 |   0     0 |  74   623 >
  0   0 100   0   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   572 >
  0   0  14  86   0   0:  0   0   0 100   0   0|   0    56k:   0     0 :   0     0 :   0     0 |   0     0 |  82   717 >
  1   0   0  99   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  69   583 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  69   576 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  73   586 >
  1   0   0  99   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  70   587 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  70   593 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  70   578 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  64   586 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  66   586 >
  1   0  11  88   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  63   575 >
  0   1  99   0   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  64   568 >
  0   0  79  21   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   571 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  64   574 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   581 >
  1   0   0  99   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   579 >
  0   0   0 100   0   0:  0   1   0  99   0   0|   0   228k:   0     0 :   0     0 :   0     0 |   0     0 | 115  1230 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  66   579 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   574 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   583 >
  1   0   0  99   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  64   581 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0    16k:   0     0 :   0     0 :   0     0 |   0     0 |  67   595 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  64   583 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   584 >
  0   0   0 100   0   0:  0   0   0 100   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 |  65   584 >
 10  17   8  66   0   0:  0   0   0 100   0   0|   0  4096B:   0     0 :   0     0 :   0     0 |   0     0 |  67   595 >
  1   3  17  76   1   2:  1   3  19  72   1   4|   0  3336k:3488k    0 :   0     0 :   0     0 |   0     0 |3386  4124 >
  2   3  86   9   0   0:  1   4  72   7   1  15|   0     0 :  11M    0 :   0     0 :   0     0 |   0     0 |8925  5683 >
  0   3  94   3   0   0:  1   3  80   4   2  10|   0     0 :6624k    0 :   0     0 :   0     0 |   0     0 |5744  4896 >
  1   4  92   3   0   0:  0   7  81   6   1   5|8192B  412k: 448k    0 :   0     0 :   0     0 |   0     0 |2927  4535 >
  1   1  97   1   0   0:  1   3  96   0   0   0|8192B   72k:  32k    0 :   0     0 :   0     0 |   0     0 | 131  3931 >
  1   2  97   0   0   0:  0   2  98   0   0   0|   0    24k:   0     0 :   0     0 :   0     0 |   0     0 | 112  3889 >
  1  16  83   0   0   0:  2  21  74   2   0   1|  12k   32k:  32k    0 :   0     0 :   0     0 |   0     0 | 130  4828 >
  1   2  97   0   0   0:  0   2  98   0   0   0|   0     0 :  48k    0 :   0     0 :   0     0 |   0     0 | 127  3838 >
  0   1  99   0   0   0:  1   2  97   0   0   0|4096B   44k:   0     0 :   0     0 :   0     0 |   0     0 | 113  3925 >
  0   3  95   2   0   0:  0   1  99   0   0   0|8192B   44k:  16k    0 :   0     0 :   0     0 |   0     0 | 136  3973 >
  1   1  96   1   0   1:  0   1  99   0   0   0|4096B   40k:  32k    0 :   0     0 :   0     0 |   0     0 | 124  3926 >
  0   2  98   0   0   0:  1   1  98   0   0   0|   0     0 :   0     0 :   0     0 :   0     0 |   0     0 | 107  3792 >

Additional info: iotop normally shows this:

Total DISK READ: 3.82 K/s | Total DISK WRITE: 26.72 K/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
13516 be/2 root        3.82 K/s   11.45 K/s  0.00 %  1.44 % vmware-vmx -C /personal/VMWare
13514 be/2 root        0.00 B/s    7.63 K/s  0.00 %  0.01 % vmware-vmx -C /personal/VMWare
13517 be/2 root        0.00 B/s    7.63 K/s  0.00 %  0.01 % vmware-vmx -C /personal/VMWare
13519 be/4 root        0.00 B/s   26.72 K/s  0.00 %  0.00 % vmware-vmx -C /personal/VMWare
    1 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % init [5]
    2 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kthreadd]
    3 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/0]
    4 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/0]
    5 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/0]
    6 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/1]
    7 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/1]
    8 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/1]
    9 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [events/0]

When blocked it shows this:

Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 3013 be/4 nobody      0.00 B/s    0.00 B/s  0.00 % 99.99 % httpd.vmware -DSSL -DSSL_ONLY 
 9828 be/4 root        0.00 B/s    0.00 B/s  0.00 % 99.99 % vmware-vmx -C /personal/VMWare/
16464 be/4 niels       0.00 B/s    0.00 B/s  0.00 % 99.99 % java -Xmx768M -Djava.encoding=UT~e/niels/pms-linux-1.10.5/pms.jar
14013 be/4 niels       0.00 B/s    0.00 B/s  0.00 % 99.99 % java -Xmx768M -Djava.encoding=UT~e/niels/pms-linux-1.10.5/pms.jar
13516 be/2 root        0.00 B/s    0.00 B/s  0.00 % 99.99 % vmware-vmx -C /personal/VMWare/
13555 be/2 root        0.00 B/s    0.00 B/s  0.00 % 99.99 % vmware-vmx -C /personal/VMWare/
    1 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % init [5]
    2 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kthreadd]
    3 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/0]
    4 be/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/0]
    5 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/0]
    6 rt/3 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/1]

So there is no actual IO to wait for and still everything is waiting for their IO.

In the past I've setup a PXE server several times by making the normal dhcp server return everything needed for network booting: both the regular ip config AND bootinfo in a single dhcp setup. That way it is easy to do.

According to the documentation i've read it (wikipedia has a nice summary) should be possible to have an unmodified dhcp server (for example if the admin at your location denies access) and a separate server that ONLY returns the boot information. This is usually called the "PXE server" or "Proxy DHCP" server. In the past I've seen (not used) this pxe server (last release was 5 years ago).

Yesterday I installed Cobbler to see what it really does. Now I would like to use it (love the ease of use) but modifying the 'main' dhcpd server to get PXE is not an option.

I've not been able to find any documentation on how I can get cobbler to run using a real PXE (==proxy dhcp) server.

Can this be done with cobbler?

Can I configure the isc dhcp server to behave as a PXE server (i.e. NO ip info, only boot info)?

Or should I go for a completely different approach (if so, what is your suggestion) ?

Thanks

New discovery I just made: The changelog for dnsmasq version 2.4.8 says:

"Proxy-DHCP, where dnsmasq just supplies the PXE information and another DHCP server does address allocation, is also allowed."

VERY INTERESTING. Especially because cobbler can already control dnsmasq.

Update:

dnsmasq 2.51 will support this config that fully does the trick I wanted.

My /etc/cobbler/dnsmasq.template now looks like this:

# Cobbler generated configuration file for dnsmasq
# $date
#

# resolve.conf .. ?
#no-poll
#enable-dbus
read-ethers
addn-hosts = /var/lib/cobbler/cobbler_hosts

# Be a proxyDHCP server
dhcp-range=10.10.0.0,proxy

# Only respond to clients that are known (i.e present in /etc/ethers)
dhcp-ignore=#known

# Set this (and domain: see below) if you want to have a domain
# automatically added to simple names in a hosts-file.
expand-hosts
domain=test.basjes.nl,10.10.15.0

# Loads <tftp-root>/pxelinux.0 from dnsmasq TFTP server.
pxe-service=x86PC, "Boot PXELinux (=Cobbler controlled)", pxelinux ,$next_server

$insert_cobbler_system_definitions

Update: 2012-04-30

A few months ago I got an email from someone stating he couldn't get the above to work. Turns out that I had done some fixing and changing of my own cobbler that I had forgotten about. So I provided the crucial fix to the mainline cobbler which has just become part of the trunk. I also created some additional supporting documentation.

I can create by own CA and generate a self signed SSL certificate this way. But what does it take to make the browser show the certificate as being an "Extended Validation SSL certificate" ?

Can I create one myself and teach my browser to show it as EV?

In DNS an SRV record really is a pretty generic form of telling a remote client where a specific service is hosted. At this moment I use it for allowing people to call me using a SIP client over the internet (works quite nicely).

One of the advantages of the SRV record is that you can specify a different port for the service and you can have multiple systems for the same (or the same system and multiple instances on a different port). So IFF it works you can have several webservers on a single system without any interferences.

So you could have this in your DNS:

_http._tcp SRV 10 50 8080 myserver.basjes.nl
_http._tcp SRV 10 50 8081 myserver.basjes.nl

There is however one tiny snag to all this goodness ... I've not been able to find any HTTP, FTP, SMTP, ... clients that support it.

So other than SIP and the few other protocols mentioned in the Wikipedia page: Will the SRV record ever become really useful?

Or have I missed something?

Is there something better out there that does work?

I have a couple of CentOS 5.3 systems and I have the same problem on all of those systems. When I send an email to 'root' (without specifying a domain) the email is rejected by the email provider. The funny thing is that the email I get back from my mail system is sent to effectively the same user and that email I do get.

This makes that both emails from my cron and asterisk are first rejected and then I get them as an attachment in the mail from my mailerdeamon.

As far as I can tell this is because of the 'From' that is classified as 'Does not exist'. I do not understand how the 'firewall.basjes.nl' hostname (= the system hooked to my cable modem) gets into the email verification.

My question is: What generic setting must I change to make this work for all programms (not just cron and asterisk) in one go?

I've placed as much relevant info below.

(I replaced all @ with #$# to avoid additional spam)

Notes:

• My ISP is Ziggo and must relay via their outgoging smtp relay.
• My mail is hosted via google apps.
• I have a local /root/.forward that contains LogFilesHobby#$#basjes.nl

Command to create this email:

# mail -s TEST root
asdfasdfa

.
Cc:

Relevant section of /var/log/maillog

Jun 22 09:54:51 hobby sendmail[3055]: n5M7sooT003055: from=root, size=35, class=0, nrcpts=1, msgid=<200906220754.n5M7sooT003055#$#hobby.basjes.nl>, relay=root#$#localhost
Jun 22 09:54:51 hobby sendmail[3056]: n5M7spZh003056: from=<root#$#hobby.basjes.nl>, size=317, class=0, nrcpts=1, msgid=<200906220754.n5M7sooT003055#$#hobby.basjes.nl>, proto=ESMTP, daemon=MTA, relay=hobby.basjes.nl [127.0.0.1]
Jun 22 09:54:51 hobby sendmail[3055]: n5M7sooT003055: to=root, ctladdr=root (0/0), delay=00:00:01, xdelay=00:00:00, mailer=relay, pri=30035, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (n5M7spZh003056 Message accepted for delivery)
Jun 22 09:54:54 hobby sendmail[3057]: n5M7spZh003056: to=LogFilesHobby#$#basjes.nl, ctladdr=<root#$#hobby.basjes.nl> (0/0), delay=00:00:03, xdelay=00:00:03, mailer=relay, pri=30530, relay=smtp.ziggo.nl [212.54.34.9], dsn=5.0.0, stat=Service unavailable
Jun 22 09:54:54 hobby sendmail[3057]: n5M7spZh003056: n5M7ssZh003057: DSN: Service unavailable
Jun 22 09:54:54 hobby sendmail[3057]: n5M7ssZh003057: to=LogFilesHobby#$#basjes.nl, delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=31554, relay=smtp.ziggo.nl, dsn=2.0.0, stat=Sent (OK id=1MIeMf-0002Z6-UP)

The reply email I received:

The original message was received at Mon, 22 Jun 2009 09:54:51 +0200
from hobby.basjes.nl [127.0.0.1]

  ----- The following addresses had permanent fatal errors -----
LogFilesHobby#$#basjes.nl
   (reason: 550-Verification failed for <root#$#firewall.basjes.nl>)
   (expanded from: <root#$#hobby.basjes.nl>)

  ----- Transcript of session follows -----
... while talking to smtp.ziggo.nl:
>>> MAIL From:<root#$#hobby.basjes.nl> SIZE=530
<<< 550-Verification failed for <root#$#firewall.basjes.nl>
<<< 550-Called:   209.85.219.47
<<< 550-Sent:     RCPT TO:<root#$#firewall.basjes.nl>
<<< 550-Response: 550-5.1.1 The email account that you tried to reach does not exist. Please try
<<< 550-550-5.1.1 double-checking the recipient's email address for typos or
<<< 550-550-5.1.1 unnecessary spaces. Learn more at
<<< 550-550 5.1.1 http://mail.google.com/support/bin/answer.py?answer=6596 27si11493246ewy.99
<<< 550 B002 Sender e-mail address invalid / Afzender e-mail adres bestaat niet
554 5.0.0 Service unavailable

Final-Recipient: RFC822; root#$#hobby.basjes.nl
X-Actual-Recipient: RFC822; LogFilesHobby#$#basjes.nl
Action: failed
Status: 5.0.0
Diagnostic-Code: SMTP; 550-Verification failed for <root#$#firewall.basjes.nl>
Last-Attempt-Date: Mon, 22 Jun 2009 09:54:54 +0200


---------- Doorgestuurd bericht ----------
From: root <root#$#hobby.basjes.nl>
To: root#$#hobby.basjes.nl
Date: Mon, 22 Jun 2009 09:54:50 +0200
Subject: TEST
asdfasdfa

[root@hobby /etc/mail]# cat sendmail.mc | grep -v '^dnl '

divert(-1)dnl
include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
VERSIONID(`setup for linux')dnl
OSTYPE(`linux')dnl
define(`SMART_HOST', `smtp.ziggo.nl')dnl
define(`confDEF_USER_ID', ``8:12'')dnl
define(`confTO_CONNECT', `1m')dnl
define(`confTRY_NULL_MX_LIST', `True')dnl
define(`confDONT_PROBE_INTERFACES', `True')dnl
define(`PROCMAIL_MAILER_PATH', `/usr/bin/procmail')dnl
define(`ALIAS_FILE', `/etc/aliases')dnl
define(`STATUS_FILE', `/var/log/mail/statistics')dnl
define(`UUCP_MAILER_MAX', `2000000')dnl
define(`confUSERDB_SPEC', `/etc/mail/userdb.db')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confAUTH_OPTIONS', `A')dnl
define(`confTO_IDENT', `0')dnl
FEATURE(`no_default_msa', `dnl')dnl
FEATURE(`smrsh', `/usr/sbin/smrsh')dnl
FEATURE(`mailertable', `hash -o /etc/mail/mailertable.db')dnl
FEATURE(`virtusertable', `hash -o /etc/mail/virtusertable.db')dnl
FEATURE(redirect)dnl
FEATURE(always_add_domain)dnl
FEATURE(use_cw_file)dnl
FEATURE(use_ct_file)dnl
FEATURE(local_procmail, `', `procmail -t -Y -a $h -d $u')dnl
FEATURE(`access_db', `hash -T<TMPF> -o /etc/mail/access.db')dnl
FEATURE(`blacklist_recipients')dnl
EXPOSED_USER(`root')dnl
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl
FEATURE(`accept_unresolvable_domains')dnl
LOCAL_DOMAIN(`localhost.localdomain')dnl
MASQUERADE_AS(`basjes.nl')dnl
FEATURE(masquerade_envelope)dnl
FEATURE(masquerade_entire_domain)dnl
MASQUERADE_DOMAIN(localhost)dnl
MASQUERADE_DOMAIN(localhost.localdomain)dnl
MAILER(smtp)dnl
MAILER(procmail)dnl

I'm thinking about building an (experimental) system that depends on realtime information from a 'standard' ISC DHCP server on a Linux box. I want to log (and process) events that are related to PXE clients trying to get address and boot information. So I'll need several additional attributes in my logging (i.e. the GUID and such). So when a system gets a new lease I would like my software to know about it immediately (lets just say < 5 seconds later).

Now I have something that places the required values in /var/log/messages so I can do something in the line of:

tail -f /var/log/messages | xargs processdhcpevents.sh

Is there a better solution? I've been looking for some kind of 'hook' in the dhcpd to handle events 'immediately' but I haven't found it yet. Does something like this exist? Or is my 'tail -f' solution the only way to do this?

Niels Basjes