Is it possible to...

• run a container with podman
• which has an IP on the same subnet that the host is on
• is accessible from every other host on the subnet
• including the host itself?

I had partial success using both a macvlan and ipvlan drivers - I am able to run a container, give it an IP address, and this IP is then accessible from every other host on the network.

However, I cannot access to or from the host running the container.

I even tried with the latest versions of podman, netavark, cni-plugins built from source, which enabled the ipvlan driver with the same outcome.

I would prefer a pointer to a guide describing how to accomplish this, though troubleshooting advice is welcome, too.

How can I check if an email address is linked to a "personal" or a "business" Google account?

In other words, I need to know if a domain is registered for business use in Google or not.

I would like this to be a standalone method (i.e., I am not looking for an addon like Domain Checker, but rather to replicate what it does on my own).

Please note that I am looking for a method that can identify even if the domain is not using Google Workspace for email (unlike this question). Specifically, I know that I can check the MX records, but this does not cover cases when the domain is not using Gmail at all, or is not even live on Google but is already registered.

My domain has a strict DMARC policy (p=reject) and a standard SPF policy with ~all catchall.

My email provider is Google Workspace.

I received an email spoofing my sender address. It came from 41.174.75.34 - a Zambian IP address which is blacklisted in a few spam databases. Gmail did flag the email as spam. Running the headers through Google's Messageheader analyzer I get the following verdict:

• SPF: softfail
• DMARC: fail

My understanding from some online documentation and this question or this question is that this email should have been completely rejected because it failed DKIM and the source IP was not in the allowed SPF range. I did not expect it to make it to my mailbox at all, even if it did get flagged as spam.

What is the correct DMARC and SPF configuration to reject spoofed emails, considering the example I witnessed? Ideally, please provide references for your answer.

Update 1

Here are the values of the relevant DNS records for reference:

DMARC record

v=DMARC1; p=reject; rua=mailto:<redacted>; ruf=mailto:<redacted>; fo=1

SPF record

v=spf1 include:_spf.google.com include:_spf.salesforce.com include:<redacted> ~all

Update 2

Here are the full mail headers with minor redactions:

Delivered-To: <user-redacted>@<domain-redacted>
Received: by 2002:a05:6214:aaa:0:0:0:0 with SMTP id ew10csp427451qvb;
        Thu, 25 Feb 2021 07:46:07 -0800 (PST)
X-Google-Smtp-Source: ABdhPJxFPuLVmXBoRU+vQaU0r38lcka6Mby1RlA787SCOD8Rb7QkX5EOXGpiMC/Tenm4pXpACNBR
X-Received: by 2002:a62:68c5:0:b029:1ee:863:8c55 with SMTP id d188-20020a6268c50000b02901ee08638c55mr3732841pfc.37.1614267967132;
        Thu, 25 Feb 2021 07:46:07 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1614267967; cv=none;
        d=google.com; s=arc-20160816;
        b=wFGGZzBJaDIOxatWEKE7LE2TBMVhqNpktIexiXikL9SJWCMTxvXV7EAooox3TKoWI2
         JpquJ/v+fsFHvHGp/AaFghBTQcvEv9MDRPtdDKDX1zulGa2vGF5P4pE8EVcPrCZ7OEZ6
         hjky81yTDLBMnowYRefHs9/UhjkrrS944a/HnLJeYN2E/UEQW7a0YjXjmFRo87g9l35g
         fBml7hP9NFmW9ZECvOU6K/cXYr/W/Fl/53X6t+kfFbschLf3/NoB0KvIi8cBA+9NiXtF
         p12/7MjR2M4gd9gqYqGzpYyfZ2FK2T/3fgJFNa07CQldfLW8qkGNBJJIFAG0boPQlEs1
         enJQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=importance:mime-version:date:subject:to:from:message-id;
        bh=1beLMfc/QIXWzdWj0auhMhJreQONuvV9O2Srp9s2OQY=;
        b=u/KHc8DIEQxX0NzfQWqE8d/wu3hGQR0UHANF4dZZjSpAKgrVeAki+WYNaXdk3s5owS
         82yew5mZcS6PJ9b3k8LnjXX1gALocBFqZcn23FbnxFoIT/9WQdYWya/dqYI42nDLByB7
         5O/c4f9tKwZjF3VDHLAVg97P6hSPhCWrIhElqIjev60huc6jm/+FjPgBq1Umpbjv1720
         hZbi95+vetbIkPZMmhXw6iOwiB/YG3RDhhUyLwCnJhr8ixf3dg3MwgCMgqHb+ky/wn5b
         x0F4D4PHZ+3MCuIFPMkMR1kvhUFPidY6RRzjL72nWGVGJ1NbQs+aVgviVTqBJKWCGxI9
         fhBQ==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=softfail (google.com: domain of transitioning <user-redacted>@<domain-redacted> does not designate 41.174.75.34 as permitted sender) smtp.mailfrom=<user-redacted>@<domain-redacted>;
       dmarc=fail (p=REJECT sp=REJECT dis=REJECT) header.from=<domain-redacted>
Return-Path: <<user-redacted>@<domain-redacted>>
Received: from [41.174.75.34] ([41.174.75.34])
        by mx.google.com with ESMTP id p11si6119467plo.125.2021.02.25.07.46.00
        for <<user-redacted>@<domain-redacted>>;
        Thu, 25 Feb 2021 07:46:07 -0800 (PST)
Received-SPF: softfail (google.com: domain of transitioning <user-redacted>@<domain-redacted> does not designate 41.174.75.34 as permitted sender) client-ip=41.174.75.34;
Authentication-Results: mx.google.com;
       spf=softfail (google.com: domain of transitioning <user-redacted>@<domain-redacted> does not designate 41.174.75.34 as permitted sender) smtp.mailfrom=<user-redacted>@<domain-redacted>;
       dmarc=fail (p=REJECT sp=REJECT dis=REJECT) header.from=<domain-redacted>
Message-ID: <20644A25380E0B57133D524F797C2064@8AIF179>
From: <<user-redacted>@<domain-redacted>>
To: <<user-redacted>@<domain-redacted>>
Subject: 情報リクエストに関する個人的な
Date: 26 Feb 2021 04:37:43 +0100
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_003F_01D70BF1.06CD2753"
X-Priority: 3
X-MSMail-Priority: Normal
Importance: Normal
X-Mailer: Microsoft Windows Live Mail 16.4.3505.912
X-MimeOLE: Produced By Microsoft MimeOLE V16.4.3505.912

Update 3

Here is the rua report - this is the only report which mentions the offending 41.174.75.34 IP address. I cut out the irrelevant <record> entries. The softfail is right there, but the question is why?

<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
  <report_metadata>
    <org_name>google.com</org_name>
    <email>[email protected]</email>
    <extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info>
    <report_id>...redacted...</report_id>
    <date_range>
      <begin>1614211200</begin>
      <end>1614297599</end>
    </date_range>
  </report_metadata>
  <policy_published>
    <domain>...redacted...</domain>
    <adkim>r</adkim>
    <aspf>r</aspf>
    <p>reject</p>
    <sp>reject</sp>
    <pct>100</pct>
  </policy_published>
... redacted multiple <record> entries ...
  <record>
    <row>
      <source_ip>41.174.75.34</source_ip>
      <count>7</count>
      <policy_evaluated>
        <disposition>reject</disposition>
        <dkim>fail</dkim>
        <spf>fail</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>...redacted...</header_from>
    </identifiers>
    <auth_results>
      <spf>
        <domain>...redacted...</domain>
        <result>softfail</result>
      </spf>
    </auth_results>
  </record>
... redacted multiple <record> entries ...
</feedback>

I was looking for a tool to display the continuous ping loss to multiple hosts, and I looked at mtr as it already has the display I was thinking about. I saw that it has this option which sounded promising:

  -F FILENAME, --filename FILENAME
          Reads the list of hostnames from the specified file.

Unfortunately, I cannot understand what mtr does with this list of hostnames. It seems to only pick up the first one from the list and report on it.

How is the "list of hostnames" used by mtr?

Official Oracle docs say that for a machine with more than 16GiB of RAM we need to allocate 16GiB of swap.

Our servers are RHEL 7 and have 256GiB of RAM.

DBAs do not want to see the system swap, so they want us to monitor the 16GiB of swap very aggressively.

I suggested that we double the RAM to 512GiB (the expense is approved), and disable the swap. However, this is against Oracle recommendation of having 16GiB of swap, even though we double the RAM.

Honestly, I don't get it how having 3% of swap makes any sense, or why if I am adding more RAM than we had swap, we have to keep the swap.

So, are there any good arguments I can use to justify running Oracle without swap?

P.S. The only reason I mention the doubling of RAM is to demonstrate the ridiculousness of the argument I am having a hard time arguing. What I am really looking for is arguments to justify disabling of the swap.

The problem

A server crashed after multiple "Out of memory" messages and I am trying to pinpoint the culprit. If it is in userland - which process. If it is in the kernel - which kernel module.

Details

I am trying to find out how to use the crash utility to investigate what triggered an OOM on a server.

As part of the installation of a fresh pair of servers, I started the initialization of a 14TB DRBD device. Around that time, while playing with the DRBD syncer rate configuration, and bringing some of the bonded network interfaces up and down, one of the servers crashed. Over a 30 seconds period it produced 39 Out of memory: Kill process #### messages. Then it crashed with:

Kernel panic - not syncing: Out of memory and no killable processes...

The system crash triggered a kdump. Now I have a nice vmcore.flat file which should be straightforward to use to investigate the issue, but I have hard time finding out where all the memory went.

The only resource I am aware of is Dedoimedo's site, which has nice instructions, and the Kernel Crash Book. These also happen to be the only resources suggested in the answers so I assume that crash is the only way to investigate.

If there is another way to do a post-mortem on the incident, I am willing to accept it. It is just that crash is the only utility that I am aware of. All I have right now is the vmcore.flat file, and all I need to know is which component ate up all that memory. I am suspecting a kernel module problem, and more specifically one of the bonding module (as it got triggered as I brought an interface down), the DRBD module (version 8.3.15 built out of tree on CentOS 6.3), or one of the 10G ethernet modules (mlnx_en built out of tree which is the interface I brought down, or the in-tree bnx2x which is the interface that remained active). All I need to know is if there is a way to verify my suspicion.

So far, I only managed to extract the following information with the crash utility:

Checked how much memory is used

$ crash /usr/lib/debug/lib/modules/2.6.32-279.5.2.el6.x86_64/vmlinux vmcore.flat
....
crash> kmem -i
              PAGES        TOTAL      PERCENTAGE
 TOTAL MEM  16482587      62.9 GB         ----
      FREE    54610     213.3 MB    0% of TOTAL MEM
      USED  16427977      62.7 GB   99% of TOTAL MEM
    SHARED     4683      18.3 MB    0% of TOTAL MEM
   BUFFERS      118       472 KB    0% of TOTAL MEM
    CACHED       82       328 KB    0% of TOTAL MEM
      SLAB    46635     182.2 MB    0% of TOTAL MEM

TOTAL SWAP        0            0         ----
 SWAP USED        0            0  100% of TOTAL SWAP
 SWAP FREE        0            0    0% of TOTAL SWAP

Obviously, it ran out of memory. All 64G is gone... but where?

Tried to see see if any processes are leaking memory

The only command that seems relevant is ps (this is the ps subcommand of crash). It shows nothing abnormal, but it also shows nothing about kernel threads.

crash> ps
   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
      0      0   0  ffffffff81a8d020  RU   0.0       0      0  [swapper]
>     0      0   1  ffff88102c456040  RU   0.0       0      0  [swapper]
>     0      0   2  ffff88082c772aa0  RU   0.0       0      0  [swapper]
>     0      0   3  ffff88102c456aa0  RU   0.0       0      0  [swapper]
      0      0   4  ffff88082c7b8ae0  RU   0.0       0      0  [swapper]
>     0      0   5  ffff88102c457500  RU   0.0       0      0  [swapper]
>     0      0   6  ffff88082c7d6aa0  RU   0.0       0      0  [swapper]
>     0      0   7  ffff88102c506080  RU   0.0       0      0  [swapper]
>     0      0   8  ffff88082c016ae0  RU   0.0       0      0  [swapper]
>     0      0   9  ffff88102c506ae0  RU   0.0       0      0  [swapper]
>     0      0  10  ffff88082c05caa0  RU   0.0       0      0  [swapper]
>     0      0  11  ffff88102c507540  RU   0.0       0      0  [swapper]
>     0      0  12  ffff88082c09cae0  RU   0.0       0      0  [swapper]
.....
   4926      1   5  ffff880828a38ae0  ??   0.0       0      0  mingetty
   4928      1   1  ffff88102a4e8040  ??   0.0       0      0  mingetty
   4930      1  19  ffff880827af4080  ??   0.0       0      0  mingetty
   4932      1   2  ffff88100f122040  ??   0.0       0      0  mingetty
   4934      1  18  ffff8810296ea080  ??   0.0       0      0  mingetty
   4936   1047   4  ffff880ff342d540  IN   0.0   11184    948  udevd
   4937   1047   5  ffff88082a240080  IN   0.0   11184    948  udevd
   5060   3772   2  ffff88082881d540  ??   0.0       0      0  sshd
   5078      1   1  ffff88100f060ae0  ??   0.0       0      0  sshd
   5079      1   1  ffff88082b882ae0  ??   0.0       0      0  bash

And if I take out the kernel threads (which all show zeroes for %MEM anyway), we can see that I was left with almost nothing running in the end:

crash> ps -u
   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
      1      0   1  ffff88082c41b500  ??   0.0   19348    348  init
   1047      1   2  ffff881029524040  IN   0.0   11188    948  udevd
   3171      1   3  ffff880826ccaaa0  IN   0.0   27636    240  auditd
   3172      1  17  ffff881029d1b500  IN   0.0   27636    240  auditd
>  3772      1   0  ffff88102b257500  RU   0.0   64072    668  sshd
   4800      1   0  ffff88100f061540  ??   0.0       0      0  dsm_om_shrsvcd
   4842      1  16  ffff88100f012ae0  ??   0.0       0      0  cmcld
   4854      1  17  ffff88082a241540  ??   0.0       0      0  cmlogd
   4855      1   3  ffff88082796cae0  ??   0.0       0      0  cmfileassistd
   4856      1  18  ffff88082809d500  ??   0.0       0      0  cmnetd
   4860      1   0  ffff88082705aae0  ??   0.0       0      0  cmresourced
   4924      1   9  ffff88102a4e8aa0  ??   0.0       0      0  mingetty
   4926      1   5  ffff880828a38ae0  ??   0.0       0      0  mingetty
   4928      1   1  ffff88102a4e8040  ??   0.0       0      0  mingetty
   4930      1  19  ffff880827af4080  ??   0.0       0      0  mingetty
   4932      1   2  ffff88100f122040  ??   0.0       0      0  mingetty
   4934      1  18  ffff8810296ea080  ??   0.0       0      0  mingetty
   4936   1047   4  ffff880ff342d540  IN   0.0   11184    948  udevd
   4937   1047   5  ffff88082a240080  IN   0.0   11184    948  udevd
   5060   3772   2  ffff88082881d540  ??   0.0       0      0  sshd
   5078      1   1  ffff88100f060ae0  ??   0.0       0      0  sshd
   5079      1   1  ffff88082b882ae0  ??   0.0       0      0  bash
   5257      1   1  ffff8808279e6aa0  ??   0.0       0      0  jnx_mlnxsnmp_da

Update:

Including some more output, as suggested by Soham. Unfortunately I cannot derive any further conclusions from it. The best I can do is suspect something in the kernel leaking memory, since the userland processes are almost all dead anyway.

The (almost full) output of log -m is here

crash> ps -G | tail -n +2 | cut -b2- | gawk '{mem += $8} END {print "total " mem/1048576 "GB"}'
total 0.00391006GB

Note that almost all userland processes are dead at this time, so the low reported usage is expected.

Out of memory messages:

As I mentioned above, there are 39 "Out of memory:" messages, and here they are:

crash> log -m | grep Out
<3>[  223.556616] Out of memory: Kill process 3189 (portreserve) score 1 or sacrifice child
<3>[  223.787234] Out of memory: Kill process 3196 (rsyslogd) score 1 or sacrifice child
<3>[  224.237119] Out of memory: Kill process 3728 (dbus-daemon) score 1 or sacrifice child
<3>[  228.771770] Out of memory: Kill process 3758 (snmpd) score 1 or sacrifice child
<3>[  229.033466] Out of memory: Kill process 3782 (xinetd) score 1 or sacrifice child
<3>[  229.257710] Out of memory: Kill process 3782 (xinetd) score 1 or sacrifice child
<3>[  229.484321] Out of memory: Kill process 3782 (xinetd) score 1 or sacrifice child
<3>[  229.711169] Out of memory: Kill process 3782 (xinetd) score 1 or sacrifice child
<3>[  229.934955] Out of memory: Kill process 3801 (cmproxyd) score 1 or sacrifice child
<3>[  230.159542] Out of memory: Kill process 3812 (ntpd) score 1 or sacrifice child
<3>[  230.382083] Out of memory: Kill process 3953 (master) score 1 or sacrifice child
<3>[  230.606613] Out of memory: Kill process 3953 (master) score 1 or sacrifice child
<3>[  230.829515] Out of memory: Kill process 3953 (master) score 1 or sacrifice child
<3>[  230.832105] Out of memory: Kill process 3961 (crond) score 1 or sacrifice child
<3>[  236.749746] Out of memory: Kill process 3974 (atd) score 1 or sacrifice child
<3>[  236.969421] Out of memory: Kill process 4272 (dsm_sa_datamgrd) score 1 or sacrifice child
<3>[  237.192102] Out of memory: Kill process 4492 (dsm_sa_datamgrd) score 1 or sacrifice child
<3>[  237.746301] Out of memory: Kill process 4552 (dsm_sa_eventmgr) score 1 or sacrifice child
<3>[  237.968308] Out of memory: Kill process 4613 (dsm_sa_snmpd) score 1 or sacrifice child
<3>[  238.190550] Out of memory: Kill process 4614 (dsm_sa_snmpd) score 1 or sacrifice child
<3>[  238.644020] Out of memory: Kill process 4643 (dsm_om_connsvcd) score 1 or sacrifice child
<3>[  238.865658] Out of memory: Kill process 4643 (dsm_om_connsvcd) score 1 or sacrifice child
<3>[  251.285450] Out of memory: Kill process 4643 (dsm_om_connsvcd) score 1 or sacrifice child
<3>[  251.506601] Out of memory: Kill process 4800 (dsm_om_shrsvcd) score 1 or sacrifice child
<3>[  251.727570] Out of memory: Kill process 4842 (cmcld) score 1 or sacrifice child
<3>[  251.947085] Out of memory: Kill process 4842 (cmcld) score 1 or sacrifice child
<3>[  252.167096] Out of memory: Kill process 4854 (cmlogd) score 1 or sacrifice child
<3>[  252.384090] Out of memory: Kill process 4855 (cmfileassistd) score 1 or sacrifice child
<3>[  252.603324] Out of memory: Kill process 4924 (mingetty) score 1 or sacrifice child
<3>[  252.820757] Out of memory: Kill process 4926 (mingetty) score 1 or sacrifice child
<3>[  253.037558] Out of memory: Kill process 4928 (mingetty) score 1 or sacrifice child
<3>[  253.254908] Out of memory: Kill process 4930 (mingetty) score 1 or sacrifice child
<3>[  253.257391] Out of memory: Kill process 4932 (mingetty) score 1 or sacrifice child
<3>[  253.259357] Out of memory: Kill process 4934 (mingetty) score 1 or sacrifice child
<3>[  253.261353] Out of memory: Kill process 5060 (sshd) score 1 or sacrifice child
<3>[  253.263365] Out of memory: Kill process 5060 (sshd) score 1 or sacrifice child
<3>[  253.264392] Out of memory: Kill process 5079 (bash) score 1 or sacrifice child
<3>[  253.266352] Out of memory: Kill process 5257 (jnx_mlnxsnmp_da) score 1 or sacrifice child
<0>[  253.529344] Kernel panic - not syncing: Out of memory and no killable processes...

sys output:

crash> sys
      KERNEL: /usr/lib/debug/lib/modules/2.6.32-279.5.2.el6.x86_64/vmlinux
    DUMPFILE: pcdata03.vmcore.flat  [PARTIAL DUMP]
        CPUS: 32
        DATE: Wed Feb  6 02:11:52 2013
      UPTIME: 00:04:12
LOAD AVERAGE: 3.03, 0.95, 0.34
       TASKS: 578
    NODENAME: ....
     RELEASE: 2.6.32-279.5.2.el6.x86_64
     VERSION: #1 SMP Fri Aug 24 01:07:11 UTC 2012
     MACHINE: x86_64  (2700 Mhz)
      MEMORY: 64 GB
       PANIC: "[  253.529344] Kernel panic - not syncing: Out of memory and no killable processes..."

kmem -z

crash> kmem -z
NODE: 0  ZONE: 0  ADDR: ffff88000000a0c0  NAME: "DMA"
  SIZE: 4095  PRESENT: 3839  MIN/LOW/HIGH: 5/6/7
  VM_STAT:
          NR_FREE_PAGES: 3936
       NR_INACTIVE_ANON: 0
         NR_ACTIVE_ANON: 0
       NR_INACTIVE_FILE: 0
         NR_ACTIVE_FILE: 0
         NR_UNEVICTABLE: 0
               NR_MLOCK: 0
          NR_ANON_PAGES: 0
         NR_FILE_MAPPED: 0
          NR_FILE_PAGES: 0
          NR_FILE_DIRTY: 0
           NR_WRITEBACK: 0
    NR_SLAB_RECLAIMABLE: 0
  NR_SLAB_UNRECLAIMABLE: 0
           NR_PAGETABLE: 0
        NR_KERNEL_STACK: 0
        NR_UNSTABLE_NFS: 0
              NR_BOUNCE: 0
        NR_VMSCAN_WRITE: 0
    NR_VMSCAN_IMMEDIATE: 0
      NR_WRITEBACK_TEMP: 0
       NR_ISOLATED_ANON: 0
       NR_ISOLATED_FILE: 0
               NR_SHMEM: 0
               NUMA_HIT: 0
              NUMA_MISS: 0
           NUMA_FOREIGN: 0
    NUMA_INTERLEAVE_HIT: 0
             NUMA_LOCAL: 0
             NUMA_OTHER: 0
NR_ANON_TRANSPARENT_HUGEPAGES: 0

NODE: 0  ZONE: 1  ADDR: ffff880000012780  NAME: "DMA32"
  SIZE: 1044480  PRESENT: 756520  MIN/LOW/HIGH: 1030/1287/1545
  VM_STAT:
          NR_FREE_PAGES: 30117
       NR_INACTIVE_ANON: 0
         NR_ACTIVE_ANON: 0
       NR_INACTIVE_FILE: 1
         NR_ACTIVE_FILE: 0
         NR_UNEVICTABLE: 0
               NR_MLOCK: 0
          NR_ANON_PAGES: 0
         NR_FILE_MAPPED: 0
          NR_FILE_PAGES: 1
          NR_FILE_DIRTY: 0
           NR_WRITEBACK: 0
    NR_SLAB_RECLAIMABLE: 4
  NR_SLAB_UNRECLAIMABLE: 4150
           NR_PAGETABLE: 0
        NR_KERNEL_STACK: 0
        NR_UNSTABLE_NFS: 0
              NR_BOUNCE: 0
        NR_VMSCAN_WRITE: 0
    NR_VMSCAN_IMMEDIATE: 0
      NR_WRITEBACK_TEMP: 0
       NR_ISOLATED_ANON: 0
       NR_ISOLATED_FILE: 0
               NR_SHMEM: 0
               NUMA_HIT: 575606
              NUMA_MISS: 3
           NUMA_FOREIGN: 0
    NUMA_INTERLEAVE_HIT: 0
             NUMA_LOCAL: 575598
             NUMA_OTHER: 11
NR_ANON_TRANSPARENT_HUGEPAGES: 0

NODE: 0  ZONE: 2  ADDR: ffff88000001ae40  NAME: "Normal"
  SIZE: 7602176  PRESENT: 7498240  MIN/LOW/HIGH: 10217/12771/15325
  VM_STAT:
          NR_FREE_PAGES: 10443
       NR_INACTIVE_ANON: 134
         NR_ACTIVE_ANON: 197
       NR_INACTIVE_FILE: -47
         NR_ACTIVE_FILE: 42
         NR_UNEVICTABLE: 0
               NR_MLOCK: 0
          NR_ANON_PAGES: 219
         NR_FILE_MAPPED: 115
          NR_FILE_PAGES: 45
          NR_FILE_DIRTY: 0
           NR_WRITEBACK: 0
    NR_SLAB_RECLAIMABLE: 908
  NR_SLAB_UNRECLAIMABLE: 18771
           NR_PAGETABLE: 91
        NR_KERNEL_STACK: 556
        NR_UNSTABLE_NFS: 0
              NR_BOUNCE: 0
        NR_VMSCAN_WRITE: 0
    NR_VMSCAN_IMMEDIATE: 0
      NR_WRITEBACK_TEMP: 0
       NR_ISOLATED_ANON: 0
       NR_ISOLATED_FILE: 0
               NR_SHMEM: 34
               NUMA_HIT: 8243991
              NUMA_MISS: 648
           NUMA_FOREIGN: 4593726
    NUMA_INTERLEAVE_HIT: 20066
             NUMA_LOCAL: 8243829
             NUMA_OTHER: 810
NR_ANON_TRANSPARENT_HUGEPAGES: 0

NODE: 0  ZONE: 3  ADDR: ffff880000023500  NAME: "Movable"
  [unpopulated]

NODE: 1  ZONE: 0  ADDR: ffff880840000040  NAME: "DMA"
  [unpopulated]

NODE: 1  ZONE: 1  ADDR: ffff880840008700  NAME: "DMA32"
  [unpopulated]

NODE: 1  ZONE: 2  ADDR: ffff880840010dc0  NAME: "Normal"
  SIZE: 8388608  PRESENT: 8273920  MIN/LOW/HIGH: 11274/14092/16911
  VM_STAT:
          NR_FREE_PAGES: 10114
       NR_INACTIVE_ANON: 417
         NR_ACTIVE_ANON: 83
       NR_INACTIVE_FILE: 47
         NR_ACTIVE_FILE: 32
         NR_UNEVICTABLE: 0
               NR_MLOCK: 0
          NR_ANON_PAGES: 436
         NR_FILE_MAPPED: 22
          NR_FILE_PAGES: 154
          NR_FILE_DIRTY: 0
           NR_WRITEBACK: 0
    NR_SLAB_RECLAIMABLE: 863
  NR_SLAB_UNRECLAIMABLE: 21939
           NR_PAGETABLE: 134
        NR_KERNEL_STACK: 27
        NR_UNSTABLE_NFS: 0
              NR_BOUNCE: 0
        NR_VMSCAN_WRITE: 3
    NR_VMSCAN_IMMEDIATE: 5
      NR_WRITEBACK_TEMP: 0
       NR_ISOLATED_ANON: 0
       NR_ISOLATED_FILE: 23
               NR_SHMEM: 20
               NUMA_HIT: 4332488
              NUMA_MISS: 4593726
           NUMA_FOREIGN: 665
    NUMA_INTERLEAVE_HIT: 20007
             NUMA_LOCAL: 4309300
             NUMA_OTHER: 4616914
NR_ANON_TRANSPARENT_HUGEPAGES: 0

NODE: 1  ZONE: 3  ADDR: ffff880840019480  NAME: "Movable"
  [unpopulated]

kmem -f

crash> kmem -f
NODE
  0
ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  0   DMA         4095    3936  ffffea0000000038      1000          0     
AREA    SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
  0       4k  ffff880000012128       2      2
  0       4k  ffff880000012138       0      0
  0       4k  ffff880000012148       0      0
  0       4k  ffff880000012158       0      0
  0       4k  ffff880000012168       0      0
  1       8k  ffff880000012180       1      2
  1       8k  ffff880000012190       0      0
  1       8k  ffff8800000121a0       0      0
  1       8k  ffff8800000121b0       0      0
  1       8k  ffff8800000121c0       0      0
  2      16k  ffff8800000121d8       1      4
  2      16k  ffff8800000121e8       0      0
  2      16k  ffff8800000121f8       0      0
  2      16k  ffff880000012208       0      0
  2      16k  ffff880000012218       0      0
  3      32k  ffff880000012230       1      8
  3      32k  ffff880000012240       0      0
  3      32k  ffff880000012250       0      0
  3      32k  ffff880000012260       0      0
  3      32k  ffff880000012270       0      0
  4      64k  ffff880000012288       1     16
  4      64k  ffff880000012298       0      0
  4      64k  ffff8800000122a8       0      0
  4      64k  ffff8800000122b8       0      0
  4      64k  ffff8800000122c8       0      0
  5     128k  ffff8800000122e0       0      0
  5     128k  ffff8800000122f0       0      0
  5     128k  ffff880000012300       0      0
  5     128k  ffff880000012310       0      0
  5     128k  ffff880000012320       0      0
  6     256k  ffff880000012338       1     64
  6     256k  ffff880000012348       0      0
  6     256k  ffff880000012358       0      0
  6     256k  ffff880000012368       0      0
  6     256k  ffff880000012378       0      0
  7     512k  ffff880000012390       0      0
  7     512k  ffff8800000123a0       0      0
  7     512k  ffff8800000123b0       0      0
  7     512k  ffff8800000123c0       0      0
  7     512k  ffff8800000123d0       0      0
  8    1024k  ffff8800000123e8       1    256
  8    1024k  ffff8800000123f8       0      0
  8    1024k  ffff880000012408       0      0
  8    1024k  ffff880000012418       0      0
  8    1024k  ffff880000012428       0      0
  9    2048k  ffff880000012440       0      0
  9    2048k  ffff880000012450       0      0
  9    2048k  ffff880000012460       0      0
  9    2048k  ffff880000012470       1    512
  9    2048k  ffff880000012480       0      0
 10    4096k  ffff880000012498       0      0
 10    4096k  ffff8800000124a8       0      0
 10    4096k  ffff8800000124b8       3   3072
 10    4096k  ffff8800000124c8       0      0
 10    4096k  ffff8800000124d8       0      0

ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  1   DMA32     1044480   30117  ffffea0000038000    1000000        4095   
AREA    SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
  0       4k  ffff88000001a7e8      24     24
  0       4k  ffff88000001a7f8       4      4
  0       4k  ffff88000001a808      13     13
  0       4k  ffff88000001a818       0      0
  0       4k  ffff88000001a828       0      0
  1       8k  ffff88000001a840       2      4
  1       8k  ffff88000001a850       2      4
  1       8k  ffff88000001a860       4      8
  1       8k  ffff88000001a870       0      0
  1       8k  ffff88000001a880       0      0
  2      16k  ffff88000001a898       0      0
  2      16k  ffff88000001a8a8       3     12
  2      16k  ffff88000001a8b8       4     16
  2      16k  ffff88000001a8c8       0      0
  2      16k  ffff88000001a8d8       0      0
  3      32k  ffff88000001a8f0       0      0
  3      32k  ffff88000001a900       3     24
  3      32k  ffff88000001a910       3     24
  3      32k  ffff88000001a920       0      0
  3      32k  ffff88000001a930       0      0
  4      64k  ffff88000001a948       1     16
  4      64k  ffff88000001a958       3     48
  4      64k  ffff88000001a968       6     96
  4      64k  ffff88000001a978       0      0
  4      64k  ffff88000001a988       0      0
  5     128k  ffff88000001a9a0       0      0
  5     128k  ffff88000001a9b0       3     96
  5     128k  ffff88000001a9c0       7    224
  5     128k  ffff88000001a9d0       0      0
  5     128k  ffff88000001a9e0       0      0
  6     256k  ffff88000001a9f8       0      0
  6     256k  ffff88000001aa08       1     64
  6     256k  ffff88000001aa18       6    384
  6     256k  ffff88000001aa28       0      0
  6     256k  ffff88000001aa38       0      0
  7     512k  ffff88000001aa50       1    128
  7     512k  ffff88000001aa60       0      0
  7     512k  ffff88000001aa70       8   1024
  7     512k  ffff88000001aa80       0      0
  7     512k  ffff88000001aa90       0      0
  8    1024k  ffff88000001aaa8       1    256
  8    1024k  ffff88000001aab8       1    256
  8    1024k  ffff88000001aac8       5   1280
  8    1024k  ffff88000001aad8       0      0
  8    1024k  ffff88000001aae8       0      0
  9    2048k  ffff88000001ab00       0      0
  9    2048k  ffff88000001ab10       1    512
  9    2048k  ffff88000001ab20       3   1536
  9    2048k  ffff88000001ab30       1    512
  9    2048k  ffff88000001ab40       0      0
 10    4096k  ffff88000001ab58       0      0
 10    4096k  ffff88000001ab68       0      0
 10    4096k  ffff88000001ab78      22  22528
 10    4096k  ffff88000001ab88       1   1024
 10    4096k  ffff88000001ab98       0      0

ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  2   Normal    7602176   10443  ffffea0003800000   100000000     1048575  
AREA    SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
  0       4k  ffff880000022ea8     365    365
  0       4k  ffff880000022eb8     274    274
  0       4k  ffff880000022ec8     274    274
  0       4k  ffff880000022ed8       0      0
  0       4k  ffff880000022ee8       0      0
  1       8k  ffff880000022f00      99    198
  1       8k  ffff880000022f10      94    188
  1       8k  ffff880000022f20     360    720
  1       8k  ffff880000022f30       0      0
  1       8k  ffff880000022f40       0      0
  2      16k  ffff880000022f58      30    120
  2      16k  ffff880000022f68      41    164
  2      16k  ffff880000022f78     204    816
  2      16k  ffff880000022f88       0      0
  2      16k  ffff880000022f98       0      0
  3      32k  ffff880000022fb0       9     72
  3      32k  ffff880000022fc0      19    152
  3      32k  ffff880000022fd0     138   1104
  3      32k  ffff880000022fe0       0      0
  3      32k  ffff880000022ff0       0      0
  4      64k  ffff880000023008       7    112
  4      64k  ffff880000023018       4     64
  4      64k  ffff880000023028      77   1232
  4      64k  ffff880000023038       0      0
  4      64k  ffff880000023048       0      0
  5     128k  ffff880000023060       3     96
  5     128k  ffff880000023070       3     96
  5     128k  ffff880000023080      43   1376
  5     128k  ffff880000023090       0      0
  5     128k  ffff8800000230a0       0      0
  6     256k  ffff8800000230b8       0      0
  6     256k  ffff8800000230c8       0      0
  6     256k  ffff8800000230d8      13    832
  6     256k  ffff8800000230e8       0      0
  6     256k  ffff8800000230f8       0      0
  7     512k  ffff880000023110       0      0
  7     512k  ffff880000023120       0      0
  7     512k  ffff880000023130       5    640
  7     512k  ffff880000023140       0      0
  7     512k  ffff880000023150       0      0
  8    1024k  ffff880000023168       0      0
  8    1024k  ffff880000023178       0      0
  8    1024k  ffff880000023188       0      0
  8    1024k  ffff880000023198       0      0
  8    1024k  ffff8800000231a8       0      0
  9    2048k  ffff8800000231c0       0      0
  9    2048k  ffff8800000231d0       0      0
  9    2048k  ffff8800000231e0       1    512
  9    2048k  ffff8800000231f0       0      0
  9    2048k  ffff880000023200       0      0
 10    4096k  ffff880000023218       0      0
 10    4096k  ffff880000023228       0      0
 10    4096k  ffff880000023238       0      0
 10    4096k  ffff880000023248       1   1024
 10    4096k  ffff880000023258       0      0

ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  3   Movable        0       0         0               0            0     

--------------------------------------------------------------------------

NODE
  1
ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  0   DMA            0       0         0               0            0     

ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  1   DMA32          0       0         0               0            0     

ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  2   Normal    8388608   10114  ffffea001ce00000   840000000        0     
AREA    SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
  0       4k  ffff880840018e28     405    405
  0       4k  ffff880840018e38     162    162
  0       4k  ffff880840018e48     317    317
  0       4k  ffff880840018e58       0      0
  0       4k  ffff880840018e68       0      0
  1       8k  ffff880840018e80     106    212
  1       8k  ffff880840018e90      70    140
  1       8k  ffff880840018ea0     269    538
  1       8k  ffff880840018eb0       0      0
  1       8k  ffff880840018ec0       0      0
  2      16k  ffff880840018ed8      24     96
  2      16k  ffff880840018ee8      18     72
  2      16k  ffff880840018ef8     207    828
  2      16k  ffff880840018f08       0      0
  2      16k  ffff880840018f18       0      0
  3      32k  ffff880840018f30      20    160
  3      32k  ffff880840018f40       4     32
  3      32k  ffff880840018f50     148   1184
  3      32k  ffff880840018f60       0      0
  3      32k  ffff880840018f70       0      0
  4      64k  ffff880840018f88      17    272
  4      64k  ffff880840018f98       2     32
  4      64k  ffff880840018fa8      95   1520
  4      64k  ffff880840018fb8       0      0
  4      64k  ffff880840018fc8       0      0
  5     128k  ffff880840018fe0       4    128
  5     128k  ffff880840018ff0       1     32
  5     128k  ffff880840019000      37   1184
  5     128k  ffff880840019010       0      0
  5     128k  ffff880840019020       0      0
  6     256k  ffff880840019038       0      0
  6     256k  ffff880840019048       0      0
  6     256k  ffff880840019058       8    512
  6     256k  ffff880840019068       0      0
  6     256k  ffff880840019078       0      0
  7     512k  ffff880840019090       0      0
  7     512k  ffff8808400190a0       0      0
  7     512k  ffff8808400190b0       1    128
  7     512k  ffff8808400190c0       0      0
  7     512k  ffff8808400190d0       0      0
  8    1024k  ffff8808400190e8       0      0
  8    1024k  ffff8808400190f8       0      0
  8    1024k  ffff880840019108       1    256
  8    1024k  ffff880840019118       0      0
  8    1024k  ffff880840019128       0      0
  9    2048k  ffff880840019140       0      0
  9    2048k  ffff880840019150       0      0
  9    2048k  ffff880840019160       1    512
  9    2048k  ffff880840019170       1    512
  9    2048k  ffff880840019180       0      0
 10    4096k  ffff880840019198       0      0
 10    4096k  ffff8808400191a8       0      0
 10    4096k  ffff8808400191b8       0      0
 10    4096k  ffff8808400191c8       1   1024
 10    4096k  ffff8808400191d8       0      0

ZONE  NAME        SIZE    FREE      MEM_MAP       START_PADDR  START_MAPNR
  3   Movable        0       0         0               0            0     

nr_free_pages: 54610  (found 54742)

A little exercise with foreach bt

crash> foreach bt | awk '$1 == "#0" { $2 = ""; print }' | sort | uniq -c
     31 #0  crash_nmi_callback at ffffffff81029df6
      1 #0  machine_kexec at ffffffff8103281b
    546 #0  schedule at ffffffff814fda62

Indeed, they are either crashed or waiting for memory (or I am not reading it correctly).

Short version

Server A (OpenBSD 4.7) connects to server B (Windows). IP of server B changes. Server A should be able to connect to server B to both the old and new IP. We cannot configure multiple IPs on server B.

Long version

We have an OpenBSD server acting as an access point (ssh + authpf rules) where external clients connect and then open a connection to a service on another internal server. The internal server IP is going to change.

To give us more time to reconfigure all clients to use the new IP address, I thought we can implement the equivalent of a DNAT on the OpenBSD box. If this was a Linux box, I could use the following DNAT rule which lets me connect out from the box itself to the remote service on either the real IP (10.68.32.215) or the new IP.

$ sudo iptables -t nat -A OUTPUT -d 10.68.99.99 -j DNAT --to-dest 10.68.32.215
$ ssh-keyscan -t rsa 10.68.32.215
# 10.68.32.215 SSH-2.0-OpenSSH_4.3
10.68.32.215 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAy/GCd47aaRkBOu72v9Ysqk48Ngd6budStvdwnvMOTLiYoz6M81cTq7SskWctXx57cz6Ijnv1sbzcmDpFMUsN5vHk+6NxfrLzO0M1zh7UezY54FakgaavSdCiy15vGw/Lifntp5kMKkjgC5o42O+RUVw5iCpR8nsu/2/kR2smcVR1G3R8EunjCZWEptOCHz3Iup7FTMd4Pw/xmt+8u+5ZyHKu+uaLWQl6I12rzLiQJNyMLVdhba54FGiJDFUfcXtgM7cFli6xlrE3dnbboQE/7/cuj/N11QwTvHuU07NtrubefZE1VahWb146ph31blsW5NSiyFwL2I7rxFFoPQMbuQ==
$ ssh-keyscan -t rsa 10.68.99.99
# 10.68.99.99 SSH-2.0-OpenSSH_4.3
10.68.99.99 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAy/GCd47aaRkBOu72v9Ysqk48Ngd6budStvdwnvMOTLiYoz6M81cTq7SskWctXx57cz6Ijnv1sbzcmDpFMUsN5vHk+6NxfrLzO0M1zh7UezY54FakgaavSdCiy15vGw/Lifntp5kMKkjgC5o42O+RUVw5iCpR8nsu/2/kR2smcVR1G3R8EunjCZWEptOCHz3Iup7FTMd4Pw/xmt+8u+5ZyHKu+uaLWQl6I12rzLiQJNyMLVdhba54FGiJDFUfcXtgM7cFli6xlrE3dnbboQE/7/cuj/N11QwTvHuU07NtrubefZE1VahWb146ph31blsW5NSiyFwL2I7rxFFoPQMbuQ==

Our version of OpenBSD is 4.7, but we can upgrade if necessary. If this DNAT is not possible we can probably do a NAT on a firewall along the way.

The closest I was able to accomplish on a test box is:

pass out on em1 inet proto icmp from any to 10.68.31.99 nat-to 10.68.31.247

Unfortunately, pfctl -s state tells me that nat-to translates the source IP, while I need to translate the destination.

$ sudo pfctl -s state
all icmp 10.68.31.247:7263 (10.68.30.199:13437) -> 10.68.31.99:8       0:0

I also found lots of mentions about rules that start with rdr and include the -> symbol to express the translation, but it looks like this syntax has been obsoleted in 4.7 and I cannot get anything similar to work. Attempts to implement a new-syntax redirect rule fail with:

$ echo match out on em1 to 10.68.31.99 rdr-to 10.68.31.247 | sudo pfctl -f -
stdin:1: rdr-to can only be used inbound

Of course, since I am trying to redirect outgoing traffic, modifying the above rule to "pass in" does not work either.

Current status

Ended up applying a NAT on a firewall between the two servers. Did the trick, though from academic interest, I am still curious if this is doable in OpenBSD.

I need to make a driver disk for anaconda. This disk would contain an updated module for the network card of a somewhat recent laptop and the goal is to get a network install of CentOS 6.2 working. I am using cobbler so even the kickstart file itself is on the network. I don't mind if the installation is not completely unattended and I don't mind sticking a USB stick with the driver disk image during the install and manually selecting the correct item, but I do want to keep everything else on the network (kernel image, initrd, kickstart file, RPM repositories).

I already got the updated driver built, and I even have a "kmod" RPM built with the help of /usr/lib/rpm/redhat/kmodtool. Unfortunately I have found absolutely no documentation on what a driver disk looks like.

http://www.phil.muni.cz/~letty/linuxfaq/pracovni/driver-disk-howto is the only link I could find, and it describes the struggle of someone who did the same thing I did - downloaded driver disks and tried to explore them. Unfortunately the format in CentOS 6 seems to have changed and the best I could do is to get anaconda to tell me that: "Driver disk is invalid for this release of CentOS.". The log itself says "cannot find /tmp/drivers/rpms/x86_64/repodata" and until now I have not seen a driver disk with "repodata" on it.

Currently diving into the anaconda source code but I hope someone can point me in the right direction.