Artur Meinild's questions

I'm using Ubuntu 22.04.3 Server. I've installed nala using the Volian Scar repo. Recently, this repo has included a number of updated Python 3 packages, including python3-anyio.

However, this particular package will not upgrade:

$ sudo apt full-upgrade
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  python3-mdurl
The following packages have been kept back:
  python3-anyio
The following packages will be upgraded:
  python3-click python3-httpcore python3-markdown-it python3-pygments python3-rich python3-typer python3-typing-extensions
7 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
...

And if I try to force install python3-anyio, I get the exact reason:

$ sudo apt install --reinstall --dry-run python3-anyio 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python3-anyio : Depends: python3-exceptiongroup but it is not installable or
                          python3 (> 3.11) but 3.10.6-1~22.04 is to be installed
E: Unable to correct problems, you have held broken packages.

It's now evident that this updated package requires Python 3 "Exception Groups", which are either included in Python 3.11 (Ubuntu 22.04 uses Python 3.10), or in the backported package python3-exceptiongroup.

Searching for this package on Ubuntu Package archive reveals that this package is part of the Universe repository, but only for Ubuntu 23.04 (Lunar) or 23.10 (Mantic).

What options do I have to avoid dependency issues in this situation when I still want to upgrade my packages?

FYI, this has already been reported as an issue with Volian. I assume the package python3-exceptiongroup should be added to the Volian repo in the future.

ALAS, python3-exceptiongroup has already been added to the Volian Scar repo, so now this dependency error does not show up anymore. Still, the method in this Q&A could be used in other similar situations.

In Ubuntu LTS, applications are locked to the version fixed to the release date. For Ubuntu 22.04 "Jammy" those are:

• nano 6.2-1
• htop 3.0.5-7
• tmux 3.2a-4
• lnav 0.9.0-2

I would like to install the latest upstream versions of these applications on my home server.

How can I do this in an easy and reliable way?

There recently was a couple of questions asking why Ubuntu is seemingly using more memory than accounted for by processes/services - here and here.

However, I have the exact opposite situation. On my server, the memory used by all processes seems to be higher than the total memory used. Let me provide the evidence.

Output of free -m:

               total        used        free      shared  buff/cache   available
Mem:           64277       15014       34424          31       14837       48530
Swap:           8191           0        8191

Output of arcstat:

    time  read  miss  miss%  dmis  dm%  pmis  pm%  mmis  mm%  size     c  avail
09:38:57     0     0      0     0    0     0    0     0    0  9.8G  9.8G    41G

Output of ps aux | awk '{sum +=$6}END{print sum}':

6351512

So what this means for me is:

free -m reports a total of 15.0 GB memory used (shared memory is neglible). Subtract from this 9.8 GB reported by arcstat for ZFS ARC Cache. The remaining result is around 5.2 GB of "real" used memory. But the sum of column 6 in ps aux (RSS - resident set size) totals to 6.35 GB. This would mean that reported memory used by all processes is 1.15 GB higher than it actually is shown by free -m.

I wonder if there is a way I can correctly calculate the memory used by all processes, so it matches the used memory (after ZFS cache and shared ramdisks are subtracted)?

There is an existing post on Unix & Linux about including Unicode characters in the Bash prompt, but the method it gives for using the UTF-16 code (syntax \uXXXX) doesn't work for me.

Let's take this arrow as an example:

Unicode Character 'BLACK RIGHTWARDS ARROWHEAD' (U+27A4)

As I can see, there should be 3 ways to do Unicode characters in the Bash PS1 prompt:

• UTF-8 Octal value - in this case: \342\236\244
• UTF-8 HEX value - in this case: \xe2\x9e\xa4
• UTF-16 HEX value (combined) - in this case: \u27A4

However, only the first option works in my Ubuntu prompt, as indicated in the image below:

Explanation:

• With PS1='[\u@\h \W]\$ \342\236\244 ', the Unicode character is applied correctly.
• With PS1='[\u@\h \W]\$ \xe2\x9e\xa4 ', the exact string is applied instead.
• With PS1='[\u@\h \W]\$ \u27A4 ', it applies my username (am) and then the exact value 27A4.

I understand that \u is a Bash shortcut for username, which is why I'm confused the other answer states that this works - for me it doesn't.

Is there a way to make the UTF-16 syntax work for PS1 on Ubuntu (since this is both shorter and easier to read)? I'm on Ubuntu 22.04.1 Server.

If you run sar without parameters, the output is in color, but with extra headers inserted after each page. If you run sar | cat the extra headers are removed, but so is the coloring.

If you would like to pipe into less, running sar | less have the same effect as using cat - extra headers are removed, but so are colors. If you would like to preserve color with less, you can use unbuffer (which is installed as a part of expect) and less -R, but then the extra headers appear again.

I believe this is better shown in pictures below (the extra headers are marked in image 2).

The result of sar | less:

The result of unbuffer sar | less -R:

So is it at all possible to preserve colors with the sar command using less, and at the same time not have the extra header lines for each page (as marked in image 2)?

I reckon it might be possible to do this with some elaborate awk/grep/sed piping (and maybe chained commands), but I would prefer a more straight solution with the commands themselves, if this is at all possible. This is also due to the fact that I would like to pipe the result to less using a single command.

I'm using Ubuntu Server 22.04.

Today, I noticed that the directory /lib/modules still have all the directories from previously installed kernels, thus:

$ ls /lib/modules
5.4.0-29-generic  5.4.0-31-generic  5.4.0-33-generic  5.4.0-37-generic  5.4.0-39-generic
5.4.0-40-generic  5.4.0-42-generic  5.4.0-45-generic  5.4.0-47-generic  5.4.0-48-generic
...  etc.

In each of these directories, the following files are left (Intel/AMD system):

$ ls -l
-rw-r--r--  1 root 143K 2022-04-28 18:36 modules.alias
-rw-r--r--  1 root 154K 2022-04-28 18:36 modules.alias.bin
-rw-r--r--  1 root 8.0K 2022-04-08 10:44 modules.builtin
-rw-r--r--  1 root  25K 2022-04-28 18:36 modules.builtin.alias.bin
-rw-r--r--  1 root  11K 2022-04-28 18:36 modules.builtin.bin
-rw-r--r--  1 root  63K 2022-04-08 10:44 modules.builtin.modinfo
-rw-r--r--  1 root  85K 2022-04-28 18:36 modules.dep
-rw-r--r--  1 root 123K 2022-04-28 18:36 modules.dep.bin
-rw-r--r--  1 root  268 2022-04-28 18:36 modules.devname
-rw-r--r--  1 root 215K 2022-04-08 10:44 modules.order
-rw-r--r--  1 root  489 2022-04-28 18:36 modules.softdep
-rw-r--r--  1 root 279K 2022-04-28 18:36 modules.symbols
-rw-r--r--  1 root 326K 2022-04-28 18:36 modules.symbols.bin

And for a Raspberry Pi (notice the size differences):

-rw-r--r--  1 root 859K 2022-04-25 17:31 modules.alias
-rw-r--r--  1 root 893K 2022-04-25 17:31 modules.alias.bin
-rw-r--r--  1 root  11K 2022-04-04 13:33 modules.builtin
-rw-r--r--  1 root  27K 2022-04-25 17:31 modules.builtin.alias.bin
-rw-r--r--  1 root  14K 2022-04-25 17:31 modules.builtin.bin
-rw-r--r--  1 root  84K 2022-04-04 13:33 modules.builtin.modinfo
-rw-r--r--  1 root 399K 2022-04-25 17:31 modules.dep
-rw-r--r--  1 root 548K 2022-04-25 17:31 modules.dep.bin
-rw-r--r--  1 root  250 2022-04-25 17:31 modules.devname
-rw-r--r--  1 root 209K 2022-04-04 13:33 modules.order
-rw-r--r--  1 root  950 2022-04-25 17:31 modules.softdep
-rw-r--r--  1 root 354K 2022-04-25 17:31 modules.symbols
-rw-r--r--  1 root 425K 2022-04-25 17:31 modules.symbols.bin

For a system that's been running for 2 years with all kernel upgrades, this amounts to ~50 old directories. For a normal Intel/AMD system, it's around 100 Megabytes, and for a Raspberry Pi it's over 300 Megabytes in total - it will amount to something over time.

So I wonder why these old module directories and files for removed kernels aren't removed as part of the cleanup process?

I recently upgraded from Ubuntu 20.04 to 22.04 (which is why I'm doing some cleanup stuff anyway).

As things currently stand, I wonder if I should make my own cleanup script for these leftover directories. Still, I'm curious why it isn't done automatically.

I'm using Docker 20.10.14 on Ubuntu 22.04 Server where I'm running a number of containers, and I'm using Watchtower to automatically update the containers.

Since I updated to Ubuntu 22.04, I've been getting excessive log messages from networkd-dispatcher when the containers are updated. This is the log messages I get:

May  3 01:03:06 xb networkd-dispatcher[1327]: ERROR:Unknown state for interface veth44e6185: initialized
May  3 01:03:06 xb networkd-dispatcher[1327]: Traceback (most recent call last):
May  3 01:03:06 xb networkd-dispatcher[1327]:   File "/usr/bin/networkd-dispatcher", line 428, in _receive_signal
May  3 01:03:06 xb networkd-dispatcher[1327]:     self.handle_state(iface_name,
May  3 01:03:06 xb networkd-dispatcher[1327]:   File "/usr/bin/networkd-dispatcher", line 345, in handle_state
May  3 01:03:06 xb networkd-dispatcher[1327]:     raise UnknownState(administrative_state)
May  3 01:03:06 xb networkd-dispatcher[1327]: UnknownState: initialized
May  3 01:03:06 xb networkd-dispatcher[1327]: ERROR:Unknown state for interface veth6d82244: initialized
May  3 01:03:06 xb networkd-dispatcher[1327]: Traceback (most recent call last):
May  3 01:03:06 xb networkd-dispatcher[1327]:   File "/usr/bin/networkd-dispatcher", line 428, in _receive_signal
May  3 01:03:06 xb networkd-dispatcher[1327]:     self.handle_state(iface_name,
May  3 01:03:06 xb networkd-dispatcher[1327]:   File "/usr/bin/networkd-dispatcher", line 345, in handle_state
May  3 01:03:06 xb networkd-dispatcher[1327]:     raise UnknownState(administrative_state)
May  3 01:03:06 xb networkd-dispatcher[1327]: UnknownState: initialized

In this particular case, the interface veth44e6185 is the one that's persisting after the container upgrade, while I assume the interface veth6d82244 is from the previous container, since this interface no longer exist.

I believe this is because the Docker network interface is reinitialized/restarted for each container when it is updated. The Docker network interfaces are easily recognizable, since the have they format vethXXXXXXX, where X is a hexadecimal digit.

Everything is working fine after the containers update, but the amount of verbosity in the error messages have gone drastically up with the update from Ubuntu 20.04 to 22.04.

So is there any simple and smart way to disable the networkd-dispatcher log messages for the Docker interfaces only?

The canonical-livepatch snap does not feature autocompletion for its subcommands. As per the help page, the valid subcommands are:

$ canonical-livepatch help
...
COMMANDS:
   config - configure livepatching on the machine
   disable - disable livepatching on the machine
   enable - enable livepatching on the machine
   help - display help
   kernel-upgrade-required - indicate whether a kernel upgrade is required
   refresh - immediately download and apply any available livepatch
   status - show kernel's livepatch status

Is there an easy way to add autocompletion for the subcommands in the canonical-livepatch snap that works in Bash, without having to modify the snap itself?

A little detail is that the autocompletion should also work for the snap alias livepatch I have for the command.

$ snap aliases
Command              Alias      Notes
canonical-livepatch  livepatch  manual
lxd.lxc              lxc        -

There recently was talks on Meta about the support period of LTS and non-LTS releases on this site. In addition, I saw the mention of Ubuntu Core in a comment, and this got me wondering what the support period for Ubuntu Core actually is, and if it follows the support period for Ubuntu LTS.

Canonical states this on the Ubuntu Core page:

Ubuntu Core gets 10 years of Canonical maintenance. Your smallest devices are now as secure as your servers. No other embedded Linux comes close.

Also, as far as I understand, Ubuntu Core 16 is still the default "core" release for snaps. This is supported by the fact that Core 16 (just named "core" in snap) has update cycle and versioning that exactly follows the snapd version.

Output of snap info snapd:

<...>
type:         snapd
snap-id:      PMrrV4ml8uWuEUDBT8dSGnKUYbevVhc4
tracking:     latest/stable
refresh-date: 9 days ago, at 23:27 CEST
channels:
  latest/stable:    2.51.4                  2021-08-25 (12883) 33MB -
  latest/candidate: 2.51.7                  2021-08-31 (13170) 33MB -
  latest/beta:      2.51.7                  2021-08-27 (13170) 33MB -
  latest/edge:      2.51.7+git1522.gc980bde 2021-08-31 (13216) 42MB -
installed:          2.51.4                             (12883) 33MB snapd

Output of snap info core:

<...>
type:         core
snap-id:      99T7MUlRhtI3U0QFgl5mXXESAiSwt776
tracking:     latest/stable
refresh-date: 9 days ago, at 23:27 CEST
channels:
  latest/stable:    16-2.51.4                  2021-08-25 (11606) 104MB -
  latest/candidate: 16-2.51.7                  2021-08-31 (11743) 104MB -
  latest/beta:      16-2.51.7                  2021-08-27 (11743) 104MB -
  latest/edge:      16-2.51.7+git3168.3a0f4248 2021-09-01 (11771) 112MB -
installed:          16-2.51.4                             (11606) 104MB core

Another element in the mix is that Ubuntu Core isn't released completely in parallel with the LTS releases. For instance, Ubuntu Core 20 was released in January 2021, i.e. 9 months after the first release of Ubuntu 20.04.

So do Ubuntu Core releases follow the same 5-year support period as LTS releases, or is it indeed 10 years, per the Canonical security maintainance period?

This also ties into the recent topic of How long do we support Ubuntu flavors?, as well as my suggestion to rephrase the Help Center explanation.

For an answer to be considered satisfactory, the following elements should be addressed:

• The relation between Ubuntu LTS and Core releases, including the difference in release times
• The "special" status of Ubuntu Core 16 (named core in snap) as the base that follows snapd versioning
• Reflections on how this should affect support time on Ask Ubuntu
• Reference to official sources and/or Ubuntu team members (as opposed to speculation)

Let's say I want to create an alias called ss for the command sudo -s. But ss is already an existing command, so I want to create another alias called sst for the command ss.

If using just the normal command names, this is not possible, since aliases:

1. Are not set in a way that respects the order, and
2. Reference other aliases, instead of only referencing commands

So if I try the following:

alias sst='ss'
alias ss='sudo -s'

Running the command sst results in running sudo -s, which is not my intention.

How can this be done?

I'm making a script to print ZFS filesystem info - currently in the testing phase, and I'm getting a strange error.

The relevant bit of my initial script is this:

zfs_human="$(zfs list | head -n 2 | tail -n 1)"
dfs_human="$(df -h | grep 'zfs' | head -n 1)"
zfs_usedh="$(echo $zfs_human | cut -d ' ' -f2)"
zfs_totah="$(echo $dfs_human | cut -d ' ' -f2)"
echo "$zfs_human"
echo "$dfs_human"
echo "$zfs_usedh"
echo "$zfs_totah"

Giving the following output:

zfs                      2.31M  5.27T     34.4K  /mnt/zfs
zfs                      5.3T  128K  5.3T   1% /mnt/zfs
2.31M
5.3T

However, when I run shellcheck, it says I should double-quote the variable names inside the command substitution, this is the output from shellcheck:

In zfsspace.sh line 5:zfs_usedh="$(echo $zfs_human | cut -d ' ' -f2)"                  ^--------^
SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
zfs_usedh="$(echo "$zfs_human" | cut -d ' ' -f2)"

In zfsspace.sh line 6:zfs_totah="$(echo $dfs_human | cut -d ' ' -f2)"                  ^--------^
SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
zfs_totah="$(echo "$dfs_human" | cut -d ' ' -f2)"

Then I of course change my code to shellcheck's recommendation:

zfs_human="$(zfs list | head -n 2 | tail -n 1)"
dfs_human="$(df -h | grep 'zfs' | head -n 1)"
zfs_usedh="$(echo "$zfs_human" | cut -d ' ' -f2)"
zfs_totah="$(echo "$dfs_human" | cut -d ' ' -f2)"
echo "$zfs_human"
echo "$dfs_human"
echo "$zfs_usedh"
echo "$zfs_totah"

But now the output is this:

zfs                      2.31M  5.27T     34.4K  /mnt/zfs
zfs                      5.3T  128K  5.3T   1% /mnt/zfs
 
 

Line 3 and 4 is blank, which means the 3rd and 4th command substitution does not work when following shellcheck's recommendation, but works when not quoting the variable that I echo.

I'm using Bash 5.0.17 on Ubuntu 20.04.1

Can anyone explain this please??? Thanks.

I'm booting Ubuntu server 20.04 from a single nvme disk. Since a couple of days ago, multipath has been spamming my syslog with these messages:

Jun 11 20:10:14 xb multipath: nvme0n1: failed to get udev uid: Invalid argument
Jun 11 20:10:14 xb multipath: nvme0n1: uid = eui.0000000001000000e4d25c7f117c5001 (sysfs)
Jun 11 20:10:17 xb multipath: nvme0n1: failed to get udev uid: Invalid argument
Jun 11 20:10:17 xb multipath: nvme0n1: uid = eui.0000000001000000e4d25c7f117c5001 (sysfs)
Jun 11 20:10:20 xb multipath: nvme0n1: failed to get udev uid: Invalid argument
Jun 11 20:10:20 xb multipath: nvme0n1: uid = eui.0000000001000000e4d25c7f117c5001 (sysfs)
Jun 11 20:11:34 xb multipath: nvme0n1: failed to get udev uid: Invalid argument
Jun 11 20:11:34 xb multipath: nvme0n1: uid = eui.0000000001000000e4d25c7f117c5001 (sysfs)
Jun 11 20:11:37 xb multipath: nvme0n1: failed to get udev uid: Invalid argument
Jun 11 20:11:37 xb multipath: nvme0n1: uid = eui.0000000001000000e4d25c7f117c5001 (sysfs)

I can get the following list of devices to confirm it monitors my nvme disk:

multipathd> show devices
available block devices:
loop1 devnode blacklisted, unmonitored
nvme0n1 devnode whitelisted, monitored
loop6 devnode blacklisted, unmonitored
loop4 devnode blacklisted, unmonitored
loop2 devnode blacklisted, unmonitored
loop0 devnode blacklisted, unmonitored
loop7 devnode blacklisted, unmonitored
loop5 devnode blacklisted, unmonitored
loop3 devnode blacklisted, unmonitored
multipathd> 

I tried the command:

remove map nvme0n1

But got the message: fail

So now the question is, how can I get it to stop monitoring my nvme disk and spamming the syslog?