A release upgrade asked me whether I would like to use the new or old version of /etc/grub.d/10_linux file. I chose the new one, but noticed now that the old (slightly manually patched) file is still there, named 10_linux.dpkg-old.

My understanding is that update-grub will run all the scripts in /etc/grub.d. Since there have been a few bugs fixed in the new version of the 10_linux script, I do not want update-grub to run 10_linux.dpkg-old.

I assume making the old script non-executable using chmod would allow me to keep it in that folder. But I would like to know whether there is some other mechanism preventing update-grub from using it.

Reference of my current understanding: help.ubuntu

The scripts in this directory are read during execution of the update-grub command and their instructions are incorporated into /boot/grub/grub.cfg.

The placement of the menu items in the grub.cfg menu is determined by the order in which the files in this directory are run. Files with a leading numeral are executed first, beginning with the lowest number. 10_linux is run before 20_memtest, which would run before 40_custom. If files with alphabetic names exist, they are run after the numerically-named files.

Custom menu entries can be added to the 40_custom file or in a newly created file. Based on its name, 40_custom entries by default appear at the bottom of the menu. A custom file beginning with 06_ would appear at the top of the menu since its alphanumeric sorting would place it ahead of 10_ through 40_ files.

Only executable files generate output to grub.cfg during execution of update-grub. By default the files in the /etc/grub.d folder are executable.

I would like to open a (text) file with Vim in nautilus. This part is easy: Right-Click -> Open With Vim starts a terminal with vim inside running, file open already.

When I close vim, the terminal closes too. I would like the terminal to remain open, with the working directory set to the directory the file is in.

I'm sure there is a simple way to achieve this by creating a custom bash script and adding it to the context menu (or even as the default action on Enter when the file is selected) but I don't know how. So, How to do this?

I am on ubuntu 18.04 but I hope it is similar for later versions.

As discussed here, if an apt PPA changes its label and we try to apt-get update, we get something like this:

E: Repository 'http://ppa.launchpad.net/ondrej/php/ubuntu artful InRelease' changed its 'Label' value from '*****
The main PPA for PHP (5.6, 7.0, 7.1) with many PECL extensions *****' to '*****
The main PPA for supported PHP versions with many PECL extensions *****'
N: This must be accepted explicitly before updates for this repository can be applied.
See apt-secure(8) manpage for details."

The solution is to run apt update instead and manually accept the change.

This question is about the reasoning. Why is it important that the label does not change without my explicit awareness? I don't know where I would even see this label in normal day-to-day usage...

I have tried to upgrade docker, and received an error, so I did a reinstall. However, the issue persists. The folder it fails to stat does indeed not exist. What can I do to fix this issue?

I could of course simply create that directory and hope it was not a sign of something being terribly broken which will haunt me later. But I'd like to have input from somebody who knows more than I.

generic@motorbrot:~$ sudo apt-get install docker.io
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils
The following NEW packages will be installed:
  docker.io
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/36.9 MB of archives.
After this operation, 192 MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package docker.io.
(Reading database ... 539898 files and directories currently installed.)
Preparing to unpack .../docker.io_20.10.2-0ubuntu1~18.04.2_amd64.deb ...
Unpacking docker.io (20.10.2-0ubuntu1~18.04.2) ...
Setting up docker.io (20.10.2-0ubuntu1~18.04.2) ...
mv: cannot stat '/var/lib/docker': No such file or directory
dpkg: error processing package docker.io (--configure):
 installed docker.io package post-installation script subprocess returned error exit status 1
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

Docker seems to be installed:

generic@motorbrot:~$ which docker
/usr/bin/docker
generic@motorbrot:~$ docker --version
Docker version 20.10.2, build 20.10.2-0ubuntu1~18.04.2

But when I run sudo apt upgrade it shows it as "Not fully installed or removed".

generic@motorbrot:~$ sudo apt upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up docker.io (20.10.2-0ubuntu1~18.04.2) ...
mv: cannot stat '/var/lib/docker': No such file or directory
dpkg: error processing package docker.io (--configure):
 installed docker.io package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

generic@motorbrot:~/algolab$ dpkg -s docker.io | grep Status
Status: install ok half-configured

System

I'm on Ubuntu 18.04

$ uname -a
Linux motorbrot 5.4.0-72-generic #80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Trying to use docker

but I cannot start the daemon. it says it is "masked".

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
generic@motorbrot:~/algolab$ sudo systemctl docker start
Unknown operation docker.
generic@motorbrot:~/algolab$ sudo service docker start
Failed to start docker.service: Unit docker.service is masked.
generic@motorbrot:~/algolab$ sudo systemctl unmask docker
Removed /etc/systemd/system/docker.service.
generic@motorbrot:~/algolab$ sudo systemctl start docker
Failed to start docker.service: Unit docker.socket is masked.

Creating directory /var/lib/docker

I tried creating that directory that it failed to stat (after making a zfs snapshot so that I could rollback in the worst case). Now it is complaining that docker.migrating/docker is not empty.

generic@motorbrot:~$ sudo apt upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up docker.io (20.10.2-0ubuntu1~18.04.2) ...
mv: cannot move '/var/lib/docker' to '/var/lib/docker.migrating/docker': Directory not empty
dpkg: error processing package docker.io (--configure):
 installed docker.io package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

And even if I remove that directory, I get problems:

generic@motorbrot:~$ sudo rm -r /var/lib/docker.migrating/docker
generic@motorbrot:~$ sudo mkdir /var/lib/docker
generic@motorbrot:~$ sudo apt upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up docker.io (20.10.2-0ubuntu1~18.04.2) ...
mv: target '/var/lib/docker/' is not a directory
dpkg: error processing package docker.io (--configure):
 installed docker.io package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

The Post-Install Script

It's located at /var/lib/dpkg/info/docker.io.postinst according to this answer, and it seems to me that it deletes the directory even if I create it ahead of time, then complains that it does not exist.

A quick look at it shows that it does special handling for ZFS, so I should point out that I am indeed running on ZFS on root.

Apt Purge

I'm going somewhat nuclear now, I rarely use docker anyway. I have removed all the docker datasets that were annyoing me in zfs list anyway:

for dataset in $(zfs list | grep legacy | cut -f1); do zfs destroy -R "$dataset" || true ; done

and also did a purge

$ sudo apt purge docker.io

generic@motorbrot:~$ sudo apt purge docker.io
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  bridge-utils cgroupfs-mount containerd pigz runc ubuntu-fan
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  docker.io*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 192 MB disk space will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 540125 files and directories currently installed.)
Removing docker.io (20.10.2-0ubuntu1~18.04.2) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
(Reading database ... 539920 files and directories currently installed.)
Purging configuration files for docker.io (20.10.2-0ubuntu1~18.04.2) ...
dpkg: warning: while removing docker.io, directory '/etc/docker' not empty so not removed

There remains a file /etc/docker/key.json

Reinstall again, get a new issue:

generic@motorbrot:~$ sudo apt install docker.io
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils
The following NEW packages will be installed:
  docker.io
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/36.9 MB of archives.
After this operation, 192 MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package docker.io.
(Reading database ... 539920 files and directories currently installed.)
Preparing to unpack .../docker.io_20.10.2-0ubuntu1~18.04.2_amd64.deb ...
Unpacking docker.io (20.10.2-0ubuntu1~18.04.2) ...
Setting up docker.io (20.10.2-0ubuntu1~18.04.2) ...
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
invoke-rc.d: initscript docker, action "start" failed.
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2021-04-23 14:31:06 CEST; 3ms ago
     Docs: https://docs.docker.com
  Process: 32306 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
 Main PID: 32306 (code=exited, status=1/FAILURE)
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

generic@motorbrot:~$ sudo systemctl restart docker

The tlp documentation describes tlp recalibrate as follows:

sudo tlp recalibrate [ BAT0 | BAT1 ]
This command works as follows:

• Resets the charge thresholds to factory defaults 96/100 %
• Discharges the selected battery completely (see description of tlp discharge above)
• When discharging is complete the command terminates; it does not wait for the charge to complete
• Important: to complete the recalibration process, let the battery charge to 100 % subsequently (you may power off but not remove AC power)

I have /etc/UPower/UPower.conf set up to go into HybridSleep at a battery percentage of 3%.

If I want to recalibrate my battery, using sudo tlp recalibrate BAT0, do I have to temporarily disable upower by changing the upower config to as follows? Or does upower not interfere because the battery can still finish discharging while the laptop is in HybridSleep?

UsePercentageForPolicy=true
PercentageAction=0

(This is a follow-up question to the explanations given in What kind of battery management is preinstalled on a clean Ubuntu 18.04 install?)

A friend recently told me that it is important to not charge the laptop battery fully, so that it lives longer. I didn't know if that is true, and if it is I would expect the battery hardware or the OS to handle that without me fiddling with it.

To back this claim up, here's a quote from Lenovo's Vantage Tool on Windows:

If you primarily use your computer with the AC adapter attached and only infrequently use battery power, you can increase the lifespan of the battery by setting the maximum charge value to below 100%. This is useful because batteries that are used infrequently have a longer lifespan when they are maintained at less than a full charge.

I have a ThinkPad X1 Yoga with Ubuntu 18.04, but I expect the answer not to be strongly dependent on the laptop model and the Ubuntu version.

What kind of battery management is preinstalled on Ubuntu without further configuration?

I am thinking of things like the following:

• stops charging at XX%
• warns at low battery
• hibernates on critical battery level

I am worried I might be destroying my battery's life by leaving the laptop constantly plugged in over a few weeks.

In the case that Ubuntu does not already have precautions set up, please also mention in your answer a suggestion regarding this.

Question

I am running Ubuntu 18.04 on a Lenovo Yoga ThinkPad X1 Gen4. The audio quality is noticeably lacking compared to what the same hardware shows off on windows. And the volume maximum on Ubuntu is quieter than 30% volume on Windows.
How can I improve this? Ideally while maintaining the functionality of the volume keys.

What I tried

alsamixer

Increasing the master volume in alsamixer does slightly raise the volume of the laptop, but not sufficiently. Also, it does not persist.

alsamixer reports only one sound card option when I press F6 (which is reasonable) and does not offer a slider for the often-mentioned "pcm" that some answers advised to modify. E.g. this answer.

Interestingly, none of the alsamixer sliders move when I use the volume keys on the keyboard to increase/decrease the volume, although they clearly do affect the volume level.

System Settings

The sound settings have a togglebutton for "Over-Amplification" which increases the volume level maximum by 50% at the cost of audio quality. This is not only not sufficiently loud, it is also even worse audio quality than before.

The speaker is the only device listed, called Speaker-sof-skl_hda_card.

pavucontrol

Increasing the volume in pavucontrol seems to be doing the same thing as increasing the master volume level in alsamixer.

upgrading PulseAudio to v13

I uninstalled PulseAudio v11 (latest in apt) and tried building the latest PulseAudio myself. The ../configure succeeded with warnings about d-bus being too old or not available. I think neither of those is true, so I didn't proceed due to feeling unsure.

PulseEffects with PulseAudio v12

Using the PPA advised for in this answer I could successfully install PulseAudio v12 and PulseEffects.
After a reboot, the update itself did not seem to have any noticeable effect.

I activated autogain in the PulseEffects GUI as advised in this comment on another answer but while it certainly did have an effect and maybe improved quality a little, the volume was decreased by having this option active.

Installing Correct Drivers

Lenovo only has drivers listed for windows, although lenovo states that the laptop is "linux certified". This includes "Proper functionality of the device" under "Audio".

It is maybe worth noting that the lenovo website talks about a "Realtek Audio Driver" while lspci only reports an "Intel Corporation Device 9dc8 (rev 11)" to me.

6.0.8854.1 (Realtek Audio)/ 10.23.0.3672 (Intel Smart Sound Technology)

According to certification.ubuntu.com the Audio hardware is

Audio

Intel Corp. Cannon Point-LP High Definition Audio Controller (8086:9dc8)

Intel Corp. Unknown

nomodeset

For a different issue, I booted with nomodeset as a grub parameter, which in turn broke my volume keys. But this seems to have been fixed in a recent kernel update. Both with and without nomodeset, the sound itself is low quality though.

System Information

uname -a

generic@motorbrot:~$ uname -a
Linux motorbrot 5.3.0-62-generic #56~18.04.1-Ubuntu SMP Wed Jun 24 16:17:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

lspci

generic@motorbrot:~$ lspci | grep -i Audio
00:1f.3 Audio device: Intel Corporation Device 9dc8 (rev 11)

aplay -l

generic@motorbrot:~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: sofsklhdacard [sof-skl_hda_card], device 0: HDA Analog (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: sofsklhdacard [sof-skl_hda_card], device 1: HDA Digital (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: sofsklhdacard [sof-skl_hda_card], device 3: HDMI1 (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: sofsklhdacard [sof-skl_hda_card], device 4: HDMI2 (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: sofsklhdacard [sof-skl_hda_card], device 5: HDMI3 (*) []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

/etc/modprobe.d/alsa-base.conf

# autoloader aliases
install sound-slot-0 /sbin/modprobe snd-card-0
install sound-slot-1 /sbin/modprobe snd-card-1
install sound-slot-2 /sbin/modprobe snd-card-2
install sound-slot-3 /sbin/modprobe snd-card-3
install sound-slot-4 /sbin/modprobe snd-card-4
install sound-slot-5 /sbin/modprobe snd-card-5
install sound-slot-6 /sbin/modprobe snd-card-6
install sound-slot-7 /sbin/modprobe snd-card-7

# Cause optional modules to be loaded above generic modules
install snd /sbin/modprobe --ignore-install snd $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-ioctl32 ; /sbin/modprobe --quiet --use-blacklist snd-seq ; }
#
# Workaround at bug #499695 (reverted in Ubuntu see LP #319505)
install snd-pcm /sbin/modprobe --ignore-install snd-pcm $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-pcm-oss ; : ; }
install snd-mixer /sbin/modprobe --ignore-install snd-mixer $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-mixer-oss ; : ; }
install snd-seq /sbin/modprobe --ignore-install snd-seq $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq-midi ; /sbin/modprobe --quiet --use-blacklist snd-seq-oss ; : ; }
#
install snd-rawmidi /sbin/modprobe --ignore-install snd-rawmidi $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq-midi ; : ; }
# Cause optional modules to be loaded above sound card driver modules
install snd-emu10k1 /sbin/modprobe --ignore-install snd-emu10k1 $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-emu10k1-synth ; }
install snd-via82xx /sbin/modprobe --ignore-install snd-via82xx $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist snd-seq ; }

# Load saa7134-alsa instead of saa7134 (which gets dragged in by it anyway)
install saa7134 /sbin/modprobe --ignore-install saa7134 $CMDLINE_OPTS && { /sbin/modprobe --quiet --use-blacklist saa7134-alsa ; : ; }
# Prevent abnormal drivers from grabbing index 0
options bt87x index=-2
options cx88_alsa index=-2
options saa7134-alsa index=-2
options snd-atiixp-modem index=-2
options snd-intel8x0m index=-2
options snd-via82xx-modem index=-2
options snd-usb-audio index=-2
options snd-usb-caiaq index=-2
options snd-usb-ua101 index=-2
options snd-usb-us122l index=-2
options snd-usb-usx2y index=-2
# Ubuntu #62691, enable MPU for snd-cmipci
options snd-cmipci mpu_port=0x330 fm_port=0x388
# Keep snd-pcsp from being loaded as first soundcard
options snd-pcsp index=-2
# Keep snd-usb-audio from beeing loaded as first soundcard
options snd-usb-audio index=-2

# sound fix 03. Jun 2020
# did help when combined with running "pulseaudio --start" after reboot.
# But broke birghtness keys.
# See askubuntu.com/questions/1245587/sound-card-not-detected-on-ubuntu-18.04/1246476
# options snd-hda-intel dmic_detect=0

Loaded Audio Drivers

As per this question my loaded sound drivers are:

generic@motorbrot:~$ /sbin/lsmod | grep snd
snd_soc_skl_hda_dsp    24576  5
snd_soc_hdac_hdmi      32768  1 snd_soc_skl_hda_dsp
snd_soc_dmic           16384  1
snd_hda_codec_hdmi     57344  1
snd_hda_codec_realtek   118784  1
snd_hda_codec_generic    81920  1 snd_hda_codec_realtek
snd_sof_intel_hda_common    73728  1 sof_pci_dev
snd_soc_hdac_hda       24576  1 snd_sof_intel_hda_common
snd_sof_intel_hda      20480  1 snd_sof_intel_hda_common
snd_sof_intel_byt      24576  1 sof_pci_dev
snd_sof_intel_ipc      20480  1 snd_sof_intel_byt
snd_sof                98304  4 snd_sof_intel_hda_common,snd_sof_intel_byt,snd_sof_intel_ipc,sof_pci_dev
snd_sof_xtensa_dsp     16384  1 sof_pci_dev
snd_hda_ext_core       28672  4 snd_sof_intel_hda_common,snd_soc_hdac_hdmi,snd_soc_hdac_hda,snd_sof_intel_hda
snd_soc_acpi_intel_match    32768  2 snd_sof_intel_hda_common,sof_pci_dev
snd_soc_acpi           16384  2 snd_soc_acpi_intel_match,sof_pci_dev
snd_soc_core          241664  6 snd_sof,snd_sof_intel_hda_common,snd_soc_hdac_hdmi,snd_soc_hdac_hda,snd_soc_dmic,snd_soc_skl_hda_dsp
snd_compress           24576  1 snd_soc_core
ac97_bus               16384  1 snd_soc_core
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_hda_intel          53248  0
snd_intel_dspcfg       24576  2 snd_hda_intel,sof_pci_dev
snd_hda_codec         131072  6 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec_realtek,snd_soc_hdac_hda,snd_soc_skl_hda_dsp
snd_hda_core           90112  11 snd_hda_codec_generic,snd_hda_codec_hdmi,snd_hda_intel,snd_hda_ext_core,snd_hda_codec,snd_hda_codec_realtek,snd_sof_intel_hda_common,snd_soc_hdac_hdmi,snd_soc_hdac_hda,snd_sof_intel_hda,snd_soc_skl_hda_dsp
snd_hwdep              20480  1 snd_hda_codec
snd_pcm               102400  11 snd_hda_codec_hdmi,snd_hda_intel,snd_hda_ext_core,snd_hda_codec,snd_sof,snd_sof_intel_hda_common,snd_soc_hdac_hdmi,snd_soc_core,snd_hda_core,snd_pcm_dmaengine
ledtrig_audio          16384  3 snd_hda_codec_generic,snd_hda_codec_realtek,thinkpad_acpi
snd_seq_midi           20480  0
snd_seq_midi_event     16384  1 snd_seq_midi
snd_rawmidi            36864  1 snd_seq_midi
snd_seq                69632  2 snd_seq_midi,snd_seq_midi_event
snd_seq_device         16384  3 snd_seq,snd_seq_midi,snd_rawmidi
snd_timer              36864  2 snd_seq,snd_pcm
snd                    86016  23 snd_hda_codec_generic,snd_seq,snd_seq_device,snd_hda_codec_hdmi,snd_hwdep,snd_hda_intel,snd_hda_codec,snd_hda_codec_realtek,snd_timer,snd_compress,thinkpad_acpi,snd_soc_core,snd_pcm,snd_rawmidi
soundcore              16384  1 snd

Reactions To Comments

Issue 207407

From bugzilla.kernel.org/show_bug.cgi?id=207407 you can test the patch to the file /usr/share/alsa/ucm2/sof-hda-dsp/HiFi.conf (comment #16) – laugeo

This file does not exist on my machine. What does exist however, is a file /usr/share/alsa/ucm/sof-skl_hda_card/HiFi.conf. Notable differences:

• it's ucm instead of ucm2
• it's the card name as reported by aplay -l

The linked diff suggests changing the PlaybackMixerElem and PlaybackVolume entries for "Speaker" and for "Headphones" to PlaybackMixerElem "Master" and PlaybackVolume "Master Playback Volume".
In my file, this is already the case. Here is my complete file:

# Use case Configuration for skl-hda-card

SectionVerb {

    EnableSequence [
        cdev "hw:sofsklhdacard"
    ]

    DisableSequence [
        cdev "hw:sofsklhdacard"
    ]
}

SectionDevice."Speaker" {
    Comment "Speaker"

    EnableSequence [
        cdev "hw:sofsklhdacard"
        cset "name='Speaker Playback Switch' on"
    ]

    DisableSequence [
        cdev "hw:sofsklhdacard"
        cset "name='Speaker Playback Switch' off"
    ]

    Value {
        PlaybackPCM "hw:sofsklhdacard,0"
        PlaybackVolume "Master"
        PlaybackSwitch "Master Playback Switch"
        JackHWMute "Headphone"
        PlaybackChannels "2"
        PlaybackPriority "100"
    }
}

SectionDevice."Headphone" {
    Comment "Headphone"

    EnableSequence [
        cdev "hw:sofsklhdacard"
        cset "name='Headphone Playback Switch' on"
    ]

    DisableSequence [
        cdev "hw:sofsklhdacard"
        cset "name='Headphone Playback Switch' off"
    ]

    Value {
        PlaybackPCM "hw:sofsklhdacard,0"
        PlaybackVolume "Master"
        PlaybackSwitch "Master Playback Switch"
        PlaybackChannels "2"
        JackName "sof-skl_hda_card Headphone"
        JackType "gpio"
        JackSwitch "12"
        JackControl "Headphone Mic Jack"
        PlaybackPriority "200"
    }
}

<ucm/sof-skl_hda_card/Mics.conf>

The statement

I think firmware and EC aren't the issue here, since everything works correctly prior to kernel 5.4.10.

confuses me a bit, since my kernel is currently only 5.4.0 which is before that.

Reboot From Windows

However, since kernel 5.4.10 (as recent as 5.6.6) on Arch, the two speakers on the bottom does not work when the machine is booted directly into Linux. The same issue also exists on Ubuntu 20.04 beta. However, if I boot into Windows 10 first, then reboot into Linux, all speakers work correctly. I'm using the SOF firmware and PulseAudio 13.99.1, but the problem still exists if I boot with 'snd_intel_dspcfg.dsp_driver=1' or 'snd_hda_intel.dmic_detect=0'.

This comment from the bugtracker indicates that updating PulseAudio and upgrading to Ubuntu 20.04 does not resolve the issue. More importantly though, it says that the sound works on all speakers when rebooting from windows into linux instead of booting cold.
I can reproduce that - but on further inspection I'm pretty sure all four speakers have worked for me since the beginning. The problem is just the sound quality and volume.

Issue 208133

On the kernel bugtracker, there is another issue, marked as duplicate. Thinkpad X1 Yoga gen 4 sound issues: no bass, max volume too low, only stereo channels visible to PA

With a Thinkpad X1 Yoga gen 4, Kernel 5.6.14, and PulseAudio 14, The sound situation is as follows:

Good:

• The microphone array works properly out of the box
• The front and rear speakers all produce sound out of the box

Bad:

• The maximum volume is a bit low
• The bass speaker is not working at all
• The speaker system shows up as a stereo device in PulseAudio, not a 4.1 channel audio device (see also https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/914)
• Useless disconnected devices for the DisplayPort and HDMI audio devices are visible to PulseAudio, cluttering up the display (see also https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/913)

Possibly relevant useful: if I set snd_hda_intel.dmic_detect=0 in the kernel parameters, the mic array is no longer detected, but https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/914 and https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/issues/913get fixed.

So I have tried what the effect of snd_hda_intel.dmic_detect=0 is on my laptop. The diff between (set -u && aplay -l && pacmd list-cards) &> out.txt before and after adding this kernel boot argument is hosted with coloring on github
There is quite some change in the profiles: section but I don't know if it's relevant and I'm having a hard time telling whether this modification changed the sound quality at all. It seems to be louder now, but that is hard to tell without an objective measure.
One of the changes that happened is that it is now a HDA Intel PCH instead of alsa.card_name = "sof-hda-dsp".

Contacting The Manufacturer

I have contacted lenovo through their customer support ticket system and their reply was

We are sorry to inform you we can only offer support for devices running Windows operating system.
If the device is working without an issue in Windows we can not provide any further service.

I am disappointed since the lenovo website explicitly states that this laptop is "linux certified" and that in general

Lenovo will also upstream device drivers directly to the Linux kernel, to help maintain stability and compatibility throughout the life of the workstation.
news.lenovo.com

but it seems like the just don't have set up any support team for linux.

(Edit:) I have now received another response from lenovo stating that I could contact their service partners in my country. They gave me two phone numbers which I will call once I get to it.

(Edit:) There is a new post on the bugtracker that I was not yet able to verify.

On kernel 5.8.12 with SOF firmware, the issue seems to be resolved. Big thanks to everyone involved!

I am running Ubuntu 18.04 with encrypted ZFS on / and have now decided that I would like to make my /home directory a separate dataset.

Initial Issue

As per Proper way to create a zfs off an existing directory? I created a new dataset with mount point at /home.

# backup data
sudo mv /home /home-backup
# create empty dataset
sudo zfs create tank/ds1/home # you could already specify the mountpoint here.
# oops, I forgot to set the mountpoint so it is now /tank/ds1/home
sudo zfs set mountpoint=/home tank/ds1/home
# copy data to newly created dataset
sudo rsync -aAX /home-backup/ /home

After reboot I cannot login in the GUI anymore - entering the correct password just throws me back to the login screen. But that's not a big problem: In the Terminal that opens when I press CtrlAltF2 I can login and run sudo zfs mount tank/ds1/home and after that, logging in in the GUI works fine. The problem is just that this dataset is not automatically mounted at boot.

Some advice in the arch wiki advises to use certain services, but I don't know if that also applies to my ubuntu and also I would like to only automount tank/ds1/home (mountpoint /home) and tank/ds1/u18 (mountpoint /, is already automounted somehow) but not tank/ds1 (which only exists for hierarchical purposes) if that can be avoided.

Questions

• How can I automount my home directory dataset at boot?
  "Solved". See the section Solution below.
• Why is my root directory dataset automatically mounted at boot?
  I'm happy that it is, but I am curious why. If it is within the scope of the first question, please answer this too.
• Which datasets get auto-mounted by the zfs mount service?
  The basis for this question is outlined in the section Remaining Question) below.

System Information

$ uname -a
Linux motorbrot 5.3.0-62-generic #56~18.04.1-Ubuntu SMP Wed Jun 24 16:07:03 UTC 2020 x86_64 x86_64 GNU/Linux

$ zfs version
zfs-0.8.4-1
zfs-kmod-0.8.4-1

Listing the zfs datasets right after booting (before temporarily fixing anything) shows the datasets and their mountpoints and they are as expected. Using zfs get all tank/ds1/home | grep mount and similar commands shows that all are canmount on but only tank/ds1/u18 is mounted (on /).

My /etc/fstab contains only my swap partition and my EXT2 /boot partition.

My /etc/default/zfs contains ZFS_MOUNT='yes' but that seems to have no effect, since not all datasets are being mounted. That can perhaps be explained by the comment Changing some of these settings may not make any difference on systemd-based setup, e.g. setting ZFS_MOUNT=no will not prevent systemd from launching zfs-mount.service during boot..

$ systemctl list-unit-files | grep zfs
zfs-import.service         generated
zfs-mount.service          generated
zfs-share.service          generated
zfs-zed.service            masked

Solution

Following How can I make ZFS datasets mount automatically on boot I am informed that zfs.target does not exist and that both zfs-import.service and zfs-mount.service are "not native services, redirecting to systemd-sysv-install".
After running these commands, the output of systemctl list-unit-files | grep zfs is still the same. I would have expected the status to change to enabled.

However, after a reboot the login works again - /tank/ds1/home is mounted correctly to /home.

Now that I am finding this question again, a year later, having the exact same problem with a new encrypted dataset, this "solution" is not helping anymore.

Remaining Question

The outputs of zfs get all DATASET | grep mount show that tank and tank/ds1 are not mounted after boot while tank/ds1/u18 and tank/ds1/home are mounted on their respective mountpoints. That's great! But why are only those two automatically mounted? It's exactly as it should be, but I fail to see why it is behaving that way.

So what I am asking in the end is: Please explain to me how ubuntu decides which zfs datasets are automatically mounted during boot. How could I specify specific datasets to be automatically mounted?

How can I automount a specific encrypted dataset at boot?

Backstory

I followed this answer about how to enable hibernation two months ago and it worked. As documented in my question on How to resume Ubuntu 18.04 after hibernation it worked the first time, but not after a complete reinstall.

xenoid kindly pointed out a generally helpful article on the ubuntu wiki.

Dead, blank, or black screen on resume In some cases, a machine can hibernate just fine, and resume without issue, with the exception of waking up to a blacked-out screen. In other words, the computer is running just fine, but the display appears dead.

They recommend adding nomodeset to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub.

Hibernation worked. Sometimes the laptop woke itself up, but lenovo has addressed an issue like that in the latest bios and fimware updates now.

Then I realized that I had sound problems. I first found one fix that broke my brightness keys in turn and then undid it when I found out that sound and brightness keys worked flawlessly on a clean install. nomodeset was the culprit.

So now I am back to square one regarding hibernation. Or, you would think so. But actually, I'm even worse off than before: In How to resume Ubuntu after hibernation it seemed like the hibernate part worked and the resume part did not.

Problem Statement

When I run sudo systemctl hibernate or sudo pm-hibernate the laptop shuts down. Closing and Reopening the lid does start it again, but like a reboot it brings me to the grub menu instead of restoring my session as it was.

Testing

kernel.org states that one can debug hibernation issues using this snippet

# echo devices > /sys/power/pm_test
# echo platform > /sys/power/disk
# echo disk > /sys/power/state

with one of those options

freezer
test the freezing of processes
devices
test the freezing of processes and suspending of devices
platform
test the freezing of processes, suspending of devices and platform global control methods [1]
processors
test the freezing of processes, suspending of devices, platform global control methods [1] and the disabling of nonboot CPUs
core
test the freezing of processes, suspending of devices, platform global control methods[1], the disabling of nonboot CPUs and suspending of platform/system devices

[1] (1, 2, 3) the platform global control methods are only available on ACPI systems and are only tested if the hibernation mode is set to “platform”

They state that this is only available on kernels compiled with CONFIG_PM_DEBUG but I assume this is the case for me since the file /sys/power/pm_test exists.

root@motorbrot:~# echo freezer > /sys/power/pm_test
root@motorbrot:~# echo disk > /sys/power/state
## results in a short freeze of the laptop

root@motorbrot:~# echo devices > /sys/power/pm_test
root@motorbrot:~# echo disk > /sys/power/state
## results in a black screen for a few seconds. Power Button LED stays on. Then screen turns back on.

root@motorbrot:~# echo platform > /sys/power/pm_test
root@motorbrot:~# echo disk > /sys/power/state
## takes longer than before until the screen turns dark, otherwise the same behaviour as with the "devices" test.

root@motorbrot:~# echo processors > /sys/power/pm_test
root@motorbrot:~# echo disk > /sys/power/state
## nothing noticeably different to the previous two tests

root@motorbrot:~# echo core > /sys/power/pm_test
root@motorbrot:~# echo disk > /sys/power/state
## Again nothing new. The screen turns dark, takes a moment, turns on again.

I interpret those results as "all tests succeed".

However, there is a log file in /var/crash/susres.2020-06-08_17:46:27.952596.crash. Either from when I last actually ran hibernate, or from the "devices" test. It starts with

ProblemType: KernelOops
Annotation: This occurred during a previous hibernation, and prevented the system from resuming properly.
Architecture: amd64
Date: Mon Jun  8 17:46:27 2020
DistroRelease: Ubuntu 18.04
ExecutablePath: /usr/share/apport/apportcheckresume
ExecutableTimestamp: 1589407934
Failure: hibernate/resume
InterpreterPath: /usr/bin/python3.6
Package: linux-image-5.3.0-53-generic 5.3.0-53.47~18.04.1

LED

Looks fine. One last question, now that it works, what does the Power button LED indicates when you hibernate? Completely off? dimmed? "Breathing"? – xenoid Mar 19 at 20:07

@xenoid it blinks twice short, once long, three times short and then it remains off like everything else (apart from the "I'm charging!-led" – lucidbrot Mar 19 at 21:07

This quote form the last time I asked about the hibernation problem no longer applies. Currently, the power button LED stays on for a while after systemctl hibernate, blinks twice, then is on for a while, and then off like after a shutdown.

Does systemctl suspend work?

When I do systemctl suspend the power button LED goes into breathing mode and the screen turns dark. Touching the trackpad or clicking, or typing does not cause any reaction.
Closing and reopening the lid works: The display turns back on and presents the login screen.

System Info

When I started writing this question, I was on

root@motorbrot:~# uname -a
Linux motorbrot 5.3.0-53-generic #47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

root@motorbrot:~# swapon --show
NAME           TYPE       SIZE USED PRIO
/dev/nvme0n1p6 partition 18.6G   0B   -2

I have since upgraded to the 5.3.0-59 kernel, but the behaviour remains the same.

/etc/fstab and friends

Perhaps that is helpful?

generic@motorbrot:~$ blkid
/dev/nvme0n1p1: LABEL="Recovery" UUID="885E1DE35E1DCAB8" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="2aac3226-cbc2-46d4-9bd4-5ce4695533fc"
/dev/nvme0n1p2: UUID="C21E-C17B" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="beb37963-eae9-4a51-a29e-8850fd392a7f"
/dev/nvme0n1p5: UUID="81e29369-ff69-4424-858c-3489283588d7" TYPE="ext2" PARTUUID="f68aad7f-bfc7-404d-b64b-072ee19d56dd"
/dev/nvme0n1p6: UUID="1ba104f0-35be-42f7-bf71-65e43f6fbcc3" TYPE="swsuspend" PARTUUID="1a651275-5ad0-4ead-9bdf-66e5e9a28b85"
/dev/nvme0n1p7: UUID="d2a1f26d-4442-4eae-9a62-2c176268208d" TYPE="ext2" PARTUUID="d296b2e6-2f0e-4393-9b54-6ed0d11f22ac"
/dev/nvme0n1p8: UUID="003fdbc2-f3fe-4e5d-a945-c2568048589a" TYPE="ext4" PARTUUID="503fddfe-0898-4ed1-a43e-1866a29abd8f"
/dev/nvme0n1p9: LABEL="aquarium" UUID="13317063534776643493" UUID_SUB="8783866549508142042" TYPE="zfs_member" PARTUUID="9bdfdcf4-f586-4603-90fb-a51ed366de3e"
/dev/nvme0n1p10: LABEL="tank" UUID="2491194531467516534" UUID_SUB="922708726245396109" TYPE="zfs_member" PARTUUID="77970109-18ef-4ebd-b49d-6ae04c88aeaa"

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/nvme0n1p8 during installation
# but that was before I moved it over to zfs
#UUID=003fdbc2-f3fe-4e5d-a945-c2568048589a /               ext4    errors=remount-ro 0       1
# I haven't specified the zfs volume yet because I'm not sure I'd need it... in fact I don't need it, as proven by usage.
#
# /boot is its own partition distinct from /boot/efi, but for some reason it was not set up correctly, it seems. Trying to fix that now with the following line:
UUID=81e29369-ff69-4424-858c-3489283588d7  /boot           ext2    defaults        0       2
# /boot/efi was on /dev/nvme0n1p2 during installation
UUID=C21E-C17B  /boot/efi       vfat    umask=0077      0       1
# Use swap partition instead of swap file
#/swapfile                                 none            swap    sw              0       0
UUID=1ba104f0-35be-42f7-bf71-65e43f6fbcc3  none            swap    sw              0       0

I have a custom grub menuentry because generating it with update-grub fails for the encrypted zfs partition (that's a different problem which seems to be a bug with the update-grub script, not my current question).

#/etc/grub.d/40_custom
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.


menuentry "Encryptioned Magic 18.04" --id encryptioned-magic-id {
setparams 'Ubuntu'

    recordfail
    load_video
    gfxmode $linux_gfx_mode
    insmod gzio
    if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
    insmod part_gpt
    insmod ext2
     # not needed. because grub has already set its root correctly to that partition(hd0,gpt5).
     # but also not harmful.
     # Find the /boot partition
     if [ x$feature_platform_search_hint = xy ]; then
       search --no-floppy --fs-uuid --set=root 81e29369-ff69-4424-858c-3489283588d7 
     else
       search --no-floppy --fs-uuid --set=root 81e29369-ff69-4424-858c-3489283588d7 
     fi
    # The relevant part
    # nomodeset was for (hopefully) fixing hibernation startup problems where screen stayed black, but I removed it again since it caused sound and brightness problems.
    # resume=SWAP-PARTITION-UUID-AS-PER-lsblk is for restoring from swap space after hibernation
    # acpi_backlight=video fixed my screen brightness keys, but no longer needed since I removed nomodeset
    linux /vmlinuz-5.3.0-59-generic root=ZFS=tank/ds1/u18 ro resume=1ba104f0-35be-42f7-bf71-65e43f6fbcc3
    initrd /initrd.img-5.3.0-59-generic
}

policy kit in local authority

As recommended in the answer about how to enable hibernation

#/etc/polkit-1/localauthority/50-local.d/com.ubuntu.enable-hibernate.pkla
[Re-enable hibernate by default in upower]
Identity=unix-user:*
Action=org.freedesktop.upower.hibernate
ResultActive=yes

[Re-enable hibernate by default in logind]
Identity=unix-user:*
Action=org.freedesktop.login1.hibernate;org.freedesktop.login1.handle-hibernate-key;org.freedesktop.login1;org.freedesktop.login1.hibernate-multiple-sessions;org.freedesktop.login1.hibernate-ignore-inhibit
ResultActive=yes

[Enable hibernate to be run via cron]
Identity=unix-user:*
Action=org.freedesktop.login1.hibernate;org.freedesktop.login1.hibernate-multiple-sessions
ResultAny=yes

Output from update-initramfs -u -k all

generic@motorbrot:~$ sudo update-initramfs -u -k all
update-initramfs: Generating /boot/initrd.img-5.3.0-59-generic
I: The initramfs will attempt to resume from /dev/nvme0n1p6
I: (UUID=1ba104f0-35be-42f7-bf71-65e43f6fbcc3)
I: Set the RESUME variable to override this.
update-initramfs: Generating /boot/initrd.img-5.3.0-53-generic
I: The initramfs will attempt to resume from /dev/nvme0n1p6
I: (UUID=1ba104f0-35be-42f7-bf71-65e43f6fbcc3)
I: Set the RESUME variable to override this.
update-initramfs: Generating /boot/initrd.img-5.3.0-51-generic
I: The initramfs will attempt to resume from /dev/nvme0n1p6
I: (UUID=1ba104f0-35be-42f7-bf71-65e43f6fbcc3)
I: Set the RESUME variable to override this.

Output from update-grub

The device-mapper issue is (as mentioned earlier) likely due to me using an encrypted zfs root and I circumvent that issue by using a custom grub entry.

generic@motorbrot:~$ sudo update-grub
Sourcing file `/etc/default/grub'
Generating grub configuration file ...
Found theme: /boot/grub/themes/poly-dark/theme.txt
Found linux image: /boot/vmlinuz-5.3.0-59-generic
Found initrd image: /boot/initrd.img-5.3.0-59-generic
Found linux image: /boot/vmlinuz-5.3.0-53-generic
Found initrd image: /boot/initrd.img-5.3.0-53-generic
Found linux image: /boot/vmlinuz-5.3.0-51-generic
Found initrd image: /boot/initrd.img-5.3.0-51-generic
Found linux image: /boot/vmlinuz-4.15.0-101-generic
Found initrd image: /boot/initrd.img-4.15.0-101-generic
device-mapper: reload ioctl on osprober-linux-nvme0n1p10  failed: Device or resource busy
Command failed
Found Windows Boot Manager on /dev/nvme0n1p2@/EFI/Microsoft/Boot/bootmgfw.efi
Found Ubuntu 18.04.4 LTS (18.04) on /dev/nvme0n1p8
Adding boot menu entry for EFI firmware configuration
done

Question

How do I fix this or how do I go about finding out where the problem lies? The end goal is to be able to write the ram to the swap partition and resume right where I was after hibernation.

Edit (14. Jul - One month after posting this question)

It's working now:

• systemctl hibernate works correctly from a running system without nomodeset to a rebooted system without nomodeset.
• systemctl hibernate works correctly from a running system without nomodeset to a rebooted system with nomodeset.
• The sound and brightness keys work in both cases.

I hope this was fixed in a kernel update. My current system information is

$ uname -a
Linux motorbrot 5.3.0-62-generic #56~18.04.1-Ubuntu SMP Wed Jun 24 16:17:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

The Question remains How did it become fixed and how would one go about that if it were to happen again?

generic@motorbrot:~$ sudo dkms status
zfs, 0.8.4, 5.3.0-51-generic, x86_64: installed (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!)
zfs, 0.8.4, 5.3.0-53-generic, x86_64: installed (WARNING! Diff between built and installed module!) (WARNING! Diff between built and installed module!)

Why does this message appear?

I have tried removing and reinstalling zfs on one of these two kernels.

generic@motorbrot:~$ sudo dkms remove -m zfs -v 0.8.4 -k 5.3.0-53-generic

-------- Uninstall Beginning --------
Module:  zfs
Version: 0.8.4
Kernel:  5.3.0-53-generic (x86_64)
-------------------------------------

Status: Before uninstall, this module version was ACTIVE on this kernel.

zavl.ko:
 - Uninstallation
   - Deleting from: /lib/modules/5.3.0-53-generic/
rmdir: failed to remove '': No such file or directory
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.

# [skipping long output that all looks similar but for different something.ko]

spl.ko:
 - Uninstallation
   - Deleting from: /lib/modules/5.3.0-53-generic/updates/dkms/
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.

depmod...

DKMS: uninstall completed.
generic@motorbrot:~$ sudo dkms install -m zfs -v 0.8.4 -k 5.3.0-53-generic

Kernel preparation unnecessary for this kernel.  Skipping...

Running the pre_build script:

# [Skipping "checking for blah... blah" outputs]

Building module:
cleaning build area...
make -j8 KERNELRELEASE=5.3.0-53-generic............

Running the post_build script:
Signing module:
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/spl.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/icp.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/zfs.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/zcommon.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/zlua.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/znvpair.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/zunicode.ko
 - /var/lib/dkms/zfs/0.8.4/5.3.0-53-generic/x86_64/module/zavl.ko
Secure Boot not enabled on this system.
cleaning build area...

DKMS: build completed.

zavl.ko:
Running module version sanity check.

Good news! Module version 0.8.4-1 for zavl.ko
exactly matches what is already found in kernel 5.3.0-53-generic.
DKMS will not replace this module.
You may override by specifying --force.

znvpair.ko:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/5.3.0-53-generic/updates/dkms/

zunicode.ko:
Running module version sanity check.

Good news! Module version 0.8.4-1 for zunicode.ko
exactly matches what is already found in kernel 5.3.0-53-generic.
DKMS will not replace this module.
You may override by specifying --force.

zcommon.ko:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/5.3.0-53-generic/updates/dkms/

zfs.ko:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/5.3.0-53-generic/updates/dkms/

icp.ko:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/5.3.0-53-generic/updates/dkms/

zlua.ko:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/5.3.0-53-generic/updates/dkms/

spl.ko:
Running module version sanity check.
 - Original module
 - Installation
   - Installing to /lib/modules/5.3.0-53-generic/updates/dkms/

depmod...

DKMS: install completed.

but the output of dkms status remains the same.

Why does this happen? Should I worry? And if I should worry, how would I go about fixing this?

The sound on my ThinkPad Yoga X1 used to work a while ago. I'm not sure what changed, but now it doesn't work anymore. It still works continued to work for some time on the parallel installation of ubuntu 18.04 (yes, I have the same distro installed twice). It does still work on the parallel installation of windows 10.

When I press the volume buttons on my keyboard, it changes the volume of a "Dummy Output" device.

lspci lists the audio device:

$ lspci | grep -A8 Audio
00:1f.3 Audio device: Intel Corporation Device 9dc8 (rev 11)
00:1f.4 SMBus: Intel Corporation Device 9da3 (rev 11)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Device 9da4 (rev 11)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (6) I219-LM (rev 11)
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 2263 (rev 03)
05:00.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
06:00.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
06:01.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)
06:02.0 PCI bridge: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] (rev 02)

But aplay does not find it

sudo aplay -l
[sudo] password for generic: 
aplay: device_list:270: no soundcards found...

$ pacmd list-sinks
1 sink(s) available.
  * index: 0
    name: <auto_null>
    driver: <module-null-sink.c>
    flags: DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
    state: SUSPENDED
    suspend cause: IDLE 
    priority: 1000
    volume: front-left: 46332 /  71% / -9.04 dB,   front-right: 46332 /  71% / -9.04 dB
            balance 0.00
    base volume: 65536 / 100% / 0.00 dB
    volume steps: 65537
    muted: no
    current latency: 0.00 ms
    max request: 344 KiB
    max rewind: 344 KiB
    monitor source: 0
    sample spec: s16le 2ch 44100Hz
    channel map: front-left,front-right
                 Stereo
    used by: 0
    linked by: 0
    configured latency: 0.00 ms; range is 0.50 .. 2000.00 ms
    module: 14
    properties:
        device.description = "Dummy Output"
        device.class = "abstract"
        device.icon_name = "audio-card"

I am running ubuntu 18.04 with ZFS on root. That means I can try a solution and roll back if it did not work. Which means I tried several solutions.

Related links that did not help:

• Ubuntu 18.04 soundcard not detected

• Sound card not detecting

• https://www.linuxuprising.com/2018/06/fix-no-sound-dummy-output-issue-in.html?m=1

• https://askubuntu.com/a/815516/606260

• No Sound (Dummy Output) error Ubuntu 19.10

• Sound card not detected on LTS 18.04

• No sound (dummy output) Ubuntu 19.10

• No sound on Ubuntu 19.10. Dummy output

• https://askubuntu.com/a/1027405/606260

• https://askubuntu.com/a/811461/606260

What I tried:

• sudo apt-get --purge remove linux-sound-base alsa-base alsa-utils && reboot

• sudo apt-get install build-essential linux-headers-uname -ralsa-base alsa-firmware-loaders alsa-oss alsa-source alsa-tools alsa-tools-gui alsa-utils alsamixergui

• adding the user generic to the group audio

• the three above combined

• a variation of the above

• both fixes in the linuxuprising post

• modprobe snd_hda_intel && modprobe sof_pci_dev

• Adding snd_hda_intel.dmic_detect=0 to the GRUB boot arguments

• Installing a collection of packages from ppa:nilarimogard/webupd8 as advised here

• Reinstalling alsa-related packages, using dkms. See here

• Reinstalling pavucontrol and pulseaudio as per this QA

My system is

$ uname -r
5.3.0-51-generic

Possibly unrelated, possibly related:
My screen brightness controls have disappeared as well. They were broken once before, and I fixed that by adding acpi_backlight=video to the GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and updating grub and rebooting. But that fix is still in place and still it is broken again.

How can I get the sound working again?

When I boot off a live stick of ubuntu 18.04, the sound does work again.

# live system version:
$ uname -a
Linux ubuntu 5.3.0-28-generic #30-18.04.1-Ubuntu SMP Fri Jan 17 06:14:09 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

# aplay output on live system:
ubuntu@ubuntu:~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC285 Analog [ALC285 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 7: HDMI 1 [HDMI 1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 8: HDMI 2 [HDMI 2]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 9: HDMI 3 [HDMI 3]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 10: HDMI 4 [HDMI 4]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

(On the non-live system)/var/log/apt/term.log and /var/log/apt/history.log and /var/log/dpkg.log are all empty when viewed with sudo vim <filepath>. But there is a file history.log.1.gz which contains all the history. I'm reviewing it, but I don't know what to look out for.

I have booted off the oldest zfs snapshot I currently have on the laptop (not incredibly old, due to me getting complications with zfs back then) and the sound does not work there either. It also does not work when I boot with the older kernel version 5.3.0-28 that is the same as on the live usb stick.

git diff --no-index apt_live.txt apt_tank.txt where the files contain the respective outputs of apt list --installed gives me too much output. Many version numbers have changed.

I have added the tags 19.04 and 20.04 in retrospect since the fix applies to those as well, according to the sources.

I'm running Ubuntu 18.04 on ZFS 0.8.4 and am playing around with encryption. I'd like to have a menuentry in grub that lets me boot to the encrypted root.

System Setup

A checkmark indicates that there is a corresponding menu entry in grub.

• aquarium/ds1/u18 ✓
  contains my ubuntu 18.04 and has its mountpoint set to /
• tank/ds1/u18
  is copied from aquarium with the sole difference being that ds1 is encrypted and u18 inherits that property.
• Windows 10 ✓
  I hope it doesn't have anything to do with my issues though.
• /dev/nvme0n1p8 ✓
  An EXT4-formatted system that is still there from when I switched to the ZFS aquarium

I have an EXT2 partition for /boot and another partition for /boot/efi.

Manual Workarounds

When I select and edit the aquarium entry by replacing root=ZFS=aquarium/ds1/u18 with root=ZFS=tank/ds1/u18 I can boot fine into initramfs where I am prompted for the passphrase. After entering the passphrase, I get my system as expected.

Inside that system (or any other ubuntu system for that matter), I can create a custom entry in /etc/grub/40_custom.

menuentry "Encryptioned Magic 18.04" {
    linux /vmlinuz-5.3.0-51-generic root=ZFS=tank/ds1/u18 ro acpi_backlight=video resume=UUID-OF-SWAP-PARTITION
    initrd /initrd.img-5.3.0-51-generic
}

I'd like to avoid this, because a manual entry means I need to modify it when the UUID of my swap filesystem changes, or something else. I mean, all the reasons why we don't create custom entries for every OS basically also apply here.

Solution Attempt

Ideally, this would use sudo update-grub, but even with the encryption key loaded or even when I am within the encrypted system, it doesn't seem to find it. This log is from running it in aquarium.

$ sudo update-grub
Sourcing file `/etc/default/grub`
Generating grub configuration file ...
Found theme: /boot/grub/themes/poly-dark/theme.txt
Found linux image: /boot/vmlinuz-5.3.0-51-generic
Found initrd image: /boot/initrd.img-5.3.0-51-generic
Found linux image: /boot/vmlinuz-5.3.0-28-generic
Found initrd image: /boot/initrd.img-5.3.0-28-generic
Found Windows Boot Manager on /dev/nvme0n1p2@/EFI/Microsoft/Boot/bootmgfw.efi
Found Ubuntu 18.04.4 LTS (18.04) on /dev/nvme0n1p8
Adding boot menu entry for EFI firmware configuration
done

When I run this same command in the encrypted tank system, I get additionally a message between the second Found initrd image and the Found Windows Boot Manager lines:

device-mapper: reload ioctl on osprober-linux-nvme0n1p10   failed: Device or resource busy
Command failed

The mentioned partition nvme0n1p10 is the single partition where the pool tank with the encrypted dataset tank/ds1/u18 is set up on.

This same error message seems to happen whenever zfs is using a partition when I run update-grub since it opens it for exclusive access, no matter which system I'm logged into. This behaviour is a confirmed bug in os-prober but they call it

not actually failing, just spewing errors during os-prober. We want to clean it up (it is cosmetic).

How can I autogenerate a grub entry for my encrypted zfs dataset?

The man zpool create page states

-O file-system-property = value
Sets the given file system properties in the root file system of the pool. See the Sx Properties section of zfs(8) for a list of valid properties that can be set.

Searching the man zfs page for Sx Properties leads me to

set property = value [property = value ... ] filesystem | volume | snapshot ...

Sets the property or list of properties to the given value(s) for each dataset.
Only some properties can be edited. See the Sx Properties section for more information on what properties can be set and acceptable values.
[...]

I saw this on the alpinelinux wiki when I was looking for how to do something similar on ubuntu 18.04 with zfs 0.8.4 (self-built).

What do the big-O options actually do?

I have experimented and created a new pool with -O compression=lz4 and tested a few hypotheses of what they might do, but:

• zfs get compression tank does not show that compression is enabled

• zfs create tank/ds does not create a dataset with compression activated by default.

So what do the file-system-property arguments to zpool create actually impact?

Edit

I have tried again and this time everything behaved as I expected.

Creating a pool with -O compression=lz4 does have the effect that zfs get compression mypool shows lz4. And it is inherited to future created child datasets.

Similar to this question from 2012, I'm looking for a way to automate the setup of a key combinations that should trigger the execution of a command the same way as setting it in the gui does. (example usage)

I'm on Ubuntu 18.04 and the linked answer's gsettings schema does not seem to exist:

$ gsettings list-schemas | grep keyboard
org.gnome.settings-daemon.plugins.keyboard
org.gnome.settings-daemon.plugins.a11y-keyboard
org.gnome.shell.keyboard
org.gnome.settings-daemon.peripherals.keyboard
org.gnome.libgnomekbd.keyboard
com.canonical.indicator.keyboard
org.gnome.desktop.peripherals.keyboard
org.gnome.desktop.a11y.keyboard

How can I use bash to set it up as if I went in the GNOME GUI to Settings > Devices > Keyboard and set it up manually?

After struggling quite some time with figuring out how preseeding on Ubuntu 18.04 Desktop works, I figured it out. I was hoping that I could use the same procedure for 20.04 Desktop. Can I?

The ubuntu wiki page on Automated Server Installs for 20.04 sounds like it has been removed in favor of a new, and hence different, mode of operation:

The server installer for 20.04 supports a new mode of operation: automated installation, autoinstallation for short. You might also know this feature as unattended or handsoff or preseeded installation. [...]

preseeds are the way to automate an installer based on debian-installer (aka d-i).

autoinstalls for the new server installer differ from preseeds in the following main ways:

• the format is completely different (cloud-init config, usually yaml, vs debconf-set-selections format)
• when the answer to a question is not present in a preseed, d-i stops and asks the user for input. autoinstalls are not like this: by default, if there is any autoinstall config at all, the installer takes the default for any unanswered question (and fails if there is no default). You can designate particular sections in the config as "interactive", which means the installer will still stop and ask about those.

Questions

1. Can I use the old procedure of preseeding with a preseed file with d-i/ubiquity directives?
2. Does this new "autoinstall" work only on Ubuntu 20.04 Server or also on Ubuntu 20.04 Desktop?
3. Or if the previous answers are both "No, that doesn't work on Ubuntu 20.04 Desktop.", what is the recommended way to automate an installation of Ubuntu 20.04 Desktop?

dconf read /org/gnome/desktop/input-sources/sources

has the same output as

dconf read /org/gnome/desktop/input-sources/mru-sources

which is

[('xkb', 'ch+de_nodeadkeys'), ('xkb', 'us')]

What is the difference?
Should I always modify both? If not, when/why not?

The example file for preseeding bionic as linked to here contains both

# Individual additional packages to install
#d-i pkgsel/include string openssh-server build-essential ```

and

# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.
#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh ```

How do I decide whether I want to use the preseed/late_command version or the pkgsel/include version?

My question What is the difference between apt-install and in-target apt-get install? is related but complementary. I expect it to be separate enough of a question.

The ubuntu help on preseeding links to an example file which contains this:

# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.
#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh

Is there a difference between using apt-install zsh and in-target apt-get install zsh?

I have a dual-boot setup on my laptop: Ubuntu 18.04 and Windows 10. From within ubuntu, I can boot to my existing windows partition using qemu with the help of this guide. The command I am using is this:

sudo qemu-system-x86_64 --enable-kvm -cpu host -smp 8 -m 8192 \
-drive format=raw,file=/dev/nvme0n1,index=1 \
-bios /usr/share/ovmf/OVMF.fd \
-display sdl -vga qxl \
-device qemu-xhci,id=xhci \
-netdev user,id=vmnic,smb=/home/generic/Downloads \
-device virtio-net,netdev=vmnic -cdrom ~/Downloads/virtio-win-0.1.171.iso

The last line was only to install the virtio drivers so I'm pretty sure I don't need it every time.
The second line determines the partition /dev/nvme0n1p1.

The guide mentions SPICE, which is apparently good for

dynamic screen resize, clipboard between guest/host, etc.

I'd like to have the virtual screen change its size and adapt the resolution when I resize the window. I tried to follow the guide on https://www.linux-kvm.org/page/SPICE and realized that spicec is no longer a thing - that means I cannot connect to the guest with spicec -h 127.0.0.1 -p 5900.
The Spice User Manual agrees:

The recommended client is remote-viewer (which is shipped with virt-viewer).
GNOME Boxes can also be used as a Spice client.
spicec is an obsolete legacy client, and spicy is only a test application.

So I had a go with adding -spice port=5900,addr=127.0.0.1,disable-ticketing to my qemu command and running remote-viewer on spice://127.0.0.1:5009. That actually opened a spice window - so now I have two windows that show my win10 guest.
The Spice window reacts more nicely when it is resized, but it keeps the same resolution and just adds white borders when necessary.

The question

How can I get a borderless view of my guest system that adapts well to being resized?
And if that is not a completely distinct question: How can I have only one window for that, instead of both the qemu and the spice window?

Further Details

When I connected remote-viewer, its console window showed

** (remote-viewer:3066): CRITICAL **: 15:54:52.441: virt_viewer_util_extract_host: assertion 'uri != NULL' failed

And the qemu output showed

main_channel_link: add main channel client
main_channel_client_handle_pong: net test: latency 0.110000 ms, bitrate 34711864406 bps (33103.813559 Mbps)
inputs_connect: inputs channel client create
red_qxl_set_cursor_peer:

Is there a keyboard shortcut to send a window into a different workspace, while remaining in the current workspace, so that I can ignore it for now?

I'm using a fresh install of Ubuntu 18.04 with the default GNOME interface. On win10, I am used to sending a windows to a different workspace so that it won't bother me for now, thanks to virtualdesktopenhancer.

This question is exactly the same, except that it's not about GNOME but about XFCE. I have not tried whether the answers there work also for GNOME, because I'm hoping for an easier answer.
If there are no easier answers, I would appreciate an answer that also explains how to set up such a keyboard shortcut.

This question is an exact duplicate on a sister site - without valid answers. I'm asking here because this is specific to ubuntu.