Answering

I gave it some time from originally asking my question to see if there were any other solutions, but it looks like the solution I came up with while waiting is the only viable solution I've seen so far.

Misconception

Since this was the source of a lot of confusion, I will try to clear it up. It seems that some of the answers tried to use the Ubuntu 16 Server image when I was specifically trying to create an unattended Ubuntu 16 Desktop image install. The issue arises because of the implementation differences of d-i (debian installer) and ubiquity. Since the Server image will pay attention and use all of the d-i commands in the preseed file, most of the questions I asked are irrelevant to the Server image. However, due to the implementation of ubiquity as the installer of the Desktop image a lot of d-i commands are ignored and you are very limited and a lot of documentation is missing.

Documentation Links Found

• Here is the link for Ubiquity ignoring the preseed/late_command (I believe it ignores the preseed/early_command too)
• Here is the Ubiquity Documentation it talks about which installer components will not be used in Ubiquity, but as you will notice even in this documentation it says the preseed/early_command but I would test that thoroughly to verify since it didn't seem to work for me (I admit I didn't test the early_command rigorously so I may be wrong).

Successful Process

Here is my process for successfully creating an unattended Ubuntu Desktop 16.04 LTS iso.

Mount Ubuntu ISO

You will need to mount the ISO files so that you can edit the pertinent files.

mkdir -p /mnt/iso
mount -o loop ~/Downloads/ubuntu-16.04.1-desktop-amd64.iso /mnt/iso

Copy ISO Files

We will need to copy the files in mounted ISO to a different directory so that we can edit them. Feel free to use any directory you like, I chose the /opt directory due to another how-to, but /tmp could just as easily be used.

mkdir -p /opt/ubuntuiso
cp -rT /mnt/iso /opt/ubuntuiso

Edit the txt.cfg File

Here we will edit the /opt/ubuntuiso/isolinux/txt.cfg file and customize our boot parameters to get a completely unattended install which will include a preseed file. Use any editor of your choice:

#default live
#label live
#  menu label ^Try Ubuntu without installing
#  kernel /casper/vmlinuz.efi
#  append  file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash ---
#label live-install
#  menu label ^Install Ubuntu
#  kernel /casper/vmlinuz.efi
#  append  file=/cdrom/preseed/ubuntu.seed boot=casper only-ubiquity initrd=/casper/initrd.lz quiet splash ---
#label check
#  menu label ^Check disc for defects
#  kernel /casper/vmlinuz.efi
#  append  boot=casper integrity-check initrd=/casper/initrd.lz quiet splash ---
#label memtest
#  menu label Test ^memory
#  kernel /install/mt86plus
#label hd 
#  menu label ^Boot from first hard disk
#  localboot 0x80

default live-install
label live-install
  menu label ^Install Ubuntu
  kernel /casper/vmlinuz.efi
  append  file=/cdrom/ks.preseed auto=true priority=critical debian-installer/locale=en_US keyboard-configuration/layoutcode=us ubiquity/reboot=true languagechooser/language-name=English countrychooser/shortlist=US localechooser/supported-locales=en_US.UTF-8 boot=casper automatic-ubiquity initrd=/casper/initrd.lz quiet splash noprompt noshell ---

Please take note of a few things:

• I commented out all original text in the file.
• I added all text after and including "default live-install" at the bottom
• I named the preseed file "ks.preseed" and it will be in the top level directory of the ISO (/opt/ubuntuiso)

Use or Create a Preseed File

Use an already existing preseed file with caution! I have not found any that just work. This isn’t to say they don’t exist, I just didn’t find any with quite a bit of searching. There are many ways to configure a preseed file, but I found many options to be ignored, outlined in the Ubiquity Documentation I linked above and in the link for preseed/late_command vs ubiquity/success_command. I am including my simple working preseed file that works with the above txt.cfg file.

For netcfg/get_hostname string and netcfg/get_domain string you can put whatever you want it to be. I used unassigned-hostname and unassigned-domain since I will just change it later through a scripting process.

For any custom command you want to run after installation you need to use the:

ubiquity ubiquity/success_command string

Followed by any command you want to run. Pay attention to continuing the strings with ";\" and using the "/target" to change anything pertaining to the new system installed.

# Partitioning
# Old style using d-i command
#d-i partman-auto/disk string /dev/sda
#d-i partman-auto/method string regular
#d-i partman-lvm/device_remove_lvm boolean true
#d-i partman-md/device_remove_md boolean true
#d-i partman-auto/choose_recipe select atomic

# Newer ubiquity command
ubiquity partman-auto/disk string /dev/sda
ubiquity partman-auto/method string regular
ubiquity partman-lvm/device_remove_lvm boolean true
ubiquity partman-md/device_remove_md boolean true
ubiquity partman-auto/choose_recipe select atomic

# This makes partman automatically partition without confirmation
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

# Locale
d-i debian-installer/locale string en_US
d-i console-setup/ask_detect boolean false
d-i console-setup/layoutcode string us

# Network
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/choose_interface select auto

# Clock
d-i clock-setup/utc-auto boolean true
d-i clock-setup/utc boolean true
d-i time/zone string US/Pacific
d-i clock-setup/ntp boolean true

# Packages, Mirrors, Image
d-i mirror/country string US
d-i apt-setup/multiverse boolean true
d-i apt-setup/restricted boolean true
d-i apt-setup/universe boolean true

# Users
d-i passwd/user-fullname string User
d-i passwd/username string user
d-i passwd/user-password-crypted password yourEncryptedPasswd
d-i passwd/user-default-groups string adm audio cdrom dip lpadmin sudo plugdev sambashare video
d-i passwd/root-login boolean true
d-i passwd/root-password-crypted password rootEncryptedPasswd
d-i user-setup/allow-password-weak boolean true

# Grub
d-i grub-installer/grub2_instead_of_grub_legacy boolean true
d-i grub-installer/only_debian boolean true
d-i finish-install/reboot_in_progress note

# Custom Commands
ubiquity ubiquity/success_command string \
  sed -i -e 's/dns=dnsmasq/#dns=dnsmasq/' /target/etc/NetworkManager/NetworkManager.conf ;\
  cp -a /cdrom/scripts/ /target/root/ ;\
  cp -a /cdrom/salt/ /target/root/

Please note these things as I left them in for illustrative purposes and they will likely be different in your preseed command.

• The user/root password need to be added by you. Here is the link that shows you 3 different ways to make a crypt 3 password.
• You may want to change the groups your user is assigned to.
• You will definitely want to change the success_command. I left it to show how it can be formatted and how to use the /target environment.

Create New ISO

Create the ISO so you can test out your preseed file. If you use one or make your own, you will need to test it as this will LIKELY be the place your process will fail. I wrote a script for quick testing, but you could just point the preseed file at an http:// hosted preseed file and test very quickly that way.

mkisofs -D -r -V "UNATTENDED_UBUNTU" -cache-inodes -J -l -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o /tmp/ubuntu16-desktop-unattended-install.iso /opt/ubuntuiso

Feel free to change the output name and the the directory in which you will save it.

Burn ISO

I would suggest testing on virtualbox or the like and once you have it working, then burn it to a DVD. You should now have a working unattended Ubuntu Desktop 16.04 LTS install DVD.

Feedback, Corrections, Errors

I wrote this all in one go and there may be errors, typos, or things that got messed up along the way. If anyone tries this, please let me know if you encounter an error in the process. And please remember that if you create your own preseed file, I will probably not be able to answer why your unattended install is broken and not working since ubiquity likes to ignore and not do certain things in the preseed file. I hope this helpful to someone.

Have a look at: https://github.com/core-process/linux-unattended-installation

This project provides all you need to create an unattended installation of a minimal setup of Linux, whereas minimal translates to the most lightweight setup - including an OpenSSH service and Python - which you can derive from the standard installer of a Linux distribution. The idea is, you will do all further deployment of your configurations and services with the help of Ansible or similar tools once you completed the minimal setup.

Great job Brandon Authier for your post and instructions, you helped me a lot.

I had one issue with your method though: I found out that after install finishes and PC reboots the GRUB hangs.

So I added to the ks.preseed the following:

# Due notably to potential USB sticks, the location of the MBR can not be
# determined safely in general, so this needs to be specified:

d-i grub-installer/bootdev  string /dev/sda

# To install to the first device (assuming it is not a USB stick):
#d-i grub-installer/bootdev  string default

This prevents crashing the grub install if you do not remove the USB stick. I got this from https://www.debian.org/releases/stable/example-preseed.txt

First off big thanks to Brandon for working through all this!! There are many, many preseed files on the web, showing the demand for skipping through the install screens.

With 20.04 out, I wanted to add a couple of notes from my tests. I'm using Kubuntu, but it looks identical for other Buntus:

• CURRENTLY EDITING: if you use virt-manager, you can install to a raw (ext4) partition and install to entire disk, skipping partitioning partman 'recipes' if your able to. (I use a single swap partition for all Linux OSes, or a large swap file on another partition, so I can backup whole copies of OS partitions quicker). It would be faster to copy this partition over a network than install on many machines with USB (unless you had a big USB cloning system).
• I use AIOBOOT as a boot menu (includes Refind, Clover, etc), so I never change the system GRUB. And installing into a VM renders that null, but use efibootmgr --help, if you mess up your boot order.
• I couldnt get the default isolinux.cfg menu selection to work (below). Didnt need boot=casper. Change default vesamenu.c32 to default install or what you name your menu entry, from here..still didnt work. BTW, there wasnt an entry for direct install in 20.04 originally.
• I happened to try Kickstarter first, so it gave me a ks.cfg (which didnt work) and a ks.preseed (with a single line). Since there was a kubuntu.preseed exisiting in the ISO, I copied those lines to ks.preseed (contents below) and then added Brandon's preseed magic, changing the custom commands. It also encrypted my generic starter password to $1$D2dImaFN$AEBMU.qOh2qer0v/alzeH0 which is 123. You can plug that in as I did below. Dont forget to change it, though ;)!
• More official info from Ubuntu on Automatic Install here. May be old, as it claims Kickstarter works, though "This method is not as flexible as the preconfiguration file method above, but it requires less knowledge of how the installer works." An official example of just a preseed from Ubuntu is here.
• Changed timeout 30 in isolinux/isolinux.cfg to 1
• An important change in isolinux/txt.cfg to the maybe-ubiquity to automatic-ubiquity. This is the installer. Possibly only-ubiquity, more info here.
• I used the lines for partitioning in ks.preseed from here, because they all use the new ubiquity command. Only one extra line appears different partman-lvm/confirm string boolean true. I didnt test Brandon's.

default install
...all the original entries...
label install
  menu label ^Install kUbuntu auto
  kernel /casper/vmlinuz
  append  file=/cdrom/ks.preseed auto=true priority=critical debian-installer/locale=en_US keyboard-configuration/layoutcode=us ubiquity/reboot=true languagechooser/language-name=English countrychooser/shortlist=US localechooser/supported-locales=en_US.UTF-8 automatic-ubiquity initrd=/casper/initrd --

# Enable extras.ubuntu.com.
d-i apt-setup/extras    boolean true
# Install the Kubuntu desktop.
tasksel tasksel/first   multiselect kubuntu-desktop
#tasksel tasksel/first   multiselect ubuntu-desktop
# Install KDE translation packages.
d-i pkgsel/language-pack-patterns   string language-pack-kde-$LL kde-i18n-$LL
# Install the KDE oem-config frontend (if in OEM mode).
d-i oem-config-udeb/frontend    string kde
# On live DVDs, don't spend huge amounts of time removing substantial
# application packages pulled in by language packs. Given that we clearly
# have the space to include them on the DVD, they're useful and we might as
# well keep them installed.
ubiquity    ubiquity/keep-installed string icedtea6-plugin openoffice.org

# Partitioning
# Not needed if only one disk
#ubiquity   partman-auto/disk string /dev/sda

ubiquity    partman-auto/method string regular
ubiquity    partman-lvm/device_remove_lvm string boolean true
ubiquity    partman-md/device_remove_md string boolean true
ubiquity    partman-lvm/confirm string boolean true
ubiquity    partman-auto/choose_recipe select atomic

ubiquity    partman/confirm_write_new_label string boolean true
ubiquity    partman/choose_partition select finish
ubiquity    partman/confirm string boolean true
ubiquity    partman/confirm_nooverwrite string boolean true

# Locale
d-i debian-installer/locale string en_US
d-i console-setup/ask_detect boolean false
d-i console-setup/layoutcode string us

# Network
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/choose_interface select auto

# Clock
d-i clock-setup/utc-auto boolean true
d-i clock-setup/utc boolean true
d-i time/zone string US/Pacific
d-i clock-setup/ntp boolean true

# Packages, Mirrors, Image
d-i mirror/country string US
d-i apt-setup/multiverse boolean true
d-i apt-setup/restricted boolean true
d-i apt-setup/universe boolean true

# Users
d-i passwd/user-fullname string User
d-i passwd/username string user
d-i passwd/user-password-crypted password $1$D2dImaFN$AEBMU.qOh2qer0v/alzeH0
d-i passwd/user-default-groups string adm audio cdrom dip lpadmin sudo plugdev sambashare video
d-i passwd/root-login boolean true
d-i passwd/root-password-crypted password $1$D2dImaFN$AEBMU.qOh2qer0v/alzeH0
d-i user-setup/allow-password-weak boolean true

# Grub
d-i grub-installer/grub2_instead_of_grub_legacy boolean true
d-i grub-installer/only_debian boolean true
d-i finish-install/reboot_in_progress note

# Custom Commands
ubiquity ubiquity/success_command string \
  echo 'command1' ;\
  read -p "Press enter to continue" ;

A little more about installing to a raw partition via a VM.. Currently, the partition isnt recognized by gparted, but I can create a new VM in virt-manager and it boots. To install in virt-manager, just select ISO install, enter the partition /dev/sda# (or more safely: /dev/disk/by-id/...) into custom storage field. Click customize before install and you can add more hardware (button at bottom) for storage as IDE or SATA devices. I had to change the Direct Kernel Boot in Boot Options to add the paths and arg pointing to the correct UUID BOOT_IMAGE=/boot/vmlinuz-4.18.0-14-generic root=UUID=..., but that was after I had converted a VirtualBOX VDI, and this time it just boots without those fields. I need to test if it boots from harddisk via my boot manager, though. Also, I've heard Vagrant is a newer VM manager. Also, change Network in virt-manager to NAT.

I am sorry to hear that my former method did not work. Fortunately, I found a script designed by Rinck Sonnenberg (netson) that would create unattended AMD64 ISOs of Ubuntu Server on GitHub, and I forked it. I then improved the script by adding the capability to create I386 ISOs. I also changed the OS from Ubuntu Server to Ubuntu Desktop. You need not visit the GitHub repository, you can just follow the below instructions.

Run these commands:

$ wget https://raw.githubusercontent.com/iPlus-TechNet/ubuntu-unattended/master/create-unattended-iso.sh
$ chmod +x create-unattended-iso.sh
$ sudo ./create-unattended-iso.sh

Sometimes wget is not available. If this is the case, use curl:

$ curl -O https://raw.githubusercontent.com/iPlus-TechNet/ubuntu-unattended/master/create-unattended-iso.sh

If none of these work, download it and move it to your home folder.

You will then be asked which version of Ubuntu to install, and then the questions Ubuntu would ask if you were doing a manual install. The script will then download the Ubuntu ISO, and then apply the changes requested to it. You now have an ISO that is all ready for an unattended installation!

This should answer your question, because it worked just fine for me. It is also good that this method is far easier than my former answer. I hope it works for you as it did for us.

Thanks so much! This is the best answer I've found to this question so far. The automatic installation part all work like a charm. Just want to add to the part of creating new iso file. The iso image created with the command in your answer works perfectly for VM installation in my tests, but not for USB boot drives. The one I tried below works fine for creating USB installation media and boot in UEFI mode.

OLD_IMAGE=/path/to/iso
MBR_FILE=/tmp/ubuntu_isohybrid_mbr.img
dd if="$OLD_IMAGE" bs=1 count=446 of="$MBR_FILE"

IMAGE=custom.iso
BUILD=/mnt/iso

xorriso -as mkisofs -r -V "Custom Ubuntu Install CD" \
            -cache-inodes -J -l \
            -isohybrid-mbr "$MBR_FILE" \
            -c isolinux/boot.cat \
            -b isolinux/isolinux.bin \
               -no-emul-boot -boot-load-size 4 -boot-info-table \
            -eltorito-alt-boot \
            -e boot/grub/efi.img \
               -no-emul-boot -isohybrid-gpt-basdat \
            -o "$IMAGE" \
            "$BUILD"

rm "$MBR_FILE"

And to have the automation working in UEFI mode, you would also need to edit boot/grub/grub.cfg in the image pretty much the same way as the isolinux.cfg mentioned in the answer above. Mine looks like

if loadfont /boot/grub/font.pf2 ; then
    set gfxmode=auto
    insmod efi_gop
    insmod efi_uga
    insmod gfxterm
    terminal_output gfxterm
fi

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray

menuentry "Customized Automatic Install" {
    set gfxpayload=keep
    linux   /casper/vmlinuz  file=/cdrom/preseed/custom.seed boot=casper automatic-ubiquity initrd=/casper/initrd quiet splash noprompt noshell ---
    initrd  /casper/initrd
}

where custom.seed is the preseed file I am using, and I put some of the locale settings into the preseed file instead of leaving them in this line.

PLEASE DO NOT TRY THIS METHOD. DOES NOT WORK (UNLESS YOU ARE TRYING TO KILL TIME):

I have seen the answer to this all around. I will tell you the most common solution. I have never tried it before myself, so it may be outdated. I really do not understand the point of it, as just doing a manual install itself would be easier, but anyways...

Login as root or run $ sudo su -

Download and mount the ISO. Do NOT do this manually from the Ubuntu website.

# mkdir -p /mnt/iso
# mount -o loop ubuntu.iso /mnt/iso

back up and move relevant files.

# mkdir -p /opt/ubuntuiso
# cp -rT /mnt/iso /opt/ubuntuiso

Block the language part of the GUI from appearing

# cd /opt/ubuntuiso
# echo en >isolinux/lang

Add a "Kickstart" file.

# apt install system-config-kickstart
# system-config-kickstart

[OPTIONAL] Add packages for your installation

# vim /path/to/ks.cfg #[OPTIONAL]

Or just search for it in your files and edit it directly. Add the %packages section, and put your packages under it.

%packages
# Add your packages below. Example:
@ ubuntu-server
apache2
mysql-server
php7.0
php-pear
libapache2-mod-php7.0 
php7.0-mysql
php7.0-curl
php7.0-json
php7.0-cgi

As I said, I have never done this myself, so if you had a configuration like the above, it may not actually install the LAMP stack and you would have to do it manually later.

Suppress questions with a "preseed" file.

# echo 'd-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition \
select Finish partitioning and write changes to disk
d-i partman/confirm boolean true' > ks.preseed

Activate the files:

# vi isolinux/txt.cfg

Then searching for:

label install
  menu label ^Install Ubuntu Server
  kernel /install/vmlinuz
  append  file=/cdrom/preseed/ubuntu-server.seed vga=788 initrd=/install/initrd.gz quiet --

You will then need to add ks=cdrom:/ks.cfg and preseed/file=/cdrom/ks.preseed. Remove words quiet and vga=788, so it looks like this:

append file=/cdrom/preseed/ubuntu-server.seed initrd=/install/initrd.gz ks=cdrom:/ks.cfg preseed/file=/cdrom/ks.preseed --

Now you can create the new answer:

# mkisofs -D -r -V "ATTENDLESS_UBUNTU" -cache-inodes -J -l -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o /opt/autoinstall.iso /opt/ubuntuiso

And there you go!