Alex Reynolds's questions

I rebooted a m5d.xlarge EC2 instance running Ubuntu 16.04.5 LTS (xenial) using shutdown -r.

Upon reboot, various drive mounts defined in /etc/fstab did not line up with the device assignments given to them initially.

This caused various services to fail, which rely upon the drive mount with the assigned name containing specific data associated with that name.

In this case, /data needed to contain what is normally expected to be in /data, and not some other drive mount, like /foo or /bar or /whatever-else.

Running lsblk helped with manually redefining the /etc/fstab file to reflect the new device assignments and bringing services back online.

Problem: My worry is that rebooting will randomly reassign device labels to drive mounts, and this problem could or will occur again when the server requires rebooting.

Question: How do I ensure that drive mounts and their respective device paths are preserved between reboots, without needing manual (re)intervention?

Edit: The c5_m5_checks_script.sh script referenced in the answer provided below returns the following information:

# ./c5_m5_checks_script.sh 
------------------------------------------------

OK     NVMe Module is installed and available on your instance


ERROR  NVMe Module is not loaded in the initramfs image.
        - Please run the following command on your instance to recreate initramfs:
        # sudo update-initramfs -c -k all


OK     ENA Module with version 2.0.3K is installed and available on your instance


OK     fstab file looks fine and does not contain any device names.

------------------------------------------------

I don't understand the last OK statement, as the fstab file contains device names. Or Amazon's use of the term "device names" means something different from the device names in the fstab file, perhaps?

To give a specific and concrete example of the nature of the problem, I have an /etc/fstab that looks like this:

...
/dev/nvme2n1    /staging        ext4    defaults,nofail 0       0
/dev/nvme3n1    /data   ext4    defaults,nofail 0       0
...

However, lsblk shows that these two volumes are mounted to each other's device ID:

# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
...
nvme2n1     259:0    0     1T  0 disk /staging
nvme3n1     259:3    0   512G  0 disk /data

In reality, nvme2n1 is the 1T volume /data, and nvme3n1 is the 512G volume /staging.

Something unknown happens between reboots which reassigns the underlying drives to a permutation of the available device IDs.

The only way I know to fix this is to manually umount these two volumes, edit /etc/fstab, and mount them again.

Bounty question: Is there a permanent fix that would let me use a different and permanent identifier in /etc/fstab, which persists consistently between reboots?

I have a machine instance and attached volume that I would like to clone, but apply new network settings to it.

What is the best process for doing this, which will leave the original instance unchanged?

Can I take a snapshot of a running instance, and use that snapshot to instantiate a new instance of the same type, but with a different network address? How do I go about doing that, if so?

EDIT:

From looking at the EC2 console, the snapshot functionality clones a volume (by volume ID), not an instance (by instance ID). Is there a way to create a new volume made from a snapshot of an existing volume and attach it to a new machine instance that has different network settings assigned to it, without detaching/reattaching volumes and manually cp/scp-ing thousands of files?

I am basically looking for a way to duplicate an existing machine instance and the full contents of its attached EBS volume, with the only difference being new network settings. (If I'm using the wrong terminology for describing what I am trying to do, please let me know!)

We would like to have an SGE-based compute cluster with a queue that gives access to all nodes for the computational staff, and a second cluster queue that gives access to, say, half the nodes for occasional (but heavy) use by other staff.

We want to limit the resources of the second queue so that computational staff can keep doing some work, even when there is occasional (but heavy) use by non-comp. staff.

Is there a way to set up two (or more) SGE queues of one collection of nodes, such that one queue can contain all nodes, and a second queue contains a subset of the same nodes, and both queues operate simultaneously?

What specific SGE configuration parameters would I research to set up something like this?

I installed boost.x86_64 on my Linux system:

$ sudo yum install boost.x86_64
Loaded plugins: fastestmirror, langpacks, presto, refresh-packagekit
Adding en_US to language list
Loading mirror speeds from cached hostfile
 * fedora: mirror.uoregon.edu
 * updates: mirror.uoregon.edu
Setting up Install Process
Package boost-1.44.0-8.fc14.x86_64 already installed and latest version
Nothing to do

But when I looked for the boost files, I couldn't find them:

$ sudo repoquery --list boost-1.44.0-8.fc14.x86_64
$

This is a bit strange, as I have boost and several of its dependencies installed:

$ sudo rpm -qa | grep boost
boost-signals-1.44.0-8.fc14.x86_64
boost-graph-1.44.0-8.fc14.x86_64
boost-program-options-1.44.0-8.fc14.x86_64
boost-system-1.44.0-8.fc14.x86_64
boost-date-time-1.44.0-8.fc14.x86_64
boost-wave-1.44.0-8.fc14.x86_64
boost-serialization-1.44.0-8.fc14.x86_64
boost-1.44.0-8.fc14.x86_64
boost-thread-1.44.0-8.fc14.x86_64
boost-test-1.44.0-8.fc14.x86_64
boost-python-1.44.0-8.fc14.x86_64
boost-regex-1.44.0-8.fc14.x86_64
boost-random-1.44.0-8.fc14.x86_64
boost-filesystem-1.44.0-8.fc14.x86_64
boost-iostreams-1.44.0-8.fc14.x86_64

Is there another destination for files that repoquery cannot find?

The reason I ask is that I have a boost dependency in a software tool I am trying to compile, and it cannot locate the boost installation (and neither can repoquery, apparently).

Thanks for any suggestions or tips.

EDIT

I overlooked installing the headers, as well:

$ sudo yum install boost-devel.x86_64

Once I did this, I could find those headers:

$ repoquery --list boost-devel.x86_64
/usr/include/boost
/usr/include/boost/accumulators
/usr/include/boost/accumulators/accumulators.hpp
/usr/include/boost/accumulators/accumulators_fwd.hpp
/usr/include/boost/accumulators/framework
/usr/include/boost/accumulators/framework/accumulator_base.hpp
...

I am running Apache 2.2.17 on Fedora v14. I cannot access the running web server from a browser, and I also have difficulties setting it up to listen on port 443.

The server httpd is running:

[me@host ~]$ sudo ps -U root -u root u | grep httpd
root      6592  0.0  3.4 404620 17552 ?        Ss   10:50   0:00 /usr/sbin/httpd -k graceful

I then stop the service:

[me@host ~]$ sudo apachectl stop

I then set up the Listen directive in /etc/httpd/conf/httpd.conf to listen on ports 80 and 443:

Listen 1.2.3.4:80
Listen 1.2.3.4:443 https

(The IP is not 1.2.3.4 but the actual IP of this host.)

I checked the other Apache conf directives to make sure I have a document folder, that it is accessible by all users, and that it will show a directory listing of the folder, if I do not specify index.html etc.

When attempting to start httpd, I get a configuration error:

[me@host conf]$ sudo apachectl graceful
(98)Address already in use: make_sock: could not bind to address [::]:443
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:443
no listening sockets available, shutting down
Unable to open logs

No other services appear to be running on TCP port 443 (or port 80):

[me@host conf]$ sudo netstat -tulpn | grep 443
[me@host conf]$ sudo netstat -tulpn | grep 80

If I comment out the 443 Listen directive and graceful-restart the httpd service, it starts and is visible in the process list (via ps, as above) but I am still unable to access the host via web browser.

Is there anything I have overlooked in setting up Apache? Thanks for your advice.

I would like to make a Zip64 archive from a bunch of bzip2 files, using 7za's -so option to send the compressed archive to standard output:

$ 7za a dummy -tzip -so myFiles*.bz2 > test.zip

Here is the output I get:

7-Zip (A) [64] 9.13 beta  Copyright (c) 1999-2010 Igor Pavlov  2010-04-15
p7zip Version 9.13 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)
Scanning

Creating archive stdout

System error:                                                                                                                                                                                                                              
E_NOINTERFACE 

The file test.zip is empty. There is no information in the documentation about the error E_NOINTERFACE. What can I do within the application to fix this? Thanks for your advice!

EDIT

I do not want to extract bz2 files to standard output. I want to (a)dd to an archive called test.zip, made from a set of bz2 files, by sending the bundled archive to standard output. This is to use 7za to archive bz2 files in a manner similar to tar, as zip files provide random access while tar archives do not.

EDIT 2

Unfortunately, it looks like the developer confirms that 7-Zip does not support writing to standard output.

We would like to encrypt the entire disk of a Snow Leopard workstation running some sensitive file, database and web services. PGP does not yet work for 10.6, and we cannot use FileVault as it is not compliant with FIPS and would require non-standard installation of services.

What whole-disk options are available that are known to work with Snow Leopard?

I am using tar to archive a group of very large (multi-GB) bz2 files.

If I use tar -tf file.tar to list the files within the archive, this takes a very long time to complete (~10-15 minutes).

Likewise, cpio -t < file.cpio takes just as long to complete, plus or minus a few seconds.

Accordingly, retrieving a file from an archive (via tar -xf file.tar myFileOfInterest.bz2 for example) is as slow.

Is there an archival method out there that keeps a readily available "catalog" with the archive, so that an individual file within the archive can be retrieved quickly?

For example, some kind of catalog that stores a pointer to a particular byte in the archive, as well as the size of the file to be retrieved (as well as any other filesystem-specific particulars).

Is there a tool (or argument to tar or cpio) that allows efficient retrieval of a file within the archive?

I have a C application that occasionally fails to open a file which is stored on a /tmp share.

Here is the relevant chunk of code:

  // open file and start parsing

  notStdin = strcmp(inFile, "-");
  if (notStdin) {
     coordsIn = fopen(inFile, "r");   <----- inFile = file that I want to open
     if (coordsIn == NULL) {
        fprintf(stderr, "ERROR: Could not open coordinates file: %s\n\t%s\n", inFile, strerror(errno));
        exit(EXIT_FAILURE);
     }
  }
  else
     coordsIn = stdin;

Once out of eight to ten trials, I get a NULL FILE pointer. Here is an example error message:

ERROR: Could not open coordinates file: /tmp/coordinates.txt
       File or directory does not exist

However, the file /tmp/coordinates.txt does indeed exist, as I can open it with standard utilities like head, cat or more, etc.

The permissions of different /tmp/coordinates.txt trial files are the same.

Here is the result from uname -a:

$ uname -a
Linux hostname 2.6.18-128.2.1.el5 #1 SMP Wed Jul 8 11:54:47 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

If I use a different inFile that is stored in a different, non-/tmp share, then I do not observe this symptom.

Is there anything that would cause fopen() to fail on a file stored in the /tmp share? Are there other troubleshooting steps I can pursue?