Arronical's questions

I have a text file made from the output of the repository management tool aptly, which lists my published repositories, from which I need to extract information.

The file format is as follows:

Published repositories:
 * test_repo_one/xenial [i386,amd64] publishes {main: [xenial-main_20190311]: Snapshot from mirror [xenial-main]: http//gb.archive.ubuntu.com/ubuntu/ xenial [src]}, {multiverse: [xenial-multiverse_20190311]: Snapshot from mirror [xenial-multiverse]: http//gb.archive.ubuntu.com/ubuntu/ xenial [src]}, {restricted: [xenial-restricted_20190311]: Snapshot from mirror [xenial-restricted]: http//gb.archive.ubuntu.com/ubuntu/ xenial [src]}, {universe: [xenial-universe_20190311]: Snapshot from mirror [xenial-universe]: http//gb.archive.ubuntu.com/ubuntu/ xenial [src]}
 * test_repo_one/xenial-security [i386,amd64] publishes {main: [xenial-security-main_20190311]: Snapshot from mirror [xenial-security-main]: http//gb.archive.ubuntu.com/ubuntu/ xenial-security[src]}, {multiverse: [xenial-security-multiverse_20190311]: Snapshot from mirror [xenial-security-multiverse]: http//gb.archive.ubuntu.com/ubuntu/ xenial-security[src]}, {restricted: [xenial-security-restricted_20190311]: Snapshot from mirror [xenial-security-restricted]: http//gb.archive.ubuntu.com/ubuntu/ xenial-security[src]}, {universe: [xenial-security-universe_20190311]: Snapshot from mirror [xenial-security-universe]: http//gb.archive.ubuntu.com/ubuntu/ xenial-security[src]}
 * test_repo_two/trusty [i386,amd64] publishes {main: [trusty-main_20190312]: Snapshot from mirror [trusty-main]: http//gb.archive.ubuntu.com/ubuntu/ trusty[src]}, {multiverse: [trusty-multiverse_20190312]: Snapshot from mirror [trusty-multiverse]: http//gb.archive.ubuntu.com/ubuntu/ trusty[src]}, {restricted: [trusty-restricted_20190312]: Snapshot from mirror [trusty-restricted]: http//gb.archive.ubuntu.com/ubuntu/ trusty[src]}, {universe: [trusty-universe_20190312]: Snapshot from mirror [trusty-universe]: http//gb.archive.ubuntu.com/ubuntu/ trusty[src]}
...

The last line of the output ends in a new line.

The "Published repositories:" line is not required.

For each of the lines starting ' *' I need to remove extraneous information, leaving only snapshot names. There is no way to do this in aptly. The desired output for the first of these lines is.

test_repo_one/xenial [xenial-main_20190311] [xenial-multiverse_20190311] [xenial-restricted_20190311] [xenial-universe_20190311]

The square brackets are not essential either so a solution that retains or removes these is fine. I'd prefer a sed or awk solution but anything that works would be highly appreciated.

I have several files with text that needs replacing. The text starts and ends with the same pattern each time, but the content in between the patterns is variable. The patterns can appear in the middle of lines, and the content between them often spans multiple lines.

There will only be a single occurrence of the start and end pattern in each file.

I need a command line method to replace the text between the patterns, including the patterns themselves. Outputting to a new file, or editing in place is fine.

A command that operates on a single file will work, as I can loop through the files and apply the command myself. I attempted a sed solution but could only manage to replace entire lines.

An example of text would be:

Cable Type ID:135, Installation ID:62, Alpha Conductor Origin:
Tolerance Report B74 - 3rd June 1996, Beta Conductor Origin: 
Tolerance Report B74 - 3rd June 1996, Phase Conductor Size: 
45mm, Security: Security-Start Bs86gKI-734Lw#32_nP/5589Zfb8Wj-
sW93j9b Security-End, Location ID:889, Protective Earth Size:
67mm, Protective Earth Max Current (A): 4, Overload Time...

The start pattern is Security-Start and the end pattern is Security-End. I want to replace the patterns and everything in between with the word REDACTED.

I would like the output to be:

Cable Type ID:135, Installation ID:62, Alpha Conductor Origin:
Tolerance Report B74 - 3rd June 1996, Beta Conductor Origin: 
Tolerance Report B74 - 3rd June 1996, Phase Conductor Size: 
45mm, Security: REDACTED, Location ID:889, Protective Earth Size:
67mm, Protective Earth Max Current (A): 4, Overload Time...

Please note that the text between the two patterns may be so long that it spans several lines, it is fairly random in length. This is not clear in the example above

Any language which is available by default on an Ubuntu system will be fine. My first thoughts are 'sed' or 'awk', but whatever you're comfortable with will be fine.

If I have the UUID of a drive partition, how would I go about finding out whether it is mounted or not, using the command line?

I would like a method to capture the disk usage of a particular partition, by using the directory where the partition is mounted. The output should just be an integer with no padding or following symbols, as I'd like to save it in a variable.

I've used df --output=pcent /mount/point, but need to trim the output as it has an unnecessary header, single space padding before the value, and a % symbol following the value like so:

Use%
 83%

In this case the output I would like would simply be 83. I'm not aware of any drawbacks to using the output of df, but am happy to accept other methods that do not rely on it.

I like the way a variable can be set for a single command, such as:

LANG=fr_FR.utf8 df -h

Is it possible to use the shopt -s command, or some other method, to set a shell option, such as nullglob or extglob, for a single command in the same manner?

I have a space seperated string which is output by a command, which I like to pipe to cut, using -fd ' ' to split on spaces. I know I can use -f <n> to display field number <n>, but can I make it display the last field if I don't know the length of the string?

Or do I need to use a more flexible text editing tool like sed or awk?

I've had a strange issue with an Ubuntu 14.04 server installation recently, and would like some pointers to diagnose the cause of the problem.

The OS will not boot unless a monitor is connected via the VGA port.

I believe that the boot process may be stalled at the grub menu, as the server turn off immediately when the power button is pressed, whereas power off when the OS is running takes a few seconds through the normal ACPI process.

As the OS isn't starting, there are no logs to guide me through a debugging process. What can I do to begin to understand where the problem lies?

I have the following function to count the number of files in a directory, within my bash script.

file_count() {
  no_of_files=$(find "$1" -maxdepth 1 -type f -printf '.' | wc -c)
}

I'd like to use it repeatedly on different directories and save the count into a variable for each directory. Currently to do this, I use

file_count $somedir
files_in_somedir="$no_of_files"

I'm aware that I'm setting the no_of_files variable outside of the function each time, and would like to make it local to the function, not settign an intermediate variable in the main script. This is just in case there's some mistake meaning that the variable doesn't change between calls of the function (mistyping the function name maybe), and the old value of no_of _files is used.

If my function were:

file_count() {
  local no_of_files=$(find "$1" -maxdepth 1 -type f -printf '.' | wc -c)
}

How would I easily set these directory count variables?

I would like to use a local mirror of the Ubuntu repositories when I'm completing an Ubuntu Server installation. I have a mirror already created to serve the repository over the LAN using http, this works for updates/upgrades with machines using custom a /etc/apt/sources.list file.

How do I ensure that any package updates/upgrades which are used in a normal Ubuntu installation come from this source? I could complete the installation with the network cable unplugged, replace the sources file, and then upgrade the packages, but would rather do it all as part of the initial installation phase.

Ideally the answer will not involve building my own custom version of the installer, but if that's a necessary process feel free to describe it. As this is a server version installation, it's not possible to simply change the sources file in a live session.

I'd like a method to find and print the modified time of a file, for use within a bash script.

I have come up with:

ls -l $filename | cut -d ' ' -f '6-8'

Which outputs:

Jul 26 15:05

Though I'd like to avoid parsing ls, also it'd be useful to have the year in there.

Ideally I'd like to see an output similar to the default output of the date command.

Tue Jul 26 15:20:59 BST 2016

What other useful methods are available?

I have a full mirror of the Trusty repositories, made with apt-mirror. I keep a backup on another disk by using an rsync command. Whenever I do the backup, I notice that no files are ever deleted. This leads me to believe that new versions of packages are added, index files are modified, but no packages are ever deleted. This seem to be backed up by the size growth of the initial repository download by apt-mirror.

Are the repositories only ever added to continuously?

Edit: Comments suggest that the lack of deleted files is due to an error with usage the rsync command. This is not the case. This is the command I use:

rsync -av --delete --log-file=/home/user/test_mirror_update_20160610.txt /path/to/raw_mirror/ /path/to/test_mirror/ &

I have VirtualBox running on several server installations, always without any GUI, administering using the vboxmanage commands.

I used to install from the Oracle PPA, but since the end of the two different versions of vbox, I've been installing from the Ubuntu repos. I've found the package for installing the guest additions iso, but never been aware of the extension packs being there.

I'm asking as the virtualbox version for 14.04 moved from 4.3.10 to 4.3.34 a little while back, so added a step for me running apt-get dist-upgrade.

I know how to get hold of these from Oracle and install them manually, but is there another method, without using GUI? Are the extension packs anywhere in the Ubuntu repos?

I want to count the number of lines in a file using wc -l but only output the number, without the following space and filename. Edit from comments: The filename may have numbers in it.

Currently if I do:

wc -l /path/to/file

On a file with 176 lines, the output is:

176 /path/to/file

This is within a bash script, and the resulting number will be assigned as the value of a variable.

I have a server which receives a file per client each day into a directory. The filenames are constructed as follows:

uuid_datestring_other-data

For example:

d6f60016-0011-49c4-8fca-e2b3496ad5a7_20160204_023-ERROR

• uuid is a standard format uuid.
• datestring is the output from date +%Y%m%d.
• other-data is variable in length but will never contain an underscore.

I have a file of the format:

#
d6f60016-0011-49c4-8fca-e2b3496ad5a7    client1
d5873483-5b98-4895-ab09-9891d80a13da    client2
be0ed6a6-e73a-4f33-b755-47226ff22401    another_client
...

I need to check that every uuid listed in the file has a corresponding file in the directory, using bash.

I've got this far, but feel like I'm coming from the wrong direction by using an if statement, and that I need to loop through the files in the source directory.

The source_directory and uuid_list variables have been assigned earlier in the script:

# Check the entries in the file list

while read -r uuid name; do
# Ignore comment lines
   [[ $uuid = \#* ]] && continue
   if [[ -f "${source_directory}/${uuid}*" ]]
   then
      echo "File for ${name} has arrived"
   else
      echo "PANIC! - No File for ${name}"
   fi
done < "${uuid_list}"

How should I check that the files in my list exist in the directory? I'd like to use bash functionality as far as possible, but am not against using commands if need be.

I've been running a 14.04 installation on an old Dell machine for a few months without issue, but decided to add another larger disk drive, and reinstall. I removed an unneeded floppy drive in the process, and closed it in the bios too.

The installation went well, but I'm seeing input output errors from the newly added disk, it doesn't contain any system stuff, so booting seems to continue happily enough.

Why does the message System information disabled due to load higher than 1.0 appear when I log in?

Today I ran:

apt-get update
apt-get dist-upgrade

And noticed that the following packages were upgraded:

linux-headers-3.13.0-65-generic:i386
linux-image-3.13.0-65-generic:i386
linux-image-extra-3.13.0-65-generic:i386
linux-headers-3.13.0-65:i386

I know that these are updates to the kernel, and checked the /var/log/apt/history file, finding these lines:

linux-headers-3.13.0-65-generic:i386 (3.13.0-65.105, 3.13.0-65.106)
linux-image-3.13.0-65-generic:i386 (3.13.0-65.105, 3.13.0-65.106)
linux-image-extra-3.13.0-65-generic:i386 (3.13.0-65.105, 3.13.0-65.106)
linux-headers-3.13.0-65:i386 (3.13.0-65.105, 3.13.0-65.106)

I figure that this is telling me it's upgraded from the 105 to the 106 build, and assume it's in relation to CVE-2015-7613.

Why does this get released as an upgrade rather than becoming 3.13.0-66? I noticed some very quick jumps in kernel (minor?) releases a couple of weeks ago, and am curious as to how this is governed.

I use the following line to find all sub-directories of the PWD and run svnadmin verify on each directory (I already know that they're Subversion repositories)

find ./* -maxdepth 0 -exec svnadmin verify {} \;

This works well, other than the fact that the output looks like this:

* Verifying repository metadata ...
* Verifying metadata at revision 1 ...
* Verifying metadata at revision 2 ...
* Verifying metadata at revision 4 ...
* Verifying metadata at revision 5 ...
* Verifying metadata at revision 6 ...
* Verifying metadata at revision 9 ...
* Verifying metadata at revision 10 ...
* Verifying metadata at revision 12 ...
* Verifying metadata at revision 14 ...
* Verifying metadata at revision 15 ...
* Verifying metadata at revision 18 ...
* Verifying metadata at revision 20 ...
* Verifying metadata at revision 22 ...
* Verified revision 0.
* Verified revision 1.
* Verified revision 2.
* Verified revision 3.
* Verified revision 4.
* Verified revision 5.
* Verified revision 6.
* Verified revision 7.
* Verified revision 8.
* Verified revision 9.
* Verified revision 10.
* Verified revision 11.
* Verified revision 12.
* Verified revision 13.
* Verified revision 14.
* Verified revision 15.
* Verified revision 16.
* Verified revision 17.
* Verified revision 18.
* Verified revision 19.
* Verified revision 20.
* Verified revision 21.
* Verified revision 22.
* Verified revision 23.
* Verified revision 0.
* Verifying repository metadata ...
* Verifying metadata at revision 4 ...
* Verifying metadata at revision 5 ...
* Verifying metadata at revision 6 ...
* Verifying metadata at revision 7 ...
* Verifying metadata at revision 9 ...
* Verified revision 0.
* Verified revision 1.
* Verified revision 2.
* Verified revision 3.
* Verified revision 4.
* Verified revision 5.
* Verified revision 6.
* Verified revision 7.
* Verified revision 8.
* Verified revision 9.

I'd really like find to print the filename before executing the svnadmin verify command, to make logging easier.

I've tried to squeeze a little ls in there but bodged it up, how should I do this (preferably simply)?

I use the following command to clear a directory, of files and directories over 30 days old, and move them to an archive directory which I can delete after a few weeks if nobody asks for their files back. The target directory has subdirectories by user name, so will the archive directory.

This is the command I use:

find /path/to/directory/username/ -mtime +30 -exec mv "{}" /path/to/archive/username/ \;

I suggested a modified version of this to answer a question on ask ubuntu, another user edited the code to change the end of line \; for + as it's faster(and more correct?). See here

However, using + in this way works if the -exec command is ls -lh but not in the actual command that I use. If I try it with + I get an error message:

find: missing argument to '-exec'

I don't understand why it's behaving this way, or what the correct command would be. Please don't just post a command correction, I'd like to understand rather than just follow a suggestion blindly.

Do the Ubuntu 14.04 Server installation .iso files come with a 'Try Ubuntu' option, or is that only on the Desktop versions? In other words, can I use this media as a live CD, for tasks such as administering and imaging partitions?

I've set up an Ubuntu 14.04 server, using Samba to share files to our Windows clients. Everything appears to be working correctly, but while checking my auth logs, I see the line 'session closed for user nobody' repeated many many times, though no sessions opened for that user. All of the session opened messages have the correct user name included.

I understand the role of the nobody user, but am unsure why Samba keeps generating this message. On one line the 'session closed' message is repeated 58 times with no other auth log messages interrupting.

Is it likely that Samba is incorrectly configured?

I'm happy to post any logs/confs/outputs necessary, if that helps.

EDIT:

I've previously disabled Samba password sync using pam-auth-update to stop the 'No Talloc Stackframe' message, like so, could that be generating all of these session closed messages?