Script or program to log the disk usage

Using the df command to keep track of the disk space, and the lsblk command to keep track of the mounted drives, the script below, run in the background, will log changes in the free space of all mounted drives. It creates a log file: ~/disklog where it writes changes to (in k).

If you run it in the terminal, it will output the result simultaneously.

The content of the log file looks like:

[mountpoint / change / date/time / used]

/ . . . . . . . . . . . . . . . . . .            36 k       Fri Mar 27 08:17:30 2015    used 87989352 k
/media/intern_2 . . . . . . . . . . .         -1792 k       Fri Mar 27 08:17:32 2015    used 562649592 k
/ . . . . . . . . . . . . . . . . . .            -4 k       Fri Mar 27 08:17:39 2015    used 87989356 k
/ . . . . . . . . . . . . . . . . . .           -36 k       Fri Mar 27 08:17:43 2015    used 87989392 k
/ . . . . . . . . . . . . . . . . . .            -4 k       Fri Mar 27 08:17:55 2015    used 87989396 k
/ . . . . . . . . . . . . . . . . . .             4 k       Fri Mar 27 08:18:11 2015    used 87989392 k
/ . . . . . . . . . . . . . . . . . .           -32 k       Fri Mar 27 08:18:13 2015    used 87989424 k

How to use

1. Copy the script below into an empty file, safe it as log_diskusage.py

2. In the head section of the script, set the time interval the treshold and the max number of lines in the logfile:

   #--- set time interval in seconds, threshold in k, and the max number of lines in the logfile
   interval = 20        # the interval between the checks
   threshold = 0        # in K, you'd probably set this higher
   max_lines = 5000     # if you want no limit, comment out the line line_limit() in the script
   #---
   

   • The interval to run the disk-space checks, as it is, 20 seconds
   • The treshold: You probably wouldn't want to keep record of all (very) small changes, since the disk has a lot of small changes in free disk space. As it is, it is set to 10k
   • The max_lines, since the logfile will grow quickly, especially if you set the treshold to zero

3. Test-run the script with the command:

   python3 /path/to/log_diskusage.py
   

4. If all works fine, add it to your Startup Applications: Dash > Startup Applications > Add.

The script

#!/usr/bin/env python3
import subprocess
import os
import time
log = os.environ["HOME"]+"/disklog.txt"
#--- set time interval in seconds, threshold in k
interval = 1
threshold = 0
max_lines = 5000
#---

def line_limit():
    lines = open(log).readlines()
    if len(lines) > max_lines:
        with open(log, "wt") as limit:
            for l in lines[-max_lines:]:
                limit.write(l)

get = lambda cmd: subprocess.check_output([cmd]).decode("utf-8")

def disk_change():
    mounted = [l[l.find("/"):] for l in get("lsblk").splitlines() if "/" in l]
    data = get("df").splitlines()
    matches = [("/", data[1].split()[-4])]
    for l in mounted:
        if l != "/":
            match = [(l, d.replace(l, "").split()[-3]) for d in data if l in d][0]
            matches.append(match)
    return matches

disk_data1 = disk_change()
while True:
    time.sleep(interval)
    disk_data2 = disk_change()
    for latest in disk_data2:
        try:
            compare = [(latest[0], int(latest[1]), int(item[1])) for item in disk_data1 if latest[0] == item[0]][0]
            if not compare[1] == compare[2]:
                diff = compare[2]-compare[1]
                if abs(diff) > threshold:
                    with open(log, "a") as logfile:
                        drive = compare[0]; lt = 18-int((len(drive)/2)); lk = 14-len(str(diff))
                        s = drive+" ."*lt+lk*" "+str(diff)+" k   \t"+str(time.strftime("%c"))+"\t"+"used "+str(compare[1])+" k\n"
                        logfile.write(s)
                    print(s, end = "")
                    # if you don't want to set a limit to the max number of lines, comment out the line below
                    line_limit()
        except IndexError:
            pass
    disk_data1 = disk_data2

You should use vmstat which is already installed on ubuntu, or iostat but you've to install it. iostat, vmstat and mpstat Examples for Linux Performance Monitoring