SnapOverflow

SnapOverflow Logo SnapOverflow Logo

SnapOverflow Navigation

  • Home
  • Server
  • Ubuntu

Mobile menu

Close
  • Home
  • System Administrators
    • Hot Questions
    • New Questions
    • Tags
  • Ubuntu
    • Hot Questions
    • New Questions
    • Tags
  • Help
Home / ubuntu / Questions / 154255
Accepted
Jorge Castro
Jorge Castro
Asked: 2012-06-22 09:28:08 +0800 CST2012-06-22 09:28:08 +0800 CST 2012-06-22 09:28:08 +0800 CST

How can I tell if I am out of inotify watches?

  • 772

I use an application that consumes inotify watches. I've already set

fs.inotify.max_user_watches=32768

in /etc/sysctl.conf but last night the application stopped indexing unless I ran it manually, which leads me to suspect I am out of watches.

Since I don't know what the trade off is when I increase this number (does it consume more RAM?), I don't know if I should just increase this number, so I'd like to know if there's a way I can tell if it's using all these watches and what the tradeoffs might be for increasing it.

kernel
  • 2 2 Answers
  • 53750 Views

2 Answers

  • Voted
  1. Best Answer
    ish
    2012-06-25T07:17:34+08:002012-06-25T07:17:34+08:00

    How do you know if you are out of watches? tail will tell!

    • Start tail with the -f (follow) option on any old file, e.g. tail -f /var/log/dmesg:
      • If all is well, it will show the last 10 lines and pause; abort with Ctrl-C
      • If you are out of watches, it will fail with this somewhat cryptic error:
        tail: cannot watch '/var/log/dmsg': No space left on device

    For the curious: Why is tail a "telltail"?

    • Actually, any well-written app should have the courtesy of telling you, since the inotify API/calls clearly tells them what the deal is.
    • Try strace tail -f ... instead, and when it succeeds, it ends with:
      inotify_add_watch(4, "/var/log/dmesg", IN_MODIFY...) = 1
      
    • but if it fails, i.e. you are out of watches, it'll say:
      inotify_add_watch(4, "/var/log/dmesg", IN_MODIFY..)
      = -1 ENOSPC (No space left on device)
      

    Can you increase the watches? By how much? Any tradeoffs?

    Short answer: Sure, no sweat. Go to straight to a half-million (524288) if you want...the additional memory used should be negligible on a modern system with 4GB+ of memory.

    • Each used inotify watch takes up 540 bytes (32-bit system), or 1 kB (double - on 64-bit) [sources: 1, 2]
    • This comes out of kernel memory, which is unswappable.
    • So, assuming you set the max at 524288, and all were used (improbable), you'd be using approx. 256MB/512MB of 32-bit/64-bit kernel memory

      • Note that your application will also use additional memory to keep track of the inotify handles, file/directory paths, etc. -- how much depends on its design.
    • What's the max value? I guess none, in theory, as long as you have enough RAM. In practice, 524288 has been officially recommended by apps, and people have been setting it to 2 million, with the accompanying memory usage, of course.

    • 75
  2. Bruno Pereira
    2012-06-25T02:13:58+08:002012-06-25T02:13:58+08:00

    I don't know if I should just increase this number

    The easy way of checking if you reached your max_user_watches value is, with your user, to use inotifywatch, from the package inotify-tools, and check if you can still collect information from a file.

    For example inotifywatch -v /home/bruno/.profile for me returns:

    Establishing watches...
    Total of 1 watches.
    Finished establishing watches, now collecting statistics.
    

    So inotify has no issues creating a new watch, no issues here.

    If you have reached your maximum limit in inotify watches it will return something like

    Failed to watch /home/bruno/.profile; upper limit on inotify watches reached!
    

    If you see something like this then you have reached the limit and will need to increase the allowed watches limit.

    Does it consume more RAM?

    Yes, it does. But according to this old article the amount it consumes is minimal compared with other aspects of a running desktop.

    --MEMORY USAGE--

    The inotify data structures are light weight:

    inotify watch is 40 bytes inotify device is 68 bytes inotify event is 272 bytes

    So assuming a device has 8192 watches, the structures are only going to consume 320KB of memory. With a maximum number of 8 devices allowed to exist at a time, this is still only 2.5 MB

    Each device can also have 256 events queued at a time, which sums to 68KB per device. And only .5 MB if all devices are opened and have a full event queue.

    So approximately 3 MB of memory are used in the rare case of everything open and full.

    Each inotify watch pins the inode of a directory/file in memory, the size of an inode is different per file system but lets assume that it is 512 byes.

    So assuming the maximum number of global watches are active, this would pin down 32 MB of inodes in the inode cache. Again not a problem on a modern system.

    I am of course assuming things did not change a lot since the article was written but looking at the numbers I would not worry and increasing the limit will not increase RAM consumption by much.


    Related posts about inotify

    • Dropbox error - 'echo 100000 | sudo tee / proc/sys/fs/inotify/max_user_watches'

    • kernel-inotify-watch-limit-reached

    • 13

Sidebar

Stats

  • Questions 681965
  • Answers 980273
  • Best Answers 280204
  • Users 287326
  • Popular
  • Answers
  • Marko Smith

    How to delete a non-empty directory in Terminal?

    • 4 Answers
  • Marko Smith

    How to unzip a zip file from the Terminal?

    • 9 Answers
  • Marko Smith

    How can I copy the contents of a folder to another folder in a different directory using terminal?

    • 8 Answers
  • Marko Smith

    How do I install a .deb file via the command line?

    • 11 Answers
  • Marko Smith

    How do I run .sh scripts?

    • 16 Answers
  • Marko Smith

    How do I install a .tar.gz (or .tar.bz2) file?

    • 14 Answers
  • Marko Smith

    What command do I need to unzip/extract a .tar.gz file?

    • 8 Answers
  • Marko Smith

    How to list all installed packages

    • 24 Answers
  • Marko Smith

    Unable to lock the administration directory (/var/lib/dpkg/) is another process using it?

    • 25 Answers
  • Marko Smith

    Change folder permissions and ownership

    • 9 Answers
  • Martin Hope
    ubuntu-nerd How to unzip a zip file from the Terminal? 2011-12-11 20:37:54 +0800 CST
  • Martin Hope
    pandisvezia How can I copy the contents of a folder to another folder in a different directory using terminal? 2011-12-11 17:19:37 +0800 CST
  • Martin Hope
    Scott Szretter Where is the cron / crontab log? 2011-08-12 04:06:43 +0800 CST
  • Martin Hope
    TheXed How do I install a .deb file via the command line? 2011-05-07 09:40:28 +0800 CST
  • Martin Hope
    EmmyS What command do I need to unzip/extract a .tar.gz file? 2011-02-09 14:50:41 +0800 CST
  • Martin Hope
    Ivan How to list all installed packages 2010-12-17 18:08:49 +0800 CST
  • Martin Hope
    La Ode Adam Saputra Unable to lock the administration directory (/var/lib/dpkg/) is another process using it? 2010-11-30 18:12:48 +0800 CST
  • Martin Hope
    David Barry How do I determine the total size of a directory (folder) from the command line? 2010-08-06 10:20:23 +0800 CST
  • Martin Hope
    jfoucher "The following packages have been kept back:" Why and how do I solve it? 2010-08-01 13:59:22 +0800 CST
  • Martin Hope
    David Ashford How can PPAs be removed? 2010-07-30 01:09:42 +0800 CST

Related Questions

Trending Tags

10.10 10.04 gnome networking server command-line package-management software-recommendation sound xorg

Explore

  • Home
  • Questions
    • Hot Questions
    • New Questions
  • Tags
  • Help

Footer

SnapOverflow

About Us

  • About Us
  • Contact Us

Legal Stuff

  • Privacy Policy

Help

© 2022 SOF-TR. All Rights Reserve