how to rotate nohup.out file without killing my application

You should take a look at flog in combination with logrotate. If you pipe your application output through this you can then SIGHUP the flog process without having to kill your running application.

flog (file logger) is a program that reads input from STDIN and writes to a file.

If SIGHUP is received, the file will be reopened, allowing for log rotation [see logrotate(8) on RH.]

The log file will only be reopened if flog detects that rotation has occurred (ie, old file gone or inode changed). flog is extremly small (less than 500 bytes memory footprint.)

Not sure if I'm too late on this. But for others who stumble upon this forum now, I didn't have to try out anything with flog or anything.

My job starts simply like this:

nohup <my process> &

This starts my job and starts appending to a nohup.out in that directory.

All I had to do was to setup a logrotate which usually comes with the distro and configure something like this in /etc/logrotate.conf

~/my abosolute path/nohup.out {
    size 1024k
    copytruncate
    rotate 100
    maxage 100
}

And it worked!! nohup.out was getting truncated while getting the new appends on the same file. The process didn't die either. And a new nohup.out like nohup.out-20190528 this was created.

Hope this helps.

You cannot rotate it. You can truncate it with the command >nohup.out, which will delete all contents in the file.

You can copy the file first and then truncate it, if you need to save the output. But there is a small window for a race, where output can be written after you copied the file but before you truncated it. Any output written during that window will be forever lost.

You can do something like this:

cp nohup.out nohup.out.save # save it somewhere
echo "" > nohup.out

You don't need to kill your app.

You can't.

You can unlink the file (rm) but the data still has a footprint on the disk and will continue to be written to as long as there is an open file handle.

You can rename the file - but again this does not stop it being written to (but if you are starting background jobs regularly the newer ones will write to the same file).

Really you should explicitly redirect the output of the job to something designed to handle this (e.g. apache rotatelogs).

Logrotate has a copytruncate option available, which will truncate (empty) your specified file after it has been copied into the normal rotation (and compression, if set) scheme.

   copytruncate
          Truncate  the  original log file in place after creating a copy,
          instead of moving the old log file and optionally creating a new
          one,  It  can be used when some program can not be told to close
          its logfile and thus might continue writing (appending)  to  the
          previous log file forever.  Note that there is a very small time
          slice between copying the file and truncating it, so  some  log-
          ging  data  might be lost.  When this option is used, the create
          option will have no effect, as the old log file stays in  place.

From the logrotate manpage.

You can use below script to rotate/limit logs. You can change the limit of file size or how often to check the size in the script. You can add other commands too in the while loop to copy this file to another location. Current version only truncates the file when its size is found to be greater than 10000 bytes. This will run the script in the background. If you want to keep it running after logging out, use nohup to run this script so that it does not stop after logging out.

while true; do SIZE=`stat --printf="%s" nohup.out`; if [ $SIZE -gt 10000 ]; then echo "">nohup.out; fi; sleep 60; done; &