Home / server / Questions / 294218
In Process
James Dean
Asked: 2011-07-27 06:34:05 +0800 CST

Is there a way to redirect output to a file without buffering on unix/linux?

  • 772

I have a long running batch process that outputs some debug and process information to stdout. If I just run from a terminal I can keep track of 'where it is' but then the data gets too much and scrolls off the screen.

If I redirect to output to a file '> out.txt' I get the whole output eventually but it is buffered so I can no longer see what it is doing right now.

Is there a way to redirect the output but make it not buffer its writes?

unix stdout

8 Answers

  1. You can explicitly set the buffering options of the standard streams using a setvbuf call in C (see this link), but if you're trying to modify the behaviour of an existing program try stdbuf (part of coreutils starting with version 7.5 apparently).

    This buffers stdout up to a line:

    stdbuf -oL command > output

    This disables stdout buffering altogether:

    stdbuf -o0 command > output
    • 67

  2. You may achieve line buffered output to a file by using the script command like so:

    stty -echo -onlcr   # avoid added \r in output
script -q /dev/null batch_process | tee output.log        # Mac OS X, FreeBSD
script -q -c "batch_process" /dev/null | tee output.log   # Linux
stty echo onlcr
    • 10

  3. On Ubuntu, the unbuffer program (from the expect-dev) package did the trick for me. Just run:

    unbuffer your_command

    and it won't buffer it.

    • 9

  4. The easiest solution that I found (didn't need any third-party packages installed) was mentioned in a similar thread over on at Unix & Linux site: use the script command. It's old, and likely already installed.

    $ script -q /dev/null long_running_command | print_progress       # FreeBSD, Mac OS X
$ script -q -c "long_running_command" /dev/null | print_progress  # Linux

    Note that the first filename parameter for the script command is the log file to be written. If you simply run script -q your_command, you'll overwrite the command you indented to run with the log file. Check man script, to be safe, before trying it.

    • 6

  5. try the script command; if your system has it, it takes a file name as argument, all text dumped to stdout gets copied to the file. It's very useful when a setup program requires interaction.

    • 4

  6. Personally I prefer piping output of a command I want to examine through tee.

    script records too much information, including timing of key presses, and a lot of non-printable characters. What tee saves is much more human readable for me.

    • 3

  7. You can use the tee command, just magic !

    someCommand | tee logFile.log will both display in the console and write into the log file.

    • 3

  8. Redirect the output into a file and follow the file with the tail -f command.

    Edit

    If this still suffers from buffering, then use the syslog facility (which is generally unbuffered). If the batch process runs as a shell script, you can use the logger command to do this. If the batch job runs in a scripting language, there should be a logging facility anyway.

    • 2