I am passing the following simple shell script to bash on an LXC container:
apt-get update
apt-get install postgresql -y
sudo -u postgres psql -c 'create database dvdrental;'
The actual command I'm using to run it is:
cat sample.sh | lxc-attach -n test-container -- /bin/bash
The reason I'm doing it this way instead of uploading the script into the container and executing that way is that this is just the proof of concept for a much more complex application we're building that has to take commands over stdin and run them in the container.
It seems to work great, except for one thing. It moves onto the psql
command while postgresql is still installing, i.e.,
[...]
Get:21 http://archive.ubuntu.com/ubuntu/ trusty/main ssl-cert all 1.0.33 [16.6 kB]
Get:22 http://archive.ubuntu.com/ubuntu/ trusty-updates/main postgresql-common all 154ubuntu1 [103 kB]
Get:23 http://archive.ubuntu.com/ubuntu/ trusty-updates/main postgresql-9.3 amd64 9.3.10-0ubuntu0.14.04 [2,669 kB]
Get:24 http://archive.ubuntu.com/ubuntu/ trusty-updates/main postgresql all 9.3+154ubuntu1 [5,038 B]
Fetched 5,834 kB in 28s (207 kB/s)
Preconfiguring packages ...
sudo -u postgres psql -c 'create database dvdrental;'
Selecting previously unselected package libroken18-heimdal:amd64.
(Reading database ... 14599 files and directories currently installed.)
Preparing to unpack .../libroken18-heimdal_1.6~git20131207+dfsg-1ubuntu1.1_amd64.deb ...
Unpacking libroken18-heimdal:amd64 (1.6~git20131207+dfsg-1ubuntu1.1) ...
Selecting previously unselected package libasn1-8-heimdal:amd64.
[...]
Note the existence of the sudo -u postgres psql -c 'create database dvdrental;'
line in the middle of the output. Interestingly it always shows up right after the downloading portion of the apt-get command is completed...
Anyone know what could be causing this?
Ooooh, this is a fun one.
The short answer: it happens there because apt (or something it forks) is reading stdin at that point in its execution, and it reads the remaining lines of the script because that's what is still sitting in stdin at that point. The short fix: put
</dev/null
at the end of theapt-get install
line, and move on with your day.The long answer (seriously, this is a biggun): there is nothing special about stdin/stdout/stderr, from the point of view of a running process. They're just file descriptors, and file descriptors are shared between processes when they're forked. So, what happens (more-or-less) is:
The copy of bash running interactively in your terminal opens a new
pipe
(2), then forks a new process, which closes the existing stdout, and then makes the stdout file descriptor (1) the writer end of the pipe (seedup2
(2)). That child process thenexec
scat sample.sh
, which reads the file and writes it to what it thinks is stdout (but is actually the writer end of a pipe).The copy of bash running interactively in your terminal forks another new process, this time closing the existing stdin, and then makes the stdin file descriptor (0) the reader end of the same pipe discussed previously (again, with a call to
dup2
). This process thenexec
s yourlxc-attach
process.If nothing interferes with stdin along the way (which it doesn't, in this particular case) then every process which is forked from the one which got the reader end of the pipe as stdin will also have that exact same file descriptor, attached to the same pipe, which had the contents of
sample.sh
stuffed into it, as its stdin. Any process which reads from that file descriptor will now have consumed the bytes read, and no other process which reads from that file descriptor will get those specific bytes. Take careful note of this; you will see this material again.When the bash at the far end of your Italian water-closet-style plumbing extravaganza finally gets going, it will read "some" of the data from the pipe which is its stdin (because that's what bash does, when called without arguments and without a tty as stdin). Through the magic of
strace
, I've just confirmed that bash does actually read its input one character at a time (rather than reading it in, say, 4k chunks), so every individual character that isn't part of a command that bash has, or is currently, executing, will still be sitting in the pipe-which-bash-has-as-its-stdin.When bash executes the second command in your script,
apt-get install
tra la la, it forks a new process. Which inherits all the file descriptors of bash, including (most importantly) our good friend the pipe-which-is-stdin. This then also happens for any processes whichapt-get
forks (which is, let me assure you, quite a lot). One of those, orapt-get
itself, is deciding to read stdin and write whatever it reads to stdout (or possibly stderr).When the
apt-get install
finishes, bash finds out what the next thing to execute is by reading from stdin once more. Because something else already read everything out of the pipe, though, there's nothing left, and bash figures "oh well, I guess I'm done then" and exits. Again, the pipe is empty because something else already read it dry, and everything that shares a single file descriptor shares the bounty from it.The solution to the "shared stdin" problem is, unsurprisingly, to stop passing stdin around like a bong at a frat party. Since you can't stop
fork
(2) from automatically giving everyone the same file descriptors, you need to tell bash, instead, to giveapt-get
(and anything else taking an illicit sip from that sweet, sweet pipe) something else to toke on, instead. The easiest thing to give is/dev/null
-- that ever-faithful, never-full, source of all your "Dave's not here, man" delights. That is the domain of "input redirection", which is what the</dev/null
does -- it says, "hey bash, before youexec
thatapt-get
, swap out stdin (file descriptor 0) with the file descriptor you get from opening/dev/null
".An exercise for the reader, to finish off: try putting
</dev/zero
after theapt-get install
command, and explain why what happens happens.