Home / server / Questions / 36432
In Process
Kyle
Asked: 2009-07-07 07:36:34 +0800 CST

svnstat script

  • 772

So I'm building out a shell script to check out all of our relevant svn repositories for analysis in svnstat. I've gotten all of this to work manually, now I'm writing up a bash script in cygwin on my Vista laptop, as I intend to move this to a Linux server at some point.

Edit: I gave up on this and wrote a simple .bat script. I'll figure out the Linux deployment some other way.

Edit: added the sleep 30 and svn log commands. I can tell now, with the svn log command, that it's not getting to the svn log ... this time, it did Applications, and ran the log, and then check out Database, and froze. I'll put the sleep 30 before and after the log this time.

co2.sh

#!/bin/bash
 function checkout {
        mkdir $1
        svn checkout svn://dev-server/$1 $1
        svn log --verbose --xml >> svn.log $1
        sleep 30
}


cd /cygdrive/c/Users/My\ User/Documents/Repos/wc

checkout Applications
checkout Database
checkout WebServer/www.mysite.com
checkout WebServer/anotherhost.mysite.com
checkout WebServer/AnotherApp
checkout WebServer/thirdhost.mysite.com
checkout WebServer/fourthhost.mysite.com
checkout WebServer/WebServices

It works, for the most part - but for some reason it has a tendency to stop working after a few repositories, usually right after finishing a repository before going to the next one. When it fails, it will not recover on its own. I've tried commenting out the svn line, it goes in and creates all the directories just fine when I do that - so its not that.

I'm looking for direction as well as direct advice. Cygwin has been very stable for me, but I did start using the native rxvt instead of "bash in a cmd.exe window" recently. I don't think that's the problem, as I've left top on remote systems running all night and rxvt didn't seem to mind. Also I haven't done any bash scripting in cygwin so I suppose this might not be recommended; though I can't see why not. I don't want all of WebServer, hence me only checking out certain folders like that.

What I suspect is that something is hanging up the svn checkout. Any ideas here?

Edit: this time when I hit ctrl+z to cancel out, I forgot I was on Windows and typed ps to see if the job was still running; and as you can see there are lots of svn processes hanging around... strange.

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ 

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ jobs
[1]-  Stopped                 bash co2.sh
[2]+  Stopped                 ./co2.sh

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ kill %1

[1]-  Stopped                 bash co2.sh

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ 
[1]-  Terminated              bash co2.sh

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ 

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ 

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ ps
      PID    PPID    PGID     WINPID  TTY  UID    STIME COMMAND
     7872       1    7872       2340    0 1000   Jun 29 /usr/bin/svn
     7752       1    6140       7828    1 1000   Jun 29 /usr/bin/svn
     6192       1    5044       2192    1 1000   Jun 30 /usr/bin/svn
     7292       1    7452       1796    1 1000   Jun 30 /usr/bin/svn
     6236       1    7304       7468    2 1000   Jul  2 /usr/bin/svn
     1564       1    5032       7144    2 1000   Jul  2 /usr/bin/svn
     9072       1    3960       6276    3 1000   Jul  3 /usr/bin/svn
     5876       1    5876       5876  con 1000 11:22:10 /usr/bin/rxvt
      924    5876     924      10192    4 1000 11:22:10 /usr/bin/bash
     7212       1    7332       5584    4 1000 13:17:54 /usr/bin/svn
     9412       1    5480       8840    4 1000 15:38:16 /usr/bin/svn
S    8128     924    8128       9452    4 1000 17:38:05 /usr/bin/bash
     9132    8128    8128       8172    4 1000 17:43:25 /usr/bin/svn
     3512       1    3512       3512  con 1000 17:43:50 /usr/bin/rxvt
I   10200    3512   10200       6616    5 1000 17:43:51 /usr/bin/bash
     9732       1    9732       9732  con 1000 17:45:55 /usr/bin/rxvt
     3148    9732    3148       8976    6 1000 17:45:55 /usr/bin/bash
     5856    3148    5856        876    6 1000 17:51:00 /usr/bin/vim
     7736     924    7736       8036    4 1000 17:53:26 /usr/bin/ps

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ jobs
[2]+  Stopped                 ./co2.sh

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$

Here's an strace on the PID of the hung svn program, it's been like this for hours. Looks like its just doing nothing. I keep suspecting that some interruption on the server is causing this; does svn have a locking mechanism I'm not aware of?

Kyle Hodgson@KyleHodgson-PC ~/winUser/Documents/Repos
$ strace -p 7304
**********************************************
Program name: C:\cygwin\bin\svn.exe (pid 7304, ppid 6408)
App version:  1005.25, api: 0.156
DLL version:  1005.25, api: 0.156
DLL build:    2008-06-12 19:34
OS version:   Windows NT-6.0
Heap size:    402653184
Date/Time:    2009-07-06 18:20:11
**********************************************
svn bash cygwin

3 Answers

  1. I've occasionally had problems with a server timeout on large checkouts. You should see this in your server logs (I'm running under apache httpd, I believe it was in the error log files).

    Suggest that you capture the output of the svn checkout command (svn checkout svn://dev-server/$1 $1 > svncolog.txt 2>&1).

    If this is the problem (or even if it isn't), this might be affected by the svn client version you are using under cygwin (are you just using the cygwin package?) -- try upgrading to the latest if you haven't already, or try installing a non-cygwin command-line client and using that in your script.

    I also feel compelled to point out that your bash script is extremely simple at this point: you could easily test this with a cmd (batch) script instead, to take cygwin out of the picture.

    • 1

  2. Wild guess: Does cygwin or windows limit the number of connections? Including the TIME_WAIT status when closed? Try inserting sleep 60 after your checkout.

    • 0

  3. Debugging suggestions:

    • add -x to your #!/bin/bash at the top so you will get a trace of the commands when they run in the script
    • if you've got spare processes hanging around, figure out what they're hanging on by running strace -p on their PIDs
    • if possible, see if it works locally on the svn server to take network issues out of the equation
    • 0