Jonik's questions

In a config file, I'm trying to replace PREFIX=/jenkins with, say, PREFIX=/foobar.

On the (bash) command line, this works fine:

sed -i.bak s/PREFIX=\\/jenkins/PREFIX=\\/foobar/g /etc/default/jenkins

However, in a Puppet script (.pp file):

exec { 'Change-prefix':
  command => "sed -i.bak s/PREFIX=\\/jenkins/PREFIX=\\/foobar/g /etc/default/jenkins",
  path    => ["/bin"],
}

produces:

err: /Stage[main]//Exec[Change-prefix]/returns: change from notrun to 0 failed:
sed -i.bak s/PREFIX=\/jenkins/PREFIX=\/foobar/g /etc/default/jenkins 
returned 1 instead of one of [0] 

How to escape the sed command properly? Or is something else the matter?

I also tried with \\\/, but that yields: warning: Unrecognised escape sequence '\/'

With default Ubuntu/Debian installation of Jenkins, is there an easy way to change the deployment path? I'd like Jenkins to be available somewhere like http://localhost:8090/foobar

I'm looking for a way to configure it in /etc/default/jenkins, similarly to setting the HTTP port:

# port for HTTP connector (default 8080; disable with -1)
HTTP_PORT=8090

/etc/default/jenkins contains this bit which seems related, but I'm not sure how to use it:

# servlet context, important if you want to use apache proxying  
PREFIX=/jenkins

I have an instance store (not EBS) based EC2 instance, launched from an official Ubuntu 12.04 AMI (specifically ami-25e8d351: eu-west-1, 32-bit, instance root store).

It is up and running, and I've made some changes to it (installed software; tweaked config files).

Now I'd like to bundle the setup as an AMI (on Amazon S3), i.e., save the changes I've made. But I can't do this on the AWS Console:

Why is the "Bundle Instance (instance store AMI)" option greyed out?

And more importantly, is there any way to save this instance as an AMI?

I have an EC2 instance, with instance store root device (i.e., not EBS), running Linux (Ubuntu).

What happens if I restart the instance like so:

sudo reboot

• Will the instance storage still be there after reboot, or does it get wiped?
• How about the public IP address of the EC2 instance (assuming an Elastic IP is not used): does it change or stay the same?
• If an Elastic IP is used, will it still be associated with this instance after reboot?

My Tomcat 7 process, which I run on a server on Amazon EC2, has settings such as these in CATALINA_OPTS which should allow me to connect for JMX monitoring remotely:

-Dcom.sun.management.jmxremote.port=8086
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

However, connecting remotely does not work with either jconsole or jvisualvm. It just times out.

I've triple-checked that the EC2 security group allows access to the JMX remote port from my IP (and only from my IP).

Are there any settings missing?

I'm trying to move data files of a MySQL installation to another place, but it won't work.

When trying to start mysqld, I get this in /var/log/mysql/error.log:

110922  7:27:40 [Note] Plugin 'FEDERATED' is disabled.
/usr/sbin/mysqld: Can't find file: './mysql/plugin.frm' (errno: 13)
110922  7:27:40 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
110922  7:27:40  InnoDB: Initializing buffer pool, size = 512.0M
110922  7:27:40  InnoDB: Completed initialization of buffer pool
110922  7:27:40  InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
InnoDB: File name ./ibdata1
InnoDB: File operation call: 'open'.
InnoDB: Cannot continue operation.

The above happens even if I try something as innocent as:

sudo cp -a /var/lib/mysql /var/lib/mysql2

...and change the datadir setting in /etc/mysql/my.cnf to /var/lib/mysql2

(I get the same result if I leave my.cnf as it is, and create a symlink named mysql2 to point to mysql.)

This is somewhat perplexing. File permissions are exactly the same in the copied datadir. And obviously I'm stopping/starting the daemon before doing these changes (sudo service mysql stop etc.) Any idea what I'm doing wrong?

This is Ubuntu 11.04 on Amazon EC2 (64-bit m1.large instance).

(In reality I'd like to move MySQL data to another EBS volume, to a path such as /mnt/data/mysql or /data/mysql, but the above minimal scenario suffices to reproduce the problem.)

I have customised (EBS-backed) AMI for running a demo version of an application of ours. (The AMI contains Ubuntu 11.04 with a Tomcat & MySQL setup for the app itself, and Jenkins for easy one-click update of the demo).

This is a 32-bit AMI, which means I have the following instance type options:

• Micro (t1.micro)
• Small (m1.small)
• High-CPU medium (c1.medium)

We've noticed that we'd like some more performance for the demo server than what c1.medium can offer. (Specifically, I suspect "I/O Performance: Moderate" might be a bottleneck, although I'm not sure if improving that would help given that we use EBS for everything.)

Anyway, in order to use the more powerful instance types (e.g. "m1.large" or "c1.xlarge"), I would need a 64-bit AMI.

One way to do this is to create a new instance from clean 64-bit Ubuntu AMI, then re-setup my system there, and finally save that as a new AMI. I could mount a volume with the current setup, and then cp -a some stuff over to the new instance's root disk, which would help somewhat. But even so, this approach can be somewhat tedious and time-consuming.

So, my question is, is there any easier, automated way of converting a 32-bit AMI to a 64-bit one?

After Amazon's Aug 8 outage, all (EBS based) AMIs stopped working for many users. This is due to corruption of some sectors in snapshots that the AMIs are based on.

However, Amazon created recovery snapshots where the disk problems should be fixed. Those are named along the lines of "Recovery snapshot for vol-xxxxxxxx".

I created a new AMI from recovery snapshot which worked fine, but instances launched from this new AMI do not work: their state is "Running", but I cannot ssh into the machine nor access any of the web services that should be running there. It boils down to this (from System Log, accessible through AWS management console):

EXT3-fs: sda1: couldn't mount because of unsupported optional features (240).

EXT2-fs: sda1: couldn't mount because of unsupported optional features (244).

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1)

I've mounted a volume created from that recovery snapshot in a another server on AWS, and everything looks quite normal though. For example, fsck says:

$ sudo fsck -a /dev/xvdg
fsck from util-linux-ng 2.17.2
uec-rootfs: clean, 53781/524288 files, 546065/2097152 blocks

In one of the AWS forum discussions, I found this advice from someone with similar problems:

A work around will be to make a volume from the snapshot and attach it to a running instance, use fsck --force to force the checking of the filesystem and once cleared, you can make a snapshot and use it for the AMI.

But I don't know how to force fsck on Ubuntu (11.04):

$ sudo fsck --force /dev/xvdg
fsck from util-linux-ng 2.17.2
fsck.ext3: invalid option -- 'o'

Anyone know how to force file system check on the volume on Ubuntu? Any other ideas on how to launch working instances that are based on the recovery snapshot?

Right now it looks like it might be quicker to just start over from a clean Ubuntu AMI and re-setup all our services. :-( But of course I would prefer not to do that if there's any way to get the recovery snapshot to actually work.

I would like a user to have sudo rights (without password check) to a couple of shell scripts under a specific directory (in my case, /usr/local/tomcat7/bin), and to nowhere else. What's the simplest way to accomplish this?

Something like this in /etc/sudoers didn't seem to work:

jsmith ALL=(ALL) NOPASSWD: /usr/local/tomcat7/bin

Is there any way to deploy my Java application .war file at root on Tomcat 7 server, using the Deploy plugin for Jenkins?

Currently the app gets deployed at http://server/foo/ if it comes in a file named foo.war.

I'm running an EBS-backed instance which acts as a software development team's build server (running Jenkins and host of other services). The server is running Linux (latest Ubuntu from the official AMIs).

I'd like to take regular, automated snapshots of the instance's associated EBS volume. I only need to keep one latest backup (i.e. old snapshots should be pruned), and a good frequency would be once a day.

It seems that Amazon does not provide such backup service out of the box, so you have to either go with 3rd party scripts or roll your own solution.

My question is, what is the simplest way to achieve this? I'd like a minimal amount of hassle, configuration, and external dependencies. Setting this up as some kind of timed script on the Linux box itself is, to my knowledge, a valid option.

I have an EC2 instance running, and it belongs to a security group. If I add a new allowed connection to that security group through AWS Management Console, should that change be effective immediately? Or perhaps only after restart of the instance?

In my case, I'm trying to allow access to PostgreSQL's default port (tcp 5432 5432 0.0.0.0/0), and I'm not sure if it's the EC2 firewall or PostgreSQL's settings that are refusing the connection.

I'm looking for a quick way to protect a Tomcat instance and all webapps running on it, so that accessing any page requires credentials (simple username/passwd).

I guess Realm is the "proper" way to do it, but that doesn't seem very simple to set up. We would prefer a way where you don't have to change the webapps themselves at all. Anyone know if there's a "quick and dirty" way to achive that?

What is the recommended way of getting command-line Amazon EC2 tools on Debian? So, basically the same as this question, but for EC2 instead of S3.

Ubuntu has ec2-ami-tools and ec2-api-tools, but I couldn't find equivalent packages for Debian. A blog post titled "Install EC2 AMI & API tools in Debian" talks about installing Amazon's packages outside package management, but that seems a little clumsy.

I need to (programmatically, in a shell script) upload an EAR file to an Amazon S3 bucket on Debian (5.0.4). What, if any, Debian package provides simple, scriptable tools for that?

Ideally this would be a set of command line tools akin to those in Ubuntu's ec2-api-tools package.

(I want raw S3 bucket access, so please don't suggest solutions like Jungle Disk.)

I'm looking into running clustered JBoss on Amazon Web Services (AWS).

I'd like to try out S3_PING, i.e. making JBoss use an S3 bucket for dynamic node discovery etc, since no multicast is available. I found a piece of example config XML related to S3_Ping, but I'm not sure where in JBoss installation you're supposed to configure this.

So, what JBoss config files would I need to tweak to get S3_PING working? Can anyone point me to a more complete example?

JBoss 5.1.0 GA. (This is probably more a JGroups/JBoss question than anything else. I've already got the S3 bucket for this set up, so no problem there.)

A simple newbie JBoss question.

I've got JBoss 5.1.0 GA installed (from here) and running, and I'm trying to access the administration console:

What are the default credentials (if any) and where do you configure these? I didn't find an answer in the JBoss Installation And Getting Started Guide.

The server is running Ubuntu 9.10 but I believe that is irrelevant, as JBoss is installed outside package management.

Possible Duplicate:
Run a shell script as a different user

While writing scripts to automate the intialisation of SVN mirror repositories (svnsync), I again ran into problems trying to start another script with another user's privileges.

#!/bin/sh
# This outer script is always run with root privileges ("sudo create-repo.sh")

# Create repository, set privileges, etc
[...]

# Set up remote mirror; this works fine! (SSH key authentication)
ssh svnsync@anotherserver /var/lib/svn/init-mirror.sh $REPO

# Set up local mirror as "svnsync" user
# ???

How can I call the other script, on the same machine, as a different user? Note that the outer script is always executed as root so su'ing as anyuser should not be a problem.

I tried something like:

su svnsync; /path/to/init-mirror.sh $REPO; exit

and get weird results. How are you supposed to do it? When it's so easy remotely it's probably possible locally, right? (For now, let's pretend running the outer script as the same "svnsync" user is not an option.)

I think the question title sucks, so please help me reword it if you can.

On a Debian (5.0.3) server I have a user svnsync who owns a certain shell script:

-rwsrwsr-x 1 svnsync users  119 Dec 21 13:45 mirror-svn.sh

I'd like everyone in the users group to be able to execute this script with svnsync's privileges. This is related to triggering "svnsync synchorize" commands from post-commit scripts; quoting svnbook:

[...] you might wish to have your primary repository push changes to one or more blessed mirrors as part of its post-commit and post-revprop-change hook implementations. This would enable the mirror to be up to date in as near to real time as is likely possible.

Anyway, I can't get SUID working, apparently because Linux / Debian is one of the modern Unix systems referred to here:

Some modern UNIX systems ignore the SUID and SGID bits on shell scripts for this reason.

This SF question suggests the same thing: "you cannot use SUID root for shell scripts". So, here's my follow-up question:

If I really really want to run a script with the privileges its owner, regardless of any potential risks, is there any hassle-free way to do that? Compiling the script into a binary was suggested, but I'd prefer a simpler way if at all possible. How about calling the shell script from e.g. a Perl script (I actually tried this but couldn't get it working)? Adding everyone to sudoers file is not really a good option either.

Update: got it working by installing perl-suid as 0x89 suggested and using a Perl wrapper script like the following.

#!/usr/bin/suidperl -T
$ENV{PATH} = "/bin:/usr/bin";
system("/path/to/mirror-svn.sh");

chmod +s is set on this wrapper script. Also note that $ENV{PATH} needs to be set in the script; otherwise you'll get a complain that it's insecure.

A Debian Stable (5.0.3) server is running ntpd, and connected to the internet. Still, the system clock is about 5 minutes wrong.

$ /etc/init.d/ntp status
NTP server is running..

Relevant parts (I think) of /etc/ntp.conf:

driftfile /var/lib/ntp/ntp.drift

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

server 0.europe.pool.ntp.org
server 1.europe.pool.ntp.org
server 2.europe.pool.ntp.org
server 3.europe.pool.ntp.org

I know NTP doesn't necessarily bring the clock in time immediately. Still, how many hours or days you need to wait in order to reasonably expect that NTP has done its job and synced the clock?

Am I missing some other configuration file or option, or just doing something wrong? Is ntp (instead of e.g. ntpdate) the right tool for this? Is there any quick way to check if configuration is correct and whether the chosen NTP servers return the correct time?

Edit: output of ntpq -p is:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 ns1.nexellent.n .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 dnscache-madrid .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 sinister.wzw.tu .INIT.          16 u    - 1024    0    0.000    0.000   0.000
 dnscache-frankf .INIT.          16 u    - 1024    0    0.000    0.000   0.000

Edit 2: Turns out ntpdate -u 0.europe.pool.ntp.org command (suggested by brent) returns

17 Dec 17:37:29 ntpdate[14195]: no server suitable for synchronization found

...even though on other machines that command works fine. So we'll be looking at network/firewall settings for this particular server (which is in a different network, accessed over VPN).

Resolution: The culprit wasn't local firewall on our server, but firewall settings somewhere in the surrounding network. So we asked the server hosting provider to allow NTP for our machines, and now it works fine. For example, ntpq -p now returns:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 ns1.eunet.fi    192.36.144.23    2 u   10   64    1    1.043    0.258   0.001
 ns2.eunet.fi    62.142.10.44     2 u    9   64    1    0.671    0.135   0.001
 ns3.eunet.fi    62.142.10.44     2 u    8   64    1    0.750    0.277   0.001

(We also switched to eunet.fi servers recommenced by the hosting company, but that is beside the point.) The commands in brent's answer were helpful because they made me realise the problem was in network access to the NTP servers, not in NTP configuration itself. Thanks everyone!