I'm trying to start glassfish on an EBS-based AMI of Ubuntu 10.04 64-bit. I have used glassfish on non-ec2 servers with no problems, but on ec2 I get this message:
$ sudo -u glassfish bin/asadmin start-domain domain1 There is a process already using the admin port 4848 -- it probably is another instance of a GlassFish server. Command start-domain failed.
I know that ec2 has requires that firewall rules be modified using ec2-authorize to let outside traffic thru the firewall, as I had to do to make ssh work. This still doesn't explain the port error when all I'm trying to do is start glassfish so I can try
$ wget localhost:8080and make sure it's working.
This is very frustrating and I'd really appreciate any help.
Thanks.
FINAL UPDATE:
Sorry if you came here looking for answers. I never figured out what was causing the problem. I created another fresh instance, installed the same stuff, and Glassfish worked perfectly. Something obviously got boned during installation, but I have no idea what. I guess it will remain a mystery.
UPDATE:
Here's what I get from netstat:
# netstat -nuptl Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 462/sshd tcp6 0 0 :::22 :::* LISTEN 462/sshd udp 0 0 0.0.0.0:5353 0.0.0.0:* 483/avahi-daemon: r udp 0 0 0.0.0.0:1194 0.0.0.0:* 589/openvpn udp 0 0 0.0.0.0:37940 0.0.0.0:* 483/avahi-daemon: r udp 0 0 0.0.0.0:68 0.0.0.0:* 377/dhclient3
UPDATE: One more thing...
I know that the "net.ipv6.bindv6only" kernel option can cause problems with java networking, so I did set this:
# sysctl -w net.ipv6.bindv6only=0
UPDATE: I also verified that it has nothing at all to do with the port number (4848). As you can see here, when I changed the admin-listener port in domain.xml to 4949, I get a similar message:
# sudo -u glassfish bin/asadmin start-domain domain1 There is a process already using the admin port 4949 -- it probably is another instance of a GlassFish server. Command start-domain failed.
UPDATE:
Here are the contents of /etc/hosts:
127.0.0.1 localhost # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts
I should mention that I have another Ubuntu Lucid 10.04 64-bit slice that is NOT hosted on ec2, and set it up the exact same way with no problems whatsoever.
Also server.log doesn't offer much insight either:
# cat ./server.log Nov 20, 2010 8:46:49 AM com.sun.enterprise.admin.launcher.GFLauncherLogger info INFO: JVM invocation command line: /usr/lib/jvm/java-6-sun-1.6.0.22/bin/java -cp /opt/glassfishv3/glassfish/modules/glassfish.jar -XX:+UnlockDiagnosticVMOptions -XX:MaxPermSize=192m -XX:NewRatio=2 -XX:+LogVMOutput -XX:LogFile=/opt/glassfishv3/glassfish/domains/domain1/logs/jvm.log -Xmx512m -client -javaagent:/opt/glassfishv3/glassfish/lib/monitor/btrace-agent.jar=unsafe=true,noServer=true -Dosgi.shell.telnet.maxconn=1 -Djdbc.drivers=org.apache.derby.jdbc.ClientDriver -Dfelix.fileinstall.dir=/opt/glassfishv3/glassfish/modules/autostart/ -Djavax.net.ssl.keyStore=/opt/glassfishv3/glassfish/domains/domain1/config/keystore.jks -Dosgi.shell.telnet.port=6666 -Djava.security.policy=/opt/glassfishv3/glassfish/domains/domain1/config/server.policy -Dfelix.fileinstall.poll=5000 -Dcom.sun.aas.instanceRoot=/opt/glassfishv3/glassfish/domains/domain1 -Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory -Dosgi.shell.telnet.ip=127.0.0.1 -Djava.endorsed.dirs=/opt/glassfishv3/glassfish/modules/endorsed:/opt/glassfishv3/glassfish/lib/endorsed -Dcom.sun.aas.installRoot=/opt/glassfishv3/glassfish -Djava.ext.dirs=/usr/lib/jvm/java-6-sun-1.6.0.22/lib/ext:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/ext:/opt/glassfishv3/glassfish/domains/domain1/lib/ext -Dfelix.fileinstall.bundles.new.start=true -Djavax.net.ssl.trustStore=/opt/glassfishv3/glassfish/domains/domain1/config/cacerts.jks -Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as -Djava.security.auth.login.config=/opt/glassfishv3/glassfish/domains/domain1/config/login.conf -DANTLR_USE_DIRECT_CLASS_LOADING=true -Dfelix.fileinstall.debug=1 -Dorg.glassfish.web.rfc2109_cookie_names_enforced=false -Djava.library.path=/opt/glassfishv3/glassfish/lib:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.22/lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib com.sun.enterprise.glassfish.bootstrap.ASMain -domainname domain1 -asadmin-args start-domain,,,domain1 -instancename server -verbose false -debug false -asadmin-classpath /opt/glassfishv3/glassfish/modules/admin-cli.jar -asadmin-classname com.sun.enterprise.admin.cli.AsadminMain -upgrade false -domaindir /opt/glassfishv3/glassfish/domains/domain1 -read-stdin true
You're right - firewalling has nothing to do with the open (presumed TCP?) port.
netstat is your friend. Something similar to
netstat -nupt -lwill give you an indication of what processes are listening on what ports.Let us know the output of that command, and whether it helps point you in the right direction.
I don't know why your hosts file was brought up, as these have nothing to do with each other.
This sounds to me (based on your revisions) that its not that something is currently listening on the port--but that you may have configured TWO things to listen on the same port in the same Glassfish instance. So if you start Glassfish and it tries to launch listeners on each of the ports, it will fail, and consequently shutdown the previously started services. So when you use netstat before and after, it reports correctly that nothing is currently listening on the ports.
Check that each of your listeners is on a different port using
asadmin, and I'll bet you find that the port number is used twice. Or it could simply be that the initialization is causing two instances to start up simultaneously, which would give you the same behavior. Can you try configuring a brand new domain on different ports to see if you experience a similar issue?Best of luck!
UPDATE: Try using the following to get all the listeners:
same problem here. It is related to /etc/hosts i did a cat /etc/hostname and added this for 127.0.0.1
This error is thrown not only when the port is taken, but also erroneously when there is an error with the
/etc/hostsfile.Make sure you have proper definitions for your hostname (have you changed it recently?) in the file, and that it is not missing. For example, if you just changed your hostname (either via
hostnameor in/etc/hostname) toserver.example.com, then make sure that your/etc/hostsfile has:Recently, I found that launching a server into VPC requires quite a bit of extra configuration (once off) to get started. A key clue:
should work. If it doesn't work, that's likely to be the reason.
You can either: 1) launch a server into EC2-classic 2) configure your VPC / host to be able to run the command above successfully.
Hope that helps.