I've read these docs over multiple times but still don't think I understand what startingDeadlineSeconds does.

I'm trying to figure out if it's possible to avoid rescheduling a job if a current one is running (with CronJob) and instead just let the next scheduled job pick up where it left off.

I have some extremely erratic job completion times (a separate problem I need to solve). It may take 1 hour or up to 24. So sometimes, if a Job takes a whole day to complete, but I've scheduled it to run every 12 hours, I get a queue of another job that immediately starts when the last one is complete.

I'd really just like it to never queue up a job and only create new jobs on the next scheduled time.

Here's a sample of the config

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: my-job
spec:
  schedule: "0 */2 * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 1
  jobTemplate:
    spec:
      backoffLimit: 1
      template:
        spec:
          containers:
          - name: my-job
            image: my-image
            command: ["do_something"]
          restartPolicy: OnFailure

Note that do_somethign could take 1 hour, or it could take 4 hours. What I'd like, is when it takes 4 hours, to not have queued up the 2nd job that would have normally run if it had taken less.

Is this possible with Kubernetes CronJob?

I'm just trying to wrap my head around queueing/timeouts in HaProxy between a frontend, backend and a server. I'm still learning about TCP, sockets, connections etc so forgive me if any of these questions don't make sense. I'm just looking for confirmation/clarification on a few of these items:

Client connections and frontend load balancing

Haproxy itself has a global maxconn limit (I'll call this gmc). From what I understand, this gmc is on incoming connections from clients (routed to front-ends).

Each frontend then has its own maxconn setting (I'll call it femc).

If my gmc is 40, and I have two frontends each with femc of 10 and I get 50 requests in at once.

Questions:

• Is it true that 20 will be accepted (10 by each frontend), 20 will be backlogged (10 on each frontend), and the leftover 10 will be backlogged globally at the proxy level?
• Is there a global backlog AND a frontend backlog? Or is the backlog parameter only relevant at the frontend.
• If there is a global backlog, is this just OS specific based on # of file descriptors available?
• If we set our max backlog to be 1, what would happen to the leftover requests coming in? Would they be denied right away? (this depends I guess on previous answer)
• does the global maxconn also apply to backend connections? Is it the upper bound on ALL connections across the whole system?

Server connections and request queueing

Continuing with the example above, we'll add one backend for each frontend, both with two servers with a maxconn of 2 each (smc) and a maxqueue of 1 (smq).

So each backend can handle 4 simultaneous requests (2 per server) and will queue 2 requests (1 per server) 6 requests total per backend

Questions:

• What happens with the extra 4 requests that the frontend has received? Are they "queued" there? Is there a parameter on the frontend that controls this queueing?
• Does a backend have its own queue that isn't configurable? (maxqueue only seems to be at a server level)

I'm asking all of this because we have a legacy server with VERY heterogeneous request properties. Some can happen in 10ms, some 10s. We currently don't have any limits at all on server connections, and we're seeing some of our web server's backlogs filling up behind really slow requests while others are almost idle. Ultimately I'm trying to smooth this out a bit.

Any answers to the above questions would be SUPER helpful!

We're currently running our application on Heroku and are seeing the limitations of its platform (mainly due to the cost of scaling).

We're looking to run our own setup on something like Amazon. There are a few competitors out there in the Cloud that offer pure SSD storage etc. that look attractive.

I wanted to basically find a standard test that I could run to spec out the performance of each of these platforms running a comparable instance.

In no way am I a performance expert so I'm not 100% sure what I'm looking for, I just wanted to see some side-by-sides to make sure we make an informed decision.

Can anyone suggest a test suite (or a few) that might give us a relative benchmark for things like disk, memory, cpu performance etc...?

I just started up a new instance that i build using debian squeeze on amazon last night. This morning I just got an email saying that I reached a limit on the volume of email you were able to send out of SMTP port 25 on your instance

I'm not sys admin min by any means but I'm relatively comfortable using linux, but I've never had this problem before. I tried searching for some mta's installed that i know of, postfix, exim etc.. and saw nothing, so I think in need to go a bit lower level and just monitor port 25 to see what's happening.

Can someone enlighten me as to some tools I can use to find out what mail activity is going on on my server? And maybe some tips on tightening things up, I don't use a mail server at all, i really only run webapps.

Many thanks.

I'm using Hudson as my CI server and I'm trying to get my commits to trigger a build. When I was using gitosis I just had a post-receive that ran a

curl -u gituser:password http://my.hudson.url/project/?token=someToken

I can't seem to find out how to run commands such as this with Github, it only appears to support certain services by sending a POST to a particular url. The reason I'm using curl is for the user authentication. I'm not sure how I can accomplish this with Github.

Has anyone used a similar setup at all?

I have a db server db.example.com CNAME'd to its external ec2 domain ec2-some-ip.aws.com

I have a client ec2-another-ip.aws.com trying to connect to my db.example.com. I granted privileges:

grant all privileges on *.* to 'some_user'@'ec2-another-ip.aws.com';

and in the client try:

mysql -u some_user -h db.example.com

only to get:

ERROR 1130 (00000): Host 'my-client-internal-mac.compute-1.internal' is not allowed to connect to this MySQL server

So it obviously maps its external domain to the internet domain. So How do I get mysql to allow my client given the restriction below.

NOTE I can't use the internal ip as all these servers are ebs root mounted and i start/stop them all the time, so the internal ip changes, but i map an external elastic ip each time i spin it up so its external ip is always the same

I may be wrong in assuming this, but if I've set an expires header of access plus 1 year on a file, and I can see that in the header, when I make another request for that file, shouldn't it just be cached in the browser so I never actually see the request on the server?

I have these response headers on an img:

Connection:Keep-Alive
Date:Mon, 05 Jul 2010 16:17:08 GMT
Etag:"9cc4b2-73b-48a43cbb63300"
Expires:access plus 1 year
Keep-Alive:timeout=15, max=98
Server:Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny6 with Suhosin-Patch

But when I refresh, I still see the request to that image in my apache access log. Is that the expected behaviour? I figured the request would never even make it to my server.

BTW this is what i'm using to set expires headers:

RewriteCond %{REQUEST_FILENAME} ^\/(images|assets|pdfs) [NC]
RewriteRule ^(.*)$ $1 [E=set_expires_header:true,L]
Header add Expires "access plus 1 year" env=set_expires_header

I have an application that has the following rewrite rule:

    RewriteCond /tmp/maintenance.html -f
    RewriteCond %{HTTP_HOST} !^backdoor.myapp.com$
    RewriteRule ^(.*)$ /tmp/maintenance.html [L] 

This is our 'maintenance mode'. so I enable maintenance by linking /tmp/maintenance.html with a real file (and keep backdoor access using backdoor.myapp.com).

Problem is, this page seems to be getting cached by browsers, so when maintenance is turned off, the user has to do a hard refresh in order to get back to the main app.

I'm wondering how I can set the headers for this particular file only in Apache2. I believe there's a no-cache header? I'm assuming I want to set that header for this file, and all should be good.

can someone direct me how to do that, or explain the mechanism by which this static file should never be cached in the user's browser?

OK this question is asked all the time I know: "What's the best way to connect Tomcat through apache". But I still haven't found a definitive answer and maybe there isn't one, but I'm just looking for a viable, high performance setup. So much of the documentation I've read references articles from 2005 and older. I'm looking for a 2010 solution :)

I have a current setup (not created by me) that uses mod_rewrite to rewrite all traffic from apache to tomcat using:

RewriteRule ^(.*) ajp://localhost:8009$1 [P,L]

We have one app as the ROOT app in tomcat. So right now I'm using mod_rewrite to send requests over ajp (is this different from mod_proxy_ajp??). Are there any reasons why this might be less than satisfactory?

Also, I'm looking to get Apache to serve up static assets to take some load off Tomcat, also so I can easily use mod_expires/deflate etc... I'm unsure of how to do this.

I have an 'assets' (ie. js/css) and 'images' folder in the webroot (same directory as WEB-INF) so I'm assuming I need some sort of match on /images and /assets that says "don't pass this on to Tomcat" ??

Any help/suggestions/comments on the current setup are greatly appreciated.

btw i'm using:
apache2 - 2.2.9-10
Tomcat - 5.5.29

Does anyone know if the PrintGCDetails affects java performance much? I've been monitoring our java garbage collecting on a staging server with the same setup as the production server.

I assumed it was safe to say that I shouldn't have this enabled on production but I don't know if there's really any affect on performance.

I've been reading a lot about java memory management, garbage collecting et al and I'm trying to find the best settings for my limited memory (1.7g on a small ec2 instance) I'm wondering if there is a direct correlation between my code size and the permgen setting. According to sun:

The permanent generation is special because it holds data needed by the virtual machine to describe objects that do not have an equivalence at the Java language level. For example objects describing classes and methods are stored in the permanent generation.

To me this means that it's literally storing my class def'ns etc... Does this mean there is a direct correlation between my compiled code size and the permgen I should be setting? My whole app is about 40mb and i noticed we're using 256mb permgen. I'm thinking maybe we're using memory that could be better allocated to dynamic code like object instances etc...

I'm having issues with a JRuby (rails) app running in tomcat. Occasionally page requests can take up to a minute to return (even though the rails logs processed the request in seconds so it's obviously a tomcat issue).

I'm wondering what settings are optimal for the java heap size. I know there's no definitive answer, but I thought maybe someone could comment on my setup.

I'm on a small EC2 instance which has 1.7g ram. I have the following JAVA_OPTS:

-Xmx1536m -Xms256m -XX:MaxPermSize=256m -XX:+CMSClassUnloadingEnabled

My first thought is that Xmx is too high. If I only have 1.7gb and I allocated 1.5gb to java, i feel like I'll get a lot of paging. Typically my java process shows (in top) 1.1g res memory and 2g virtual.

I also read somewhere that setting the Xms and Xmx to the same size will help as it eliminates time spend on memory allocation.

I'm not a java person but I've been tasked with figuring out this problem and I'm trying to find out where to start. Any tips are greatly appreciated!!

update
I've started analyzing the garbage collection dumps using -XX:+PrintGCDetails

When i notice these occasional long load times, the gc logs go nuts. the last one I did (which took 25s to complete) I had gc log lines such as:

1720.267: [GC 1720.267: [DefNew: 27712K->16K(31104K), 0.0068020 secs] 281792K->254096K(444112K), 0.0069440 secs]
1720.294: [GC 1720.294: [DefNew: 27728K->0K(31104K), 0.0343340 secs] 281808K->254080K(444112K), 0.0344910 secs]

about 300 of them on a single request!!! Now, I don't totally understand why it's always GC'ng from ~28m down to 0 over and over.

I've set up a VMWare instance to run CastIron Integration Appliance. I allocated 2gb of memory to the instance, assuming it would take this as physical memory (my server has 8gb total).

When I view top however on the server, the vmware-vmx process has about 100m Resident memory and 1900m Virtual.

Running CastIron it reports that the appliance often hits 50% memory usage. Does this mean I'm using 900mb of harddrive space as memory? I wanted VMWare to use 2gb of physical memory, no swap. Can anyone tell me how to achieve this?

Setup
Debian Lenny 5.0.3
VMWare Server 2.0.2

I'm using hudson to auto-deploy my Rails app to heroku. In my main build job I pull from a Git repo (hosted using gitosis on the same machine), master branch with the following:

URL of repository: /home/git/repositories/my_app.git
Name of repository: origin
Refspec: +refs/heads/master:refs/remotes/origin/master
Branches to build: master

Then, assuming all tests pass, I want to kick off a new build that is the deploy to Heroku.

I can't however figure out how to get that deploy build to checkout the particular revision that this build was using. I understand there's a parameterized trigger plugin that would allow me to pass this revision number, but I don't know how I can tell hudson to checkout this particular revision on the deploy build.

I'm pretty sure this just has to do with my limited knowledge of git, but where in the hudson git config's is there an option to checkout a particular revision?

Otherwise, I could have many commits happen whilst a build is happening, and when it kicks off a deploy build, that deploy build would just check out the HEAD of the branch, which may not be the same as the code that was pushed that triggered this build.

I don't fully understand why I have a refspec in Hudson, then also specify a branch to build, I thought this was the same thing. Can refspec somehow specify the revision number? How would this be referenced if it was passed through with the parameterized trigger plugin?

(I've never used that plugin, but someone else recommended it as a way to pass in vars to a new build, if there's another way I'm all ears)

Are there any dangers of skipping a particular revision if I know we don't need it when using svnadmin dump?

I'm migrating to a new SVN server, and one foolish commit of about 1gb worth of images is causing the dump to take forever. Furthermore, I'm setting up an svnsync to do incremental syncs on each commit, but my initial sync ends up timing out, just on that particular revision.

If I do a dump up to that revision, then do an incremental dump thereafter, could this in any way affect revisions after that??

I'm using hudson as my CI and it works great, builds run their tests, code metrics, all that good stuff. But at the moment, that's it, no automated deployment, I have to manually do that after. I haven't found any sort of capistrano plugin for hudson and I can't even see where I can just run my cap deploy after a successful build in Hudson.

Does anyone have any idea what I need in order to automate a deployment to a testing server on a successful build? I'd like each commit to force a build and in term deploy to testing so I can see everything right away.

I'm on Debian Lenny using apache2. in my proxy.conf I tried adding

Allow from localhost

as suggested in some other forums to get proxying to work. Didn't work. It only worked if I say

Allow from all

My question is this. Are there any security implications to this Allow from all directive? Most people were saying to make this as limited as possible, but "all" is the client right? I want anyone regardless of their IP to be forwarded properly. Is there a better way to configure this?

So I have jira installed and I access i through mydomain.com:8080/jira. I'd really like to access i through jira.mydomain.com. I enabled mod_proxy and put the following in an apache vhost:

ServerName jira.mydomain.com

ProxyPreserveHost On
ProxyPass / http://localhost:8080/jira

But, it forwards me to jira.mydomain.com/jira. Which doesn't work. I changed the base URL in jira to jira.mydomain.com also. What's going on here, why does the extra /jira exist?

I installed jira as a war installation as I'm running other apps through tomcat (confluence, hudson etc)

I tried googling and the only articles I found comparing these two were from 2005 down to 2002. I just set up my first Tomcat instance to run Jira for a project I'm doing. I want to proxy this through port 80 and apache. From what I can tell, I can easily forward the traffic using mod_proxy. What's the difference iwth using mod_jk? Are there any performance/security differences between these two mods? Anyone have any tips/experience setting up forwarding through apache? I'm running debian Lenny.

I'm trying to get Confluence wiki running on my Debian box inside Tomcat. Now, I've never used Tomcat before so I really don't understand a lot of it. Here's my setup:

Debian Lenny 5.0
Tomcat 5.5.26-5
Open jdk openjdk-6-jre

all installed with apt

Tomcat itself works and I can view the admin and manager screens, but when I try to start the confluence app, the logs say:

Oct 6, 2009 1:00:00 AM org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
Oct 6, 2009 1:00:00 AM org.apache.catalina.core.StandardContext start
SEVERE: Context [/confluence] startup failed due to previous errors

Great help :(

I have in my /var/lib/tomcat5.5/conf/Catalina/localhost directory a file named confluence.xml with the following

<Context path="/confluence" docBase="/usr/share/tomcat5.5/atlassian/confluence-3.0.1/confluence" debug="0" reloadable="true">
</Context>

and the confluence dir as specified in that context. Tomcat sees it in the manager, but can't start it.
My server.xml contains the following connector:

<Connector port="8080" maxHttpHeaderSize="8192"
           maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
           enableLookups="false" redirectPort="8443" acceptCount="100"
           connectionTimeout="20000" disableUploadTimeout="true" URIEncoding="UTF-8" />

Which I think is pretty standard. My question is, why can't I see any more info in the logs? How can I get a more specific logging for this app. The Tomcat docs say that a Logger directive inside this context is no longer valid. but telling me "because of previous errors" doesn't really help. Also, if anyone has any tips on setting up Confluence, I'm all ears