pablo's questions

We are running apache solr through sunspot.

What are the best metrics to use to look at the performance of solr in regards to needs to cluster/scale?

I am looking at monitoring these with nagios or sensu and was planning on just graphing requests per second and response time, then apache bench it and see where the breaking point is.

Are there any specific values I should look at or best practice for monitoring? I've seen a few articles around monitoring with JMX but I dont think I can enable that with sunspot and most nagios plugins seem to just monitor if its up or not and response time, not performance data.

I've been looking online for what practices people are using when scaling their site using chef/capistrano and have not found much procedural information on this so thought I would ask here.

We are using chef in a limited way across our servers and we are using capistrano to deploy. For various reasons we will not be using the chef deploy resource to deploy for our app. We will be using chef to bootstrap app servers to scale our website.

From what I can gather what I would do is bootstrap the node with an app server role, then following the run, cap deploy. This will be done with ruby and the fog gem with some custom scripts.

cap HOSTFILTER:ip.add.ress.here production deploy

My questions are:

1. What are some common approaches other people are using when scaling their site and deploying - outside of using chef to deploy
2. Is host filter the best way to accomplish this?
3. Should I be redeploying to all nodes when scaling instead of just single node?

Any other gotchas/etc appreciated.

I currently have a production web site which hosts a large volume 50GB+ of images on s3. For my staging environment at first I was cloning the production bucket into a staging bucket but this has become too large to maintain.

The changes that occur on staging must continue to be isolated to the staging environment and not touch production.

What are the best options for redirecting missing staging assets to production? I have seen this article which is an example: http://containerdiv.wordpress.com/2009/05/01/safely-using-production-assets-in-staging/

This is rails + unicorn + nginx + S3.

Using opscode chef, is there a way to find or loop over all user accounts?

I need to create a configuration file in each users home directory. I do not want to loop over any directory structure for this.

When looking at the output of SHOW INNODB STATUS on mysql what are the most important values to be looking at, and what would be an indicator of poor performance?

Specifically under FILE I/O and BUFFER POOL AND MEMORY.

The DB server is:

High-CPU Medium Instance

1.7 GB of memory
5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
350 GB of instance storage
32-bit platform
I/O Performance: Moderate
API name: c1.medium

This is a follow up to this: Mysql InnoDB table size performance

I have a table that is closely approaching 2 million records. This table stores a history of transactions. This is on a high traffic website but also the table is not accessed regularly. We currently have no slow queries due to this table, but am looking as to when I should expect to need to migrate data from this table to a data archival method.

The server is an amazon ec2 high cpu medium box.

High-CPU Medium Instance

1.7 GB of memory
5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
350 GB of instance storage
32-bit platform
I/O Performance: Moderate
API name: c1.medium


         Engine: InnoDB
        Version: 10
     Row_format: Compact
           Rows: 1677878
 Avg_row_length: 71
    Data_length: 120209408
Max_data_length: 0
   Index_length: 246497280
      Data_free: 0
 Auto_increment: 1914179
    Create_time: 2011-08-07 20:15:29
    Update_time: NULL
     Check_time: NULL
      Collation: latin1_swedish_ci
       Checksum: NULL
 Create_options: 
        Comment: InnoDB free: 7168 kB

Few questions:

• At which point might I start seeing performance degrade due to the amount of records in this table?

• What settings should I be checking on the mysql server, via SHOW INNODB STATUS and on the table itself(ie, row format) to ensure I am maximizing performance as I need it?

• What metrics should I be gathering to calculate performance over time?

I want to try mounting a folder with NFS - I set up the NFS sharing on a windows 2008 R2 server, specified hosts in the NFS permissions (by IP address) and mounted via /etc/fstab

it mounts, but when I try to list the folder, I get 'input/output error'

the owner/group on the local mount point look weird too?

drwx------ 2 4294967294 4294967294 4096 2011-02-10 19:15 data/

I mounted in /etc/fstab as:

10.0.6.55:/share$ /media/data nfs soft,intr,rsize=8192,wsize=8192

What am I doing wrong?