Zuhaib's questions

Trying to import ~400m docs in to Elasticsearch from couchdb using the couchdb river plugin. Everything starts out great with indexing time around 5k/s but after a few hours come back and find its hitting the floor around 20/s. We have the system on a beefy box, a x1.xlarge, and all its doing is Elasticsearch. We have a 20 shard with no replication to help with the indexing and disable index refreshing. Heap is setup to use 65% of memory and we are using Java 7 latest from oracle.

What setting do i need to tune to help the initial data importing? I have played with bluk timeouts/size but still cant find the sweet spot.

Any help would be great. Zuhaib

So I got a unique problem. Our app is designed to server large amount of files using a special link that is one time use. We have been playing with ELB but notice an interesting "bug", if you start to get a lot of traffic or load it looks like ELB will try to scale out and retry the connection. This in turns causes our system to invalidate the link before the download is done.

So quickly I put together a nginx proxy server to test the issue and in that setup the connection does not get invalided but I am seeing performance drop by about 2/3 to 1/2. Seeing as I cant throw more bandwidth at the instance as I am locked to what AWS gives me what alternatives do I have? Maybe there is a configuration with the instance/nginx I am not doing correctly or do I need to move to something like RR DNS and pay the price with that.

I am playing around with Amazons EC2 new free tier plan and noticed the default ubuntu install comes with a user ubuntu that is preconfigured to work with your key you are provided from AWS. One thing I would like to do is add a new user and remove the default "ubuntu" user. From my experience with other VPS (VMWare and Rackspace cloud) you only have root on your fresh install (tho that is with Red Hat and not Ubuntu).

Should I have any problems removed the default ubuntu user?

Or should I just not touch the user "ubuntu" and create a new user for my to login and do my work.

I am in the process of playing around with ESX and one of the projects I took on is converting my old NAS to a VM. This was pretty straight forward but now I have some problem with storage.

The NAS had 3 drives, one was OS/Apps only and the rest was Data. Now I need the VM to connect to the 2 drives directly and cant seem to see how to do this in ESX4. When I go to "Add Hardware" and select SCSI Drive I only see my CD Rom Drive. The ESX installation itself see the added drive, but it seems I cant get the VM (Ubuntu) to access it. Can ESX be setup to allow VM access to physical hard drive, and if so how.

Thanks