undefined's questions

apologies for the newbie nature of this question .. I am a web developer though don't have too much experience maintaining servers or debugging and fixing performance issues.

I have a Moodle site (Moodle 2.2.3) running on an Apache server (LAMP stack) that uses a MySQL database.

Recently, when certain users try to log in they have been reporting a timeout - they simply see a blank page after a given time. Some users are able to log in fine. This appears to be determined by which courses the user is enrolled in. An admin user with no courses for example can log in fine.

I have been able to extend the timeout value in php.ini from 60 seconds to 120 seconds and see the same behaviour. Using $top I can see that the CPU used by www-data spikes around 99% until the timeout kicks in.

I am assuming there is a long call to the database being made, or similar.

How can I see error messages logged by mysql? how can I see error messages logged by PHP? The default error_log (/var/log/php.log) which is defined in my php.ini doesn't appear to be there? How can I get PHP to log errors to this file so I can see whats happening?

Moodle also logs errors to logs/error.log but this doesn't shed any light on things.

All and any good suggestions welcome - thanks!

As part of my web application users can upload photos, audio and video files from a Flash SWF to my EC2 based upload server. I want to find a tool that I can use to stress test the EC2 instance (which runs behind an Elastic Load Balancer). Are there any tools that can simulate concurrent users uploading a range of media files to PHP scripts on this server?

I want to get some advice / best practices for how to go about developing a strategy for scaling up my a web application. I waffle on a bit here and show my limited knowledge but want to fill in my knowledge gaps. It is tempting to learn as much of this as possible myself but realise I need to look at getting some outside help so as a general question it would be good to know what the easiest things to outsource are.

My background - I am a developer who has primarily worked developing user interfaces and have been working in Flash and PHP to develop the functionality of the app which sees users uploading images and video to share online.

The system architecture is as follows -

1. a single web server which also acts as database server (MySQL). This server is on a managed hosting package with a trusted and reliable hosting company. The webserver serves up PHP pages and Flash SWFs which are the main UI components
2. Amazon S3 buckets for storage of user's images, video and audio files.
3. User interface components are either PHP pages, or Flash SWFs, for example images and video are viewed through Flash swfs which query the database through an AMFPHP service to get urls of the images and video files to load. These are then received from Amazon S3 buckets. Another FLash SWF handles uploads and POSTS files to a PHP script running on an EC2 instance on Amazon Cloud.
4. The upload server for managing image, video and audio uploads. This is an Amazon EC2 instance running behind an Elastic Load Balancer set to add more instances when it sees CPU capacity reach 80%.
5. We also use a third party service which also runs on Amazon EC2 for transcoding video files.

So for the large part I think things are set up ok to be able to scale up. But I have no experience of how to scale up or manage a high traffic web application and so will be relying on our webserver hosting company to manage the setup of a scalable configuration of web/app server and database server.

Hardware / architecture scaling -

As I understand it the first steps here will be to separate the web server and database and have a database server running on its own, place the webserver behind a load balancer and ultimately to have a master/slave configuration for the database servers. What should I ask my web hosting company to do? What are the issues around doing this and what are the implications for things like my AMFPHP services, the different types of queries - writes and reads? I have a separate script which contains the database connection details which gets included in the globals.php script so I can easily update the connection details in a single step. Am I right that in a master / slave config all writes usually go to the master server and reads come from a slave? would this mean I would need to look at all my database queries and make sure that if its an "UPDATE" or "DELETE" query that it goes to the master database server? In my PHP code the database queries are spread throughout the scripts and called from within functions as needed. I have read a little about database abstraction but do not fully understand the significance of this approach.

Code optimisation for scaling -

What are the things I need to think about changing in my code to make things more scalable? What are the common things in PHP that are affected by scaling?

Security -

What are the common things relating to security that I need to be aware of when thinking of dealing with large volume.

Database optimisation, backup and recovery procedures -

What are the best ways to implement an automated database backup and recovery strategy for a MySQL database for a large database. Should I think about splitting my database - I keep all data on a single server though there are several databases such as a database of user membership information, a database for holding data about user uploads, a database with data about the running of the site such as things that are more related to administrative functions. I want to be able to analyse the databases to be able to see how things are working and get a picture of how the web app is being used, developing so assume it is better to take a snapshot of the database on a periodic basis and store these so I have a timeline of how things have developed over time and use some sort of database analysis tool to query these snapshots to track how things have changed. Is there any good advice about how to go about this, tools that allow both myself and other non-technical people to analyse the data and produce reports.

This about covers my understanding of the areas I will need to address when developing a scale-up strategy. What do people think i need to do to successfully manage the above issues? Am I right in my assumptions? What have I missed? What other things should I consider? Are there any good resources out there for dealing with this? If I was to recruit someone to help with these issues what are the skills / experiences that I should expect the perfect candidate to have?

All advice / experiences / suggestions gratefully received

cheers!

What are the pros/ cons of setting up a subdomain to use as a dev environment on the same server as a live webserver?

the pros for me are it's already set up - dev.mysite.com, I dont need to worry about setting up another server with same config.

What are the other considerations about this?

How can I keep updated of changes to the range of IP addresses that Amazon will use for EC2 instances. I want to add a range of IPs to my firewall settings to allow access to my 'ground based' mysql database from instances started by my autoscale policy. As i understand each instance will have an IP address that will be in this range? is this correct?

I want to know if I can reserve a number of Amazon Elastic IP addresses and assign them to instances started by Autoscale. So basically, when a new instance is started because a trigger has been triggered can I also set the API to look for a spare IP address and allocate it to the instance. I need to do this because the started instance will need to communicate to a server outside the cloud and get through a firewall which will only allow remote access from a predefined set of IP addresses. So i think i need to reserve some IPs, add them to my firewall settings then allocate them (automatically) when a new instance is started.

Any ideas?

I have a web application that consists of a web server managed by a web hosting firm, a set of EC2 instances in amazons cloud and a MySQL database (hosted on the webserver). MySQL is behind a firewall and is set to allow access from Localhost and from a single IP address which is an Amazon Elastic IP address that is attached to the EC2 instance I have been running up to now.

The problem is that I want to look at my scaling up and load balancing strategy for my EC2 instance. To this end I have been investigating the Elastic Load Balancers and Autoscaling tools that Amazon provides and have managed to set this up fine but for one thing - connecting to the MySQL database running on my webserver. I realised (thanks to answers on Serverfault) that I needed to check firewall settings and add the IP address for the load balancer, however Elastic Load Balancers provide you with a DNS name, not an IP address and infact the IP addresses change over time so this will not work. I have been told by the company hosting the database that the way the firewall works is to look up the IP address of the DNS name and store the IP rather than the DNS name. so basically this will not work and the only way to allow access would be to open up the SQL port to allow access from anyone!

Is this a viable idea? Should I look at moving my database into the cloud? Is there another firewall that the server company can use? Should I find another way of load balancing (if so what?)

tricky one eh? any help appreciated!

I have a web application that uses an EC2 instance to receive uploaded images, resize and store on S3 and update my MySQL database with the image record. This database is hosted outside Amazon Web Services and so obviously involves communication between the EC2 instance and the database. Images are posted to the upload server from a Flash client which receives the IP address of the upload server when it is loaded and so sends images to 1.12.23.34/resize_script.php

This has worked great .. until i started to try and include a load balancer.

Since the ELBs do not use an IP address but a DNS address I am now passing this to Flash. Now when I upload images I get the following response from the server -

Could not connect to MySQL: Lost connection to MySQL server at 'reading initial communication packet', system error: 111

What might be causing the lost connection to MySQL server. Is there any additional steps I need to take to allow my upload servers to be load balanced? I have set the host property of my MySQL privileges for this user to %

any pointers greatly appreciated thanks.

I have an EC2 server running that I use to process image uploads. i have a flash swf that handles uploading to the server from my local disk - while uploading about 130 images (a total of about 650MB) I got the following error in my server log file after about the 45th image.

• glibc detected /usr/sbin/httpd: double free or corruption (!prev): 0x85a6b990 **

What does this error mean?

The server has stopped responding so I will restart it. Where should I begin to find the cause of this problem?

thanks

some info -

Apache/2.2.9 (Unix) DAV/2 PHP/5.2.6 mod_ssl/2.2.9 OpenSSL/0.9.8b configured Fedora 8

I have swf files that are embedded in a PHP page using SWFObject. These swf files are in the same directory as my PHP files. for example www.myurl.com/index.php embeds www.myurl.com/flashfile.swf, index.php and flashfile.swf are in the same directory. However I want to prevent people from being able to type in www.myurl.com/flashfile.swf and viewing the swf. I want the browser to deny access to this file unless it has been embedded by the PHP file. Should I move my swfs to another folder and protect this folder somehow - is this with the .htaccess file?

I am running Apache on a linux machine. While my main concern is for swf files I would like to protect graphics used on the site too.

all help appreciated thanks

I was looking over the access logs for a server that we are running on Amazon Web Services. I noticed that someone was obviously trying to find the phpmyadmin directory - they (or a bot) were trying different paths eg - admin/phpmyadmin/, db_admin, ... and the list goes on. Actually there isnt a database on this server and so this was not a problem, they were never going to find it, but should I be worried about such snooping? Is this just a really basic attempt at getting in to our system? Actually our database is held on another managed server which I assume is protected from such intrusions. What are your views on such sneaky activity?

My new webhosting company has set up a MySQL database for me and it has the tables MySQL and Information_schema already there. I want to copy my existing database from another server (a) to the new one (b). I assume I need to overwrite the 'mysql' database on server (b) with the one from my existing server (a) or atleast copy over the users and permissions.

1. What information does the mysql database hold? users and permissions I can see, does it have the login info for phpMyAdmin? I dont want to overwrite that obviously.
2. Should I drop the table on server (b) and import my original?
3. Should I just copy the users table?
4. Do I need to worry about the information_schema table? should I copy this over too?

thanks

I have a server running on Amazon EC2 that acts as a video encoder (LAMP CentOS + FFMPEG). How can I load test it with video files and get an idea of how many encodings it can handle. I can't seem to find where FFMPEG keeps a log file or how to enable logging with FFMPEG can anyone help?

Some assumptions -

1) Only 1 video can be encoded at a time, I have set up a SQS queue to feed the processes to the server. 2) I can use Amazon Cloudwatch and Auto Scale to simply start another instance when my SQS queue gets to a certain size or performance on server slows - Amazon will take care of distributing traffic between multiple servers.

what are best practices for scaling video encoding servers, am I better off having lots of small instances or fewer larger instances?

thanks

I have a web application running on a windows 2003 IIS 6 server (PHP, MySQL, Zend, cURL, PEAR ... settings about permissions, cache etc. Lots of PHP scripts, on a dedicated machine.

I want to know what is the easiest way to take a snapshot / backup a server configuration. I have all the PHP scripts under version control and backed up locally but its all the settings etc that I want to save also.

I am not very experienced in server admin but have been using Amazon's EC2 to set up Linux machines and we are running some other jobs on a LAMP based image that I have bundled and can start whenever I need. I think this is exactly what I want to be able to do - to take a snapshot of a server and bundle it / back it up.

Also if we change hosting company I want to be able to take this with me and install on the new company and be up and running.

How easy is this, what are the best ways to make this pain free? Any other advice?

many thanks