I have a shell file which needs to perform a replace of some text in a php file.
The line that needs replacing is:
$database = $db."_db";
The actual sed command needs to contain a variable so not sure what i should be escaping and where?
#!/bin/sh
sed -i s/$db."db"/$DATABASE/ mysql_connect.php
Thanks.
We have a web app on a number of servers and we want to add an additional layer of redundancy by backing up the key data to S3. The key data is the MySQL database and a folder containing dynamically created site assets - predominantly images. Some kind of rsync based solution would initially seem the best plan. A couple of years ago we played with S3cmd (in particular s3cmd sync) with some success but we didn't find it particularly reliable although this may have changed since. Its occurred to me though that a rsync solution might not work particularly well with a single db.sql file created with mysqldump and I assume this means the whole database getting transferred each time, with multiple databases of over 1GB this is going to add up to a lot of traffic (and $s) very quickly. With the image files I could simply just transfer files modified within the last day which would be far more simple. What approach should I look at?
We have a situation with an Application which is hosted on a dedicated server where we are anticipating a potential surge in traffic. The major bottleneck on the app is MySQL which is heavy on read and write. Although we have multiple levels of caching its still a concern that MySQL might fall over due to the volume of traffic. We aren't in a position to implement massive architectural changes that would be required to put things like load balancers and replication in place. Ideally we would like to leave the architecture alone as much as possible, largely due to the fact that the traffic volume is by no means guaranteed so the costs might exceed the requirement ultimately. One thing thats occurred to me as a possibility is to move the database off this server completely and put it onto Amazon RDS. This would give us the ability to scale in a cost effective way but wouldn't mean we're stuck with expensive software/hardware that we may or may not need all the time. Really not sure whether this approach is a good idea or not so would greatly appreciate any ideas/gotcha's/pointer's that people have.
In the process of comparing files between 2 versions of an application i've noticed that a number of files seem to have gained additional blank lines throughout the file. These files were never uploaded by FTP, purely copied from one location to another using cp and its not even every single file in the directory, just seemingly random ones. When i do a diff between the files in a side by side format ignoring spaces, newlines etc i just see a bunch of caveats showing the difference as the new line, so this:
diff --suppress-common-lines -Bbwy /home/site1/public_html/file.php /home/site2/public_html/file.php
outputs something like this:
>
>
>
>
>
>
I guess this is 2 questions then:
why would the file have gained these extra new lines, is this a line ending issue somehow and how can i check?
Is it possible to do a diff that will completely ignore these lines and recognise the files as identical which essentially they seem to be?
FYI this is on a CentOS server.
I have a server that has been running at high CPU load due to MySQL activity, when i run the command mysqladmin pr i often see a few queries with the state "writing to net". I had a look around and couldn't find much out about this other than the fact i read somewhere that this shouldnt be expected in usual MySQL activity. Any ideas what this could mean?
Running MySQL 5.0.91-community on CentOS 4.8
Our servers are currently setup to run PHP as DSO with suEXEC enabled. Is it possible for us to install APC in this setup?
Servers are CentOS 5.2 running with WHM/cPanel
Our servers (all CentOS) are all restricted by IP but often i am out and about and stuck on a dynamic IP address. Using a DynDNS widget I have set this dynamic IP to always sync with a DynDNS hostname but how should i go about making this resolve to an IP in hosts.allow. For the moment i've written a cron script that runs every few minutes and checks the IP assigned to that hostname and writes the dynamic IP to hosts.allow but i'm not too keen on that as a solution. Is there a more elegant way i could be doing this?
Thanks.
I've setup sFTP access on an Ubuntu 9.10 Karmic server but i'm having what i think are permission issues.
With the sFTP account i've created it logs in automatically to:
/srv/www/domain.com/
However, i'm only actually able to upload to:
/srv/www/domain.com/public_html
This is not workable as i need to create directories etc parallel to public_html. I appreciate this is something i've probably done wrong as its patched together from a few help files.
I followed these instructions to create a group for sFTP access, i then created a user and then mod'ed their home directory using:
usermod -d /srv/www/domain.com newuser
Let me know if there's any other information you need to troubleshoot this.
OUTPUT OF COMMANDS
> ls -al /srv/www/domain.com/ | grep public_html
drwxr-xr-x 2 newuser newuser 4096 2010-08-24 12:38 public_html
> ls -al /srv/www/ | grep domain.com
drwxr-xr-x 5 root root 4096 2010-08-24 12:21 domain.com
> groups newuser
newuser : newuser filetransfer
> ls -ld /srv/www/domain.com/
drwxr-xr-x 5 root root 4096 2010-08-24 12:21 /srv/www/domain.com/
A previously working command line PHP script has stopped working - when i run it now it returns this error:
[root@server dir]# ./custom.php
Extension './custom.php' not present.
If i run the script like this it works:
[root@server dir]# php custom.php
For a number of reasons i cannot do this though. I have read something that implies this could be related to line endings but no idea how/why. i have edited this file recently on A Mac and a PC and FTP'd it to the server but i've been doing that for a long time without issues.
EDIT First line of script is:
#!/usr/bin/php -q
FYI The server is running CentOS 5.4
I'm trying to consolidate some website files on one of our CenotOS servers, the setup i want to create is something like this but i'm not sure if its actually possible
we would have a set of files used by all sites:
/home/commonfiles/
public_html/
library/
each site directory would look something like this:
/home/site1/bespoke/
css/
images/
each "site" would use the public_html and library folders from within the common files. Symlinks would obviously work for this. What i would also want though is for /home/commonfiles/public_html/css to point to /home/site1/bespoke/css where site1 is the site that the absolute link to /home/commonfiles/public_html/ is from. I know its possible to create relative symlinks but is it possible for an absolute symlink to contain a relative one.
I hope this makes sense as i'd really appreciate some advice
EDIT
to elaborate, this is the basic structure i would want (if its possible)
/home/common
/home/common/public
/home/common/library
/home/site1
/home/site1/public -> /home/common/public
/home/site1/library -> /home/common/library
/home/site1/bespoke/
/home/site1/bespoke/css
/home/site1/bespoke/images
/home/site1/public/css -> /home/site1/bespoke/css
/home/site1/public/images -> /home/site1/bespoke/images
/home/site2
/home/site2/public -> /home/common/public
/home/site2/library -> /home/common/library
/home/site2/bespoke/
/home/site2/bespoke/css
/home/site2/bespoke/images
/home/site2/public/css -> /home/site2/bespoke/css
/home/site2/public/images -> /home/site2/bespoke/images
I guess essentially the mapping of - for example - /home/site1/public/css -> /home/site1/bespoke/css does not necessarily have to be done with links, maybe this could be done in the .htaccess in some way. Hopefully you can see what i'm trying to achieve here though.
just to add to this, this is essentially what i'd like to be able to do (if it worked):
mkdir test
mkdir test/common
mkdir test/common/public
mkdir test/common/library
mkdir test/site1
mkdir test/site2
mkdir test/site1/bespoke
mkdir test/site2/bespoke
mkdir test/site1/bespoke/css
mkdir test/site1/bespoke/images
mkdir test/site2/bespoke/css
mkdir test/site2/bespoke/images
ln -s /home/user/test/common/public test/site1/public
ln -s /home/user/test/common/library test/site1/library
ln -s /home/user/test/common/public test/site2/public
ln -s /home/user/test/common/library test/site2/library
cd test/common/public
ln -s ../bespoke/css css
ln -s ../bespoke/images images
We have a system where we use a bespoke CMS on one of our servers. This CMS is used by 30-40 clients. The majority of the files for each client CMS are the same (apart from a few folders)as each time we add a new feature / patch / bug fix we roll it out to all clients. We want to consolidate these files into a single place so we can deploy our changes to it from SVN more easily.
we have a structure like this for each user account:
/home/user/public_html/css/
/images/
/admin/
index.php
etc ....
In this example, the css and images folders would be unique to each client site but all other files would be common to all. Not sure if we should be looking into symlinks / NFS / something else ??
One of our servers has had much higher than average load today, consistently running higher than normal. Our server admins are making sure all is running okay but this has got me wondering if maybe the server is just very busy today. This in turn has made me think, is there any way to get a reasonably accurate report on how many active website visitors there are across all sites on the whole server?
All sites are PHP/MySQL running on apache 2 on Centos 4.8 - obviously we do have things like google analytics / awstats / webaliser for each site but i just wondered if theres a command that can be run that will give us an active users on server stat? maybe analyse all session files for sessions active in last x minutes?
We have an ecommerce app that we develop at our company. Its a reasonably standard LAMP application that we have been developing on and off for about 3 years. We develop the application on a testing domain, here we add new features and fix bugs etc. Our bug tracking and feature development is all managed within a hosted subversion solution (unfuddle.com). As bugs are reported we make these fixes on the testing domain and then commit changes to svn when we are happy the bug has been fixed. We follow this same procedure with the addition of new features.
It is worth pointing out there the general architecture of our system and application across our servers. Each time a new feature is developed we roll this update out to all sites using our application (always a server we control). Each site using our system essentially uses exactly the same files for 95% of the codebase. We have a couple of folders within each site which contain files bespoke to that site - css files / images etc. Other than that the differences between each site are defined by various configuration settings within each sites database.
This gets on to the actual deployment itself. As and when we are ready to roll out an update of some kind we run a command on the server that the testing site is on. This performs a copy command (cp -fru /testsite/ /othersite/) and goes through each vhost force updating the files based on modified date. Each additional server that we host on has a vhost that we rsync the production codebase to and we then repeat the copy procedure on all sites on that server. During this process we move out the files we dont want to be overwritten, moving them back when the copy has completed. Our rollout script performs a number of other function such as applying SQL commands to alter each database, adding fields / new tables etc.
We have become increasingly concerned that our process is not stable enough, not fault-tolerant and is also a bit of a brute-force method. We're also aware we are not making best use of subversion as we have a position where working on a new feature would prevent us from rolling out an important bug fix as we are not making use of branches or tags. It also seems wrong that we have so much replication of files across our servers. We're also not able to easily perform a rollback on what we have just rolled out. We do perform a diff before each rollout so we can get a list of files that will be changed so we know what has been changed after but the process to rollback would still be problematic. In terms of the database i've started looking into dbdeploy as a potential solution. What we really want though is some general guidance about how we can improve our file management and deployment. Ideally we want the file management to be more closely linked to our repository so a rollout / rollback would be more connected to svn. Something like using the export command to make sure the site files are the same as the repo files. It would also be good though if the solution maybe would also stop the file replication around our servers.
Ignoring our current methods it would be really good to hear how other people approach the same problem.
to summarise ...
Thanks in advance
EDIT:
I have read a lot of good things recently about using Phing and Capistrano for these kind of tasks. Can anyone give any more info about them and how good they would be for this kind of task?
We have a site thats using MySQL fulltext searching but has an issue that a number of strings that need to be indexed are 3 characters long. I know that the default for MySQL is not to index strings of this length but I also know that it is possible to override this in my.cnf. I assume the reason for the limit is primarily performance based so i'm not sure if this is something we should consider changing? What options do we have here? There are a number of sites on this server so any performance degredation could cause issues to a number of sites.
mysql version: 5
I am trying to prevent people hotlinking to PDF AND DOC files. Usually, i would approach this with a .htaccess rule like this:
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domain.com/.*$ [NC]
RewriteRule \.(pdf|doc)$ /home/ [R=302,L]
However, many of these files are linked to through php scripts like filedownload.php?id=5 which then trigger the download of a PDF/DOC file. Is there a way to prevent hotlinking to these files via the mime of the outputted file? another way?
edit - added this source to show how files are called:
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: private",false);
header("Content-Type: $ctype");
header("Content-Disposition: attachment; filename=\"".basename($fn)."\";" );
header("Content-Transfer-Encoding: binary");
header("Content-Length: ".$fs);
echo $upload["file_contents"];
exit();
We're looking to purchase some SSL certificates to secure the login pages of ecommerce sites. It is not required to secure the actual payment process as this is protected by a third party with its own verisign certificate. rapidSSL looks like a good (and cheap) option but a salesperson has told me that they are only suitable for "test sites" and recommended that we use one that is 4 times the cost. Can anyone make any recommendations about what we should be looking for and what we should consider?
Thanks.
We need to do hundreds of 301 redirects on a site which is undergoing a massive URL restructuring. Whats the best way of doing this? at .htaccess level? at code level?, apache level? Will there be a noticeable slowdown on the site if the .htaccess becomes bloated?
Thanks
We use a third party newsletter system which updates frequently, usually around 100 files. Each time all the developers release is the whole application again with a changelog but there's usually too many files listed for it to be useful. The system - once online - generates a lot of files (caches etc) so i dont want to download the remote files to do a local diff each time either. What i want to do is upload the zip of all the newly released files and run a Diff (or similar) and copy over from new_files -> existing_files just the updated files and ignore the files in existing that aren't in new. Whats the best way to do this?
I know this can be achieved to an extent using copy but its my understanding that this is largely based on last edited date but i'm more interested in actual file differences
This is on CentOS 5 and i have shell access
Thanks.
In our apache error log (/usr/local/apache/logs/error_log) for one of our servers we are seeing a shell command running quite often which is throwing an error, it looks like this:
sh: list_price: command not found
list_price is a field we use in an ecommerce system on over 50 sites and certainly nothing suspicious. The issue is that we have no idea how this is being passed to the shell - we have checked all occurrences of exec() and system() and we just cannot see how this would be getting passed through. Is there any way we can detect more easily what the source of this could be as there is nothing more meaningful being output that what i put above. FYI this is on a server running CentOS 5 and the sites in question are all PHP.
We generate cache files for combined site assets such as javascript and css. We currently store these files in /tmp but with everything else in there it is making the cleanup routines awkward to setup. Are we safe to switch to using /home/USER/tmp instead? Does this area have an automated cleanup routine that I need to be aware of? I've noticed there are currently files in there for Awstats / webalizer / cpbandwidth.
This is using CentOS btw.