I've been using AWS for years, but have never ventured outside the Quick Start and AWS Marketplace sections when launching an EC2 instance.
The AMIs from the AWS Marketplace look trustable, they have a link to the seller profile, etc.:
Compare this to community AMIs, that seem to appear out of thin air, with no information whatsoever on who the heck created and uploaded it:
How to know where a Community AMI comes from? Can these be trusted?
I have a script that installs and sets up traffic server:
yum install -y trafficserver
systemctl start trafficserver
traffic_line -s proxy.config.url_remap.remap_required -v 0
traffic_line -s proxy.config.reverse_proxy.enabled -v 0
The problem is, traffic_line fails with:
[connect] ERROR (main_socket_fd 3): No such file or directory error: could not connect to management port, make sure traffic_manager is running
This is because systemctl start returns immediately, without waiting for traffic server to be actually started.
Is there a way to tell systemctl start to only return once the service is started?
If this is not possible, is there a command that I can run after systemctl start to actually wait for the service to be started?
I want to use the hash director in Varnish to redirect all requests for a same URL without query parameters to the same server:
For example,
http://example.com/foo/bar?a=1
http://example.com/foo/bar?a=2&b=3
Should hash to the same server. I want to hash on host and path only.
Here is my configuration:
sub vcl_init {
new workers = directors.hash();
workers.add_backend(worker_1, 1);
workers.add_backend(worker_2, 1);
workers.add_backend(worker_3, 1);
}
I understand I need to set the hash function here:
sub vcl_recv {
set req.backend_hint = workers.backend(...);
}
What should I put in there to hash on host + path, without query string?
Note: before you jump in too fast, yes I read linuxatemyram.com!
I have a server with 64GB RAM.
free -m says that my RAM is full, and it's not because of disk caching:
total used free shared buffers cached
Mem: 64458 64117 340 201 67 331
-/+ buffers/cache: 63719 739
Swap: 1532 383 1149
However, top ordered by memory usage does not add up to 64GB:
KiB Mem: 66005116 total, 65652464 used, 352652 free, 67512 buffers
KiB Swap: 1569780 total, 392656 used, 1177124 free. 337464 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6258 mysql 20 0 38.665g 0.033t 4924 S 1.3 54.3 482:26.21 mysqld
2293 root 20 0 165896 102116 101964 S 0.0 0.2 0:43.53 systemd-journal
4909 root 20 0 377548 57840 57548 S 0.0 0.1 0:18.47 rsyslogd
26639 www 20 0 650076 53348 32968 S 0.0 0.1 11:32.27 php-fpm
26640 www 20 0 648344 51912 32984 S 0.0 0.1 11:37.43 php-fpm
26642 www 20 0 648600 51472 32580 S 0.0 0.1 11:37.16 php-fpm
26669 www 20 0 648148 50696 31988 S 0.0 0.1 11:35.24 php-fpm
26643 www 20 0 648452 50616 31628 S 0.0 0.1 11:36.19 php-fpm
26641 www 20 0 648620 50496 31340 S 0.0 0.1 11:36.51 php-fpm
28121 www 20 0 648620 48820 29660 S 0.0 0.1 11:35.75 php-fpm
27231 www 20 0 647508 48804 30760 S 0.0 0.1 11:35.61 php-fpm
28029 www 20 0 648044 48752 30172 S 0.0 0.1 11:37.20 php-fpm
28117 www 20 0 647868 48700 30296 S 0.0 0.1 11:36.45 php-fpm
28122 www 20 0 648340 48568 29676 S 0.0 0.1 11:35.73 php-fpm
8569 www 20 0 649028 40268 20704 S 0.0 0.1 11:31.50 php-fpm
10126 www 20 0 648432 39420 20700 S 0.0 0.1 9:58.52 php-fpm
22386 www 20 0 647996 39400 20868 S 0.0 0.1 11:25.00 php-fpm
9643 www 20 0 647976 39220 20704 S 0.0 0.1 11:29.23 php-fpm
23077 www 20 0 647852 39084 20692 S 0.0 0.1 11:11.80 php-fpm
10139 www 20 0 647580 38808 20692 S 0.0 0.1 9:59.94 php-fpm
6326 www 20 0 647368 38396 20696 S 0.7 0.1 8:32.34 php-fpm
4727 www 20 0 646128 37304 20692 S 0.0 0.1 8:30.20 php-fpm
5459 www 20 0 645988 37156 20688 S 0.0 0.1 7:15.13 php-fpm
2173 www 20 0 645240 36408 20684 S 0.0 0.1 4:39.13 php-fpm
20752 www 20 0 644536 35428 20680 S 0.0 0.1 4:29.78 php-fpm
5396 www 20 0 644468 35324 20692 S 0.0 0.1 4:14.65 php-fpm
17558 www 20 0 642668 33816 20740 S 0.0 0.1 1:28.34 php-fpm
28133 www 20 0 642780 33636 20704 S 0.0 0.1 0:49.88 php-fpm
10925 www 20 0 479584 29264 11212 S 3.0 0.0 0:00.09 php
26632 root 20 0 552136 26072 19468 S 0.0 0.0 0:37.74 php-fpm
4946 named 20 0 697996 18748 2104 S 0.0 0.0 3:46.96 named
15609 apache 20 0 2137056 8120 1592 S 0.0 0.0 0:56.18 httpd
8584 root 20 0 133432 4864 3700 S 0.0 0.0 0:00.08 sshd
MySQL uses 54.3% alone, this is perfectly normal as it has an innodb_buffer_pool_size of 32G. The other processes' memory usage add up to roughly 2.8%, that's a total of 57.1%.
Where are the 32% remaining?
Edit: contents of /proc/meminfo:
MemTotal: 66005116 kB
MemFree: 353272 kB
Buffers: 66328 kB
Cached: 736620 kB
SwapCached: 11348 kB
Active: 34396680 kB
Inactive: 2651132 kB
Active(anon): 34223240 kB
Inactive(anon): 2228020 kB
Active(file): 173440 kB
Inactive(file): 423112 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 1569780 kB
SwapFree: 1177448 kB
Dirty: 328 kB
Writeback: 0 kB
AnonPages: 36234364 kB
Mapped: 125208 kB
Shmem: 206396 kB
Slab: 28058904 kB
SReclaimable: 28010224 kB
SUnreclaim: 48680 kB
KernelStack: 2760 kB
PageTables: 94780 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 34572336 kB
Committed_AS: 38572348 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 382304 kB
VmallocChunk: 34359353572 kB
HardwareCorrupted: 0 kB
DirectMap4k: 9000 kB
DirectMap2M: 2054144 kB
DirectMap1G: 67108864 kB
Output of slabtop:
Active / Total Objects (% used) : 147380425 / 147413026 (100.0%)
Active / Total Slabs (% used) : 7005839 / 7005839 (100.0%)
Active / Total Caches (% used) : 71 / 144 (49.3%)
Active / Total Size (% used) : 27615020.12K / 27627490.91K (100.0%)
Minimum / Average / Maximum Object : 0.01K / 0.19K / 16.12K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
146851887 146851887 12% 0.19K 6992947 21 27971788K dentry
124936 124936 100% 0.07K 2231 56 8924K Acpi-ParseExt
105144 105144 100% 0.10K 2696 39 10784K buffer_head
49920 49172 98% 0.06K 780 64 3120K kmalloc-64
29916 29916 100% 0.11K 831 36 3324K sysfs_dir_cache
29856 29661 99% 0.12K 933 32 3732K kmalloc-128
21450 21128 98% 0.18K 975 22 3900K vm_area_struct
19328 19328 100% 0.03K 151 128 604K kmalloc-32
18258 13383 73% 0.93K 537 34 17184K ext4_inode_cache
17952 11651 64% 0.04K 176 102 704K ext4_extent_status
16828 6513 38% 0.55K 601 28 9616K radix_tree_node
14400 13996 97% 0.06K 225 64 900K anon_vma
11645 7903 67% 0.05K 137 85 548K shared_policy_node
10710 7006 65% 0.19K 510 21 2040K kmalloc-192
10608 10608 100% 0.04K 104 102 416K Acpi-Namespace
9728 9728 100% 0.01K 19 512 76K kmalloc-8
...
As part of my application deployment, I have some commands in my .ebextensions config files. It might take 20 minutes or more for all the commands to complete, the first time the commands are run (cloning big repositories).
Unfortunately, this triggers a timeout during deployment:
INFO Deploying new version to instance(s).
WARN The following instances have not responded in the allowed command
timeout time (they might still finish eventually on their own).
INFO Command execution completed. Summary: [Successful: 0, TimedOut: 1].
Is it possible to increase this timeout? I can't find the option in my environment settings.
I'm using Zend Server CE on Windows 7 Pro 64-bit, and I'm trying to activate HTTPS with a self-signed certificate.
I uncommented this line in httpd.conf:
Include conf/extra/httpd-ssl.conf
I copied my server.crt and server.key to the conf directory, and then tried to restart Apache via the Apache Monitor. But then got a pop-up message saying:
The requested operation has failed!
I then headed to logs/error.log, but the most recent entry is more than 1 hour old and does not match the current problem at all.
Why isn't the cause of the error logged in error.log? Where should I look for it instead?
I'd like to add these PHP settings to my Elastic Beanstalk environment:
upload_max_filesize = 64M
post_max_size = 64M
These options cannot be set with option_settings in .ebextensions:
Namespace | Extend
---------------------------------------------|-------
aws:elasticbeanstalk:application:environment | Yes
aws:elasticbeanstalk:container:php:phpini | No
How can I add custom settings to php.ini in an Elastic Beanstalk container?
My Elastic Beanstalk container uses container_commands to run a script to patch the database on every deployment.
I run the deployments with a home-brewed script that calls the UpdateEnvironment on the Elastic Beanstalk API.
The problem is, I have no visibility on the output of my container_commands, which is pretty scary during deployment. The only way I can tell if something went wrong is when a command returns a non-zero value: in that case DescribeEvents will report a problem. But still, I have no way to get the actual output of the command.
How can I get my container_commands output after the deployment has finished?
Ideally, with the Elastic Beanstalk API, but any other programmatic way will be fine!
I have created my RDS instance before creating my Elastic Beanstalk environment. The two are working together with no problem, but I'd like them to be linked together, and have the RDS parameters accessible via the RDS_* environment variables.
The Elastic Beanstalk configuration page says:
Although the first link creates a RDS instance in-place and links it to the current environment, the second link just redirects to this documentation page., which unfortunately only explains how to create a new RDS instance, but not how to link an existing one.
How can I associate an existing RDS instance to my Elastic Beanstalk environment?
I'd like to setup a catch-all for my subdomains:
*.domain.com. CNAME domain.com
But, a few specific subdomains should have their own entry:
mail.domain.com. A X.X.X.X
*.domain.com. CNAME domain.com
The problem is, how can I specify that I want the first rule to be processed before the catch-all?
As far as I can see, there is no way to reorder the rules. So, if I create a rule after the catch-all, the catch-all might be processed before the specific rule.
There's a weight parameter, but it doesn't look like it's designed for this purpose, but for load balancing.
I'm using AWS Elastic Beanstalk, and very often when deploying an environment, the health check turns RED, but I can't find an explanation why:
Clicking on Monitor doesn't help much:
Is there a way to have more specific information about why the environment turns RED?
For information, there is a healthy instance for this environment in my EC2, I've SSH'ed it and it does respond to HTTP requests, although when loading http://xxx.elasticbeanstalk.com/ from my browser, the request times out.
I have an installation script that unzips a directory, then recursively chmods its contents.
I'm surprised it takes almost 10 times the time it takes to unzip, to run the following two commands:
find $dir -type f -exec chmod a+r "{}" \;
find $dir -type d -exec chmod a+rx "{}" \;
Am I doing something wrong, is there a faster way to change the chmod of all files and directories?
I'm running an application on AWS Elastic Beanstalk, and it looks like I need to create a new environment if I want to use the latest AMI.
If I knew what the latest AMI id was, I could update it in the environment configuration.
Is there a place where I could find the ID of the latest Elastic Beanstalk AMIs, or even better, is it possible to have the instances automatically replaced by the new version whenever a new version is rolled out?
I was running the CentOS installer, and was surprised that I couldn't find UTC when prompted for the timezone.
The only available choices are:
However, I recently learned that Etc/UTC is not the same as UTC.
So why is only Etc/UTC proposed and not UTC?
We have a PHP command-line script to version a database. We run this script whenever a developer has added a new database patch.
The script runs the patch with the MySQL command-line:
system('mysql --user=xxx --password=xxx < patch.sql');
However, MySQL 5.6 now issues the following warning:
Warning: Using a password on the command line interface can be insecure
Which is obviously true, but might or might not be a problem for the user.
Please note that I don't want to have to rely on an external password file.
Is it possible to rename an archive in Amazon Glacier?
The documentation says:
After you upload an archive, you cannot update its content or its description. The only way you can update the archive content or its description is by deleting the archive and uploading another archive.
That would lead me to think that it's not possible, but I'm not sure whether the file name is considered part of the archive description.
I'm running MySQL5.6.3 on a CentOS 6.1 virtual machine running on Windows XP in VirtualBox.
The network card is configured in bridge mode, so my physical & virtual machines share the same ethernet card.
On the virtual machine, everything works fine: internet access, DNS lookups. However, connections to the MySQL daemon take a while, and the logs keep showing this warning:
[Warning] IP address '192.168.1.201' could not be resolved: Temporary failure in name resolution
192.168.1.201 is my host machine on which I'm runnning the MySQL client.
Looks like although DNS lookups work fine, reverse DNS lookups end up in a timeout.
Here is the virtual machine configuration:
# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
HWADDR="08:00:27:4B:3D:7C"
NM_CONTROLLED="yes"
ONBOOT="yes"
NETMASK=255.255.255.0
IPADDR=192.168.1.200
GATEWAY=192.168.1.1
PEERDNS=yes
# cat /etc/resolv.conf
nameserver 192.168.1.1
Is there something wrong in my network configuration?
I have VirtualBox (Windows XP host) running a CentOS 6.0 virtual machine, with one shared folder.
Here is the list of mounted devices:
[root@localhost ~]# mount -l
...
www on /media/sf_www type vboxsf (gid=501,rw)
Looks like it's read/write.
Now if I try to write on this shared folder:
[root@localhost ~]# touch /media/sf_www/test
touch: cannot touch `/media/sf_www/test': Read-only file system
Any idea about what I'm doing wrong?
Edit: just tried to manually mount it elsewhere, same problem:
[root@localhost ~]# mount.vboxsf -w www /mnt/www
[root@localhost ~]# mount -l
...
www on /media/sf_www type vboxsf (gid=501,rw)
www on /mnt/www type vboxsf (rw)
[root@localhost ~]# touch /mnt/www/test
touch: cannot touch `/mnt/www/test': Read-only file system
Note: the www folder is writeable on my host machine.
We've been experiencing a lot of problems with our current office's internet connection lately: slowdowns, packet losses, huge ping times, etc.
(Un)fortunately, this is not happening all the day, just a few minutes here and there, several times a day, but that makes our working day a pain.
As Murphy's law dictates, when the IT guy pops up, the internet works just fine, ping is good, bandwidth is normal.
As they won't do anything more for us without further proof of failure, is there a good and simple tool (on whatever platform) that will monitor the connection for 24h (pinging Google every second, for example) and display the results as a graph of the ping time + packet losses at anytime of the day?
I gave a try PingPlotter, but that didn't work as intended for me.
I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]:
0/0/0/
0/0/1/
...
f/f/f/
This leads to around 2400 files per directory, which is very fast when we need to retrieve one file. This is moreover a practice suggested by many questions.
However, when I need to backup these files, it takes many days just to browse the 4k directories holding 10m files.
So I'm wondering if I could store these files in a container (or in 4k containers), which would each act exactly like a filesystem (some kind of mounted ext3/4 container?). I guess this would be almost as efficient as accessing directly a file in the filesystem, and this would have the great advantage of being copied to another server very efficiently.
Any suggestion on how to do this best? Or any viable alternative (noSQL, ...) ?