Axel's questions

I've an EC2 machine which is behind a ALB. Listener rules in the ALB are like

IF
Path is/blog*
Host is abc.example.com

THEN
Forward to
target-group-1

My nginx config looks like this:

##

# Default server configuration

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    root /var/www/html;
    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html;

    server_name _;

    location /blog {
        proxy_pass https://abc.webflow.io/;
    }
}

Main page opens fine at abc.example.com/blog

but embedded links are not opening and giving 502 bad gateway when opened through DNS defined in ALB listener rules and when directly opened using EC2 instance IP are giving 404 not found

When I click on embedded links inside the main page, URL in the browser shows abc.example.com/categories/something/

and gives the above HTTP codes but if I manually edit the url in the browser and type it as: abc.example.com/blog/categories/something/ then page opens fine.

I'm trying to find how to rewrite the URL in such a way that it contains the sub-folder name as well but can't find anything.

Any help will be great!

We have a bucket with more than 500,000 objects in it.

I'm assigned a job where I've to delete files which have a specific prefix. There are around 300,000 files with the given prefix in the bucket.

For eg If there are 3 files

abc_1file.txt
abc_2file.txt
abc_1newfile.txt

I've to delete the files with abc_1 prefix only. I didn't find much in AWS documentation related to this.

Any suggestions on how can I automate this?

We have multiple OU's in our AWS organisation.

We use AWS cli to create new AWS organisation members. For eg.

aws organizations create-account --email [email protected] --account-name "testaccount"

Everytime a new member account is created, it is added under root and not in any OU.

We want to add new member account in a particular OU. We can move the new member accounts manually through cli or console to any OU but we want to specify it at the time of account creation itself.

It will be really helpful if someone can point me into the right direction.

Update:

I'm trying to automate moving of new member accounts from root to a particular OU. This is what I've done, running these commands from a ps1 file. I'm not being able to pass $AccId variable in the move command. Any suggestions?

aws organizations create-account --email [email protected] --account-name test
$Id = aws organizations list-create-account-status --states IN_PROGRESS --query 'CreateAccountStatuses[*].Id' --output text
$AccId = aws organizations describe-create-account-status --create-account-request-id $Id --query 'CreateAccountStatus.AccountId' --output text
aws organizations move-account --account-id $AccId --source-parent-id r-0000 --destination-parent-id ou-0000-000000

We have certain AWS labs which we offer to our customers. Every time a user opens the lab, a new member account is created and added to the organisational account.

This only happens the first time when a user logs into the lab. This member account has no resource restrictions and can do anything the user wants.

For eg.

1. Launching any type and number of ec2 instances.
2. Create as many s3 buckets as possible and upload files of any size.
3. Launch any type of RDS and ElastiCache clusters.

This has been creating a huge problem for us and we want to limit resources depending upon what is required to perform in the labs.

After a lot of research, I came up with this:

Resource restrictions on OU level using SCP:
1. Deny every service by default.
2. Allow only those services which are used in tasks.
3. Allow those services in 1 particular region only (For e.g. us-east-1)
4. Limit what type of instances can be launched (For e.g. t2.micro only)
5. Limit specific AMI's using which instances can be launched (For e.g. Only free AMI's like ubuntu and linux AMI's, no windows AMI's)
6. Policy for limiting s3 bucket sizes is not possible.

Organisation account removal:
1. Can't remove member account if they don't have required information to become standalone account.
2. This information includes:
    - AWS Customer Agreement
    - choose a support plan
    - provide and verify the required contact information
    - provide a current payment method
3. This can't be automated so the idea is to create 2 OU's "Organisational units".
    - Working accounts
    - Disabled accounts
4. 1st OU will have required permissions to perform the lab tasks only (Principle of least privilege)
5. 2nd OU will have no permissions, Deny All for all services and actions.

Managing OU's

https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_ous.html

Moving accounts from one OU to another OU A program can be written to list account under the "working accounts OU"

https://docs.aws.amazon.com/cli/latest/reference/organizations/list-accounts-for-parent.html

From the output, filter out the "JoinedTimestamp" parameter and perform move operation on accounts which are older than xx days.

https://docs.aws.amazon.com/cli/latest/reference/organizations/move-account.html

I want to know from experienced AWS architects whether the second part of the "Organisational Unit" is possible.

If yes, can someone help me to figure out how to achieve it as I don't have much programming experience.

This is an urgent question. My AWS instance with 4GB RAM and 2vCPU is getting exhausted by too many php-fpm processes.

PHP-FPM starts creating too many processes which it didn't do earlier. This results in slow loading of the site. I'm not able to figure out why this is happening.

My php.ini config:

html errors = off
precision = 14
output_buffering = 4096
serialize_precision = 17
max_execution_time = 300
max_input_time = 60
max_input_vars = 5000
log_errors_max_len = 1024
post_max_size = 27M 
upload_max_filesize = 27M 
max_file_uploads = 20
default_socket_timeout = 60
pdo_mysql.cache_size = 2000
memory_limit = 256M

www.conf:

pm = dynamic
pm.max_children = 35
pm.start_servers = 10
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 1000

screenshot of top command:

I'm working on creating a DR setup and runbook based on AWS.

I don't have any experience with creating DR setups so it would be really helpful if the experienced veterans can guide me through it.

Our Setup:

RDS MYSQL Aurora DB
ElastiCache
Ubuntu 16.04 Linux EC2 instances
Static files stored in S3
Route 53- Total of 250 record sets.
Application Load balancer

Everything is under the same VPC. We're trying to build a PilotLight DR setup.

We have a Amazon RDS database in a different VPC and our EC2 servers in a different VPC but in the same region i.e. us-east-1.

Our servers connect to the RDS via it's publicly accessible endpoint. We are working on reducing our aws bills and we can see high data transfer cost.

Does connecting to RDS instance from an ec2 instance from a different VPC affect the data transfer cost as compared to connecting from the same VPC?

I couldn't find the answer from google or aws forums. Any information will be much appreciated.

I'm in dilemma regarding which instance type we should be using for our development server. This server is used to test our code before it goes to our production server.

This server is also used by our employees for their day to day task such as bug reporting, status filling and similar tasks.

OS: Ubuntu 16.04

Server: nginx+php-fpm

Current Instance type: c5d.large

Last 2 weeks average CPU utilization:

All cloudwatch metrics based on last 2 weeks:

We have a AWS CodeDeploy set up in our AWS environment.

Yesterday, while CodeDeploy was deploying the code on all the instances, one of the instances got terminated by the AutoScaling group and because of that our deployment got failed on that instances and codedeploy skipped all the instances after that.

How do I make sure if the deployment gets failed on one instance, CodeDeploy doesn't skip deployment on all other instances?

I'm running nginx server with php-fpm.

I'm using "c5d.xlarge" ec2 instance type to host my website.

c5d.xlarge = 4 vcpu & 8GB RAM.

If active connections go beyond 10k on my ELB, CPU Utilization goes beyond 60-70% on my all 15 servers.

php-fpm configuration:

pm = dynamic
pm.max_children = 65
pm.start_servers = 10
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 600

nginx configuration:

user  www-data;
worker_processes  4;
pid        /var/run/nginx.pid;


events {
    worker_connections  3072;
}


http {
        ##
        # Basic Settings
        ##

        charset utf-8;
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        server_tokens off;
        log_not_found off;
        types_hash_max_size 2048;
        client_max_body_size 16M;
        keepalive_timeout  70;
        client_header_timeout 3000;
        client_body_timeout 3000;
        fastcgi_read_timeout 3000;
        fastcgi_buffers 8 128k;
        fastcgi_buffer_size 128k;

        # MIME
        include /etc/nginx/mime.types;
        default_type application/octet-stream;
   log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /home/ubuntu/apps/log/default/access.log  main buffer=32k;
    error_log   /home/ubuntu/apps/log/default/error.log;


    gzip on;
        gzip_disable "MSIE [1-6]\.";

        # Only allow proxy request with these headers to be gzipped.
        gzip_proxied expired no-cache no-store private auth;

        # Default is 6 (1<n<9), but 2 -- even 1 -- is enough. The higher it is, the
        # more CPU cycles will be wasted.
        gzip_comp_level 7;
        gzip_min_length 20; # Default 20

        gzip_types text/plain text/css application/json application/javascript  application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/conf.d/*.conf;

top command output:

Please suggest if these configurations are okay. I'm not too sure if this much cpu utilization for these many active connections is alright. I'll be really grateful if someone can guide me in setting up nginx and php-fpm for optimal performance.

If anymore information is required, please let me know.

Server: Ubuntu 16.04/Nginx

Scenario: I'm trying to ban IP's based on multiple 404 requests on my server.

I've created a filter in /etc/fail2ban/filter.d/banbadrequest.conf

[definition]
failregex = ^<HOST> - .* "(GET|POST|HEAD).*HTTP.*" 404 .*$
ignoreregex =.*(robots.txt|favicon.ico|jpg|png)

added a new jail in the location /etc/fail2ban/jail.conf

[banbadrequest]
enabled = true
port = http,https
filter = banbadrequest
logpath = /var/log/nginx/error.log
logpath = /var/log/nginx/access.log
bantime = 3600
findtime = 600
maxretry = 5

But when I restart the fail2ban service, it fails to restart and exits. Can anyone point out what am I doing wrong here?

I've a task in hand where I've to run a script on all the servers behind a load balancer in aws. We have auto-scaling as well so at the time of high load, new servers get added behind the same load balancer.

The script will run to find out which git repo is set and what is the latest commit id on the server so that we can check if all the servers match the same git repo and commit id.

This is the script which will run on all the servers behind loadbalancer.

#!/usr/bin/env bash


# gets the current git branch
function parse_git_branch() {
  git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e "s/* \(.*\)/\1/"
}

# get last commit hash prepended with @ (i.e. @8a323d0)
function parse_git_hash() {
  git rev-parse --short HEAD 2> /dev/null | sed "s/\(.*\)/@\1/"
}

# DEMO
GIT_BRANCH=$(parse_git_branch)$(parse_git_hash)
echo ${GIT_BRANCH}

Anyone has any idea how can I achieve this or have any previous experience with something similar? Thanks!

we are migrating from Apache to Nginx. Everything is setup and tested. Our whole infra is hosted on AWS.

I'm little bit nervous about the final migration as to how should I start the migration on production servers with zero downtime for our users.

We've 20 running apache servers and we want to replace them with nginx based servers. Can anyone give an insight on how to go about it?

We have auto-scaling group behind an Application load balancer which launches instances if the avg. CPU Utilization goes beyond 70%.

We have created a base AMI for the auto-scaling group.

But the problem is that our website updates almost on a regular or weekly basis so what we can configure to allow AWS code-deploy to automatically deploy our latest revision from github on those instances created via auto-scaling group?

If any more information is required please let me know.

My website is on Apache which is hosted in AWS VPS.

I tried setting http-only flag and secure flag by editing security.conf file but upon checking the headers via https://hackertarget.com/http-header-check/ I see that there is no change and cookies are still without these flags.

I followed these steps:

Ensure you have mod_headers.so enabled in Apache HTTP server

Add following entry in /etc/apache2/conf-enabled/security.conf

Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure

Restart Apache HTTP server to test

Can anyone help me out?

I've searched for this issue but couldn't get any solution working from the client side.

We've mysql server with Apache on Ubuntu.

Our client (MacOS) is getting "can't get hostname for your address" error when connecting to our database. He is able to connect without any problems from Home network but not from his work network.

None of our other clients face this issue so I guess there is no problem on server side. Solutions like editing my.ini or my.cnf is not what we're looking for as server side changes are not encouraged. Any help towards solving this problem will be appreciated.

Server: mysql (Apache,Ubuntu)
Client: macOS

I have an ESXi host installed on a VMware workstation.
On the ESXi, I have two Ubuntu virtual machines and 1 freesco router.

I want to connect both the Ubuntu virtual machines via the router. The virtual machines are on different networks. I've set the static IP's of both the virtual machines so that they are on a different network.

Freesco router config:

ESXi host:

VM1 Static IP: 192.168.204.2

VM2 Static IP: 10.10.10.2

Switch topology:

VM1 settings:

Portgroups:

Both the VM's are connected to same portgroup i.e "connect" and same switch "newSwitch".

I want to connect both the VM's using router. How can I do this?