Tom Brossman's questions

TL;DR - How do I block requests for https://s3-eu-west-1.amazonaws.com/BUCKET-NAME/FILE.EXT while allowing requests coming via CloudFront?

I followed the AWS documentation "Restricting Access to Amazon S3 Content by Using an Origin Access Identity" and this appears to be tested and working almost perfectly (in my case with an Alternate Domain Name set), however when I try the URL...

https://s3-eu-west-1.amazonaws.com/BUCKET-NAME/index.html

...the index.html document is returned! What is worse, I can access any object in the bucket by following this convention. My understanding is that should not happen, and only CloudFront URLs should return objects.

Static website hosting for the S3 bucket is disabled. Visiting the S3 endpoint URL http://BUCKET-NAME.s3-website-eu-west-1.amazonaws.com/index.html returns a 404 (with code "NoSuchWebsiteConfiguration"), which is the expected result. Can someone please explain why the other links work, instead of returning a 403 or 404 error?

When I set up the CloudFront distribution, for the option "Grant Read Permissions on Bucket" I selected "Yes, Update Bucket Policy", which produced this S3 bucket policy in use now:

{
  "Version": "2008-10-17",
  "Id": "PolicyForCloudFrontPrivateContent",
  "Statement": [
    {
      "Sid": " Grant a CloudFront Origin Identity access to support private content",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity XXXXXXXXXXXXXX"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::BUCKET-NAME/*"
    }
  ]
}

The HSTS Preload List Submission website has very specific prerequisites which disallow combining protocol changes and redirects into one step (classic example - redirecting requests for http://example.com directly to https://www.example.com fails their test - a protocol upgrade to HTTPS must occur first, then the 'www' subdomain may be added in a discrete step). Is it possible to meet these rules for sites hosted on Amazon CloudFront?

A related question is here, about sending the necessary headers, but I already have that part tested and working. My issue is the redirect, for a site that uses a www subdomain.

I have CloudFront configured to 'Redirect HTTP to HTTPS' under the default behavior and that works perfectly for all HTTP requests, however requests to https://example.com first redirect to the downgraded protocol http://www.example.com before being upgraded to HTTPS, and this is where I am stuck. I need to handle HTTP and HTTPS requests for the second-level domain differently.

For clarity, here is the redirect pattern I want to achieve:

http://example.com > https://example.com > https://www.example.com
http://www.example.com > https://www.example.com
https://example.com > https://www.example.com
https://www.example.com

And here is what CloudFront is doing now, with the third line (protocol downgrade) resulting in HSTS Preload submission failure:

http://example.com > https://example.com > https://www.example.com
http://www.example.com > https://www.example.com
https://example.com > http://www.example.com > https://www.example.com
https://www.example.com

I have a command that takes domain names as options. I have a long list of domain names to include and so I'd like to automate the parsing of that list of domains into the command.

For example, the simplest usage would be command --option=example-one.com --output=file.ext. After adding the list of domains the eventual command to execute would look like command --option=example-one.com ... --option=example-99.io --output=file.ext (where '...' would be 97 more options, shown abbreviated here)

Is this possible to do in Bash and if yes, what's it called please?

Here's what I tried that didn't work so far:

1. command --option=$(cat domains.txt) --output=file.ext
2. command "$(while read baddomains ; do echo -n '--option=$baddomains ' ; done < domains.txt)" --output=file.ext
3. command --option="domains.txt" --output=file.ext

The command while read baddomains ; do echo -n '--option=$baddomains ' ; done < domains.txt > all-options.txt does output a list of perfectly formatted options into a new file all-options.txt, but I can't understand how to redirect output of the options back into the command, instead of a text file.

The list of domains includes Cyrillic characters, if that matters.

Is there a way to make Nginx proactively OCSP staple certificates each time its configuration is reloaded or it is re-started? Alternatively, can Nginx be set to save the stapled certificates across reloads or restarts instead of discarding them? Reloading or restarting Nginx appears to clear all cached OCSP stapled certificates.

I have OSCP stapling tested and working on an Ubuntu 16.04.1 server running Nginx 1.11.4 and using Certbot's OCSP Must-Staple TLS feature extension. My problem is that upon reload or restart of Nginx, the stapled response is not saved and the first visitor sees an error page instead (this is the expected outcome for "must staple" certs not yet stapled by the server).

I have to visit each website hosted by the server and reload them a couple times while Nginx automagically OCSP staples the certificates, then everything starts working again until the next restart. I'd like to automate this step or avoid it altogether.

I'd like to set a content security policy header for a Joomla website running on Apache 2.4.

Using this configuration from h5bp and setting Header set Content-Security-Policy "script-src 'self'; object-src 'self'" gives me a blank page for the Joomla login page at www.example.com/administrator/. How can I use this policy and still log in?

Checking the console, the error message is:

Content Security Policy: The page's settings blocked the loading of a resource at self ("script-src http://www.example.com").

The administrator page is entirely served from example.com, there is no third-party content. The site works perfectly except for the blank page on the login page with the policy set. Checking the /administrator page source, it looks completely ordinary except that the JS isn't run. A copy of the complete page source is here.

Because I have whitelisted example.com with "script-src 'self'; object-src 'self'" I expect that the page will render but I'm obviously missing something.

I've now re-tested this with a new VPS and clean install of Joomla with no customisations. Setting the content security policy and restarting Apache immediately reproduces the issue - totally blank admin page with accompanying console error in the browser complaining about the policy blocking the loading of resources. Changing "script-src 'self' to "script-src 'example.com' or "script-src 'IP:AD:DR:ESS' doens't help, all scripts are blocked, period.

Any idea how to get this working or further troubleshoot it?

I have an RSS feed example.com/?format=feed&type=rss from an old Joomla install. The site is now built with Jekyll, and uses a new URL example.com/rss.xml for the RSS feed.

Rewrite rules aren't working for me since the URL starts with "?". I've got it working with an if directive but want to avoid using if.

Here's the working if directive which I want to replace with a rewrite or redirect:

if ($args ~ "type=rss") {
rewrite ^ https://example.com/rss.xml? permanent;
}

Can the if directive for the URL starting with "?" be replaced with a rewrite or redirect?

I'm trying to get a rewrite rule working and I've discovered that if the URL begins with a question mark, Nginx fails to return 404 error as expected. Instead, index.html as defined in the root directive is served for all urls beginning with "?". (e.g. example.com/?page-does-not-exist works for the home page but should not)

This is making it impossible to get a redirect from example.com/?format=feed&type=rss -> example.com/rss.xml working. This is for a blog migrated from Joomla to Jekyll.

Put simply, visiting example.com/123 returns a 404, and visiting example.com/?123 does not. The latter returns the site's home page at the URL example.com?/123. (There is no file or folder '123', so requests for it should always fail.)

Everything works perfectly with the site, including the HTTPS redirect, except for the expected 404 redirects not happening for URLs starting with a question mark. How do I fix this?

Here's my configuration:

server {
    server_name example.com;
    root /var/www/example.com;
    index index.html;
    listen 443;
    ssl on;      
    ...

    rewrite "/?format=feed&type=rss" https://example.com/rss.xml permanent;

    location ~* \.(?:ico|css|js|gif|jpeg|jpg|png|txt|svg|eot|woff|ttf)$ {
            expires max;
            add_header Pragma public;
            add_header Cache-Control "public, must-revalidate, proxy-revalidate";
            add_header Access-Control-Allow-Origin *;
            valid_referers none blocked example.com;
            if ($invalid_referer) {
                    return 403;
            }
    }
}

# HTTP --> HTTPS
server {
        listen 80;
        server_name www.example.com example.com;
        return 301 https://example.com$request_uri;
}

My goal is to use GitHub Pages and also benefit from their CDN, without having to use a third-party DNS host that supports DNS ALIAS record types (or ANAME - neither are accepted standards). I want to know how they do it, can anyone explain please?

We have many questions here about DNS aliasing with CNAME records, but in this particular situation I can't CNAME the naked domain to username.github.io.

If I do this with Apache or Nginx to get www.example.com working as a GitHub Page, does it work and do I break anything like email?

Apache

<VirtualHost *:80>
        ServerName example.com
        Redirect 301 / http://username.github.io/
</VirtualHost>

Nginx

server {
    server_name example.com;
    return 301 http://username.github.io/;
}

The above assumes a CNAME for www => username.github.io and an A Record for the naked domain => one of my servers with the above configuration.

First time setting up Nginx and my goal is to have example.com with a static 'index.html' page served with a minimalist config, nothing more. I also want to drop the www subdomain. Here are my sites-available server blocks:

server {
    server_name www.example.com;
    return 301 $scheme://example.com$request_uri;
}

server {
    server_name example.com;
    root /var/www/example.com/;
    index index.html index.htm;

    location / {
            try_files $uri $uri/ /index.html;
    }
}

If I use www.example.com or example.com they work fine, with www automatically dropped.

My issue is I can type anything after example.com and the index.html page still loads, like example.com/ABC or example.com/12345. These pages don't exist, why are the URLs accepted? I would expect any URL other than the domain root to return a 404 page instead.

This is probably a very simple issue but I've tried searching here & in the docs and I'm coming up with nothing so far.