Home / user-32883

Dan Fabulich's questions

Martin Hope
Dan Fabulich
Asked: 2021-04-02 17:30:27 +0800 CST
  • 3

pip no longer works on old versions of Python 2, because https://pypi.python.org/ now requires Server Name Indication (SNI), which isn't available in Python 2.7.6, the version of Python that comes with Ubuntu 14 Trusty.

https://github.com/pypa/pypi-support/issues/978 explains:

Upgrading to the last Python 2.7 release is an option.

However, note that Python 2.7 series itself is now End of Life and support in pip was dropped with version 21.0.

I'm maintaining a legacy code base; I need to pursue the least invasive option possible to get pip working again. What are my options? What's the smallest possible change I can make to get back to (temporarily) working order?

ubuntu python pip ubuntu-14.04
Martin Hope
Dan Fabulich
Asked: 2020-08-28 21:11:53 +0800 CST
  • 2

I'm running an Apache web server, serving up large static images off of the file system. I've configured the server to return a valid cache header, like so: Header merge Cache-Control "public, max-age=31536000, immutable"

That's all well and good, but when the user reloads the image in the browser, Apache never returns a 304 Not Modified response.

I ran this curl command (I've changed the host name for confidentiality reasons):

curl -o /tmp/image.png -v -u test:example http://blah.example.com/static/images/covers/436g2gu94kxin4x7

Here are the request headers from curl.

GET /static/images/covers/436g2gu94kxin4x7 HTTP/1.1
Host: blah.example.com
Authorization: Basic dGVzdDpleGFtcGxl
User-Agent: curl/7.64.1
Accept: */*
If-Modified-Since: Thu, 30 Apr 2020 02:21:47 GMT

I expected a 304 response, but instead I got:

HTTP/1.1 200 OK
Date: Fri, 28 Aug 2020 04:59:54 GMT
Server: Apache/2.4.25 (Debian)
Content-Type: image/png
Content-Length: 3115720
Last-Modified: Thu, 30 Apr 2020 02:21:47 GMT
Vary: Accept-Encoding
Cache-Control: public, max-age=31536000, immutable

I've read the Apache Caching Guide, but none of it seems relevant to this situation… it's all about configuring Apache's own cache, either in memory or on disk, especially to support cases where Apache is serving as a reverse proxy for an origin server.

But in this case, the files are sitting there on the local disk already. There is no origin server. I don't want/need Apache to store a separate "cached" copy of the file; I just want Apache to notice that the Last-Modified date of the file on disk matches the If-Modified-Since date and return 304 Not Modified in that case.

How do I set this up? Do I have to make Apache store its own separate cached copy of the files in order to enable conditional GETs? (That will be a very large cache, presumably stored on disk, which is a total waste.)

apache-2.4
Martin Hope
Dan Fabulich
Asked: 2013-12-04 14:37:09 +0800 CST
  • 4

I've got a script that automatically installs some npm packages (specifically karma, which I'd like to install with --global) on an EC2 instance on startup. The problem is that npm sometimes randomly fails, presumably due to network hiccups, though I'm not entirely sure what's going on.

To rule out the network, what I'd like to do is download the relevant files for npm and ask it to install karma based on the files as I download them from S3.

But how do I do that? How do I tell npm not to grab files from the central repository and to instead grab them from somewhere else?

node.js
Martin Hope
Dan Fabulich
Asked: 2013-04-17 16:24:36 +0800 CST
  • 2

squid.conf

cache_effective_user squid

http_access allow all

# Squid normally listens to port 3128
http_port 3128 accel defaultsite=localhost no-vhost ignore-cc

cache_peer localhost parent 80 0 no-query originserver name=myAccel
cache_peer_access myAccel allow all

# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /usr/local/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/var/cache/squid

minimum_expiry_time 0

test.php

<?php

Header( "Cache-Control: max-age=0, must-revalidate" ); 

$headers = getallheaders();
$ims = $headers['If-Modified-Since'];
$lastModified = "Mon, 16 Apr 2012 23:01:32 GMT";
if ($ims == $lastModified) {
    Header( "HTTP/1.1 304 Not Modified" ); 
    exit;
}

Header("Last-Modified: $lastModified");
if ($_GET["vary"]) Header("Vary: Accept-Encoding");

?><html>hi</html>

Squid 3.2.9, installed on OSX via Homebrew

Squid 3.2.9, installed on OSX via Homebrew

localhost: ~ $ squid -v
Squid Cache: Version 3.2.9
configure options:  '--disable-debug' '--disable-dependency-tracking' '--prefix=/usr/local/Cellar/squid/3.2.9' '--localstatedir=/usr/local/var' 'CC=cc' 'CXX=c++' 'PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig'

Started squid with squid -f squid.conf -d 2 -N

When the server omits the "Vary: Accept-Encoding" header, Squid revalidates/refreshes its cache using an If-Modified-Since conditional GET. But when the "Vary: Accept-Encoding" header is present, Squid refuses to use If-Modified-Since, and instead just ignores its cache. Why?

localhost: ~ $ curl --silent -o /dev/null --dump-header /dev/stdout http://localhost:3128/test.php
HTTP/1.1 200 OK
Date: Wed, 17 Apr 2013 00:17:42 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
X-Powered-By: PHP/5.3.15
Cache-Control: max-age=0, must-revalidate
Last-Modified: Mon, 16 Apr 2012 23:01:32 GMT
Content-Length: 16
Content-Type: text/html
X-Cache: MISS from localhost
Via: 1.1 localhost (squid/3.2.9)
Connection: keep-alive

localhost: ~ $ curl --silent -o /dev/null --dump-header /dev/stdout http://localhost:3128/test.php
HTTP/1.1 200 OK
X-Powered-By: PHP/5.3.15
Last-Modified: Mon, 16 Apr 2012 23:01:32 GMT
Content-Length: 16
Content-Type: text/html
Date: Wed, 17 Apr 2013 00:17:43 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
Cache-Control: max-age=0, must-revalidate
Age: 0
X-Cache: HIT from localhost
Via: 1.1 localhost (squid/3.2.9)
Connection: keep-alive

localhost: ~ $ curl --silent -o /dev/null --dump-header /dev/stdout "http://localhost:3128/test.php?vary=1"
HTTP/1.1 200 OK
Date: Wed, 17 Apr 2013 00:17:58 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
X-Powered-By: PHP/5.3.15
Cache-Control: max-age=0, must-revalidate
Last-Modified: Mon, 16 Apr 2012 23:01:32 GMT
Vary: Accept-Encoding
Content-Length: 16
Content-Type: text/html
X-Cache: MISS from localhost
Via: 1.1 localhost (squid/3.2.9)
Connection: keep-alive

localhost: ~ $ curl --silent -o /dev/null --dump-header /dev/stdout "http://localhost:3128/test.php?vary=1"
HTTP/1.1 200 OK
Date: Wed, 17 Apr 2013 00:18:00 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
X-Powered-By: PHP/5.3.15
Cache-Control: max-age=0, must-revalidate
Last-Modified: Mon, 16 Apr 2012 23:01:32 GMT
Vary: Accept-Encoding
Content-Length: 16
Content-Type: text/html
X-Cache: MISS from localhost
Via: 1.1 localhost (squid/3.2.9)
Connection: keep-alive
proxy
Martin Hope
Dan Fabulich
Asked: 2013-04-17 13:36:48 +0800 CST
  • 5

squid.conf:

http_access allow all

# Squid normally listens to port 3128
http_port 3128 accel defaultsite=localhost no-vhost

cache_peer localhost parent 80 0 no-query originserver name=myAccel
cache_peer_access myAccel allow all

# Uncomment and adjust the following to add a disk cache directory.
cache_dir ufs /usr/local/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/var/cache/squid

Squid 3.2.9, installed on OSX via Homebrew

localhost: ~ $ squid -v
Squid Cache: Version 3.2.9
configure options:  '--disable-debug' '--disable-dependency-tracking' '--prefix=/usr/local/Cellar/squid/3.2.9' '--localstatedir=/usr/local/var' 'CC=cc' 'CXX=c++' 'PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig'

Started squid with squid -f squid.conf -d 2 -N

The upstream is returning valid Cache-Control headers, which I've verified are correct by visiting the page in Google Chrome 26 and watching in Chrome Dev Tools.

localhost: ~ $ curl -s -o /dev/null --dump-header /dev/stdout  http://localhost/test.php
HTTP/1.1 200 OK
Date: Tue, 16 Apr 2013 21:29:33 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
X-Powered-By: PHP/5.3.15
Cache-Control: max-age=60, public
Content-Length: 16
Content-Type: text/html
X-Pad: avoid browser bug

But when I query the page via the Squid reverse proxy, I always get a cache miss.

localhost: ~ $ curl -s -o /dev/null --dump-header /dev/stdout  http://localhost:3128/test.php
HTTP/1.1 200 OK
Date: Tue, 16 Apr 2013 21:29:34 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
X-Powered-By: PHP/5.3.15
Cache-Control: max-age=60, public
Content-Length: 16
Content-Type: text/html
X-Cache: MISS from localhost
Via: 1.1 localhost (squid/3.2.9)
Connection: keep-alive

localhost: ~ $ curl -s -o /dev/null --dump-header /dev/stdout  http://localhost:3128/test.php
HTTP/1.1 200 OK
Date: Tue, 16 Apr 2013 21:29:38 GMT
Server: Apache/2.2.22 (Unix) DAV/2 PHP/5.3.15 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8r
X-Powered-By: PHP/5.3.15
Cache-Control: max-age=60, public
Content-Length: 16
Content-Type: text/html
X-Cache: MISS from localhost
Via: 1.1 localhost (squid/3.2.9)
Connection: keep-alive

What's wrong with my configuration?

proxy
Martin Hope
Dan Fabulich
Asked: 2011-03-30 13:32:54 +0800 CST
  • 1

I'm putting together a Windows box that will act as a continuous integration build server; we'll mostly build C# stuff using MSBuild, but I think this question applies to any compiled language on Windows.

What filesystem configuration settings should I use? (e.g. disabling NTFS last access time) The build server will spend a lot of time reading and writing small files, and scanning directories to see which files have been modified.

NOTE: Data integrity is a low priority in this case; it's just a build machine ... the final artifacts will be zipped up and archived elsewhere. If the filesystem on the build machine gets corrupted and loses all data, we can just wipe and re-image; builds will continue running as before. So we're prepared to play fast and loose with data integrity to get better performance.

windows performance filesystems continuous-integration
Martin Hope
Dan Fabulich
Asked: 2011-02-04 16:02:39 +0800 CST
  • 12

I'm putting together a Linux box that will act as a continuous integration build server; we'll mostly build Java stuff, but I think this question applies to any compiled language.

What filesystem and configuration settings should I use? (For example, I know I won't need atime for this!) The build server will spend a lot of time reading and writing small files, and scanning directories to see which files have been modified.

UPDATE: Data integrity is a low priority in this case; it's just a build machine ... the final artifacts will be zipped up and archived elsewhere. If the filesystem on the build machine gets corrupted and loses all data, we can just wipe and re-image; builds will continue running as before.

performance filesystems continuous-integration build-server
Martin Hope
Dan Fabulich
Asked: 2011-01-28 18:07:06 +0800 CST
  • 1

I'm attempting to set up FreeNX on the standard Ubuntu Amazon EC2 AMI image. (I'm using ami-af7e2eea).

Specifically, I'm trying to follow these directions: http://blog.restbackup.com/how-to-use-amazon-ec2-as-your-desktop?c=1 which worked fine on Lucid, but I needed to upgrade to a large instance (64-bit) and I needed to upgrade to Ubuntu 10.10 Maverick. I've been trying and failing to transmute these directions for Maverick.

Following the standard Ubuntu FreeNX documentation: https://help.ubuntu.com/community/FreeNX I was able to get freenx installed. But when I log into it from Windows, the gnome-terminal I launch has a grey screen; I can't actually use it for anything.

I assume I'm not installing freenx correctly. What should I be doing differently? How do I troubleshoot a broken FreeNX installation?

ubuntu freenx
Martin Hope
Dan Fabulich
Asked: 2010-01-26 16:43:27 +0800 CST
  • 6

I use Google Analytics today to notify me of big traffic spikes. In my case, a big traffic spike is typically due to a link from a prominent website.

Google Analytics includes a new "Intelligence" feature to report on spikes. Unfortunately, today it only reports that I had a spike, but doesn't report the referring site; instead it reports the geographic region of the traffic, e.g. "California."

This isn't a very useful dimension to report on, because geography is almost never the cause of the traffic spike; it's almost always due to a referring site.

Is there a way to configure Google Analytics Intelligence to report on the cause (the referring site) of traffic spikes?

reporting alerts google-analytics