Richard Keller's questions

I have a redirector enabled in my Squid config file as follows:

url_rewrite_program /etc/squid/redirector.php

I need to access the original HTTP request headers from within the redirector. I'm not interested in making any changes to the request headers; I only need to be able to read them.

As far as I can tell, Squid only passes the URL to the redirector, but not the request headers. So how can I access these request headers in my redirector?

Update 1: A related side question, is it possible to access POST data within a Squid redirector? I imagine that if this is possible, the mechanism for doing so would be the same as accessing the headers.

Update 2: I believe this is possible using ICAP. I'm looking for a solution that does not involve ICAP.

I have 10 MySQL servers and each one holds a database that needs to be replicated to a single, centralised MySQL server. The complexity is that each server must replicate its database to the same database on the centralised server, essentially creating a single, centralised database of aggregated data from all the individual servers. Although various tables do contain primary keys, these columns can be safely ignored, and do not need to be replicated to the centralized server. The database structure is identical on each server. What would be the best way to accomplish this?

I'm in the process of converting an existing mail server to support encrypted SMTP for our clients, but I've run into this brick wall with very little useful log data to help me forward. Everything works fine when using regular unencrypted SMTP; it's only when trying to use encrypted SMTP that things go pear-shaped.

My exim config file contains the following:

# Allow any client to use TLS
tls_advertise_hosts                       =  *

# Specify the location of the Exim server's TLS certificate and private key.
tls_certificate                           =  /etc/exim/exim.crt
tls_privatekey                            =  /etc/exim/exim.key 

Initially, Exim appears to be working as expected, I am able to securely connect to the mail server and authenticate myself, but just after I enter the recipient section in the SMTP session, the connection is dropped. This problem does not occur when using an unencrypted connection.

To test secure SMTP I use the following command:

openssl s_client -starttls smtp -crlf -connect localhost:25

And this is the output I get:

CONNECTED(00000003)
depth=0 C = ZA, etc, etc
verify error:num=18:self signed certificate
verify return:1
depth=0 C = ZA, etc, etc
verify return:1
---
Certificate chain
 0 s:/C=ZA/etc,etc
   i:/C=ZA/etc,etc
---
Server certificate
-----BEGIN CERTIFICATE-----
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXX==
-----END CERTIFICATE-----
subject=/C=ZA/etc,etc
---
No client certificate CA names sent
---
SSL handshake has read 1275 bytes and written 444 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: zlib compression
Expansion: zlib compression
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    Session-ID-ctx: 
    Master-Key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket:
    0000 - d0 cd ff b6 0c a2 fb 6c-f6 69 dc 0b a7 aa f3 1a   .......l.i......
    0010 - 10 76 75 05 15 d8 8c 21-cb eb b8 ae ec 34 7d b3   .vu....!.....4}.
    0020 - 7a bf f0 d6 7d df 26 27-41 1e d1 2a 35 bf 2f 0c   z...}.&'A..*5./.
    0030 - 25 6a 32 15 6e 53 d2 30-31 1b d9 60 e6 11 20 73   %j2.nS.01..`.. s
    0040 - 57 e3 76 96 e7 7e dc da-98 f2 cc a7 e5 58 62 b2   W.v..~.......Xb.
    0050 - ec db 58 91 16 14 18 ff-15 64 d6 66 1f 75 92 96   ..X......d.f.u..
    0060 - 65 43 f8 2c 4a 42 81 41-0c 2f 46 84 38 0c c5 e0   eC.,JB.A./F.8...
    0070 - 8d 7b d7 7e 12 0e 28 ca-f0 f9 b5 d0 b2 a6 ab 66   .{.~..(........f
    0080 - f8 c5 33 e3 cb 16 f5 76-8f e7 49 0c 49 69 31 43   ..3....v..I.Ii1C
    0090 - 05 25 dc 75 3a 07 13 91-63 ff 13 fd b0 2c 9f 8b   .%.u:...c....,..

    Compression: 1 (zlib compression)
    Start Time: 1315250595
    Timeout   : 300 (sec)
    Verify return code: 18 (self signed certificate)
---
250 HELP
HELO localhost
250 OK
MAIL FROM:[email protected]
250 OK
RCPT TO:[email protected]
RENEGOTIATING
depth=0 C = ZA, etc, etc
verify error:num=18:self signed certificate
verify return:1
depth=0 C = ZA, etc, etc
verify return:1
421 lost input connection
read:errno=0

I've replaced the email addresses and organisation tree with garbage data in the above output, since it is irrelevant as I do not have this same issue when using regular SMTP. The above transaction occurs regardless of whether I attempt the connection from localhost or from an external source. I should also note that I am using a self-signed certificate generated using OpenSSL. Also, in the above example there is no authentication data since I am performing the test from localhost, which allows all mail without authentication required.

As you can see in the above output, Exim seems to break during/after issuing the string "RENEGOTIATING".

Since the output I receive during the SMTP session is not much help, I've also tried running Exim in debug +all mode. For the sake of brevity, I won't post the complete SMTP transaction since the entire session is quite normal, up until the point where I specify the recipient address. This is the exact snippet of Exim debug data that I get once I've typed in the recipient address and pressed enter:

21:42:10  7425 SSL info: before accept initialization
21:42:10  7425 SSL info: before accept initialization
21:42:10  7425 SSL info: SSLv3 read client hello A
21:42:10  7425 SSL info: SSLv3 write server hello A
21:42:10  7425 SSL info: SSLv3 write certificate A
21:42:10  7425 SSL info: SSLv3 write server done A
21:42:10  7425 SSL info: SSLv3 flush data
21:42:10  7425 SSL info: SSLv3 read client key exchange A
21:42:10  7425 SSL info: SSLv3 read finished A
21:42:10  7425 SSL info: SSLv3 write session ticket A
21:42:10  7425 SSL info: SSLv3 write change cipher spec A
21:42:10  7425 SSL info: SSLv3 write finished A
21:42:10  7425 SSL info: SSLv3 flush data
21:42:10  7425 SSL info: SSL negotiation finished successfully
21:42:10  7425 SSL info: SSL negotiation finished successfully
21:42:10  7425 Got SSL error 2
21:42:10  7425 SMTP>> 421 lost input connection
21:42:10  7425 tls_do_write(1db4020, 48)
21:42:10  7425 SSL_write(SSL, 1db4020, 48)
21:42:10  7425 outbytes=48 error=0
21:42:10  7425 LOG: lost_incoming_connection MAIN
21:42:10  7425   unexpected disconnection while reading SMTP command from (localhost) [127.0.0.1]
21:42:10  7425 search_tidyup called
21:42:10  7194 child 7425 ended: status=0x100
21:42:10  7194 0 SMTP accept processes now running
21:42:10  7194 Listening...

We have a number of Linux servers for which I would like to capture netflow data to be processed by a netflow analyzer. I've been spoiled by the ease in which Mikrotik routers allow netflow data generation, but I haven't managed to find an opensource tool that is able to generate netflow data for multiple interfaces on a Linux system.

I have come across fprobe but it seems quite buggy. Admittedly I haven't spent much time with it yet since I'd also like to evaluate some other possibilities. The other tool I've seen mentioned is nprobe, which appears to be GPL, but is not available as a free download since it is only offered for a fee.

The servers on which I plan to generate netflow data are all Gentoo systems, but this shouldn't really make any difference. At most it means I would have to manually compile a tool from source.

Summary: I'm looking for an opensource netflow generator that will work on Linux and allows capturing flows for multiple interfaces.

Our primary MX record points to an external server which handles spam filtering, thereafter the mail is pushed to our internal mail server. The mail is never stored on the external server for longer than it takes to process the mail and deliver it to our internal server.

The current setup requires that we connect to our internal server to retrieve mail. The problem with this setup is that roaming users are unable to access their email if our internet connectivity at the office goes down. An alternative is to host all our mail externally, but this means that users in the office must be connected to the internet for the duration it takes to download and send all their mail, which will be significantly slower than if we had a mail server on the LAN handling the mail. Additionally, this setup would mean that even internal mail has to leave the network, which is terribly inefficient.

Are there any methods whereby we can have both our internal server and our external server hosting the mail, and depending on the DNS record (from within the office the DNS record will resolve to our internal server, and externally the record will resolve to our offsite mail server) our users will always be able to send and retrieve their mail? ie. users should be able to use either of the two mail servers to retrieve their mail. I've given some thought to using IMAP in this situation, but would this not cause mail duplication when switching between servers?

The architecture is irrelevant. I'm looking for any solution, regardless of whether it's based on Exim, Exchange, or anything else that we can setup ourselves.

I've never quite understood whether or not it's possible to rate-limit incoming traffic. I realize that there is no direct method whereby to control the remote server's rate of sending packets (unless you're in control of both endpoints), but taking this limitation into account, how exactly do download managers allow me to successfully set download speed limits?

Is there any link between TCP slow-start and rate-limiting incoming traffic? Is it possible to use the methods described by slow-start to artificially limit the sender's rate of sending packets?

As an additional consideration, it should be noted that the server on which I'd like to implement traffic shaping establishes the PPPoE connection itself, and acts as a router for the rest of the network.

Update: The answers thus far have given a fair overview of the questions I've asked, but I still don't know how download managers are able to limit incoming traffic, and more specifically, whether it's possible to implement a similar strategy on a Linux gateway box.

I have a server running multiple websites, with each website configured as a separate vhost. Some trusted users require shell access to the web server, but in an effort to prevent abuse and accidental misconfiguration, each user is only allowed write access to their own website directory. The way in which this is currently implemented is that each website has the ownership set to username:apache where 'username' is the username of the person who is allowed write access to the directory, and 'apache' is the group name to which the actual Apache user belongs.

This configuration works, to an extent. Users are able to edit existing files and create new files without any problems. However, since most of the sites are driven by various CMS software, all content written to the directory by the CMS (ie. the webserver / Apache) is written as apache:apache, and the user does not receive write access to the new files.

My questions:

1. What is the best way to set up such a configuration whereby multiple users are allowed write access to their own directories only, and whereby files written by the webserver are also writable by the user?
2. Is it possible to instruct Apache to write data for certain vhosts as another user and group?

PHP on Gentoo supports the threads USE flag. As far as I know, PHP does not support true threading; I am only aware of the ability to fork using pcntl_fork(), but this functionality is provided by the pcntl USE flag. So what exactly is the point of the threads USE flag if PHP does not support threading?

tl;dr - What purpose does the threads USE flag serve for PHP on Gentoo systems?

What is the best way to handle bounces and immediate failures with Exim? An email sent to a large mailing list will generate a large number of immediate failures as well as bounces. I'd like to capture these bounces and immediate failures and process them with a script, so that I can automatically remove them from the mailing list.

Using a third party email campaign manager is not an option. I need to specifically handle these messages with Exim, either capturing or redirecting them for later processing with a custom script. What would be the best way to implement something like this with Exim?

A similar question was asked here regarding IIS, but isn't much help in implementing something similar with Exim.

I understand that 'defer' is returned if a test cannot be completed, for example when a DNS blacklist lookup times out. But what effect does defer have on the final ACL decision? Also, what happens when I manually set the return to be 'defer' instead of 'deny', for example when looking up against a local blacklist?