Nick Jennings's questions

I've got a mail setup using postfix and dovecot, with spamassassin, clamav and amavis. Incoming mail contains the correct X-Spam* headers and everything appears to be operating more or less correct, except for a few erros when sending outgoing mail.

If I compose a message From: [email protected] - To: [email protected] I get some puzzling entries in my mail.log:

spamd: connection from localhost [::1]:40516 to port 783, fd 5                                              
spamd: handle_user (getpwnam) unable to find user: 'foobar'                                               
spamd: still running as root: user not specified with -u, not found, or set to root, falling back to nobody 
spamd: processing message <[email protected]> for foobar:65534        

I've tried searching for this error (specifically that spamd seems to be looking for a local user by the name of the recipient of the email (but only the sender is a local account on myhost.com), with no luck.

At this point I'm not sure which config is relevant to paste here, some documentation says I should be running spamd, others say that clamav or amavis should handle passing the mail through spamassassin and therefore spamd should not be running (however when I stop the daemon, things don't work).

This is sort of a general question about training spamassassin. I have a newly set up mailserver which filters incoming mail through spamassassin. I recently got a flight reservation flagged as spam (score 5) and would like to tell spamassassin it's not spam. (Perhaps doing this would also re-send the mail without the modified spamassassin headers?)

I've tried searching around and am only finding stuff about either getting spamassassin to flag messages as spam (and not about fixing false positives), or for people writing emails - how not to be flagged as spam.

So in regards to giving spamassassin feedback on wrong calls:

1. Is there a way to do this from within an email client (for example: Thunderbird)

2. Is there a way to do this via the command-line on the mail server?

I'd like to make the process as fluid as possible, but whatever gets the job done.

Details from SpamAssassin regarding the email:

 0.0 FSL_HELO_NON_FQDN_1    No description available.
 0.6 HK_RANDOM_ENVFROM      Envelope sender username looks random
-0.0 RCVD_IN_DNSWL_NONE     RBL: Sender listed at http://www.dnswl.org/, no trust [82.150.225.129 listed in list.dnswl.org]
-0.0 RCVD_IN_MSPIKE_H3      RBL: Good reputation (+3) [82.150.225.129 listed in wl.mailspike.net]
 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 1.6 SUBJ_ALL_CAPS          Subject is all capitals
 1.1 MIME_HTML_ONLY         BODY: Message only has text/html MIME parts
 0.7 HTML_IMAGE_ONLY_20     BODY: HTML: images with 1600-2000 bytes of words
 0.0 HTML_MESSAGE           BODY: HTML included in message
-0.0 RCVD_IN_MSPIKE_WL      Mailspike good senders
 0.0 UNPARSEABLE_RELAY      Informational: message has unparseable relay lines
 0.0 T_REMOTE_IMAGE         Message contains an external image

Clearly the main culprits are the all-caps subject line SUBJ_ALL_CAPS and the MIME_HTML_ONLY (I guess, no text alternative).

The email was for a flight booking confirmation and the subject looked like this:

 Subject: JENNINGS/NICHOLAS KOSSOW MR 24 JAN MOF DPS

Headers:

X-Envelope-From: <[email protected]>
X-Envelope-To: <[email protected]>
Received: from mail1.amadeus.net (unknown)
    by 147-49-15-51.rev.cloud.scaleway.com(Postfix 3.1.0/8.13.0) with SMTP id unknown
    Fri, 20 Jan 2017 07:55:10 +0000
    (envelope-from <[email protected]>
Received: from obeap115 (nat-dns-mnp.amadeus.net [82.150.225.129])
    by mail1.amadeus.net (Postfix) with ESMTP id 3F7A9200042
    for <[email protected]>; Fri, 20 Jan 2017 07:55:10 +0000 (GMT)
From: [email protected]
TO: [email protected]
Message-ID: <CTS/GA/C50D54421A07/[email protected]>
FND-Request-ID: <CTS/GA/C50D54421A07/[email protected]>
Job-ID: 1
Subject: JENNINGS/NICHOLAS KOSSOW MR 24 JAN MOF DPS
Date: Fri, 20 Jan 2017 07:55:09 +0000
Content-Type: multipart/mixed; 
    boundary="----=_Part_191904_1900935199.1484898909762"
MIME-Version: 1.0

I'm currently working on an API service and one of the requirements is that we measure the amount of data used by each user of the system, and track how many times they use the API. So bandwidth usage and statistics on which parts of the API they access, and how often.

The way these users access the API is by using a special Authorization HTTP header with their unique token.

One way to do this would be to implement these measurements within the API code, however since the webserver compresses all of the responses, the JSON objects we measure within the code will be much smaller in terms of bandwidth usage when being sent to the client.

I was wondering if there were existing tools (Linux) which we could use to monitor this traffic and break it up using the unique HTTP header? I've had a cursory glance at a few tools, namely ntopng and PRTG, but these seem more geared toward real-time monitoring (with options to store a historical record) and don't seem to be able to split up the stats based on the HTTP headers.

So, I'm wondering about a few things:

(A) What would be the best way to go about this task? (internal code, or an external monitoring tool)

(B) Are there any existing tools that can handle this?

(C) Are there any other ways to go about this that I'm not aware of?

Seems like a very common problem, so I was hoping there was already something out there that we could set up. Any insight is greatly appreciated.

I have several thousand MySQL users all set to allow access from a specific host. The problem is that now I'm going to have two machines (more in the future) which will need to use the same account to access each of their databases.

I'd like to a quick and easy (as automated as possible) way to run through and modify the host portion of each user account to fit an internal network wildcard. For example:

'bugsy'@'internalfoo' has access to the 'bugy' DB.

I want to now allow bugsy access from anywhere on the internal network

'bugsy'@'10.0.0.%' has access to the 'bugsy' DB.

I tried to cheat and use phpmyadmin for one, copy the syntax, rinse repeat for the other several thousand users, but the phpmyadmin way of doing it seems overly complicated. It creates a new user with the same permissions (different host access) and then deletes the old one. This would require me to know what every users password is, which isn't a realistic option for me at this time. I'd simply like to modify the existing users host access.

If anyone has any tips, I'd be very appreciative! :) -Nick