I used the tutorial below to setup postfix and dovecot on a Digital Ocean server. But I'm getting "Connection Timed out" when postfix/smtp tries to deliver an email to a remote host like gmail or yahoo..

https://www.digitalocean.com/community/articles/how-to-set-up-a-postfix-e-mail-server-with-dovecot

RECEIVING A NEW MESSAGE /var/log/mail.log

Feb 2 22:33:38 localhost dovecot: auth-worker: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth

Feb 2 22:33:38 localhost dovecot: auth-worker: Debug: pam(app,189.63.49.XXX): lookup service=dovecot

Feb 2 22:33:38 localhost dovecot: auth-worker: Debug: pam(app,189.63.49.XXX): #1/1 style=1 msg=Password:

Feb 2 22:33:38 localhost dovecot: auth: Debug: client out: OK#0111#011user=app

Feb 2 22:33:40 localhost postfix/submission/smtpd[1045]: E9AA724264: client=unknown[189.63.49.XXX], sasl_method=PLAIN, sasl_username=app

Feb 2 22:33:43 localhost postfix/cleanup[1052]: E9AA724264: message-id=

Feb 2 22:33:43 localhost postfix/qmgr[32661]: E9AA724264: from=, size=485, nrcpt=1 (queue active)

Feb 2 22:33:45 localhost postfix/submission/smtpd[1045]: disconnect from unknown[189.63.49.XXX]

Feb 2 22:34:14 localhost postfix/smtp[1053]: connect to mta5.am0.yahoodns.net[98.136.216.25]:25: Connection timed out

Feb 2 22:24:21 localhost postfix/smtp[1013]: connect to mta6.am0.yahoodns.net[98.136.217.202]:25: Connection timed out

Feb 2 22:24:21 localhost postfix/smtp[1014]: connect to gmail-smtp-in.l.google.com[173.194.76.27]:25: Connection timed out

Feb 2 22:24:21 localhost postfix/smtp[1015]: connect to gmail-smtp-in.l.google.com[173.194.76.27]:25: Connection timed out

Feb 2 22:24:21 localhost postfix/smtp[1012]: connect to aspmx.l.google.com[173.194.68.27]:25: Connection timed out

Here is /etc/postfix/main.cf http://pastebin.com/nHQhh8Bp

/etc/postfix/master.cf http://pastebin.com/nnJBP9mh

Result from #> netstat -nlptu http://pastebin.com/PKJX6xC1

I trying to setup as part of a study a PaaS (Platform as a service) for Ruby on Rails applications. I have already played with Amazon EC2 and Chef creating EC2 instances based on Chef's recipes and roles.

Now I need to be able to give every "customer" account or every app a unique subdomain like app1.mydomain.com, app2.mydomain.com, etc. which points to my customer's app on an EC2 instance. I then need to allow the customer to point their domain to their customized subdomain (app1.mydomain.com, app2.mydomain.com).

Can someone explain to me how I can achieve that?

I couldn't find any resource about how to setup exim throttling.
I needed to control email delivery flow respecting each ISP rate limits and guidelines.

Can anyone point me some references on how to setup this?

Thanks.

While checking on Exim's log messages I found many entries of the following message "Sender verify failed" "rejected RCPT" ... I 'm not an exim expert... I'm afraid Exim is not delivering 100% emails to recipients, because our Email Marketing Application its getting a lower OPEN RATE.

Can someone helpe understand this log messages? Is it my server saying "No Such User Here" or a remote server? 174.111.111.11 represents my server IP. Thanks

Exim log

2010-10-02 14:00:19 SMTP connection from myserverdomain.com.br () [174.111.111.11]:54514 I=[174.111.111.11]:25 closed by QUIT
2010-10-02 14:00:19 SMTP connection from [174.111.111.11]:54515 I=[174.111.111.11]:25 (TCP/IP connection count = 2)
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54515 I=[174.111.111.11]:25 Warning: Sender rate 672.4 / 1h
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54515 I=[174.111.111.11]:25 sender verify fail for <[email protected]>: No Such User Here
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54515 I=[174.111.111.11]:25 F=<[email protected]> rejected RCPT <[email protected]>: Sender verify failed

2010-10-02 14:00:19 SMTP connection from myserverdomain.com.br () [174.111.111.11]:54515 I=[174.111.111.11]:25 closed by QUIT
2010-10-02 14:00:19 SMTP connection from [174.111.111.11]:54516 I=[174.111.111.11]:25 (TCP/IP connection count = 2)
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54516 I=[174.111.111.11]:25 Warning: Sender rate 673.3 / 1h
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54516 I=[174.111.111.11]:25 sender verify fail for <[email protected]>: No Such User Here
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54516 I=[174.111.111.11]:25 F=<[email protected]> rejected RCPT <[email protected]>: Sender verify failed

2010-10-02 14:00:19 SMTP connection from myserverdomain.com.br () [174.111.111.11]:54516 I=[174.111.111.11]:25 closed by QUIT
2010-10-02 14:00:19 SMTP connection from [174.111.111.11]:54517 I=[174.111.111.11]:25 (TCP/IP connection count = 2)
2010-10-02 14:00:19 H=myserverdomain.com.br () [174.111.111.11]:54517 I=[174.111.111.11]:25 Warning: Sender rate 674.3 / 1h
2010-10-02 14:00:20 H=myserverdomain.com.br () [174.111.111.11]:54517 I=[174.111.111.11]:25 sender verify fail for <Luciene_souza_vasconcellos=hotmail.com--2723--bounce@e-mydomain.com.br>: No Such User Here
2010-10-02 14:00:20 H=myserverdomain.com.br () [174.111.111.11]:54517 I=[174.111.111.11]:25 F=<Luciene_souza_vasconcellos=hotmail.com--2723--bounce@e-mydomain.com.br> rejected RCPT <[email protected]>: Sender verify failed

HI

I have an Email Marketing Rails application running on a CentOS QuadCore 16GB RAM server. But currently our webserver is taking too long to respond to requests on rush hours (Mongrel Cluster + Apache ). We monitoring it using ScoutApp ( www.scoutapp.com ).

Scout Alerts Maximum Time(3 sec) exceeded on 668 requests Maximum Time(3 sec) exceeded on 120 requests

I contracted another server Dual Xeon 4GB RAM .

What is the best setup for distributing this application between 2 servers? I'm thinking about using the SERVER-1 (16GB RAM) with Mysql and Exim and migrate the application to SERVER-2 (4GB RAM) and use it as the WEB SERVER (Mongrel cluster + Apache) only.

Can anyone suggest me a better setup , tips or ideas?

I'm trying to import some data from one server to another. But when I do it, I'm having problems with charset.

Words like Goiânia became Goiâni and conceição became conceição

My Application was set to use latin1 charset

Server 1 : MySQL Charset : UTF-8 Unicode (utf8) table collation : latin1_swedish_ci

Server 2 : MySQL Charset: UTF-8 Unicode (utf8) table collation : latin1_swedish_ci

Command I used to export data from server 1 mysqldump -u root -p --default-character-set=iso-8859-1 database_name > db.sql

Command used to restore to server 2 mysql -u root -p database_name < db.sql

We have multi-user application (like MailChimp,Constant Contact) . Each of our customers has it's own contact's list (from 5 to 100.000 contacts). Everything is stored in one BIG database (currently 25G). Since we released our product we have the following data history.

5 years of data history : - users/customers (200+) - contacts (40 million records) - campaigns - campaign_deliveries (73.843.764 records) - campaign_queue ( 8 millions currently )

As we get more users and table records increase our system/web app is getting slower and slower . Some queries takes too long to execute .

SCHEMA

Table contacts

--------------------+------------------+------+-----+---------+----------------+
| Field               | Type             | Null | Key | Default | Extra          |
+---------------------+------------------+------+-----+---------+----------------+
| contact_id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment | 
| client_id           | int(10) unsigned | YES  |     | NULL    |                | 
| name                | varchar(60)      | YES  |     | NULL    |                | 
| mail                | varchar(60)      | YES  | MUL | NULL    |                | 
| verified            | int(1)           | YES  |     | 0       |                | 
| owner               | int(10) unsigned | NO   | MUL | 0       |                | 
| date_created        | date             | YES  | MUL | NULL    |                |  
| geolocation         | varchar(100)     | YES  |     | NULL    |                | 
| ip                  | varchar(20)      | YES  | MUL | NULL    |                | 
+---------------------+------------------+------+-----+---------+----------------+  
PRIMARY KEY  (`contact_id`),
  UNIQUE KEY `owner` (`owner`,`mail`),
  KEY `contacts_index_mail` (`mail`),
  KEY `index_contacts_date_created` (`date_created`),
  KEY `index_contacts_ip` (`ip`),

Table campaign_deliveries

+---------------+------------------+------+-----+---------+----------------+
| Field         | Type             | Null | Key | Default | Extra          |
+---------------+------------------+------+-----+---------+----------------+
| id            | int(11)          | NO   | PRI | NULL    | auto_increment | 
| newsletter_id | int(10) unsigned | NO   | MUL | 0       |                | 
| contact_id    | int(10) unsigned | NO   | MUL | 0       |                | 
| sent_date     | date             | YES  | MUL | NULL    |                | 
| sent_time     | time             | YES  | MUL | NULL    |                | 
| smtp_server   | varchar(20)      | YES  |     | NULL    |                | 
| owner         | int(5)           | YES  | MUL | NULL    |                | 
| ip            | varchar(20)      | YES  | MUL | NULL    |                | 
+---------------+------------------+------+-----+---------+----------------+
INDEXES
PRIMARY KEY  (`id`),
  UNIQUE KEY `newsletter_id` (`newsletter_id`,`contact_id`),
  KEY `newsletter_delivery_FKIndex1` (`newsletter_id`),
  KEY `newsletter_delivery_FKIndex2` (`contact_id`),
  KEY `newsletter_delivery_owner` (`owner`),
  KEY `index_nd_sent_date` (`sent_date`),
  KEY `index_nd_sent_time` (`sent_time`),
  KEY `index_ip` (`ip`)

Table campaign_queue

+---------------+------------------+------+-----+---------+----------------+
| Field         | Type             | Null | Key | Default | Extra          |
+---------------+------------------+------+-----+---------+----------------+
| queue_id      | int(10) unsigned | NO   | PRI | NULL    | auto_increment | 
| newsletter_id | int(10) unsigned | NO   | MUL | 0       |                | 
| owner         | int(10) unsigned | NO   | MUL | 0       |                | 
| date_to_send  | date             | YES  |     | NULL    |                | 
| contact_id    | int(11)          | NO   | MUL | NULL    |                | 
| date_created  | date             | YES  |     | NULL    |                | 
+---------------+------------------+------+-----+---------+----------------+
INDEXES
PRIMARY KEY  (`queue_id`),
  UNIQUE KEY `newsletter_id` (`newsletter_id`,`contact_id`),
  KEY `newsletter_queue_index1` (`newsletter_id`),
  KEY `newsletter_queue_index4` (`owner`),
  KEY `newsletter_queue_index5` (`newsletter_id`),
  KEY `contacts_contact_id` (`contact_id`)

Slow queries LOG --------------------------------------------

Query_time: 350 Lock_time: 1 Rows_sent: 1 Rows_examined: 971004

SELECT COUNT(*) as total FROM contacts WHERE (contacts.owner = 70 AND contacts.verified = 1);

Query_time: 235 Lock_time: 1 Rows_sent: 1 Rows_examined: 4455209

SELECT COUNT(*) as total FROM contacts WHERE (contacts.owner = 2);

EXPLAIN

EXPLAIN SELECT COUNT(*) as total FROM contacts WHERE (contacts.owner = 112 AND contacts.verified = 1);

+----+-------------+----------+------+---------------+-------+---------+-------+--------+-------------+
| id | select_type | table    | type | possible_keys | key   | key_len | ref   | rows   | Extra       |
+----+-------------+----------+------+---------------+-------+---------+-------+--------+-------------+
|  1 | SIMPLE      | contacts | ref  | owner         | owner | 4       | const | 644817 | Using where | 
+----+-------------+----------+------+---------------+-------+---------+-------+--------+-------------+

How can we optimize it ? Queries should take no more than 30 secs to execute? Can we optimize it and keep all data in one BIG database or should we change app's structure and set one single database to each user ?

EDIT table indexes added below schema

Thanks

How can I customize the mail FROM header in our Email Marketing Application , to enable our customers to specify their OWN email ( from their domain ) .

Currently the customer specify his own domain and we use it at the Reply-To mail's header.

CURRENTLY

From: [email protected]
Reply-To: customer_email@customer_domain.com
Return-Path: [email protected]

WHAT WE NEED

From: customer_email@customer_domain.com
Reply-To: customer_email@customer_domain.com
Return-Path: [email protected]

We do it this way to avoid getting blacklisted because Mail Servers like Gmail or Hotmail would considers it as a MAIL'S HEADER FORGERY ATTEMPT.

But our customers keeps asking us to make the FROM HEADER customizable.

Can someone help us ?

We provide Email Marketing service through our online App. We have about 30 customers. And each one has it's own mailling list (5k to 20k emails each).

What we really want is to distribute email's delivery between 2 or more servers. I was wondering What kind of aproach/solutions MailChimp , Constant Contact uses to provide a great service ? use many servers ? many IPs ?

Our spam policy suspends ANY user/customer that gets 10% bounced .