So you've had some success using sendmail to send one-off emails like order receipts, password reset confirmations and welcome letters. Now, you need to be able to send 5,000 newsletter emails to your entire contact database. Hmm, sendmail alone just isn't going to cut it!

This tutorial assumes that you are already sending emails out using sendmail. This is a tutorial on changing from sendmail to ar_mailer. If you are not yet able to send email from your application with sendmail, I suggest reading this tutorial first.

Introducing ar_mailer

A great way to handle a massive amount of emails that you need to send is simply by queueing the emails and sending them in smaller bursts. You will need to install the RubyGem "ar_mailer" on your server (and/or local development machine) to get this working. How it works is simple: Whenever you attempt to send an email by calling a method like "deliver_welcome_letter," instead of using sendmail to send the email at that moment, ar_mailer will instead put all the outgoing email information into a database table. Then, in the background, the ar_sendmail process will be periodically querying that database table and sending out the emails until the table is empty.

When you pay for a shared Ruby on Rails hosting plan, you will only be allowed to send out a certain number of emails per hour. The host that we use here at Ameravant actually limits us to 250 outgoing emails per hour for each of our accounts. So, if a client of ours wants to send out a newsletter to 1,000 contacts, only the first 250 would be delivered if we used sendmail with no queue. Therefore, we need to limit the number of emails we send per hour so all 1,000 contacts will eventually receive their emails after about four hours.

Let's get started

First off, you gotta have the ar_mailer gem installed on your server. This may require contacting your hosting provider to get the gem installed on your production server. For now, we can install it on our local machine for development and testing purposes.

$ sudo gem install ar_mailer

We need to setup a model in our application that will be used to store the outgoing emails. There are two commands that ar_sendmail (note that while the gem is called ar_mailer, the actual command is ar_sendmail) provides to give us the model and migration. They are ar_sendmail --create-migration and ar_sendmail --create-model. Running these commands will print out the migration and model code for you to use. However, you can simply make the model yourself and use the code I give you here, to save some time.

The model and database migration

$ script/generate model email

001_create_emails.rb

class CreateEmails < ActiveRecord::Migration
  def self.up
    create_table :emails do |t|
      t.string :from, :to
      t.integer :last_send_attempt, :default => 0
      t.text :mail
      t.datetime :created_on # ar_mailer uses deprecated created_on field
    end
  end

  def self.down
    drop_table :emails
  end
end

You will notice that ar_mailer still uses the old "created_on" field name as opposed to the newer "created_at" convention. This is the table that will hold all of the outgoing email information. It will store the to and from addresses, the entire MIME email body, and some meta information like when the email was added to the queue and how many times it has been attempted to be sent out. When the ar_sendmail process runs, it uses the "created_on" field to determine if an email is expired. This means that if an email is unable to be delivered for a long period of time, it will simply be removed from the queue (the default expiration time is one week).

email.rb

We only have some simple validations here to ensure that no emails are added to the database that are missing information.

class Email < ActiveRecord::Base
  validates_presence_of :from, :to, :mail
end

environment.rb

We also need to include the ar_mailer gem, and change our application's ActionMailer delivery method from sendmail to activerecord.

require 'action_mailer/ar_mailer'

ActionMailer::Base.perform_deliveries = true  
ActionMailer::Base.default_charset = 'utf-8'
ActionMailer::Base.delivery_method = :activerecord

The model I have for sending email is called postoffice.rb. Yours may be called something else, like mailer.rb. You need to modify the first line in this file to use ARMailer instead of ActionMailer.

class Postoffice < ActionMailer::ARMailer
  ...
end

Now, you should test sending your emails. Once you use your site to send the emails out, go into your console and look in the Emails table for your queued emails.

$ script/console
Loading development environment (Rails 2.1.0)
>> Email.find(:all)

The magic in the background

So, we've got our emails queued and ready to be sent, but they will not actually be delivered until we start using the ar_mailer process called ar_sendmail. We have two options for making this work: We can use the process in daemon mode so it runs constantly in the background, or we can call it on a recurring schedule by setting up a cron job.

The main problem with running the process as a daemon is that it can unexpectedly and randomly halt. This is not such a problem locally, as we'll likely be developing and testing our application and will know the process has stopped. We can then simply restart it. However, on a production server, we will likely not be monitoring the processes, and therefore will usually only know it has stopped running when a client calls us, frustrated that their newsletter is not being sent.

I suggest running it as a daemon locally, and use a cron job on your production server.

Running ar_sendmail in daemon mode

This command is pretty simple. All we do is run the ar_sendmail command from our shell. To get a list of all the arguments you can pass to it, simply run ar_sendmail -h. For development purposes right now, I'm going to just have ar_sendmail one a single time, so we can test that everything is configured properly.

First, we can check out our email queue with this:

ar_sendmail -mailq

We will be shown a list of emails in our database that are ready to be sent. If you see emails in there, great! That means we've got our queue working properly. So, now let's actually send the emails out.

ar_sendmail -ov

This tells it to run one time only, and to be verbose about what it is doing. If all goes well, you should see some emails appearing in your inbox! If you want it to run on a recurring basis on your development machine, set it to run in daemon mode and give it parameters to tell it how many to send in each burst, and how often. This, for example, will send 20 emails every 5 minutes:

ar_sendmail -d --batch-size 20 --delay 300

Running as a cron job on your production server

When we run ar_sendmail on our production server, we need to be a bit more specific in our command, to ensure that the proper application path and environment is used. As explained before, I prefer to run a cron job that commands ar_sendmail to run a single time, on a recurring basis. For our host, we are limited to 250 emails an hour. So, I will setup the cron job to run at 5 minute intervals. This means we can send 20 emails every 5 minutes, giving us the safe number of sending 240 per hour.

Here's my entry in my production server's crontab:

*/5 * * * * /usr/local/bin/ruby /usr/local/bin/ar_sendmail -o --batch-size 20 --chdir /home/myusername/doohickey --environment production

If you've read my cron job post, you'll understand this command pretty easily.

That's it, we're up and rolling with queued emails. But for some extra credit, a few really neat things that I like to do is provide a section for the customer to manage their email queue. I let them see:

1. A list of all emails that are in the queue
2. A time estimate of how long it will take the remaining emails to be sent (we know how many we send an hour, so this is cake!)
3. The ability to delete a single email from the queue
4. The ability to clear the entire queue in case they mistakenly sent something and then realized there was a huge typo in the subject line

This kind of flexibility is easy to add and your customers will appreciate it!