Utah Open Source Planet

I’ve got a few servers in different places around the country and try to monitor them using the logwatch utility.  One problem that I’ve run into however is that a few of these servers are not able to send their logwatch emails to me, based on email restrictions by the ISPs.  I spent some time this afternoon researching what was required to have my servers authenticate to my gmail account and send me the mail that way.  This setup assumes Ubuntu 8.04 (or later) and Postfix.

Install the required packages

sudo aptitude install postfix libsasl2 ca-certificate

Configure Postfix

This tutorial will not outline how to configure your postfix server, but we’ll jump directly to the relayhost section.  You’ll want to add the following lines to your /etc/postfix/main.cf file:

relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CAfile = /etc/postfix/cacert.pem
smtp_use_tls = yes

The above lines are telling Postfix that you want to relay mail through gmail on a specific port, telling it to authenticate, and where to find the username and password.  The last three lines specify the authentication types supported, where the certificate authority file is and that it should use tls.

Define Username and Password

Next we’ll need to populate the sasl_passwd file.  Create the file /etc/postfix/sasl_passwd with the following contents:

[smtp.gmail.com]:587    user.name@gmail.com:password

This file should have restrictive permissions and then needs to be translated into a .db that Postfix will read.

sudo chmod 400 /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd


At this point you can restart Postfix and it should work, however it will complain about not being able to authenticate the certificate.  To take care of this issue we’ll use the ca-certificate package we installed and tell it where it can validate the certificate.

cat /etc/ssl/certs/Thawte_Premium_Server_CA.pem | sudo tee -a /etc/postfix/cacert.pem

Go ahead and reload postfix (sudo /etc/init.d/postfix reload) and you should be set.

Other Points of Interest

• November 13, 2008 — Monitor System Logs With Logwatch (3)

As many of you already know from this PLUG thread, the mail servers for MSN and Hotmail have been doing something despicable, for quite some time now. They quietly delete email that they think is spam, after accepting it for delivery.

Now, depending on your experience that may sound like no big deal, or it may rival the best plot ever for a horror movie. Let me explain why this is worse than zombies.

First, an analogy. If you put a stamp on an envelope and send a letter to your friend or business associate via the postal service, you expect it to get there. If you send a package via UPS you expect it to get there. You don't expect the recipient to show up at the post office to find the postal worker wearing the sweater you sent him. The mail may have been delayed, but it will get there. "Neither rain nor hail nor sleet nor snow nor heat of day nor dark of night shall keep this carrier from the swift completion of his appointed rounds."

If you show up to the post office with illegal mail—something that is outside the allowed dimensions or too heavy or has explosives in it—you will be told that you can't send it. You'll know you need to get a different envelope, pay more postage, etc. You have feedback. If you send mail to a bad address, or someone who hates your guts, you will get your mail back marked "Return to Sender" (assuming you provided a return address). Feedback. In neither case would you be satisfied if the post office just threw your important correspondence in the trash.

So, why would you accept this behavior from your email provider? When I send an email to fred@hotmail.com (whoever he is), and hotmail's mail server says 250 Queued mail for delivery, I expect fred will get my email. Maybe it's in his junk mailbox. Maybe fred is away on vacation and won't see it for a few months. Maybe it takes more than 5 minutes to traverse the internet. But I expect it will get there, or that I will get a bounce message if it doesn't. The reliability of email depends on it. The rules demand it.

If you use Hotmail or MSN, you have been losing email. I dare you to prove otherwise. You owe it to yourself and to the people with whom you correspond to switch right now. GMail is an excellent alternative. It has more features, better spam detection that doesn't delete mail without asking, a nicer user interface, more storage space than even you will use, and POP3 and IMAP support. While you're at it, if you use MSN for instant messaging switch over to Google Talk or, even better, Jabber.

If you have been corresponding with MSN or Hotmail users, your emails may have been trashed before the recipient ever saw them. They may not even realize this. Do your part and inform them. If they don't want to switch to a sane email service, then insist on communicating in a different manner.

And let this be a lesson to you: don't let the quest against spam (or anything else) turn into tilting at windmills. Keep your wits about you and consider the consequences of your actions. Especially if you're a system administrator. Really especially if you're a system administrator at a big email service like Hotmail. And that goes for SPF, too.

A commenter pointed out on my original post about Apple Mail and Gmail that you can manage multiple Gmail identities by separating each email address with a comma. Here are the steps: If you have already set up your Gmail identities on Gmail, you can go to the next step. You can't use an [...]

So what is all this RSS hype? I've known about RSS feeds, what they are for, and how to use them for quite a while. It's not like they are new or anything! It wasn't until recently that I started to actually use them though. I didn't know what I was [...]

After posting a few tips on completing a Masters Degree and a Thesis, I had a thought that it would be fun to post about the same topic from a lighter angle. How many emails does it take to complete a Masters Degree? I started using Gmail in 2005. Unfortunately, I don't have [...]

So its not a big surprise that an oft-requested feature for Beagle is the ability to index a users Gmail messages (like Google Desktop Search). Today we (the Beagle developers) started to investigate just how this is done. While POP3 (and now IMAP) are available, downloading all of a users mail, indexing it, and then caching the text so we can display it. Now, my initial investigation into GDS for Linux revealed that it was calling home via POP3s and downloading lots of data. I have assumed that it was simply iterating over all messages (via POP3), downloading them, indexing them, and caching the compressed content somewhere in Google’s custom indexes.

Now, I had originally planned on this post being an open plea to any and everyone at Google asking them to open up the Gmail access API, but seeing as its just the plain old ugly POP3 (maybe a cool extension), were stuck biting the bullet and implementing a remote mail access layer.

Anyways, given how incredible Google has been in a million other situations, I thought I would throw out 2 wildly out-of-this-world questions, I wouldn’t expect to get a response, but before I spend the time figuring it all out, I felt like I should at least ask.

• Are there some special POP Extensions available in Gmail? Is there some helper web api? Or does GDS really just have a POP3 crawler?
• Is your compression/text storage library open source? (or documented in some research paper at all?) Beagle has always struggled with how to best handle storing copies of a documents text so that it might be made available in interfaces. While we do have a new hybrid text cache (text over 4k on the filesystem, under in a sqlite db, all compressed) we were still no where near as small as the GDS indexes. A cursory examination reveals that the GDS indexes are some form of b-tree on disk, but how are you compressing all that text so small? Is there some substitution/reconstruction algorithm? (It seems like that would be wildly expensive, but who knows).

Anyways, its a long shot, and its pretty far out there, but for the sake of not passing up answers that I can’t seem to find elsewhere on the net, I have asked.

After I set up Apple Mail to use my Gmail account, I had a couple things I was still having to go to the Gmail web interface for. One of those was managing multiple labels for a particular message. Since that time, I did some experimenting and have figured out how to manage [...]

I recently purchased my first iMac. I’ve been having all kinds of fun learning about OS X features as well as playing with the newly certified Unix system (Darwin) that powers the operating system. Anyhow, I’ve been pleasantly surprised at the level of integration that I was able to achieve with Apple Mail and Gmail [...]