31 March 2006

Hotmail vs Thunderbird: The Quickening

Kids, sit back and let me spin a sad tale of a young, handsome man and his mail client. That incredibly good-looking man is me. That mail client is Mozilla Thunderbird 1.5.

I've been sending a lot of email to a lot of vague acquaintances lately, mostly due to trying to get a lot of people to join us and play some Ultimate. Obviously a lot of people happen to use Hotmail for their email provider. One day, one such acquaintance told me that he hadn't received an expected email from me. Intrigued and suspicious, I decided to do some controlled testing with a dummy hotmail account I've kept around for just such an occasion.

It turns out that when I send email from Mozilla Thunderbird to a Hotmail address in which my address is not already in their contact list or in the "Safe List", it simply gets dropped by Hotmail. It doesn't even get delivered to that user's Junk Folder or anything for them to deal with. No notification is sent to me that the mail wasn't delivered, since my mail was accepted by my ISP's SMTP server just fine, which then sends it to Hotmail. And the recipient obviously has know way of knowing that an email wasn't delivered, except for my personally notifying them through non-email channels.

What was even more curious though, is that when I sent mail from ssmtp or msmtp (via mutt) on my linux box at home, also going through my ISP's SMTP server, it was received.

Well it turns out that both Thunderbird and Hotmail are at fault (in my opinion) on this one. Thunderbird sends an IP address in the SMTP HELO command. This is basically asking to get flagged as spam by any decent spam catcher, like SpamAssassin. I feel comfortable saying that this is why Hotmail drops the mail. Thunderbird doesn't even try to get the client machine's hostname or FQDN. On linux, ssmtp would send the internal (non-resolvable) FQDN of that machine, and msmtp would just literally send "localhost". Both were acceptable to Hotmail. Outlook and Outlook Express send just the hostname (not FQDN). Thunderbird always sends the IP address, no matter what.

Well fine, but then why does Hotmail feel the need to simply drop the message. As I said earlier, if my email address is in the Hotmail user's Safe List or contact list, it will be received. But if the user never sees my message in the first place, they would never know to add me to their list. Plus if I'm trying to email a bunch of kids (e.g. replying-all to a multi-recipient list), I shouldn't be expected to first contact them another way and ask them to add me as a contact so Hotmail won't drop the message. It's just asinine. What is the point of the Junk folder, I ask? Rest assured that I've gone over my own Hotmail Junk Filter settings and made them as forgiving as possible. The mail doesn't get delivered. PERIOD.

Furthermore, as you'd read in the Thunderbird bug, the SMTP RFC does indeed say to send the IP address if the FQDN is not obtainable. Even though Thunderbird really isn't making an effort to obtain the FQDN, simply blackholing the email message is going to result in a lot of data loss.

What's more, it isn't like Thunderbird is sending the "bad" HELO to Hotmail. It is sending it to SBC's (my ISP) SMTP server, which requires authentication, and is accepted there just fine. Here is a header snippet to show you what I mean:
Received: from smtp104.sbc.mail.re2.yahoo.com ([]) by bay0-mc5-f16.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 30 Mar 2006 10:58:00 -0800
Received: (qmail 93766 invoked from network); 30 Mar 2006 18:57:59 -0000
Received: from unknown (HELO ? (foobar@ameritech.net@ with plain) by smtp104.sbc.mail.re2.yahoo.com with SMTP; 30 Mar 2006 18:57:59 -0000
User-Agent: Thunderbird 1.5 (Windows/20051201)
I've changed my internal and external IPs and ameritech.net login to protect the innocent. As you can see, the "bad" HELO is buried down in the message. Hotmail doesn't seem to trust SBC's authenticated SMTP server enough to just accept it.

And so, if you have Hotmail, there is a chance someone out there is trying to email you and you simply aren't seeing it. I'd suggest emailing bill@microsoft.com and let him know. Just don't do it from Thunderbird.

20 March 2006

GeekList: Games Steven Wright would like

GeekList: Games Steven Wright would like: "All those who believe in psychokinesis, raise my hand!"

Patriot Act: The Home Version

Patriot Act: The Home Version: "The board game that brings the thrill of trampling the Constitution right into your home... newly updated for 2006 to include NSA wiretaps and renewal of provisions!"

08 March 2006

jabook 1.0.0 beta1 released

Finally found the time to tag and release jabook 1.0.0beta1 for our hard-core fans. Anyone looking for a fairly decent LDAP addressbook front-end, feel free to give it a try.


07 March 2006

I'm back

The drama continues. First I wanted to host my own blog, then I said "screw it" and hosted it here. Then I wanted to host my own blog again. Then I got bombarded with comment spam every minute of the day and just got sick of worrying about maintenance.

So I'm back. I'll be moving over my old posts from http://seiler.us/blog/ when I can. The fact that I even mention that makes me feel pretentious in that I think there is someone out there who actually may read my blog and then care whether or not I'll be migrating my old posts.

Update: Migration Done.

Don out.