JangoMail In Progress

Five new API methods

noreply@blogger.com (Ajay Goel) — Thu, 25 Feb 2010 20:54:00 +0000

We've added the following five API/web service methods to improve our transactional email capabilities. All were requested by prospects evaluating JangoMail. Do you have a request for an API method that would make your developer's life easier? Let us know by filling out our Support form.

Click each method name to use its corresponding test form.

GetTransactionalGroupID
Gets a transactional group id. Returns a string.

DeleteTransactionalGroup
Deletes a transactional group from account. Returns a string.

Reports_GetBouncesByTransactionalGroup_DataSet
Retrieves list of bounced addresses for a particular transactional group. Includes SMTP Diagnostic Code and Definitive columns. Returns a .NET DataSet.

Reports_GetBouncesByTransactionalGroup_String
Retrieves list of bounced addresses for a particular transactional group. Includes SMTP Diagnostic Code and Definitive columns. Returns a string.

Reports_GetBouncesByTransactionalGroup_XML
Retrieves list of bounced addresses for a particular transactional group. Includes SMTP Diagnostic Code and Definitive columns. Returns an XML document.

New Report: Complaint Rate Dashboard

noreply@blogger.com (Ajay Goel) — Wed, 24 Feb 2010 11:09:00 +0000

JangoMail and JangoSMTP users have always been able to see the complaint rate for an individual broadcast campaign or for an individual transactional email group. However, it was difficult to see complaint rate history over time, and in aggregate across different types of email campaigns (transactional, autoresponders, recurring campaigns, single-shot broadcast campaigns).

With our new Complaint Rate Dashboard, available only in our new no-frames interface, you can see your overall complaint rate based on all emails sent from your account in aggregate.

To access the Complaint Rate Dashboard, make sure your account is using our new no-frames interface, then click Reports, then choose "Complaint Rate History" from the Account dropdown menu:

Here's an example of the Complaint Rate Dashboard from our customer, Popcorn Palace:

The report shows the complaint rate for the last 7 days and the last 30 days, and it also allows you to see how many total emails you've sent per day during this period.

Additionally, you can see what ISP has resulted in the most complaints during a given period.

Time-based complaint rates are more difficult to calculate than the complaint rate on say, an individual broadcast email campaign. There are two ways to calculate time-based complaint rates:

1. Bad way: Number of emails sent within a time period and number of complaints received within that same time period. This method is flawed, since the complaint on an email sent within time period A may not be received until after time period A. So a simple division of the complaints received within a particular 7 day period divided by the number of emails sent within that 7 day period would result in an erroneous complaint rate.

2. Good way: Instead, we tie each complaint back to the original email that resulted in it. The complaint rates shown in this new report are calculated by analyzing how many emails were sent in a given time period, and then analyzing how many complaints were received for only the emails sent within that given time period.

Your Company Logo in your JangoMail Surveys

noreply@blogger.com (Dean Goodman) — Tue, 23 Feb 2010 20:35:00 +0000

JangoMail now has support for adding your company logo to surveys that you create through our website using the Surveys tab. Adding your logo to your surveys is a great idea, not only because it makes it clear who is conducting the survey, but it also builds on your branding by putting your image in front of your survey participants once more.

Adding your logo is accomplished in three simple steps:

While logged in, click on the Company Logo selection on the Settings tab
Input the URL of your company logo from your website (or wherever your logo image is hosted)
Click the Update Company Logo button

After your company logo setting has been updated, that logo will automatically appear at the top of any survey pages you create in the Surveys section of JangoMail.

Below is a screenshot with a sample survey put together to demonstrate the new feature.

V-Day Email Tips to Enflame Customer Passions

noreply@blogger.com (Melissa Sherman) — Wed, 10 Feb 2010 16:19:00 +0000

Nothing inflames the passions like a well-written love letter. That's why for this Valentine's Day I thought I'd offer some tips on how you can take the same basic tenets of love letter writing to create email campaigns that will really get your customer and prospect hearts beating!

1. The sheer appearance of a love letter says so much about the sender and the level of his or her affection. As an email marketer, you need to choose layouts, colors, fonts, and graphics that not only accurately reflect your brand but also draw readers in and show you truly care about how they perceive you.

2. All Romeos and Juliets know how to create the right ambiance. Be sure to consider not only the mood you’re hoping your email will leave readers in, but also the mood they may already be in. Most recipients will be busy, so be considerate of their time and say what needs to be said in order for them to warm up to you.

3. Ever get a love letter and then gush about it with friends? Your friends are probably left wishing their special someones could melt their hearts like that! Know what your competitors are telling customers, and then beat their sweet nothings with sweet somethings that make customers feel special. Segment your outreach and create copy that focuses on your reader's specific needs. Coupons, giveaways, newsletters and funny videos all show you’re interested in a giving, long-term relationship while contributing to the viral effect.

4. "Let us go then you and I…" Just like the Love Song of J. Alfred Prufrock, your email must contain a clear call to action. But don’t just ask would-be customers to click on a link or buy your product. Show them how by doing so you can change their lives in deep and meaningful ways.

5. No matter how eloquently written, sending unsolicited love letters can get you into trouble. As always, be sure your recipients have opted in to receiving your affections!

JangoMail is also showing some love this Valentine's Day by offering free trials of its powerful email marketing platform and waiving its start-up fee through the end of February. JangoMail offers a variety of affordable pricing options, so anyone can send emails in a highly reliable and personalized way. Visit http://www.jangomail.com/ for more.

Two New Reports in our New User Interface

noreply@blogger.com (Ajay Goel) — Tue, 09 Feb 2010 06:19:00 +0000

Those users that have switched to our new no-frames interface now have a new report to enjoy -- two new detailed Sending Progress reports. The first shows how many emails are still pending, and which JangoMail sending server they have been assigned to, and the second shows how many emails have been sent per hour since the email campaign was launched.

To access these two new reports, you must be using the new interface. If you are on the old interface, click "Switch to No Frames" in the top blue area.

Go to Reports click the (sending) link next to an in-progress email campaign:

After clicking (sending), a popup will show you how many emails have been sent, how many are remaining, and to which sending server they've been assigned:

Clicking on [Time] will reveal how many emails have been sent per hour since the email campaign was launched.

Two bug fixes in SMTP relay service

noreply@blogger.com (Ajay Goel) — Sat, 06 Feb 2010 23:50:00 +0000

We've just deployed two fixes to the JangoSMTP transactional email service:

1. Previously, lines in an email message that begin with a single period would result in two periods when delivered to the recipient. This was a bug related to how SMTP treats the escaping of periods. This is now fixed. To read more about issues involving SMTP and periods, see this article: http://db.ilug-bom.org.in/lug-authors/philip/docs/mail-stuff/smtp-intro.html

2. Previously, if an email message was relayed to relay.jangosmtp.net and included both the Disposition-Notification-To and the Return-Receipt-To headers, then JangoSMTP would throw an error on the email message:

System.ArgumentException: An item with the same key has already been added.

This is now fixed, and email messages containing both headers will be delivered properly.

JangoMail Has Upgraded Its HTML Editor

noreply@blogger.com (Dean Goodman) — Mon, 18 Jan 2010 22:57:00 +0000

It has been a while since the last upgrade but JangoMail has just installed the latest version of the Ephox EditLive! HTML editor. What does this mean for you? TONS of new features and bugfixes, including:

Support for setting the language direction in both the applet and in inline editing sections
Improved dictionaries for spell checking
Additional support for international, non-English locales
Fixes for several table rendering problems

See here for a list of all bugfixes and enhancements.

Transactional emails now support List-Unsubscribe header option

noreply@blogger.com (Ajay Goel) — Sat, 02 Jan 2010 04:22:00 +0000

Back in July, we announced support for the List-Unsubscribe header in broadcast email campaigns sent via JangoMail.

Starting today, the List-Unsubscribe header will also be applied to transactional email messages. Transactional emails are email messages sent via any of the following methods:

The SMTP relay service
Any of the following API methods: SendTransactionalEmail, SendTransactionalEmailFromTemplate, SendTransactionalEmailRaw

See the original July blog post for information on how this header affects your email messages. It is our recommendation, that all clients leave this header turned on. If you wish to turn it off, you may do so under Settings --> Unsubscribe Options.

The Rendering/Inbox Preview tool is back in action

noreply@blogger.com (Ajay Goel) — Fri, 18 Dec 2009 01:04:00 +0000

Our rendering/inbox preview tool is back in action, after a hiatus for the last month where the only client that rendered properly was GMail.

Now, four of the clients will consistently display (Gmail, Outlook, Thunderbird, and Windows Mail), and we're working on bringing AOL, Hotmail, and Yahoo! Mail back to life.

The reason this tool is so difficult to maintain and service is because a) We coded this feature from scratch, and Inbox Rendering is an extremely difficult process from a software development perspective, and b) Webmail clients are constantly changing, and when a change is made to its login process or its interface, we have to re-code our app to render that client all over again.

SMTP service now supports large attachments

noreply@blogger.com (Ajay Goel) — Fri, 11 Dec 2009 08:08:00 +0000

The JangoMail SMTP relay service now supports sending large file attachments in email. Before, there was a limit of approximately 2 MB in total file attachments. Now, there is NO theoretical limit, so the SMTP service can handle file attachments as large as 10 or 20 MB. Of course, email may not be the best medium to transmit such large files, but it is possible with our system.

JangoSMTP: The Stand-Alone SMTP Service with open tracking, click tracking, DKIM signing, and more.

JangoMail: The SMTP Service combined with email broadcast service.

Review of Megabus WiFi Internet access

noreply@blogger.com (Ajay Goel) — Sat, 05 Dec 2009 09:24:00 +0000

Running a fast-paced high tech company like JangoMail, I need to be as productive as possible. I am usually either in Chicago (where I live) or in Dayton, Ohio (where I grew up, and where JangoMail is headquartered). Normally I drive between the two cities, because non-stop flights between Chicago and Dayton are upwards of $1,000. Recently, I was told of a bus service called Megabus, which operates a bus route between Cincinnati and Chicago with WiFi Internet access, so tonight, I gave it a shot. With the WiFi, I figured taking the bus I figured would allow me to work for six hours rater than drive for six hours.

What I wanted to know

Prior to buying my ticket, I wanted to know the following about the Megabus WiFi access:

1. How fast was it? Would it be like dialup, or a T1?

2. Were there any connectivity restrictions? Could I remote into a JangoMail server and deploy a new feature? Could I use GoToMyPC to access my main workstation in Chicago?

3. Were there power outlets available? My laptop battery only lasts four hours, and my trip was six.

Googling got me nowhere, as nobody has documented in detail experiences with Megabus's WiFi, so hence the purpose of this article.

To all tech executives, sysadmins, and those that like to make their web app better while travelling between cities, this article's for you.

Network Settings

The above screenshot shows my network settings after connecting to the wireless network with SSID MEGABUS - 64184. Note that I manually put in the DNS server of 8.8.8.8 in an attempt to get around the network's web site blocking, however my attempt was futile.

The Speed

The speed various greatly depending on whether the bus is moving or stationary. While stationary, I got reasonable high-speed access. I ran this speed test which shows that I was getting near T1 speeds on the download.

Above speed was while bus was stationary at Indianapolis stop.

Above speed test was while bus was in route, half-way between Indianapolis and Chicago.

The speed between Cincinnati and Indianapolis was the slowest of all, but unfortunately I did not capture a speed test screenshot during that portion of the trip.

Port Blocking

All connectivity, except to ports 80 and 443, and DNS lookups, are blocked:

I could not ping any of our servers.
I could not FTP to our server.
I could not connect on port 25 to any mail servers.
I could not Remote Desktop into any servers.
I could not use GoToMyPC to connect to my workstation.

Web Site Blocking

Additionally, the WiFi service uses OpenDNS to block many web sites. While I was able to read and respond to email via GMail, and browse Facebook, the following sites were blocked:

MySpace
Google Docs (at docs.google.com), which was odd, because most other Google services were available
Nerve

Because it was an OpenDNS based block, I suspected that switching to Google's new public DNS server (8.8.8.8) might be a workaround to the blocking. I manually set my Windows TCP/IP settings to use 8.8.8.8 as its DNS server rather than the one dynamically assigned by DHCP, but to no avail. The sites were still blocked, redirecting me to an OpenDNS message.

The WiFi service is provided by a company called Saucon, and it is free.

Power Outlets on Megabus

There were no power outlets on my bus from Cincinnati to Chicago, but I've read (insert link) that some buses do have outlets.

Resources

For the official Megabus Saucon WiFi Terms of Use, that you must agree to right when you connect, is here: http://www.saucontds.com/us/index.html

Wikipedia entry on Megabus - http://en.wikipedia.org/wiki/Megabus

Megabus Home page - http://www.megabus.com, which oddly enough when I just visited, gave me:

Shocking that the site goes offline EVERY DAY for 3 hours. In our world of real-time, mission-critical everything, I'm surprised that they would sacrifice 12.5% of the time they could be selling tickets. JangoMail, which I'd imagine is a far more complex app than Megabus's web site, has maintenance running on the back-end almost constantly, and we rarely go offline.

Test Out Our New Survey Tool

noreply@blogger.com (Ajay Goel) — Tue, 24 Nov 2009 22:05:00 +0000

We just launched the first ever Silverlight-based Survey Design Tool. You can now design and publish email survey invites through the JangoMail interface and view your survey results for each individual participant.

Set Up Instructions
1. The Survey feature is part of our new no-frames interface. Because both are in beta, you must set your system to use the no-frames interface. Click on Switch to No-Frames in the upper right-hand corner.

2. Click on the new Surveys tab. If you don't have Microsoft's Silverlight Platform installed, you will be prompted to install it at this time. Survey recipients see the survey on a regular webpage and will not need to download Silverlight.

3. Click the Create a New Survey button to get started. Then choose the Blank survey option.

4. To create a new question, click in the box under Create a new item.

Edit the question and answer points by deleting the text that is currently there and typing in your own.

To use a pre-defined set of answers, click the down arrow next to your answer set.

To add space for short text answers and comment sections, click on the down arrow in the box under Create a new item. Then click on the type of answer you would like.

5. When you are done creating your survey, click the Save button.

6. To make the survey available for people to respond, you must publish it. Click on the Publish tab and then click Publish Survey. Here you will get a link that you can send to people to fill out your survey.

7. Set your survey to end by navigating to the Close tab and entering in when the survey should close. You can choose to end it immediately, on a certain date, or after a given number of responses.

Send the Survey

1. To send out your survey, go to the Send Email tab and begin creating an email as you normally would.

2. On the Send Email page, you will see a new link for adding a Survey Link. Click on that link and find the link for the Survey that you would like to add.

3. Copy and paste the link into your email.

4. Send your email as you normally would and wait for results to come in.

View Results

View Recipient-Level Results by clicking on My Surveys in the Surveys section. Click on the Results button next to the survey that you would like to view results for.

We will soon add results in our Reporting and Analytics section as well.

Test our new no-frames interface

noreply@blogger.com (Ajay Goel) — Mon, 23 Nov 2009 08:12:00 +0000

Our new no-frames user interface is ready to be tested. To switch to the no-frames interface, login to your account and click the "Switch to No-Frames" link. You can always switch back to the old frames interface if you want to later.

Below is the first screen of the new interface:

Along with the absence of frames, there are three key navigational differences:

The "Account Info" section is no longer a main tab. It is a link in the top area.
The "Help" section is no longer a main tab. It is a link in the top area.
There is a new tab called "Surveys". You can now design and publish and email survey invites to your email lists. The Surveys feature is also in beta. The survey builder runs on Microsoft's Silverlight platform, so if you don't have Silverlight installed, the first time you click on the "Surveys" tab you'll be asked to install Silverlight.

The new no-frames interface will now be the basis for a whole host of user interface improvements, including improved text styling, contextual help, more videos, easier navigation, and clearer organization. Stay tuned for these improvements and future blog posts on how to use Surveys.

Error handling for the SMTP Relay Service

noreply@blogger.com (Ajay Goel) — Mon, 23 Nov 2009 05:33:00 +0000

Overview:

We've launched a new error handling system for the SMTP relay system. There are a number of reasons why a message sent through the SMTP relay may result in an error. For example:

1. If the recipient address is on your account's bounce list, then a "bounce" error will be thrown.
2. If the recipient address is on your account's unsubscribe list, then an "unsubscribe" error will be thrown.
3. If your account is over its sending limits, a "Over Account Limit" error will be thrown.
4. If a badly formed email message is passed through the relay, a "parsing" error will be thrown.

These errors are only detected after a message is received by the SMTP relay, so the error will NOT appear during the SMTP conversation with relay.jangosmtp.net.

When an error is detected, the error notification can be emailed to the Account Manager and/or the From Address on the email that resulted in the error. Additionally, the error is available for viewing under Reporting in the Transactional Emails section.

To set your error handling preferences, go to Settings --> SMTP Relay and set the two relevant checkboxes appropriately.

Recommendations:

1. If you know that you regularly attempt to relay emails to addresses on your Unsubscribe and Bounce lists, you may wish to turn off the email notifications for errors entirely, as this could result in a large volume of emails to you.

2. To decide whether or not the error should also be sent to the From Address, consider what type of transactional emails you're sending through the SMTP relay. If the From Address is an address at your organization, then it's probably okay for the From Address to also receive errors. But, if you're sending social networking "You have a new message from John" type emails, where the From Address is that of one of your users, you probably don't want your users to receive error notifications from JangoMail. In this case, uncheck the box to send errors to the From Address.

Seven Steps to Optimize Email Deliverability Using JangoMail

noreply@blogger.com (Ajay Goel) — Thu, 19 Nov 2009 23:40:00 +0000

Overview:

Email deliverability is a key concern for most email marketers, and at JangoMail, we think we do a pretty good job of ensuring the highest possible inbox placement of our clients' emails. JangoMail is a highly customizable platform, from sending speeds, to email headers, to SMTP protocol level customizations. Taking advantage of certain settings and features can help ensure high email deliverability.

Some of these steps involve separating yourself from other JangoMail clients. It's not that JangoMail accepts spammers, but JangoMail does accept different levels of opt-in email marketing. For example, some customers use single opt-in, while other customers use confirmed opt-in. Generally, customers using confirmed opt-in experience higher deliverability. Therefore, different JangoMail clients have different reputations with ISPs, so if two clients are both using a username@jangomail.com From Address for example, they both can impact the reputation of the domain jangomail.com.

The Seven Steps:

Following the below steps will solve most email delivery issues to major consumer ISPs, like Yahoo, Hotmail, GMail, and AOL.

Use a branded From Address rather than a username@jangomail.com From Address. All JangoMail accounts come with the option to use username@jangomail.com as the From Address on your email campaigns. This is provided as a convenience to our clients.

a. In order to separate your reputation from other clients using jangomail.com as their From Address, use your actual corporate email address, like john.smith@company.com.

b. An even better and safer option is to use a branded sub-domain, like john.smith@newsletters.company.com. That way newsletters.company.com can have its own SenderID and DK/DKIM records. Additionally, there is an SMTP level benefit to setting up a branded sub-domain -- it allows the SMTP MAIL FROM command during the SMTP transaction between our sender and the recipient server to show your sub-domain rather than jangomail.com. For instructions see the PDF Setting up a Branded Sub-Domain.

Who should do this? The technical person who manages your domain's DNS settings.
Use a branded tracking domain, that is based on your organization's domain name. The tracking domain is the domain referenced in the open-tracking mechanism, the click-tracking re-directs, the unsubscribe link, the forward-to-friend link, the view-as-a-web-page link, and other links that offer tracking in your email messages. By default, every new JangoMail account is assigned a system tracking domain that is shared amongst multiple clients, like x.jango5.com for example. By setting up your own, you can isolate yourself from the activities of our other clients. Set up your own by going to Settings --> Tracking Domain. If your domain is mycompany.com, then setting up x.mycompany.com makes for the perfect tracking domain.

Who should do this? The technical person who manages your domain's DNS settings.
Setup DomainKeys/DKIM for the domain use in your From Address. Both Yahoo! Mail and GMail currently look for DKIM signatures in the headers of email messages. The presence of a DKIM signature fosters a sense of trust that the email was sent by who was purported to send the email. To setup DomainKeys and DKIM in JangoMail, see our PDF entitled Setting up DomainKeys/DKIM with JangoMail.

Who should do this? The technical person who manages your domain's DNS settings.
Setup SenderID on your domain, and then set your campaigns to use your own From Address as the SMTP-level MAIL-FROM address. Instructions can be found in the PDF How to Publish SPF Records in JangoMail.

Who should do this? The technical person who manages your domain's DNS settings.
If you're sending HTML email, make sure you include a corresponding plain text message. Spam filters tend to frown upon emails that have a heavy ratio of HTML to plain text content. This is as easy as setting the Plain Text Message to "auto-generate". See screenshot below. This will cause a plain text message to be generated (based on your HTML message) at the time of email sending.

Who should do this? The person using JangoMail.
Enroll in the Sender Score Certified program. In order to be accepted, you must have a complaint rate of 0 to 0.1%. Once enrolled, your emails are sent from IP addresses that are certified. Hotmail virtually guarantees inbox placement for emails sent from certified IPs, and Yahoo prioritizes emails sent from certified IPs.

Who should do this? The person using JangoMail.
If using click-tracking, anchor text should be phrases, not URLs. Some spam filters look closely at how you link to determine whether the link is legitimate or fraudulent. They do this to prevent phishing scams--a type of scam where an email pretends to be a request from a legitimate company in order to get the login credentials of that company's user. For more information on phishing, see the Wikipedia article on phishing. The best way to explain good links versus bad links is with an example:

Good Link: <a href="http://www.browniekitchen.com/">Visit our web site.</a>

Bad Link: <a href="http://www.browniekitchen.com/">http://www.browniekitchen.com</a>

When these URLs are click-tracked in JangoMail, the final email message looks like:

Good Click-Tracked Link: <a href="http://x.jango5.com/y.z?l=http://www.browniekitchen.com">Visit our web site.</a>

Bad Click-Tracked Link: <a href="http://x.jango5.com/y.z?l=http://www.browniekitchen.com">http://www.browniekitchen.com</a>

What makes the bad link bad is that the anchor text is a URL, and that domain in that URL does not match the domain in the link destination. Phishing filters look for this domain mismatch. In the good example, however, the anchor text is not a URL to begin with, so the phishing filter will accept it as legitimate.

Who should do this? The person designing your email campaigns.

Further Reading:

For more information on general JangoMail deliverability practices, see our Deliverability page on our web site

There are basic steps that any email marketer can take with the content of their emailings to ensure high deliverability. The content-related issues are covered in the PDF document Optimizing Email Deliverability.

How we scaled up the JangoSMTP service to accommodate the ING New York City Marathon

noreply@blogger.com (Ajay Goel) — Fri, 06 Nov 2009 09:29:00 +0000

Overview of the New York City Marathon and its Email Alerts

It was a sunny mid-September day when my head of sales informed me that the systems integrator for the 2009 New York City Marathon was looking at our SMTP service for the delivery of alert emails on runners during the marathon. Given that our SMTP relay service was relatively new at the time, we saw this as an opportunity to demonstrate the powerful tracking features and performance capabilities of our service.

Having tried other email vendors and their own internal systems in the past, the marathon was looking to avoid past issues, such as large delays in the delivery of the email alerts and email blocking issues with consumer ISPs like GMail and Hotmail.

The marathon had about 42,000 total runners. Friends and family members of each runner could "subscribe" to a runner, such that whenever the runner reached a checkpoint during the marathon, an email alert would be sent to the subscribers of that runner. This would be done electronically via a Chronotrack D-Tag. For more information on the electronic tracking of runners, see http://www.nycmarathon.org/race_scoring.htm.

My team was told to expect anywhere from 400,000 to 2,000,000 email alerts to be sent out during the race. That represents an average of 10 to 50 email alerts per runner, depending on how many individuals subscribe to a particular runner.

The Speed Problem

The JangoSMTP service has been a single-node service since its launch earlier in 2009. It was serviced by one single fault-tolerant, RAID-based, multiple-CPU, high memory Windows server located at relay.jangosmtp.net. The SMTP service works by receiving email at relay.jangosmtp.net, and then passing emails along to one of JangoMail's 40 outbound SMTP senders for delivery to the final recipient. The relay.jangosmtp.net server could receive an unlimited amount of emails and saturate our upstream Internet provider's bandwidth, but it could only process, add tracking, and transmit to the outbound SMTP senders over the Internet at a rate of 250 emails/minute. I therefore calculated:

250 emails/minute x 60 = 15,000 emails/hour

Based on past marathon results, I estimated that the fastest runners would complete the race in about 2 hours, and the slowest runners would complete the race in about 6 hours. However, the marathon would be initiated in 3 waves of 14,000 runners each, distributed over an hour the morning of the marathon. So 6 hours of running time, plus an added hour for the start of the last wave, meant 7 hours of sending email.

7 hours x 15,000 emails/hour = 105,000 emails over 7 hours

Uh oh. We weren't nearly fast enough. Even at the minimum expected volume, we weren't fast enough by a factor of 4. And at the maximum expected volume, we only had 5% of the needed capacity.

The two bottlenecks were 1) the processing of an email message, meaning the dis-assembly and re-assembly to determine what user it belonged to and add tracking mechanisms, and 2) transmitting the email to a SMTP sender. Point #2 warrants more explanation. While there is no bottleneck for relay.jangosmtp.net to receive emails from the outside world, there can be a bottleneck for relay.jangosmtp.net to transmit emails, since relay.jangosmtp.net must transmit the message to separate SMTP sender located on separate networks in separate data centers. This transmission happens over the Internet, and based on where the SMTP sender is located and the routing to it, speed can fluctuate.

Time to Scale Up

Load Balancing

We needed to scale up, and scale up fast. The initial plan was to order four more servers, and have them each serve as an additional SMTP receiver and processor for relay.jangosmtp.net. The JangoMail architecture does not employ the use of appliance-based load balancers, so I decided to handle the load balancing via our Domain Name System (DNS).

We created multiple DNS "A" records for relay.jangosmtp.net, each with a Time to Live (TTL) of 60 seconds. Five "A" records were created in total, each with a different IP address. The first of the five was the original IP for relay.jangosmtp.net, and the other four were for the four additional servers we commissioned.

By keeping the TTL at a short 60 seconds, I could make certain that each of the five servers would receive an equal load of email every minute. And since not all DNS servers on the net respect TTLs and sometimes do their own caching, I confirmed with our tech contact at the marathon that their systems would NOT cache the IPs for relay.jangosmtp.net beyond the designated 60 second TTLs.

SQL Query Caching

Every email that arrives at relay.jangosmtp.net is disassembled, tracked, re-assembled, and then passed to an outbound SMTP server for actual sending. In order to determine what email belongs to what user, and what tracking options each user has selected, the originating IP address of each email message is looked up against an IP Address table, and then once the user is determined from the IP address, the UserAccounts table in the core database is queried to determine what tracking/DomainKeys options the user has selected. These two queries combined took anywhere from 0.05 seconds to 0.2 seconds, depending on the load on the database at the time.

We shaved this time down to 0.001 seconds by caching the results of these queries and refreshing every five minutes.

Multi-threading

Our custom SMTP architecture is a multi-threaded model, allowing for the simultaneous processing and delivery of emails across user accounts. We had been informed ahead of time that the marathon would trigger email alerts from two originating IP addresses. We therefore configured two dedicated processing threads on each of the 8 servers. This gave us 16 total processing threads, and also isolated the processing/delivery of the marathon's emails from our other clients' emails during the race.

Distributed Transactional Senders

The SMTP service is part of JangoMail's transactional email platform, which also includes the SendTransactionalEmail API method. All transactional emails are sent through the email sender that is assigned for a particular user. For fault-tolerance purposes, every user has a list of transactional email senders assigned to it, such that if the first email sender is unavailable or offline, the email is passed to the second, and to the third, and so on, until the email is successfully transmitted. This approach was great for fault-tolerance and redundancy, but not for scaleability. If the first server in the list was online and available, then it would receive all the transactional emails for that account.

I therefore decided to add an internally controlled user-level setting option to randomize the list of senders. Now, if a user's list of transactional senders included:

Sender1, Sender2, Sender3, Sender4

Now the relay.jangosmtp.net servers would farm out the emails for delivery to any sender in the user's list at random, ensuring that as many transactional senders as were assigned would receive an equal load of emails to deliver, rather than having the first available sender do all the delivery. This also aided in resolving the bandwidth bottleneck with the SMTP senders mentioned earlier.

Still not fast enough - need 3 more servers

Given our SQL query optimizations, multi-threading, additional servers, the timing now looked like:

400 emails/minute/server x 5 servers = 2,000 emails/minute x 60 minutes = 120,000 emails/hour.

120,000 emails/hour x 7 hours = 840,000 emails over 7 hours

However, if the load was over 1 million emails, there would still be a delay. I decided to annex 3 additional servers that are already a part of the JangoMail network but weren't active on port 25, and turn them into 3 additional SMTP receivers. There were now 8 total DNS "A" records for relay.jangosmtp.net.

With just 48 hours before the marathon, we were informed that the expected outbound volume, based on the number of subscribers so far, would be about 750,000 emails.

And just in case all else failed...

While we've always believed in the performance and reliability of our code and architecture, we decided that for a project of this caliber, a backup plan was necessary in case our custom SMTP architecture was unable to perform. The JangoSMTP custom architecture is what allows emails that pass through the SMTP relay to be open and click-tracked, and stored in a database, so that SMTP logs can be viewed and reports can be generated based on open and click timing, domains, and geo-tracking reports based on IP addresses. However, the primary issue for the marathon, was to ensure the emails were delivered, and delivered on time. Therefore, a backup plan was put into place that could guarantee the emails would be delivered, even if we had to eliminate the tracking and logging based on our own custom architecture. If we found that JangoSMTP could not handle the load, we would replace the instances of the JangoSMTP receiver with Microsoft's built-in SMTP service (part of Internet Information Services), such that emails would be received and delivered to the final recipient, without any processing needed in between.

In order to isolate the marathon from our other clients in case this backup plan had to be put in place, we setup the domain marathon.jangosmtp.net, which mimiced the 8 A records for relay.jangosmtp.net, and we asked the system integrators for the marathon to connect to marathon.jangosmtp.net instead of relay.jangosmtp.net. This would allow us to re-direct just their email on the day of the race if needed.

The Day of the Race - November 1, 2009

I awoke at 7:30 AM EST on that Sunday, after having been out the night prior for Halloween. The emails would begin trickling in at 8:30 AM EST for runner check-ins, and the first wave of the race was set to begin at 9:40 AM EST.

I was watching the race live on TV, while at the same time monitoring the traffic flows across our 8 instances of relay.jangosmtp.net. Emails were being received, processed, and sent quickly, and there was no backlog...until about 12:25 PM EST.

All three waves had been released, and runners from all three waves were triggering a massive volume of email alerts. From 12:25 PM EST to about 12:35 PM EST, there was a 4-5 minute delay with final delivery. Thankfully, the backlog period only lasted 10 minutes. It makes sense that approximately 3 hours after the release of the first wave, that the highest volume of email was passing through, since the greatest number of runners would still be running around this time.

Additionally, at about 1:00 PM EST, we discovered that comcast.net and att.net/bellsouth.net domains were blocking one of the IPs from which marathon email was sending. While we did have a mechanism by which domain-specific routes could be used, such that we could enable all comcast.net/att.net/bellsouth.net email to go through one specific non-blocked IP address, the complexity of the randomization system we had added to accomodate the bandwidth bottleneck rendered this mechanism non-functional. I called our lead developer, who was on call, asked him to make a change to the sender-determination algorithm, and re-deployed our code across all 8 SMTP server instances. We were now able to route all comcast.net/att.net/bellsouth.net email through a separate non-blocked SMTP sender. In the end, the marathon alert emails had less than a 0.3% blocking rate.

After 1:00 PM EST, no further email deliverability or backlog issues ensued.

Conclusion

We were thrilled that we were able to pull this off for the 2009 ING New York City Marathon. Even post-marathon, we continue to make performance and feature enhancements to our transactional email platform. If you have an important project for which sending email is critical, please get in touch with us, and we'll work as hard for you as we did for the marathon.

New API Method for Template-Driven Transactional Emails

noreply@blogger.com (Ajay Goel) — Fri, 06 Nov 2009 07:55:00 +0000

Overview

Marketers can now create and edit Transactional Email Templates on the Send Email Page and without the help of their IT team.

To enable this, we launched a new API method: SendTransactionalEmailFromTemplate. Instead of passing the full details of the email message using the regular SendTransactionalEmail method, you can now simply pass the email Campaign ID and let your marketers use the JangoMail interface to dictate the email's creative content.

SendTransactionalEmailFromTemplate works like the SendTransactionalEmail method, but does not require you to enter FromEmail, FromName, Subject, MessagePlain, and MessageHTML. Instead you must enter CampaignID, PersonalizationValues, PersonalizationColDelimiter and PersonalizationFields.

Input Parameters

Username
Your JangoMail account username

Password

Your JangoMail account password

CampaignID
Marketers should create an Email Campaign as they normally would, starting on the Send Email Page (or just use the SendMassEmail API method). Once they create a campaign and send a test, you can find the Campaign ID in the Reporting and Analytics section of the JangoMail interface. Look in the ID column to find the Campaign ID.

ToEmailAddress
The single email address to whom you want to send this email message.

PersonalizationFields
Example:
FirstName,LastName,Company,FromAddress

JangoMail uses the following syntax for personalization and Marketers should continue to use this when creating transactional email templates. Mail-merge personalization tags can be used in the Subject, HTML Message, Plain Text Message, From Display Name, and From Email Address:

%%FieldName%%

%%FieldName**Default Value%

The second version of the personalization syntax can be used in cases where the value of FieldName is blank, such that a default value is substituted instead of a blank.

PersonalizationValues
Example:
Ajay,Goel,Silicomm,ag@silicomm.com

PersonalizationColDelimiter
Specify a column delimiter as follows:

c for a comma
s for a space
n for a newline, or a hard break
t for a tab

Options
A comma separated list of name/value pairs. The names of the various options and their associated values are explained below.

You may specify:
TransactionalGroupID (a numeric ID of a previously created Transactional Group)
SkipUnsubCheck ("True" or "False")

You can also specify the following, but if not specified, values will pull from your original email campaign. User-specified values here will override values in your email campaign:

ReplyTo (any valid email address)
CC (any valid email address)
BCC (any valid email address)
CharacterSet (a character set like "ISO-8859-1" or "US-ASCII")
Encoding (either "7 bit" or "Quoted Printable" or "Base 64")
Priority (either "Low", "Medium", or "High")
UseSystemMAILFROM ("True" or "False")
Receipt ("True" or "False")
Wrapping (a numeric value from 0 to 100, with 0 meaning no wrapping takes place)
ClickTrack ("True" or "False")
OpenTrack ("True" or "False")
NoClickTrackText ("True" or "False")
Attachment1 (the name of a previously FTPd file, like "Invoice.pdf")
Attachment2 (the name of a previously FTPd file, like "Invoice.pdf")
Attachment3 (the name of a previously FTPd file, like "Invoice.pdf")
Attachment4 (the name of a previously FTPd file, like "Invoice.pdf")
Attachment5 (the name of a previously FTPd file, like "Invoice.pdf")

Example

In the below example, the input parameters are:

Username:jangomail

Password:******

CampaignID:265987456

ToEmailAddress:jangomail@gmail.com

PersonalizationFields:FirstName,OrderNumber

PersonalizationValues:Bob,123456

PersonalizationColDelimiter:c

Options:ReplyTo=jm@gmail.com,OpenTrack=True

This example assumes that the Subject and/or Body contains personalization placeholders for FirstName and OrderNumber. In fact, for Campaign ID 265987456, the actual message is:

------BEGIN MESSAGE-------
Dear %%FirstName**Valued Customer%%,

Your order has shipped. Your order number is %%OrderNumber%%.
------END MESSAGE-------

This transactional email will be open tracked. To ensure that it will also be click tracked, we would add "ClickTrack=True" to the Options parameter. If we wanted it to be a high priority message instead of the default medium priority, we could add "Priority=High", such that the full Options string would look like:

Options:ReplyTo=jm@gmail.com,OpenTrack=True,ClickTrack=True,Priority=High

To test the operation using the HTTP POST protocol, visit the JangoMail API page on this method and click the Invoke button.

Frequently Asked Questions

Q. What's the difference between SendTransactionalEmailFromTemplate and SendTransactionalEmail?
A. When calling SendTransactionalEmail, you are passing in the full details of that email message, including the Subject, the HTML Message, the Plain Text Message, tracking options and much more. However, if you've already created a template message in the Send Email section of the JangoMail web interface, you can call SendTransactionalEmailFromTemplate to send transactional emails and avoid having to pass in all the message details on every method call. You simply reference the Campaign ID of the email message that you created in the Send Email section.

Q. I see there's a PersonalizationColDelimiter input paramter. Where is the PersonalizationRowDelimiter input parameter?
A. Since this is a method to send a single transactional email, only one set of values can be provided for the single recipient of the email message. There is no need for a row delimiter since the data should only be for one record.

Q. What is the TransactionalGroupID?
A. This is an optional setting that can be specified as part of the Options input parameter. Transactional Groups are a way to categorize different types of transactional emails, like Order Confirmations, Shipping Notifications, and Thank You messages. You can create different Transactional Groups using the web interface or by calling AddTransactionalGroup. Every Group has a numeric ID assigned to it, and so to assign a particular transactional email to a Group, you can set the TransactionalGroupID.

Q. How do I send attachments with my transactional email?
A. You can specify the names of attachments in the Options input parameter. However, the file names you specify must already be present in your account. You should FTP your files to client.jangomail.com/Attachments prior to calling this method with attachments.

Official Launch of JangoSMTP.com, Our Stand-Alone SMTP Relay Service

noreply@blogger.com (Ajay Goel) — Tue, 27 Oct 2009 13:35:00 +0000

We have just launched JangoSMTP.com, a new stand-alone SMTP relay service and the first designed for email marketers.

Why use it?
JangoSMTP is the first SMTP relay service to offer open and click tracking. It also comes with a variety of features that contribute to its extreme deliverability:

DomainKeys/DKIM Signing
SPF/SenderID Authentication
Sender Score Certification
Feedback Loops with ISPs

Visit our site for JangoSMTP's Full Features List.

Who is it for?
JangoSMTP can be utilized by users of desktop email clients (Outlook, Thunderbird, etc.), Gmail users, and web programmers who send transactional emails through their own corporate servers.

Where can I learn more?
Go to http://www.jangosmtp.com/ for more information or Contact Us directly.

New API Method: AddTransactionalGroup

noreply@blogger.com (Ajay Goel) — Mon, 26 Oct 2009 17:55:00 +0000

We just released a new API method for the JangoMail API which allows you to add a Transactional Group to your account:

*AddTransactionalGroup
Adds a new Transactional Group to your account. Returns a string.

We also recently released 3 other transactional email API methods. These methods allow you to retrieve opens, clicks, unsubscribes, bounces, and complaint statistics for a single transactional email in your account.

Visit api.jangomail.com for a full list of our API methods.

New Design: Features Page

noreply@blogger.com (Ajay Goel) — Thu, 22 Oct 2009 17:54:00 +0000

JangoMail has a very comprehensive feature list and we've been working on a way to organize it so that it's easy to navigate. We want to make sure that our current and prospective customers are able to identify all of our capabilities and can easily find all of the information and tutorials we have on each feature.

We are excited to announce that we just relaunched our Email Features page, which now has an improved navigation system. You can expand each feature set and feature for more information.

*Use the Table of Contents to find the feature category you are interested in.

*To navigate to a certain feature's description, video, tutorial or blog entry, simply click the arrow next to that feature.

Then click on the link for the information you would like to see.

*To see a visual description of a certain feature, click on the name of that feature.

A window will pop up with a visual description of the feature.

Browse through our new page yourself at: http://www.jangomail.com/features_overview.asp. Let us know what you think!

Two SMTP Service Bug Fixes

noreply@blogger.com (Ajay Goel) — Thu, 22 Oct 2009 02:10:00 +0000

Tonight we've deployed two bug fixes to the SMTP relay service:

Previously, emails sent with very long Subject lines, such that the Subject line folded onto the next line, were improperly processed, and this could result in headers being visible in the email body. This is now fixed.
Previously, emails sent with very long From lines, such that the From line folded onto the next line, were improperly processed, and this could result in the email not being transmitted to the final recipient at all. This is now fixed.
Previously, encoded subject lines, such as:

=?utf-8?Q?You've_received_=E2=82=A8229.64_($2.00_USD)_in_your_Surveyhead_?= =?utf-8?Q?account?=

would not have their encoding preserved. This is also now fixed.

Technical Resources:

Those wishing to read about the allowance of folded lines as stated in the SMTP specification can do so here: http://www.faqs.org/rfcs/rfc822.html. Specifically, read section:

3.1.1. LONG HEADER FIELDS

New Transactional Email API Methods

noreply@blogger.com (Ajay Goel) — Mon, 19 Oct 2009 19:00:00 +0000

We have just released 3 new Transactional Email API Methods for the JangoMail Email Marketing API.

These new methods allow you to retrieve opens, clicks, unsubscribes, bounces, and complaint statistics for a single transactional email in your account:

*Reports_Transactional_GetSingleEmailStats_Dataset
Retrieves statistics for a transactional email. Returns a .NET DataSet.

*Reports_Transactional_GetSingleEmailStats_String
Retrieves statistics for a transactional email. Returns a string.

*Reports_Transactional_GetSingleEmailStats_XML
Retrieves statistics for a transactional email. Returns an XML document.

Technical Discussion of How Social Networking Sites Send Transactional Email

noreply@blogger.com (Ajay Goel) — Sun, 20 Sep 2009 03:58:00 +0000

Shortly after JangoMail launched its own transactional email platform, we took on a flood of new clients with a need to send "social networking" type transactional emails, including "You have a new message" notifications and "You have a new friend request" notifications. Then we launched our SMTP relay for email marketers, and the floodgates ripped off.

We're often asked what the best practices are from a technical standpoint, with respect to issues like Content, From Addressing, DomainKeys and DKIM, and SPF/SenderID. Should the content of a message sent from one member to another be included in the email notification, or should the recipient have to log in to the site to read the message? Should a "friend request" email come "from" the friend or "from" the social networking site? Should the email contain Plain Text, or HTML, or both? To best answer these questions, I've analyzed transactional email from the most notable social networking sites -- Facebook, LinkedIn, Friendster, and MySpace. Note that this is an article about the technical aspects of sending transactional email. If you're looking for an article on transactional email strategy (what to send, how often, to whom...), then you won't find that here, but you can in many other places.

The following are the properties of each social networking site's email that I will analyze:

Presence of a DomainKeys header.
Presence of a DKIM header.
Sender Policy Framework (SPF) compliant email.
The From Address
The From Display Name
The SMTP MAIL-FROM Address used in the SMTP transaction
The Sender header, if present
The Reply-To header, if present
Use of a unique identifier in the SMTP MAIL-FROM address
The content, particularly whether a friend's message is present in the Email Body, or whether the user must log in to read the message.
The use of HTML versus Plain Text MIME parts.

Let's get started with the biggest social network of all, Facebook:

Facebook:

When somebody sends me a message on Facebook, I get an email notification with the Subject line:

"Mary Parker sent you a message on Facebook..."

The Body of the email contains the actual message, which is convenient, because it prevents me from having to log in to Facebook to read the message.

From Address: notification+mummiv~d@facebookmail.com
Reply-To: noreply@facebookmail.com
From Name: Facebook
DKIM-Signature: v=1; a=rsa-sha1; d=facebookmail.com; s=q1-2009b; c=relaxed/relaxed;q=dns/txt; i=@facebookmail.com; t=1253415012;h=From:Subject:Date:To:MIME-Version:Content-Type;bh=fLwGefDfPrLKUK2vwWzaC5k5iNo=;b=DfJugajI0Fsc5FIppBQSfSAAiV+WuK1ugzc00USpIkJku3sUgcupO+TPua0Tcxw7qJDUE0yRohPMWz3jPwPfRA==;

A DKIM-Signature header for facebookmail.com is present, but it is noteworthy that a DomainKey-Signature header is not present.

The SMTP MAIL-FROM address is: notification+mummiv~d@facebookmail.com

Facebook Transactional Email SMTP Log:

69.63.178.170 [03A4] 22:50:13 Connected
69.63.178.170 [03A4] 22:50:13 >>> 220 mail.silicomm.com ESMTP IceWarp 9.1.0; Sat, 19 Sep 2009 22:50:13 -0400
69.63.178.170 [03A4] 22:50:13 <<< EHLO mx-out.facebook.com
69.63.178.170 [03A4] 22:50:13 >>> 250-mail.silicomm.com Hello mx-out.facebook.com [69.63.178.170], pleased to meet you.
69.63.178.170 [03A4] 22:50:13 <<< MAIL FROM:<notification+mummiv~d@facebookmail.com>
69.63.178.170 [03A4] 22:50:13 >>> 250 2.1.0 <notification+mummiv~d@facebookmail.com>... Sender ok
69.63.178.170 [03A4] 22:50:13 <<< RCPT TO:<AjayGoel@silicomm.com>
69.63.178.170 [03A4] 22:50:13 >>> 250 2.1.5 <AjayGoel@silicomm.com>... Recipient ok
69.63.178.170 [03A4] 22:50:13 <<< DATA
69.63.178.170 [03A4] 22:50:13 >>> 354 Enter mail, end with "." on a line by itself
69.63.178.170 [03A4] 22:50:13 *** <notification+mummiv~d@facebookmail.com> <ajay@us.jangomail.com> 1 2021 00:00:00 OK BLS56313
69.63.178.170 [03A4] 22:50:13 >>> 250 2.6.0 2021 bytes received in 00:00:00; Message id BLS56313 accepted for delivery

The SPF record for facebookmail.com is:

"v=spf1 mx ip4:204.15.20.0/22 ip4:69.63.176.0/20 -all"

The connecting IP is 69.63.178.170, which is a part of the "ip4:69.63.176.0/20" directive and therefore this IP is SPF-valid for sending email "FROM" facebookmail.com.

Noteworthy Points:

It's excellent that Facebook is employing both SPF and DKIM.
The email lacks a DomainKey-Signature header for backwards compatibility for receiving systems that might not be DKIM-compliant yet.
The email is from "Facebook" and not my friend "Mary Parker". This is likely to prevent people from attempting to reply to the email as a way of writing back to "Mary Parker". The only way to reply to the message is to login to Facebook and use their internal messaging system.
Facebook's transactional emails come from the domain facebookmail.com rather than the more recognizable facebook.com. This is perhaps to separate email reputation from their actual high-value business domain facebook.com. That way, should their emails be treated as spam by a network, the domain name facebookmail.com will be punished, not facebook.com.
The X-Mailer header of the email is:

X-Mailer: ZuckMail [version 1.00]

Presumably, ZuckMail is an email system written by, or an homage to, Facebook's founder Mark Zuckerberg.

LinkedIn:

LinkedIn sends transactional email differently than Facebook. While Facebook "friend request" email comes from "Facebook" and a no-reply facebookmail.com email address, LinkedIn's "Join my network" emails usually come "From" the person trying to add you.

The relevant headers from a LinkedIn transactional email are below:

DomainKey-Signature: s=prod; d=linkedin.com; c=nofws; q=dns; h=Sender:Date:From:To:Message-ID:Subject:MIME-Version: Content-Type:X-LinkedIn-fbl;b=KQVq9LfSNIZ+6tOuz7xUaUW1wx86f62EndMqyY+bbkmI8jHQDNe8HdtW ZfJsxH50d8nvAnQHryCdQKGfBEY0u2t5sXkrRmf7yCAhhg4Q9aqBo20KZ LSUvTK726opFnO0;
Sender: messages-noreply@bounce.linkedin.com
From: "Mary Parker" <maryparker@hotmail.com>
To: Ajay Goel <AjayGoel@silicomm.com>

Note that the the From Name is my friend "Mary Parker" and the From Address is Mary's email address, maryparker@hotmail.com. Because no Reply-To header is present, clicking "Reply" on my email client will auto-address the message to the From Address, maryparker@hotmail.com.

From an authentication perspective, it's interesting that the LinkedIn transactional email contains the DomainKey-Signature header, but not the more modern DKIM-Signature header. JangoMail transactional emails contain both the DomainKey-Signature and the DKIM-Signature headers, for the highest level of compatibility with all receiving email systems.

LinkedIn Transactional Email SMTP Log:

64.74.98.137 [057C] 12:16:54 Connected
64.74.98.137 [057C] 12:16:54 >>> 220 mail.silicomm.com ESMTP IceWarp 9.1.0; Thu, 03 Sep 2009 12:16:54 -0400
64.74.98.137 [057C] 12:16:54 <<< EHLO mail15-a-aa.linkedin.com
64.74.98.137 [057C] 12:16:54 >>> 250-mail.silicomm.com Hello mail15-a-aa.linkedin.com [64.74.98.137], pleased to meet you.
64.74.98.137 [057C] 12:16:54 <<< MAIL FROM:<s-vEJ2DMWLs51nh5KhMMSL430pk-MMh3YJi-KV_N5M-Qy28fihJyfjtc@bounce.linkedin.com> SIZE=3706
64.74.98.137 [057C] 12:16:54 >>> 250 2.1.0 <s-vEJ2DMWLs51nh5KhMMSL430pk-MMh3YJi-KV_N5M-Qy28fihJyfjtc@bounce.linkedin.com>... Sender ok
64.74.98.137 [057C] 12:16:54 <<< RCPT TO:<AjayGoel@silicomm.com>
64.74.98.137 [057C] 12:16:54 >>> 250 2.1.5 <AjayGoel@silicomm.com>... Recipient ok
64.74.98.137 [057C] 12:16:54 <<< DATA
64.74.98.137 [057C] 12:16:54 >>> 354 Enter mail, end with "." on a line by itself
64.74.98.137 [057C] 12:16:54 *** <s-vEJ2DMWLs51nh5KhMMSL430pk-MMh3YJi-KV_N5M-Qy28fihJyfjtc@bounce.linkedin.com> <AjayGoel@silicomm.com> 1 4048 00:00:00 OK KZT46754
64.74.98.137 [057C] 12:16:54 >>> 250 2.6.0 4048 bytes received in 00:00:00; Message id KZT46754 accepted for delivery

Noteworthy Points:

1. The SMTP MAIL-FROM address is a string of characters followed by @bounce.linkedin.com. LinkedIn is using a sub-domain of its high-value business domain, linkedin.com. JangoMail also offers the option to setup a sub-domain based off your corporate domain name. The string of randomly generated characters is presumably to track any bounce that occurs as a result of this particular transactional email. If the silicomm.com MTA sends a notification back to s-vEJ2DMWLs51nh5KhMMSL430pk-MMh3YJi-KV_N5M-Qy28fihJyfjtc@bounce.linkedin.com, the LinkedIn system will be able to immediately tie the identifier present in that email address and use that information to immediately notify Mary Parker that the email to me bounced. JangoMail also handles bounces to transactional emails, but with a unique identifier in the X-VConfig header of the message as opposed to within the MAIL-FROM address itself. Both options are legitimate for tracking bounces.
The SPF record for bounce.linkedin.com is:

"v=spf1 redirect=linkedin.com"

The redirect directive then tells us to look up the SPF record for linkedin.com, which is:

"v=spf1 ip4:70.42.142.0/24 ip4:208.111.172.0/24 ip4:64.74.220.0/24 ip4:64.74.221.0/26 ip4:64.71.153.211 ip4:64.74.221.30 ip4:69.28.149.0/24 ip4:208.111.169.128/26 ip4:64.74.98.128/26 ip4:64.74.98.16/29 mx ~all"

The sending IP 64.74.98.137 is covered by the "ip4:64.74.98.128/26" designation, therefore this email passes the SPF test.
Emails with subject line "Invitation to connect on LinkedIn" always has the user's information as the From Name / From Address.
Emails with subject line "Join my network on LinkedIn" sometimes have the From Name / From Address of the user, but other times From Name is "Mary Parker (LinkedIn Invitations)" and From Address is invitations@linkedin.com.

I have been unable to determine the reason for the ambiguity with the "Join my network on LinkedIn" emails.

Friendster:

The following are the relevant headers from a Friendster "new message" notification:

From: Friendster <message-2219486617@mail.friendster.com>
Subject: New Friendster Message from Vikki - 09/07/09 10:17 PM
To: AjayGoel@silicomm.com

Friendster Transactional Email SMTP Log:

209.11.169.65 [03A4] 01:17:58 Connected
209.11.169.65 [03A4] 01:17:58 >>> 220 mail.silicomm.com ESMTP IceWarp 9.1.0; Tue, 08 Sep 2009 01:17:58 -0400
209.11.169.65 [03A4] 01:17:58 <<< EHLO c350a-5.friendster.com
209.11.169.65 [03A4] 01:17:58 >>> 250-mail.silicomm.com Hello c350a-5.friendster.com [209.11.169.65], pleased to meet you.
209.11.169.65 [03A4] 01:17:58 <<< MAIL FROM:<message-2219486617@mail.friendster.com> SIZE=9837
209.11.169.65 [03A4] 01:17:58 >>> 250 2.1.0 <message-2219486617@mail.friendster.com>... Sender ok
209.11.169.65 [03A4] 01:17:58 <<< RCPT TO:<AjayGoel@silicomm.com>
209.11.169.65 [03A4] 01:17:58 >>> 250 2.1.5 <AjayGoel@silicomm.com>... Recipient ok
209.11.169.65 [03A4] 01:17:58 <<< DATA
209.11.169.65 [03A4] 01:17:58 >>> 354 Enter mail, end with "." on a line by itself
209.11.169.65 [03A4] 01:17:58 *** <message-2219486617@mail.friendster.com> <AjayGoel@silicomm.com> 1 10017 00:00:00 OK POZ63258
209.11.169.65 [03A4] 01:17:58 >>> 250 2.6.0 10017 bytes received in 00:00:00; Message id POZ63258 accepted for delivery

Noteworthy Points:

The complete lack of both DomainKeys and DKIM signatures.
The email is SPF compliant. The SPF record for mail.friendster.com is "v=spf1 mx ip4:209.11.168.0/23 mx:c300a-1.gbxsc.friendster.com -all" and the sending IP is 209.11.169.65, which is covered by the "ip4:209.11.168.0/23" directive.
The email is not "from" my friend but instead is from "Friendster". The From Email is also not of my friend, but a Friendster unique email address, in this case message-2219486617@mail.friendster.com. The From Address is the same as the SMTP MAIL-FROM address shown in the SMTP transaction. Therefore bounces at the MTA level will be sent to message-2219486617@mail.friendster.com, and the unique number that's a part of that From Address presumably allows the Friendster mail system to know that it was this particular transactional email that bounced.
There is no Reply-To address, so if the user does hit Reply in his email client, the reply will go to the From Address, message-2219486617@mail.friendster.com, and sending to this will presumably cause the email to be lost in the ether.

MySpace:

The following are the relevant headers from a MySpace "new message" notification:

DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=pmta; d=message.myspace.com;h=MIME-Version:From:To:Reply-To:Date:Subject:Content-Type:Content-Transfer-Encoding; i=noreply@message.myspace.com;bh=T25HyvwBLohBcz5zhA0xB0tJqdc=;b=SeqKqNrcg4jDIyy0lmlo2BJje2frO0hgd16E9M/Y7U42LNLPh+cLlr7q5muDN7ASuPzwaNIvjj9VKq9VzNSvnxNukRsF2ICj+jT7GFRHdO39feZXnyJgcGbu0eHGLF7v83PL8CApEF80w/f3HuZ8xZgzYQcagUvNeD2jUmTqj/4=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=pmta; d=message.myspace.com;b=SlJszI5cM6w78pK5nSJLzVTrrFw5YgU2vX8GraxP8yPjhbb6/hZ+oMjr7HUF21r8wyK774IEA/o/u3Y6z+kcpRHTf7D3TWD94TCZW+mQDGxl3t+FSdqLLt4ISVUvGu+LTAIMfpSOzVKNnz3C/4QHpAg1c2a2jRu1IFmrPbqOaK0=;
From: MySpace <noreply@message.myspace.com>
To: AjayGoel@silicomm.com
Reply-To: MySpace <noreply@message.myspace.com>
Subject: MaryParker has sent you a new message!

MySpace Transactional Email SMTP Log:

204.16.33.83 [0664] 18:04:01 Connected
204.16.33.83 [0664] 18:04:01 >>> 220 mx.silicomm.com ESMTP IceWarp 9.1.0; Fri, 10 Jul 2009 18:04:01 -0400
204.16.33.83 [0664] 18:04:01 <<< EHLO vmta20.myspace.com
204.16.33.83 [0664] 18:04:01 >>> 250-mx.silicomm.com Hello vmta20.myspace.com [204.16.33.83], pleased to meet you.
204.16.33.83 [0664] 18:04:01 <<< MAIL FROM:<noreply@message.myspace.com> BODY=8BITMIME
204.16.33.83 [0664] 18:04:01 >>> 250 2.1.0 <noreply@message.myspace.com>... Sender ok
204.16.33.83 [0664] 18:04:01 <<< RCPT TO:<AjayGoel@silicomm.com>
204.16.33.83 [0664] 18:04:01 >>> 250 2.1.5 <AjayGoel@silicomm.com>... Recipient ok
204.16.33.83 [0664] 18:04:02 <<< DATA
204.16.33.83 [0664] 18:04:02 >>> 354 Enter mail, end with "." on a line by itself
204.16.33.83 [0664] 18:04:02 *** <noreply@message.myspace.com> <AjayGoel@silicomm.com> 1 5669 00:00:00 OK RDO05602
204.16.33.83 [0664] 18:04:02 >>> 250 2.6.0 5669 bytes received in 00:00:00; Message id RDO05602 accepted for delivery

MySpace is the only one of the four social networking sites to use both the DomainKey-Signature and the DKIM-Signature headers, so kudos to MySpace for considering the entire email ecosystem.

Noteworthy Points:

Both DKIM and DomainKeys headers present.
SPF passes. The SPF record for message.myspace.com is "v=spf1 ip4:63.208.226.0/24 ip4:204.16.32.0/22 ip4:216.178.32.0/20 ip4:216.178.38.0/20 a ~all" and the sending IP is 204.16.33.83, which is included in the "ip4:204.16.32.0/22" directive.
The email is From "MySpace" rather than "Mary Parker", and the From Address is noreply@message.myspace.com instead of maryparker@hotmail.com. The From Address clearly implies that you should not reply to it, unlike Friendster's address. Clearly MySpace wants you to log in to the site to reply rather than by email.
The SMTP MAIL FROM is noreply@message.myspace.com, and does not contain a unique identifier.
The Reply-To header is present, but its value is the same as the From Address. So regardless, if you hit Reply on your email client, the email is going to noreply@message.myspace.com.

Conclusion:

If I had to rank the social networking sites in order of best to worse in sending authenticated, compliant, and useful transactional email, this would be the ranking:

1. Facebook (Best)

Positive: Emails are DKIM and SPF compliant. Email notifications of "You have a new message" include the message in the body of the email.
Negative: No DomainKeys signature, and the Reply-To address is one that cannot receive replies.

2. MySpace

Positive: Emails are SPF, DKIM, and DomainKeys compliant - the only social network to use all three.
Negative: Notifications of "You have a new message" force you to login to read your message. The Reply-To address is one that cannot receive replies.

3. LinkedIn

Positive: Emails are SPF and DomainKeys compliant. Sometimes emails are "From" the actual friend rather than LinkedIn. Proper use of the Sender header.
Negative: No DKIM signature, which means LinkedIn is complying with the older DomainKeys standard but not the newer DKIM standard. Notifications of "You have a new message" force you to login to read your message.

4. Friendster (Worst)

Positive: Emails are SPF compliant.
Negative: No DomainKeys signature. No DKIM signature. Notifications of "You have a new message" force you to login to read your message. The Reply-To address is one that cannot receive replies.

JangoMail Best Practice Recommendations:

When using JangoMail to send transactional emails for a social network, you can optimize the deliverability and usability of your emails by following these guidelines:

1. Ensure your emails are SPF compliant. In most cases, your emails will already be SPF compliant. Whether you're using your-username@jangomail.com as your From Address or noreply@MySocialNetwork.com as your From Address, SPF validates the SMTP MAIL-FROM Address, not the Display From Address that the recipient sees. The SMTP MAIL-FROM Address, meaning the From Address that shows in the SMTP logs, will always be your-username@jangomail.com, unless one of the following is true:

a. You've setup a custom sub-domain, in which case your sub-domain will be reflected in the SMTP MAIL-FROM.
b. You've explicitly set the SMTP MAIL-FROM by unchecking the "Use JangoMail MAIL-FROM" box on the Send Email page.
c. You've explicitly set the UseSystemMailFrom=False option when calling the SendTransactionalEmail API method.
d. You've unchecked the Use JangoMail MAIL-FROM box under My Options --> SMTP Relay, and you are using the smtp relay to send your emails.

2. Ensure your emails are signed with DKIM and DomainKeys. Once you've setup your account to use DKIM, your emails will automatically include both a DKIM and a DomainKeys signature. Read this guide to learn how to configure DomainKeys/DKIM for your emails.

3. Determine whether the content of a "You have a new message" email should appear in the Body of the email or whether you want to force the recipient to login to read the new message. I personally favor Facebook's method of including the original message in the email notification but forcing the recipient to log in to reply. After all, free social networks need to generate page-views, and they can't do that if they allow back-and-forth messaging functionality all within email.

4. Determine whether the From Address and/or Reply-To Address should be that of your social network's or that of your user's. We've seen that in the big four, LinkedIn is the only site that shows the user's name and email address in the From Line. This allows the recipient to conveniently reply to the sender of the message via email. The disadvantage to the owner of the social network is that the recipient need not visit the web site to reply to the message.

Therefore, If you wish to optimize the user experience, set the user's name and email address in the From line of the email message. If you employ this strategy, consider these points:

a. Your emails will still be SPF compliant so long as the SMTP MAIL-FROM contains a single consistent domain and SPF has been setup properly for that single domain.
b. Return Path, a large email deliverability company, recommends that if you send emails in this manner, be sure to include the "Sender" header as well. In JangoMail, you need only contact Support to have the Sender header turned on for your account.
c. There is a small chance that your emails will not be SenderID compliant, since SenderID may validate the Display From Address in addition to the SMTP MAIL-FROM address. However most SPF records for major domains are just that - SPF records and not SenderID records. In fact, all four major social networking sites use ONLY SPF, and not SenderID, as shown by the v=spf1 designation in all four records.

The best option may be to set the From Address to the social network site's address (noreply@MySocialNetwork.com), and set a Reply-To Address equal to the user's email address. The advantage of this setup is that you need not worry about any SPF/SenderID issues, and if the recipient hits Reply, the email will be directed to the user, not the social network web site. It is still important to have the Sender header inserted, however.

Data Center Move Part 3: What went right and what went wrong

noreply@blogger.com (Ajay Goel) — Wed, 02 Sep 2009 07:20:00 +0000

This is the third and final installment of my account of our data center move. This is also my favorite part to write, because I get to recount everything that went right and wrong during after the move. And I love learning from my mistakes.

I had six total people, including myself, coordinating this effort - five at the data centers, and one remote man who could test connectivity from the outside and perform tasks related to the move that didn't require being on-site. Our downtime was scheduled from 12:00 AM to 4:00 AM EST on a Saturday night / Sunday morning.

11:00 PM

The five met at the New Data Center at 11:00 PM. I wanted to make sure the New Data Center had the WAN cable drop ready, that our access card worked, that the combinations to the cabinet doors worked, and plan out our access into and out of the building and determine where to park our cars while unloading equipment. The New Data Center came equipped with remote-access Power Distribution Units (PDUs) in both cabinets, one cabinet with 30 Amps and one with 20 Amps.

11:30 PM

Our inspection of the New Data Center was complete, and we drove the eight blocks to the Old Data Center to prepare for the take-down of the core servers: the web server, the database server, and the API server.

11:40 PM

We arrived at the Old Data Center. The primary check we needed to do prior to shutting down the servers ensuring that the standby database server had up-to-date copies of files from the primary database server. The primary log-ships to the standby server, so we needed to ensure the last of the log files had been restored to the standby server.

12:00 AM

The equipment was transported in three waves:

1. At 12:00 AM, I took the firewall and the main switch to the New Data Center. I wanted to get the firewall appliance connected to the WAN connection from the New Data Center, plug my laptop in to verify that connectivity to the Internet was working. The team transporting the three core servers would wait for my call from the New Data Center signaling that connectivity was working.

2. After my call to verify connectivity, a team of two (the primary two) would begin dismantling the three core servers (web server, database server, API server), and transport them to the New Data Center.

3. Then the remaining two (the back two) would begin dismantling and transporting the last set of servers, including corporate mail, incoming mail, standby DB server, corporate web server, the rendering servers, and a few other standby servers and appliances.

12:30 AM

Testing connectivity was a critical first step, because the type of connectivity was different from that of the Old Data Center. The New Data Center required a layer 3 routing device, while the Old Data Center did not. The firewall had to be reconfigured with a static IP, a range of DMZ IPs, and a gateway. All DMZ devices would then use the firewall as the gateway. At the Old Data Center, all devices, including the firewall, were assigned IPs from the same class C, and the gateway IP for all devices, was the same IP, and an IP provided to us by the Old Data Center. While the new connectivity setup seemed complicated at first, it worked just as expected.

After the firewall and switch were plugged in and connected, I had the sixth man who was remote, connect to the firewall via web browser, and begin changing the Network Address Objects' IP addresses. Simultaneously, he logged into our third party DNS system and begin making DNS changes for the main domain names, like jangomail.com and www.jangomail.com and mail.jangomail.com and relay.jangosmtp.net.

12:45 AM

After having verified connectivity, I called the core two, and had them initiate take-down and transport of the three core servers.

1:15 AM

Within thirty minutes, the team arrived, and we began rack-mounting the three core servers. We plugged them in, connected them to the switch, but they did not come back online.

This is where we ran into problem #1: While the IPs for the New Data Center had been assigned to the second NIC on each of the core 3 servers, the second NIC on each of the servers was disabled. Each needed to be re-enabled for connectivity to the servers to be established. I needed to login to each server, and activate the NIC, but I forgot the Keyboard-Video-Mouse (KVM) unit at the Old Data Center, and had no way of configuring anything on these three servers.

1:45 AM

I took a vehicle back to the Old Data Center, retrieved the KVM switch, checked in on the back two that were moving the ancillary servers, and drove back to the New Data Center.

2:15 AM

I mounted the KVM switch, and one by one, plugged it into each of the 3 core now-mounted servers, and activated the NIC with the new IP. One problem - I could only activate the NIC on 2 of the 3 servers. On the third, the API server, the keyboard/mouse ports were USB, and our KVM unit doesn't have USB connectors for keyboard/mouse. I had no way of actually configuring the API server. The New Data Center had mentioned the availability of a KVM crash cart on the floor, so I searched for it.Ten minutes later, I found it, locked behind another customer's cage.

2:45 AM

The web server and database server were operational, 75 minutes prior to the end of our downtime window. However, the API is mission critical to 30% of our customers, so I had to find a way to get it up and running.

Unable to fabricate an immediate solution to this problem, I drove back to the Old Data Center to check on the two men dismantling the ancillary servers.

3:00 AM

I pulled up, and they were just loading up the car. All equipment had now been removed from Old Data Center's racks. I was going to lead the car back to the New Data Center.The car wouldn't start. The car's battery had died while the flashers were on in, parked in front of the building.I pulled up alongside the car, we moved the equipment from his vehicle to mine, had a quick chat with the building doorman about not towing the now defunct car, piled back into my vehicle, and the three of us drove to the New Data Center.

3:20 AM

We loaded up the ancillary equipment on carts, and rolled the carts into the New Data Center. Once the equipment was in the data center, I had my core two men rack-mount the equipment. I let the back two use my laptop, still connected directly to the firewall, to Google for a store that was still open at the hour that would carry jumper cables. They didn't find one, but left anyway in my vehicle in search of jumper cables.

The three of us continued to rack-mount the remaining ancillary equipment. One by one, we turned each device on, and re-configured the NIC for its new IP address

After all equipment was powered on and connected, I still needed to get the API up. I decided to use the standby database server as my new primary API server, until such time as I could configure the API server with a USB keyboard/mouse.

Post Move - 5:30 AM

The three of us drove back home to Dayton, Ohio.

Post Move - 6:30 AM

I arrived at my home in Dayton and fired up my laptop to do some final testing from the outside. I noticed a few problems:

My BlackBerry wasn't receiving any email.

My BlackBerry receives all my work email by having our corporate mail server, mail.silicomm.com, forward my email to my TMobile BlackBerry email account. The email wasn't being forwarded.

I first remoted into mail.silicomm.com, did an MX lookup on tmo.blackberry.net, which is the T-Mobile BlackBerry domain, and attempted to connect to port 25 on that IP. The connection was made and immediately dropped. I assumed it was due to the new IP for mail.silicomm.com never having sent email before. As a temporary work-around, I had mail.silicomm.com forward my email to my GMail account, and then I had my GMail account forward email to my TMobile BlackBerry account.

Email I was sending wasn't being received.

I sent a test email from mail.silicomm.com to my GMail account, and it wasn't received.

Our corporate email system, IceWarp, has its own DNS settings, separate from the DNS settings in Windows TCP/IP. I hadn't changed the DNS servers of IceWarp to the New Data Cetner's DNS servers, and therefore, outbound email wasn't being transmitted since the MX lookups couldn't be performed.

JangoMail notification emails weren't being received.

All JangoMail notifications, such as "Sending Complete" and "Import Complete" notifications are sent via the JangoMail SMTP relay service. You can authenticate into the SMTP relay either by IP address or by From Address. For our own system notifications, we authenticate by IP address. We had forgotten to change the authenticated IP address in our own JangoMail account - the one that handles all these customer notifications. Once that was changed, customers received the email notifications.

One-Three Days After Data Center Move

Clients were reporting that the API was still inaccessible.

The API was inaccessible to at least three clients, because they were connecting to the API by IP address, rather than the domain name api.jangomail.com. This was discovered in the 48 hours following the move.

Clients were reporting that the JangoMail web-database connectivity feature wasn't working.

Clients were unable to use the web-database connectivity feature of JangoMail because they had restricted access to their web servers/database servers by IP address, and they had our Old Data Center IP range in their firewalls. They did not have the New Data Center IPs in their firewalls.

Five Days After Data Center Move

While examining the headers of a forwarded email in my GMail account, I noticed that an email forwarded by mail.silicomm.com to my GMail account had the originating IP listed as
209.173.128.118, which is not the IP for mail.silicomm.com. The IP for mail.silicomm.com is 209.173.141.195, and the reverse lookup for this IP is appropriately, mail.silicomm.com.

Upon further examination, I discovered that all servers in the DMZ were connecting to the outside as 209.173.128.118, which is the IP of the firewall. If I pointed a web browser on any device to www.whatismyip.com, the result was the same - 209.173.128.118. I was baffled, since all devices had been assigned public, routable IP addresses. I called SonicWall support, but I was declined support because I didn't have a support contract, and they refused to do a per-incident support ticket. I posted on the SonicWall forums, and was told I had to put in a Network Address Translation (NAT) setting in order for the firewall to prevent the firewall IP from routing the outbound connections. Once, I did that, the original BlackBerry forwarding issue was resolved. The BlackBerry email server was dropping the connection immediately because the source IP of the connection, 209.173.128.118, did not have a reverse DNS entry for it. It was the firewall's IP, so it doesn't need a rDNS entry.

Lessons Learned

There were several areas where planning was weak, including planning what equipment would be moved in what phase, and under assessing the impact of switching to new IP addresses. Given the experience I've outlined, I would have done the following differently:

Activated the NIC with the New Data Center IP while still at the Old Data Center.
Made sure to have a USB keyboard/mouse with me.
A week prior to the move, informed all customers that our IP addresses were changing, including the new IP range.
Informed all API customers that anyone connecting directly to our IP address needed to connect to the domain api.jangomail.com instead.
Been more conscious of the JangoMail accounts that we ourselves use and dependencies on IP addresses.
Been more aware that some applications don't inherit the TCP/IP settings of Windows, and have them set manually.
And lastly, I would have made sure at least one vehicle carried jumper cables.

New Video Tutorial: Configuring Gmail with the JangoMail SMTP Relay

noreply@blogger.com (Ajay Goel) — Fri, 28 Aug 2009 15:36:00 +0000

We have a new Video Tutorial on how to configure Gmail with the JangoMail SMTP Relay Service:

Video: How to Configure Gmail with the JangoMail SMTP Relay

The JangoMail SMTP Relay improves your sending capabilities by providing the following features for your person-to-person emails:

Open Tracking
Click Tracking
DomainKeys/DKIM Signing
Easy web-based access to SMTP Logs
API methods to retrieve Reporting data

For more information, screenshots, and written directions, read our post on configuring your Gmail account with the JangoMail SMTP Relay Service.

If you are not a Gmail user, watch our other Video Tutorials on Configuring the JangoMail SMTP Relay with a Desktop Client or a Web Server.