How to send secure TLS email from your Ubuntu server using Postfix, SPF and DKIM records

Image for post
Image for post

Imagine Google had to use Mailchimp to send email to 2 billion Android users. How much would Mailchimp charge them? Exactly the same amount you are going to pay after setting up secure email running from your own Linux web server.

Every time I’m stuck setting things up for the first time (SSL certificates for example) I later realize that it’s because I simply don’t understand how something works.

After going through the process several times I develop a pattern I can follow next time to quickly get it done. These patterns become subjects for my tutorials.

Image for post
Image for post

This Standard Encryption (TLS) message comes at a cost of understanding how to properly set up Postfix SPF and DKIM records in your web hosting company’s DNS settings as well as directly on your Linux server.

I’ve gone through the process several times and found out what the key issues and common pitfalls are. I wanted to share it with everyone so you can save time setting up your own mail server. It’s a lot simpler if you know exact steps to follow.

Why Do We Need This Tutorial?

For me personally it was because I thought services like Mailchimp charge way too much for a handful of subscribers.

When it comes to email you have to learn from multiple tutorials. I learned Postfix configuration separately from SPF and DKIM. I’ve yet to see a tutorial that explained everything in one article. So I decided to write one for future learners.

Sending email using paid services has its benefits. The most important claim they’ve been making since 1990s is that they protect you from being black listed by ISP. But how do they do it? It almost seems like they’ve struck some sort of a pact with other companies — a magic trick not available to us mere mortals who just want to send email.

How does Amazon do it? Nintendo? Free Code Camp? How does any company do it without using Mailchimp? Every developer should know how to install, configure and send email from the web server they are already paying for.

There is a way to send clean email from your own server

If you don’t send spam, provide an easy way to unsubscribe from your service and create useful content…you should never have any issues with sending quantities of email from your own domain name without having to pay extra.

In this tutorial we will use Postfix to set up encrypted TLS email server running from your own domain’s hostname. This means when your email arrives that annoying red lock icon in recipient’s Gmail will be gone.

Login to your remote server and run postfix.

Image for post
Image for post

Whoops. We can’t. It’s not installed.

How To Install Postfix On Your System

We need to install postfix on our Linux server:

apt install postfix --yes

yes helps us automatically answer “Yes” to all installation questions.

Once installed we’re greeted by Postfix Configuration screen:

Image for post
Image for post

Hit Enter.

Image for post
Image for post

Press Enter again to choose Internet Site (default option.)

Image for post
Image for post

Replace localhost with yourdomain.com (skip www part) and press Enter.

At this point installation process will continue on the screen. Once it’s done main.cf and master.cf will be created. main.cf file contains default Postfix configuration and master.cf contains additional advanced settings.

They were both installed in: /etc/postfix/ directory.

In this tutorial we will edit both files to setup our mail server. But we won’t need to modify all settings. Just the important ones.

Just remember to run service postfix reload command every time you change Postfix configuration settings in those files.

How To Configure Postfix

First we need to edit /etc/postfix/main.cf — the default configuration file:

nano /etc/postfix/main.cf

This will open it in nano editor (or just use your favorite editor.) This is what the default main.cf file looks like (I removed all comments for clarity):

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
append_dot_mydomain = no
readme_directory = no
compatibility_level = 2
# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = localhost.members.linode.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, example.com, localhost, localhost.localdomain, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all

In this file make sure that myhostname = mail.mydomain.com. Again here just replace mydomain.com with your actual domain name. (By default it might point to some default server host name but we don’t need that.)

Note that example.com in mydestination should also be set to your own domain name without the www part (We already did this on Postfix config though.)

Add the following line just above where it says #TLS parameters:

tls_random_source = dev:/dev/urandom

Then also add or make sure following entries exist:

smtp_tls_security_level = may
smtp_tls_note_starttls_offer = yes
smtp_tls_mandatory_protocols=!SSLv2,!SSLv3
smtp_tls_protocols=!SSLv2,!SSLv3
smtp_tls_loglevel = 1
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
# Enable TLS
smtpd_tls_security_level = may
smtpd_tls_ask_ccert = yes
#smtpd_tls_session_cache_timeout = 3600s

Save the main.cf file and exit back to the command line.

Then run following commands:

postfix reload
postfix stop
postfix start

Congratulations. You’ve just earned yourself the following badge.

Image for post
Image for post

Google now considers your emails encrypted with Standard Encryption (TLS)

But you’ve only won the battle and not the war. If you try sending an email from your server now it’s likely the message would still go to Spam folder. There are a few other things we need to do.

Follow instructions below to prevent emails from your server to be treated as spam by Gmail and other inboxes.

Locating Snake Oil Key Pair Under # TLS parameters

Go down the file a bit and locate the following two lines…notice that they are currently setup with default snakeoil SSL certificate key pair:

smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key

What Is Snake Oil?

Snake Oil is a substance with no real medicinal value sold as a remedy for all diseases. Snake Oil Certificates are intentionally fake certificates that exist by default. All this means is they should not be trusted.

They are temporary placeholder certificates that encrypt the data, but they cannot be trusted because they lack root authority. Ideally these certificates need to be reconfigured with a proper signed key by an authentic CA (Certificate Authority)

In my experience commenting snake oil certificates and restarting the server had no impact on actual TLS encryption. But I could be wrong. The emails were still sent. I’m not an expert on Postfix TLS certificates but what I did was generate them with openssl anyway.

Generating Self-Signed Key Pair

For my setup I created a pair of self-signed keys using openssl. And I replaced snake oil certificates with a path to where they are located.

Create new directory in postfix just for the certificates:

mkdir /etc/postfix/ssl

Now navigate to that directory:

cd /etc/postfix/ssl

Run following command to generate the keys:

openssl req -new -nodes -keyout smtp.site.com.key -out smtp.site.com.csr

(Replace site.com with yourdomain.com) Answer the questions that appear on the screen. For most of them you can simply hit Enter. Important ones are:

  1. Common Name (e.g. server FQDN): smtp.yourdomain.com
  2. A challenge password[]: (enter something you can remember.)

Enter smtp.yourdomain.com for Common Name. (Obviously, yourdomain.com would be the actual name of your domain, not literally “yourdomain.com”)

Now enter ls command in bash to verify keys were generated. You should have smtp.domain.com.csr and smtp.domain.com.key files there.

The way I understand it is you are supposed to submit this .csr file to SSL certificate providers (Certificate Authority) and they will send you back a zip archive containing a file called caroot.crt which becomes your TLS CA file. You need to now copy it to the SSL directory and also include a path to it in main.cf file. See below.

Edit your /etc/postfix/main.cf file by replacing snake oil certificates with the keys we generated and in addition to that add the link to caroot.crt:

smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_CAfile=/etc/postfix/ssl/caroot.crt

Again…I was able to send email and see proper TLS Encryption verification on Gmail without going through this step. I tried to comment all of this out and then restarting Postfix…with TLS and DKIM still working properly without any of these certificates.

I need to look more into openssl in order to understand how it works and if there is a better way to generate .pem and .key certificates for TLS in Postfix. If you have experience with this contact me and let me know so I can update this part.

Adding Two SPF Records

Now we need to create 2 SPF records. One for your IP address in IPV4 format, and one for IPV6 (the longer one.) You need to log in to your web hosting account.

Go to your web hosting control panel and create a new TXT record. A TXT record will have hostname, value and TTL properties. For IPV4 fill them out as follows:

  1. Hostname: yourdomain.com
  2. Value: v=spf1 ip4:xx.xx.xx.xx mx a ~all
  3. TTL: 5 minutes

Just replace xx.xx.xx.xx with your remote IP address for your server.

Now create a 2ND TXT record for your IP address in IPV6 format and enter:

  1. Hostname: yourdomain.com
  2. Value: v=spf1 ip6:xxxx:xxxx::xxxx:xxxx:xxxx:xxxx mx a ~all
  3. TTL: 5 minutes

Note that some hosts will automatically remove yourdomain.com and replace that with empty space. It is assumed that a blank hostname refers to your domain name (without the www part.) So if you see it disappear from your DNS settings it means it’s properly setup. This is to be expected especially on Linode.

Replace xxxx:xxxx::xxxx:xxxx:xxxx:xxxx with your server’s IPV6 address. Your web hosting company should be able to tell you what it is on the dashboard. If not, contact customer support or create a help ticket.

Save the settings and they should kick in in a bit. Might not work immediately so give it anywhere between 5 to 30 minutes.

Testing if SPF records were properly installed

Now when you send an email from your server to let’s say a GMail address (and if you analyze header of the message) you want the recipient mail header say spf=pass. This indicates that SPF record is properly saved in your DNS settings.

To check the complete header go to “Show original option in Gmail. Goal is to make sure that both spf=pass and dkim=pass messages appear here:

Image for post
Image for post

spf=pass (google.com: domain of root@yourdomain.com designates xx.xx.xx.xx as permitted sender.)

The message designates xx.xx.xx.xx as permitted sender is important. If that’s not what you get you have to keep configuring your mail server until it does. That’s what the rest of this Postfix tutorial is about!

The x’es will be replaced with IP address of server from which email was sent. And yourdomain.com will be your hostname. This is important. If you want to properly configure mail you need to make sure your hostname is your domain name.

Hostname

On Linux you have hostname. This is the name that identifies your server by a unique name. If this hostname does not match your domain name and you send an email to a Gmail inbox, Gmail will complain with:

spf=neutral (google.com: xx.xx.xx.xx is neither permitted nor denied by best guess record for domain of root@hostname)

It says spf=neutral and you don’t want this to happen. And the neither permitted nor denied message shouldn’t be there. This is not secure email.

Go to your Linux command line and type:

hostname yourdomain.com

You just set your hostname to match your domain name. To verify type:

hostname

It should say yourdomain.com which is what will appear in email headers as your root@hostname. Restart your Postfix server and send email. This time around it should generate spf=pass message which means you’ve successfully set up SPF.

Adding DKIM Record

How does DKIM work? Like many security systems DKIM is based on one public key and one private key. The public key is added to your DNS settings as a TXT record. Your email message is signed with a private key.

When the email message is sent the recipient server (Gmail, for example) checks your DNS setting for the public key which has the name of mail._domainkey and verifies the DKIM value in it. Now we just need to actually generate DKIM keys.

Fastest way to generate your DKIM keys

If you just want to get this done fast just go to an online DKIM generator tool such as https://dkimcore.org/tools/keys.html and just type your domain name into input box and hit Enter. It will give you two parts.

One is your Private Key and it will look something like this:

-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQCv6a4bE2OWezjAk4/SmG+a+M+STPv5rid/teTjw/HknGrNFalt
c0nvW5FTiwoiofm7lG0C8OHQo0PVkXVGsrizVYShJMDfQk39/WyICxIHYt3XBB+1
ELC4bexsE8JYGJ9RwazO4otIOHBk+XPLEvNWeSx3Cj8dcVqRYPnQDv4cYwIDAQAB
AoGBAJ/d51s14LB2JVpmADOsUujsF39StT/wdHcMsAoKHf/b4vekcwhD2PJNLiJV
Xo2g1FsThYpBrYa1iUVC3ui0LUNRmOgPQfySkPSIoM3hm0HhkEfMLIwRNXHqfNT5
ayewCvsnuYGrAiwTqf5S9klsYHRlM6AD4uVR1yLDAwMQZvlxAkEA6m1PbVI610U0
om4LM0P+v7nQ3GzgtsqovPXZaTUHwgnI/zDWfH/b+0jPghjagwtIzVLqObMKvbh5
GaKrYcf96wJBAMAZ4M4LjiNKMVYfahi1Ow5+S2Xn4Kh7Rlo+GkDzqvUiqqyacGDE
XRaWBxURkRc1I6zw/0iZV0TGW0FaP9/MJWkCQQCLOYQ0fuOnOdi6pvRL7BP9tdOP
NbJ3nQB9yNzoGLCU0s7mkBCyPBVftaxXrX8I2MjR+G5W+jhk2IxDZ1K1bdslAkAT
u5TIFY/IODrRKfDwEM28M2TzVtpQ9DjJGE9GFMGe5Ky7hjG8/u7A/zBDDcblp2O+
xBK1FmtMeUOvxiqBhqrhAkApJFLcyl3mSV2hXnansn+hmJ4m7KTX1XivPk9gZUfb
j0mYWHMxhWsWCcS65XNpBn5G1ucerFKJWRtQRgm3D/q8
-----END RSA PRIVATE KEY-----

Save this into dkim-private.pem as a text file in UTF-8 format and store it somewhere on your server perhaps under /etc/postfix/ssl with the others.

Your Public Key will look similar to:

1583548738.semicolon._domainkey.yourdomain.com. IN TXT ("v=DKIM1;t=s;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCv6a4bE2OWezjAk4/SmG+a+M+S""TPv5rid/teTjw/HknGrNFaltc0nvW5FTiwoiofm7lG0C8OHQo0PVkXVGsrizVYSh""JMDfQk39/WyICxIHYt3XBB+1ELC4bexsE8JYGJ9RwazO4otIOHBk+XPLEvNWeSx3""Cj8dcVqRYPnQDv4cYwIDAQAB")

The value enclosed in double quotes should be entered this into your DNS as a TXT entry with property name mail._domainkey.

The value should be one continuous string of text. When actually entering the DKIM value into your DNS settings we generated above it would be entered as:

v=DKIM1; h=sha256; k=rsa; t=y; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCv6a4bE2OWezjAk4/SmG+a+M+STPv5rid/teTjw/HknGrNFaltc0nvW5FTiwoiofm7lG0C8OHQo0PVkXVGsrizVYShJMDfQk39/WyICxIHYt3XBB+1ELC4bexsE8JYGJ9RwazO4otIOHBk+XPLEvNWeSx3Cj8dcVqRYPnQDv4cYwIDAQAB

Save this as a new TXT entry in your DNS settings.

Now we just need to do one more thing. Modify /etc/postfix/main.cf file by adding the following 4 lines of to the very end:

#DKIM
milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

You may think we need to open port 8891 via IPTABLES on Linux server. Wrong! This is for Postfix socket. There is nothing else you need to do.

Save the file and run postfix stop command then postfix start. This will restart the Postfix server with new configuration.

Verifying if your DKIM is properly installed with DIG

It’s possible to successfully add DKIM record with a wrong public key. The email header will contain dkim=neutral (invalid public key) note. This is a silent fail and your email will still be sent. What you want it to say is dkim=pass.

There is a way to verify the actual value of the DKIM record. After changing DKIM record in DNS wait 5–10 minutes or so and run the following command in your bash:

dig mail._domainkey.yourdomain.com TXT

Just replace yourdomain.com with your actual domain name. This will output your TXT records on the console. Make sure it matches the generated DKIM value You may also want to try passing mail._domainkey or default._domainkey to the dig TXT command just to make sure. The TXT record shown on the screen should match what was generated in previous step. For my domain name I had this output:

Image for post
Image for post

Installing DKIM package

Adding DKIM record to our DNS is only first step. To put it all together…now we need to match the public domain key with our private key.

To install dkim package run the following command:

sudo apt-get install opendkim opendkim-tools

Installing opendkim-tools package has also created /etc/opendkim.conf file. This is the file we need to edit. Open it with:

nano /etc/opendkim.conf

Locate the following two lines:

#Socket                  inet:8891@localhost
Socket local:/var/run/opendkim/opendkim.sock

And change to:

Socket                  inet:8891@localhost
#Socket local:/var/run/opendkim/opendkim.sock

Ctrl-X and type Y and Enter to save the file.

Socket is for the milter to listen to at this address. Posfix will send messages to opendkim for signing and verification through this socket; 8891@localhost defines a TCP socket that listens on localhost:8891 port.

Side note: Milter is an extension to the widely used open source mail transfer agents Sendmail and Postfix. It allows administrators to add mail filters for filtering spam or viruses in the mail-processing chain.

Now edit default opendkim settings file:

nano /etc/default/opendkim

And change the line that starts with SOCKET to:

SOCKET=inet:8891@localhost

Save and exit the editor.

Edit main.cf Postfix configuration file:

nano /etc/postfix/main.cf

And add to the bottom of the file:

milter_protocol = 2
milter_default_action = accept
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Execute the following commands:

sudo service postfix restart
sudo service opendkim restart

Congratulations! At this point you should be able to send secure email from your server. In next part of this tutorial we’ll take a look at how to send email from command line and then install sendmail package so it can be send directly by your Node server.

Sending Email From Linux Command Line

Our mail server is ready for testing. But first we need to install mailutils which is a mailer utility to help us send mail directly from the command.

apt install mailutils

Type the following line (your message may be different) into your Linux bash:

echo "Hi" mail -s "My Subject Line" recipient001@huskmail.com

Simply replace recipient001@huskmail.com to the email address you want to send this message to. Hit enter and watch your inbox for message arrival. Chances are the first time around the email will end up in Spam folder.

Rewriting the From: header.

You will notice that your email arrived from root@domain. This isn’t exactly what you want. As a legitimate business, you want it to be reflected in the name of the email address from which you’re sending your news, updates, etc.

Postfix has extensive configuration that allows you to change pretty much everything you can think of re: sending email. It also allows you to rewrite email headers. It actually gives you ability to use regular expressions to do that.

But it’s best to set From: value using an email client, rather than directly messing with Postfix header rewrites. Gmail and other email servers are pretty strict about even slightly malformed headers, because it’s often a sign that a spammer has made a clumsy attempt at faking the message. We will use Node’s sendmail module.

Using Node sendmail To Send Email

Sending email from command line is nice but you also want to be able to send email directly from your Node server. This is especially useful if you want to send multiple emails and properly rewrite sender’s name, reply-to address and so on.

Installing sendmail

Just run the following command from your project’s directory:

npm install sendmail --save

And now we can use sendmail in our Node program!

// Bread and potatoes Express server
const express = require('express');
const https = require('https');
const fs = require('fs');
const path = require('path');
const app = express();
const port = 80;
const priv = '/etc/postfix/ssl/dkim-private.pem';// Send email to one or multiple recipients:
const sendmail = require('sendmail')({
logger: {
debug: console.log,
info: console.info,
warn: console.warn,
error: console.error
},
silent: false,
dkim: { // Default: False
privateKey: fs.readFileSync(priv, 'utf8'),
keySelector: '._domainkey'//'mydomainkey'
},
devPort: 25, // Default: False
devHost: 'localhost', // Default: localhost
smtpPort: 25, // 2525, // Default: 25
smtpHost: 'localhost'
});
const error = function(err, reply) {
console.dir(reply);
};
// Email body
const
body = '<p>You activation code is <b>10-' + Math.floor(Math.random()*10000) + '</b> enter it into code confirmation box within next hour to finish registering your account.</p><p><b>Glorious Chimpanzee</b><br/>Thanks from all of our team!</p>';
// Recipient list
const emails = ["recip-1@gmail.com",
"recip-2@other.com",
"FelixTheCat@catmail.com"];
const from_addr = 'Glorious Chimpanzee <hello@gloriouschimp.com>';
const subj = 'Please Confirm Your Glorious Chimpanzee Account';
emails.forEach(email => {
sendmail({from: from_addr,
to: email,
subject: subj,
html: body}, error);
});
// Run basic Express server:
// (You don't even need to do this, if you just want to send email)
let site = 'gloriouschimp.com';
const key = `/etc/letsencrypt/live/${site}/privkey.pem`;
const cert = `/etc/letsencrypt/live/${site}/fullchain.pem`;
const certificates = {
"key": fs.readFileSync(key),
"cert": fs.readFileSync(cert)
};
const server = event => {
console.log(`${site} is listening on port ${port}!`);
};
// Launch Node server with Express "app"
https.createServer(certificates, app).listen(port, server);

Save this file as express.js and run node express.js in bash.

Written by

Issues. Every webdev has them. Published author of CSS Visual Dictionary https://amzn.to/2JMWQP3 few others…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store