Deploying Signals for On-Premises: Momentum Integration – Part 3

Part 1 of this series introduced SparkPost Signals for on-premises deployments. Part 2 walked through setting up PowerMTA step-by-step. In this part, we’re going to dive into the details of connecting your Momentum server to SparkPost Signals. You’re going to need:

• A host running Momentum 4.x
• The Signals Agent rpm file and User Guide
• A SparkPost account with API key permission for “Incoming Events: Write” as per Part 1

We’ll set up Momentum to stream events up to your SparkPost account, then you’ll be able to use the following Signals Analytics reports: 

• Health Score 
• Engagement Recency 
• Spam Trap Monitoring

• Summary
• Bounce
• Accepted
• Delayed
• Engagement
• Events Search

Unlike PowerMTA, which requires external engagement-tracking, we’ll use Momentum’s built-in engagement tracking to capture recipient opens and clicks. That way, the Health Score, Engagement Recency, and Engagement reports all work immediately.

Configuring Momentum for Signals

There’s a lot of flexibility when setting up Momentum, and each setup will be different. This section will cover adding Signals integration to an existing working Momentum setup, as that’s what I expect most folks are interested in, so you don’t have to wade through a lot of basics that you already know. For the truly motivated, the details of our demo setup are covered in the “Annex: Momentum Signals demo configuration” section at the end. 

Firstly, follow the steps in the “Signals Agent User Guide” that you’ll receive with your SparkPost Signals account. On completion, you’ll have your specific API key stored within the script /etc/init.d/signals-agent

.. and your file /opt/msys/ecelerity/etc/conf/default/ecelerity.conf will have (near the end)

include “signals-agent.conf” 

.. and the file signals-agent.conf will be present in the same directory.

There is nothing special you need to do for clustered installations.  The Signals Agent must be installed on each node and each node reports events independently.

Momentum Engagement Tracking

If you’re already using Momentum Engagement Tracking, you can skip this section!

The setup of Engagement Tracking shown in some detail here, because it helps to get the most out of Momentum / Signals integration.

For this example, our Momentum demo server is on momo.signalsdemo.trymsys.net with a single elastic IP (and an A record pointing to it).

After following the Support instructions for enabling engagement tracking, I checked that mails delivered through our demo containing html have their links wrapped correctly, and an open-tracking pixel added. I chose to use port 81, and simple http (not https) tracking; your setup may vary. I saw the following, inside delivered mails.

<img border="0" width="1" height="1" alt="
" src="http://momo.signalsdemo.trymsys.net:81/q/rCZ3YpEEJBOHGGI06Rw9OA~~/AAAAAQA~/RgRfOCDVPhdCCgAAVe1WXbth_Q
BSGHN0ZXZlLnR1Y2tAc3Bhcmtwb3N0LmNvbVgEAAAAAEEIAC6rdFwAAPoJUQQAAAAARwJ7fQ~~">

As per the above instructions, I configured nginx, establishing the internal endpoints that will receive the clicks and opens on port 2081. Here’s my setup in /opt/msys/3rdParty/nginx/conf.d/click_proxy_upstream.conf:

upstream click_proxy {
  server momo.signalsdemo.trymsys.net:2081;
  least_conn;
}

This port is not exposed to the Internet. Instead, I use nginx to forward traffic on port 81 to the internal endpoint. I also set the headers to make Momentum “look like” SparkPost engagement tracking (so that my demo will follow the links). Here’s my setup in /opt/msys/3rdParty/nginx/conf.d/my_click_proxy.conf:

# Simple pass through to internal engagement tracking
# SMT 2019-08-16
#
# some basic additions to harden the server (tokens off) and
# make the endpoint behave more like SparkPost (set Server: type)
#
server {
  listen 81;
  server_name  momo.signalsdemo.trymsys.net;   # put your server name here
  location / {
    proxy_pass http://localhost:2081;
  }
  server_tokens off;
  more_set_headers 'Server: msys-http';
}

Controlling what information your server presents publicly, by using settings such as server_tokens off  is generally good security practice. Now we test the nginx configuration:

service msys-nginx configtest
nginx: the configuration file /opt/msys/3rdParty/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /opt/msys/3rdParty/nginx/conf/nginx.conf test is successful

Once iptables/firewalld configuration was done, and (in our case) AWS EC2 inbound security rule configured, we can test that our open pixel can be fetched from outside, using curl  from another host:

curl -v http://momo.signalsdemo.trymsys.net:81/q/rCZ3YpEEJBOHGGI06Rw9OA~~/AAAAAQA~/RgRfOCDVPhdCCgAAVe1WXbth_QBSGHN0ZXZlLnR1Y2tAc3Bhcmtwb3N0LmNvbVgEAAAAAEEIAC6rdFwAAPoJUQQAAAAARwJ7fQ~~
* About to connect() to momo.signalsdemo.trymsys.net port 81 (#0)
*   Trying 34.211.7.3...
* Connected to momo.signalsdemo.trymsys.net (34.211.7.3) port 81 (#0)
> GET /q/rCZ3YpEEJBOHGGI06Rw9OA~~/AAAAAQA~/RgRfOCDVPhdCCgAAVe1WXbth_QBSGHN0ZXZlLnR1Y2tAc3Bhcmtwb3N0LmNvbVgEAAAAAEEIAC6rdFwAAPoJUQQAAAAARwJ7fQ~~ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: momo.signalsdemo.trymsys.net:81
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Tue, 22 Oct 2019 18:38:46 GMT
< Content-Type: image/gif
< Content-Length: 44
< Connection: keep-alive
< Cache-Control: no-cache, max-age=0
< Server: msys-http
< 
GIF89a???????!?
* Connection #0 to host momo.signalsdemo.trymsys.net left intact

That response beginning GIF89a is the server delivering the open pixel, as a GIF file, back to our client. We can see the contents more easily by piping through hexdump :

curl http://momo.signalsdemo.trymsys.net:81/q/rCZ3YpEEJBOHGGI06Rw9OA~~/AAAAAQA~/RgRfOCDVPhdCCgAAVe1WXbth_QBSGHN0ZXZlLnR1Y2tAc3Bhcmtwb3N0LmNvbVgEAAAAAEEIAC6rdFwAAPoJUQQAAAAARwJ7fQ~~ | hexdump -C

00000000  47 49 46 38 39 61 01 00  01 00 80 00 00 ff ff ff |GIF89a..........|
00000010  ff ff ff 21 f9 04 01 0a  00 01 00 2c 00 00 00 00 |...!.......,....|
00000020  01 00 01 00 00 02 02 4c  01 00 3b 00 |.......L..;.|
0000002c

That’s all you need to have Engagement Tracking running on your server, and you should see Open and Click events appear in your linked SparkPost account:

Here’s what you’ve been waiting for …

After a day or two of running, you’ll see Health Score data building up:

Reporting facets

Here’s how Momentum attributes map onto SparkPost Signals Analytics reporting facets:

Setting Campaign

Being able to report with Campaign as a facet is really useful. There are two ways to do this:

1. Set up the X-MSYS-API header as described here. This special header provides various features as well, such as control of open and click tracking and metadata on your Momentum traffic stream. This is the method we used in this demo setup.
2. Create your own custom X-header to carry a campaign identifier, and map this in the signals-agent-config.lua file. For example, this makes Momentum accept an X-Job header carrying campaign ID, just like PowerMTA:

local cfg = {}
-- to add more custom headers it would look like this
-- custom_header = { ["X-SP-SUBACCOUNT-ID"] = "subaccount_id", ["X-CUSTOM-HEADER"] = "custom1", ["X-CUSTOM-HEADER2"] = "custom2"}
cfg.custom_header = { ["X-SP-SUBACCOUNT-ID"] = "subaccount_id", ["X-Job"] = "campaign_id" }
-- set to true if you are using your own click tracking
cfg.click = false
-- set to true if you are using your own open tracking
cfg.open = false
return cfg

That’s everything you need for Momentum / SparkPost Signals integration. If you want to know more about our demo configuration, read on.

Annex: Momentum Signals demo configuration

The config file structure can be found in /opt/msys/ecelerity/etc/conf/default/. A reference copy of selected files from our demo server config is on Github here.

Momentum offers Auth login & STARTTLS on injection

Our demo has user/password protected message injection on Port 587, as we did with the PowerMTA demo and SparkPost itself. Following the inbound TLS setup instructions, we have:

ESMTP_Listener{
  Listen ":587" {
    Enable = true
    # TLS key/cert for *.trymsys.net  
    TLS_Certificate = "/etc/pki/tls/certs/trymsys.net.crt"
    TLS_Key = "/etc/pki/tls/certs/trymsys.net.key"
    # Reference client CA bundle from https://curl.haxx.se/
    TLS_Client_CA = "/etc/pki/tls/certs/cacert.pem"
    TLS_Ciphers = "DEFAULT"
    TLS_protocols = "+ALL:-TLSv1.0:-SSLv3"
      AuthLoginParameters = [
        uri = "file:///opt/msys/ecelerity/etc/unsafe_passwd"
        log_authentication = "true"
    ]
    SMTP_Extensions = ( "ENHANCEDSTATUSCODES" "STARTTLS" "AUTH LOGIN" )

    # Engagement tracking
    tracking_domain = "momo.signalsdemo.trymsys.net:81"
    open_tracking_enabled = true
    click_tracking_enabled = true
    click_tracking_scheme = "http"
    open_tracking_scheme = "http"
  }

A fresh reference CA bundle was fetched from haxx.se. Legacy TLS versions (prior to v1.1) are disabled here for safety. You can prove that by comparing the output you see (from another host) between -tls1 , -tls1_1  and  -tls1_2  from another host using

openssl s_client -connect momo.signalsdemo.trymsys.net:587 -starttls smtp -tls1

Momentum out-of-band bounce processing

Firstly, we set up DNS MX records for our Return-Path:  (which will be test@momo.signalsdemo.trymsys.net ). Check using:

dig MX +short momo.signalsdemo.trymsys.net
10 momo.signalsdemo.trymsys.net.

The FBL and OOB listener on port 25 is separate to the injection port 587, and defined in ecelerity_mods.conf :

# FBL and OOB listener - no auth, but NOT open relay
#
  Listen "*:25" {
    Enable = true
    Open_Relay = false 
  }
}

Now we check (from an external host) that this listener is NOT open-relay:

swaks --server momo.signalsdemo.trymsys.net:25 --from steve@bouncy.test --to bob.lumreeker@gmail.com
=== Trying momo.signalsdemo.trymsys.net:25...
=== Connected to momo.signalsdemo.trymsys.net.
<-  220 2.0.0 ip-172-31-22-249.us-west-2.compute.internal ESMTP ecelerity 4.3.0.67725 r(Core:4.3.0.0) Tue, 26 Nov 2019 12:30:34 +0000
 -> EHLO steve-tuck-macbook-pro
<-  250-ip-172-31-22-249.us-west-2.compute.internal says EHLO to 81.105.42.190:52896
<-  250-8BITMIME
<-  250-PIPELINING
<-  250 ENHANCEDSTATUSCODES
 -> MAIL FROM:<steve@bouncy.test>
<-  250 2.0.0 MAIL FROM accepted
 -> RCPT TO:<bob.lumreeker@gmail.com>
<** 550 5.7.1 relaying denied
 -> QUIT
<-  221 2.3.0 ip-172-31-22-249.us-west-2.compute.internal closing connection
=== Connection closed with remote host.

That “relaying denied” message tells us we’re safe. Next, we check it does accept messages destined for the bounce processor. This is not a true bounce message, but is enough to check the routing is correct.

swaks --server momo.signalsdemo.trymsys.net:25 --from steve@bouncy.test --to test@momo.signalsdemo.trymsys.net
:
<-  250 2.0.0 OK 7D/00-30572-40C655D5
 -> QUIT

Momentum FBL processing

The file ecelerity_mods.conf contains:

#
# FBL content added to outbound mail - SMT 2019-08-15
#
Enable_FBL_Header_Insertion = enabled
fbl {
  Auto_Log = true # default is "false"
  Log_Path = "/var/log/ecelerity/fbllog.ec"     # not jlog
  Addresses = ( "^.*@fbl.momo.signalsdemo.trymsys.net" )  # default is unset
  Header_Name = "X-MSFBL"           # this is the default
  Message_Disposition = "blackhole"         # default is blackhole, also allowed to set to "pass"
}

FBLs on a subdomain (in fact any address on a subdomain) is taken care of with a wildcard CNAME record:

*.momo.signalsdemo.trymsys.net. 34 IN CNAME momo.signalsdemo.trymsys.net.

This resolves via the return-path MX:

host fbl.momo.signalsdemo.trymsys.net
fbl.momo.signalsdemo.trymsys.net is an alias for momo.signalsdemo.trymsys.net.
momo.signalsdemo.trymsys.net has address 34.211.7.3
momo.signalsdemo.trymsys.net mail is handled by 10 momo.signalsdemo.trymsys.net.

We check basic connectivity with swaks (not actually generating an FBL here, as such):

swaks --server fbl.momo.signalsdemo.trymsys.net:25 --from steve@fbl.test --to test@fbl.momo.signalsdemo.trymsys.net
:
:
<-  250 2.0.0 OK B4/80-24808-BCA12BD5

Putting test traffic through Momentum

Firstly we check with a single message:

swaks --server momo.signalsdemo.trymsys.net:587 --from test@momo.signalsdemo.trymsys.net --to test@bouncy-sink.trymsys.net --auth-user demo --auth-pass __YOUR_KEY_HERE__ -tls
:
:
<~  250 2.0.0 OK C4/80-24808-00C12BD5

Then we use this Traffic Generator to inject periodic message batches through Momentum, which delivers onwards to the Bouncy Sink. The Bouncy Sink accepts, opens, clicks, and in-band-bounces messages, and occasionally generates Out-of-Band bounces and FBLs.

. setenvs.sh
pipenv run ./sparkpost-traffic-gen.py 
Not replacing URLs for tracking, before injection
Established SMTP connection to momo.signalsdemo.trymsys.net, port 587
Successful LOGIN with user=demo, password=********************************
Sending to 42 recipients in batches of 10
Basic stats for day in month: Spam trap = 0.0077%, Spam complaint = 0.1032%
Today scaling factor = 0.6645, giving Spam trap = 0.0051%, Spam complaint 0.0686%
  To    10 recipients | campaign "Password_Reset" | ..........OK - in 0.157 seconds
  To    10 recipients | campaign "Password_Reset" | ..........OK - in 0.147 seconds
  To    10 recipients | campaign "Password_Reset" | ..........OK - in 0.137 seconds
  To    10 recipients | campaign "Welcome_Letter" | ..........OK - in 0.146 seconds
  To     2 recipients | campaign "Holiday_Bargains" | ..OK - in 0.035 seconds
Done in 0.6s.
Results written to redis

We can check this is working as expected by looking in Momentum logs; mainlog.ec  shows lots of deliveries such as

1571875206@4F/E2-01988-589E0BD5@4F/E2-01988-589E0BD5@3E/BB-01988-489E0BD5@R@test+00096118@not-hotmail.com
.bouncy-sink.trymsys.net@test@momo.signalsdemo.trymsys.net@34.210.87.20@3009@esmtp@general@generic

Momentum bouncelog.ec  shows these are processed, then “blackholed” as expected, i.e. does not try to forward them anywhere.

==> /var/log/ecelerity/bouncelog.ec <==
1571877606@4B/03-01988-6E2F0BD5@4B/03-01988-6E2F0BD5@19/DB-01988-6E2F0BD5@B@fbl@fbl.momo.signalsdemo.tryms
ys.net@test+00096899@fbl.bouncy-sink.trymsys.net@general@generic@22@25@0@@551 5.7.0 [internal] recipient
blackholed
1571877606@9C/03-01988-6E2F0BD5@9C/03-01988-6E2F0BD5@99/DB-01988-6E2F0BD5@B@test+00134362@not-yahoo.co.uk.
bouncy-sink.trymsys.net@test@momo.signalsdemo.trymsys.net@default@default@21@10@5715@@550 [internal] [oob]
The recipient is invalid.

You can deliberately cause FBLs and OOBs, by sending to specific bouncy sink subdomains (as per this table).

swaks --server momo.signalsdemo.trymsys.net:587 --from test@momo.signalsdemo.trymsys.net --to
test@fbl.bouncy-sink.trymsys.net --auth-user demo --auth-pass __YOUR_KEY_HERE__ -tls

swaks --server momo.signalsdemo.trymsys.net:587 --from test@momo.signalsdemo.trymsys.net --to 
test@oob.bouncy-sink.trymsys.net --auth-user demo --auth-pass __YOUR_KEY_HERE__ -tls

Checking results

Looking in our SparkPost Events Search report, we can see Spam Complaint and Out of Band events showing up:

Showing the reporting facets

Our demo has a set of example subaccounts. Messages are assigned to specific bindings, via injected message headers. The campaign ID is set using the X-MSYS-API  header, for example:

X-Binding: medium
X-Sp-Subaccount-Id: 3
X-MSYS-API: {"campaign_id": "Charlie's Last Minute Savings"}

We see message streams are flowing through these subaccounts on the Summary report:

We see the individual campaign names:

“Sending IP” (aka Binding) can be used as a reporting facet:

We can also use “IP Pool” (aka Binding Group) as a reporting facet:

~ Steve