Acquiring a large SMTP pcap for testing

Recently I needed a large PCAP of SMTP traffic for testing zeek scripts. Obtaining a capture from a production network would be a straightforward process. However, I needed a file that could be included in test suites, or copied to other hosts without having to worry about leaking sensitive data.

I could have taken an archived MBOX file and replayed it into a server. This would have worked, but wouldn’t have included any variability due to client differences. When testing tools that are analyzing traffic, you want the traffic to be as real world as possible. Replaying a MBOX file would have given me real world mail data, but it would have only consisted of SMTP behaviour from a single client implementation.

My solution was to set-up a SMTP server configured to accept all email sent to it. Concurrently, tcpdump would be running to capture every packet transmitted over port 25.

I set this up using a VPS and the following script

Create a locked user to act as the destination for the email

adduser spam
passwd -l spam

Install postfix with the pcre module

apt-get install postfix-pcre

Route all email to the spam user

echo '/.*/    spam' > /etc/postfix/virtual
echo 'virtual_alias_maps = pcre:/etc/postfix/virtual' >> /etc/postfix/main.cf

Disable TLS - we want to analyze SMTP commands, not TLS handshakes

perl -pi -e 's/smtpd_use_tls=yes/smtpd_use_tls=no/' /etc/postfix/main.cf

Create a directory for storing the pcap files

mkdir -p /data/pcaps

Create a wrapper script

This will create pcap files based on the date and limited to at most 100MB. The filenames aren’t super important because things can be merged and re-split later on.

cat <<'END' > /usr/local/bin/start_pcap
#!/bin/sh
exec /usr/sbin/tcpdump -i eth0 -s 0 -C 100 -w /data/pcaps/smtp.$(date +'%Y-%m-%d_%H_%m').pcap 'port 25'
END
chmod +x /usr/local/bin/start_pcap

Have systemd run this script at boot and ensure it’s always running.

cat <<END > /etc/systemd/system/pcap.service
[Unit]
Description=full pcap for smtp
After=network.target

[Service]
User=root
Group=root
ExecStart=/usr/local/bin/start_pcap
Restart=always
RestartSec=2s

[Install]
WantedBy=multi-user.target
END

systemctl enable pcap
systemctl start pcap

Roadblocks

Once I had this running I figured I would start receiving spam, but that turned out to be harder than I thought. Tweeting out an email address resulted in zero emails. Putting an address into a cryptocurrency newsletter sign-up form resulted in exactly the one email per week as promised. Various “sign-an-address-up-for-a-lot-of-email” websites failed to work at all.

To get the process going I signed an address up to the Linux kernel mailing list. Living up to its reputation as a high volume list, I have received about 13,000 emails. This isn’t ideal, but it’s a start.

Results

After two weeks I have about 100MB of MBOX and PCAP files:

96M     /data/pcaps/smtp.2019-12-13_18_12.pcap
29M     /data/pcaps/smtp.2019-12-24_02_12.pcap
97M     /var/spool/mail/spam