Frequently Asked Questions
What is spamdyke and what does it do?
In a sentence, spamdyke is a drop-in qmail filter for stopping spam at connection-time.
"drop-in" means it can be installed without patching or recompiling qmail, without installing or updating libraries, without drastically reconfiguring anything and without having to become a qmail expert.
"connection-time" means spamdyke evaluates and rejects spam while the remote server is still delivering it. Other filters and anti-spam solutions focus on classifying spam after qmail has accepted it. The spam still has to go somewhere. Even if it's filed in a folder, it still occupies disk space, consumes server resources and must be deleted by someone. When spamdyke rejects the incoming spam completely, no one has to deal with it. It's never on the server at all.
For a complete description of spamdyke and all its features, see the README page.
How do I install spamdyke?
For installation instructions, see the INSTALL.txt file.
How do I get support for spamdyke? Is there a mailing list?
Yes! Visit the mailing list page to sign up: www.spamdyke.org/mailman/listinfo/spamdyke-users. The mailing list archives are searchable at: www.mail-archive.com/spamdyke-users@spamdyke.org.
All of the documentation and releases can be found at on the spamdyke website at spamdyke.org.
If you can't find answers there, send an email to me at: samc (at) silence (dot) org
I don't use qmail. Can I still use spamdyke?
Not at this time.
Some background: Qmail starts a new process (qmail-smtpd) from a listening daemon (tcpserver or xinetd) every time a new connection is established. spamdyke slips in between those two, so the daemon starts a new copy of spamdyke for each connection and spamdyke starts the qmail process.
Most other mail servers don't work this way; they use a single long-running daemon for handling incoming requests. There's no way to insert spamdyke without rewriting the mail server daemon (no thanks!).
Having said that, spamdyke could be modified to listen for incoming connections itself (replacing qmail's tcpserver) and establish a new connection to the "real" server (presumably listening on a different port, perhaps running on a different interface or machine). This requires some refactoring in the spamdyke code but it's not an insurmountable task.
Look for this feature in a future version.
Graylisting isn't working! What am I doing wrong?
Most often, graylisting doesn't work because there are no domain folders. spamdyke is designed to allow some flexibility when configuring graylisting, so you can enable it for some domains and disable it for others. To enable graylisting for a domain, you must create a folder within your graylist folder named for the domain you want to graylist. For example, suppose you created a graylist folder:
/home/vpopmail/graylist.d
/usr/local/bin/spamdyke -g /home/vpopmail/graylist.d ...
/home/vpopmail/graylist.d/example.com
I want to use spamdyke but some of my users roam and connect from strange places. How can I allow them to send email but still filter spam?
This is definitely the most frequently asked question.
As of version 2.5.0, this is no problem. spamdyke now understands SMTP AUTH, so it can authenticate your users and bypass all of its filtering just for them. spamdyke will even provide SMTP AUTH for you even if your qmail has not been patched to provide SMTP AUTH!
See the README page for complete details.
My users authenticate with SMTP AUTH. Can I still use spamdyke?
Yes! As of version 2.5.0, spamdyke understands SMTP AUTH and disables all of its filtering for authenticated users.
See the README page for complete details.
My users authenticate with POP3-before-SMTP. Can I still use spamdyke?
Maybe, maybe not. If your POP3 server is writing authenticated IP addresses to a file, you can use that file as an IP whitelist with spamdyke. If it's keeping track of authenticated IP addresses in some other way, you're out of luck.
POP3-before-SMTP is really a kludge anyway; consider using SMTP AUTH instead.
I want to block all emails unless the sender authenticates. Can spamdyke do that?
Yes. The easiest way is to create an IP blacklist file that will block all IP addresses:
0.0.0.0/0.0.0.0
Does spamdyke support TLS?
This is the second-most-frequently asked question.
As of version 2.6.0, spamdyke supports TLS (which is just another name for SSL). spamdyke will detect TLS and pass it through seamlessly. Obviously, none of its post-connect filters will work (e.g. graylisting) because the traffic will all be encrypted.
However, if spamdyke has access to a server certificate, it will handle the TLS itself and all of its filters will work as before. spamdyke will even provide TLS for you even if your qmail has not been patched to provide TLS!
spamdyke also works very well with SMTP-over-SSL (SMTPS). The SSL decryption takes place before spamdyke receives the traffic, so spamdyke never knows it's happening.
See the README page for complete details.
Does spamdyke filter using Sender Policy Framework (SPF), Sender ID, Certified Server Validation (CSV), DomainKeys or DomainKeys Identified Mail (DKIM)?
Not at this time. Each of those systems is very complex, so adding support to spamdyke will not be a small task. They're also all different, so supporting all of them would be a major undertaking. At this point, there are other ways to improve spamdyke that will take less effort and provide much more benefit.
Moreover, I'm not convinced any of those systems will make any difference. They were each designed to prevent spam being sent from forged addreses so, for example, you won't get spam from president@whitehouse.gov. However, most spammers own their own domains (often many thousands of them) and control their own DNS, so they can (and do) add SPF/CSV/etc records so their spam follows the SPF/CSV/etc rules. (HINT: Spammers understand and administer their DNS records better than most ISPs do.) The major ISPs (AOL, Yahoo!, GMail) use these systems and still have spam problems; this is pretty good evidence they aren't living up to the hype.
There is a (small) chance these systems could stop spam coming from botnets. If that happens however, the spammers will just start relaying their spam through the compromised machine's ISP's mail servers. That's why having every ISP block all outbound port 25 traffic won't stop botnet spam either.
If you really, really want to filter your email using one or more of these frameworks, try SpamAssassin first. It already supports most of them and you'll be able to see for yourself how effective they are.
Does spamdyke filter based on message headers like "From:" or "Received:"? Can spamdyke block large messages or strip attachments?
No. To understand why, a little explanation of the SMTP protocol is necessary.
After some conversational preliminaries, a client sending a message to an SMTP server gives a sender address. This address is referred to as the "envelope sender" and it's the address a bounced message will be delivered to. Here's the part most users don't understand: The "envelope sender" doesn't have to be the same address you see on the "From" line. The two aren't related at all; the "From" line is mostly for show (this is why spam "From" lines are so often forged).
If the server accepts the "envelope sender" address, the client sends the addresses of the message recipients, one at a time. The recipient addresses dictate who the message will actually be delivered to. Again, the "To" and "Cc" lines aren't related at all; they're mostly for show. The server must accept or reject each recipient individually.
If the server accepts one or more recipients, the clients sends the message. The server accepts or rejects the message and the conversation is over.
spamdyke makes a decision whether to accept or reject senders and recipients during the first two steps because those steps must be acknowleged separately. Once the client sends the actual message, it can't be rejected.
Well, OK, technically, according to the RFCs, it can be rejected. However, most MTAs don't handle this situation correctly and bounce the message if they see anything other than a success code after sending the message. This is a Bad Thing. If the rejection was for graylisting, the client won't retry delivery. If the rejection was for one recipient out of many, the client (and therefore the sending user) won't understand why the error was given. The message will just bounce and that's it.
There are many, many tools for filtering messages based on headers and content that are far, far smarter and feature-rich than spamdyke. If you really want to filter based on message headers, please use one of them in addition to spamdyke. Personally, I use (and recommend) SpamAssassin and ClamAV.
I installed spamdyke and now I'm seeing a lot of timeouts in my logs. Why?
Badly written software on the remote hosts. It seems a lot of spam software doesn't handle error codes at all. It just attempts delivery and expects success. When an error code is sent, the software just sits and waits for the success code it wants. Eventually, the connection times out. Sometimes, a remote server will take a long time to begin delivering a large (legitimate) message, which can cause timeouts.
qmail and rblsmtpd both enforce timeouts (20 minutes idle time and 1 minute total connection time, respectively) but they do so silently. You were likely always getting timeouts before and just didn't know it until spamdyke began logging them.
If you suspect legitimate connections are timing out, there are two things you can do. First, you can increase or disable spamdyke's timeouts. Of course, qmail's 20 minute idle timeout will still apply but at least you'll be back to the same setup you had before.
Second, you can use spamdyke's full logging feature to log all incoming connections to files. The log files contain timestamps so you can see how quickly the remote server is sending data and where it's stopping. Hopefully that will yield some clues you can use to fix the problem.
I want to whitelist a large number of IP addresses; can I use wildcards?
Yes, starting with spamdyke version 2.2.0. The whitelist and blacklist IP files will work with partial IP addresses to represent ranges.
Starting with spamdyke version 2.4.0, whitelist and blacklist IP files can also contain IP addresses as dotted quad IP address followed by a netmask as a number of bits. Also, spamdyke supports dotted quad IP addresses followed by a netmask as a dotted quad.
Starting with spamdyke version 3.0.0, whitelisted IP addresses can also be found by using a DNS realtime whitelist. This is like a DNS RBL that lists whitelisted IP addresses instead of blacklisted ones.
For complete details, see the README page.
I installed spamdyke and now my server is very slow! Incoming connections have to wait 20 seconds or longer before they see the greeting banner. What can I do speed it up?
Most often, delays like these are due to DNS traffic. Depending on the enabled filters, spamdyke can perform the following DNS queries for each incoming connection:
- Find the reverse DNS name for the remote server
- Find the IP address for the remote server's reverse DNS name
- Check the realtime whitelists for TXT records matching the remote server's IP address
- Check the realtime whitelists for A records matching the remote server's IP address
- Check the realtime blacklists for TXT records matching the remote server's IP address
- Check the realtime blacklists for A records matching the remote server's IP address
- Check the righthand-side whitelists for TXT records matching the remote server's rDNS name
- Check the righthand-side whitelists for A records matching the remote server's rDNS name
- Check the righthand-side blacklists for TXT records matching the remote server's rDNS name
- Check the righthand-side blacklists for A records matching the remote server's rDNS name
- Check the righthand-side whitelists for TXT records matching the sender's domain name
- Check the righthand-side whitelists for A records matching the sender's domain name
- Check the righthand-side blacklists for TXT records matching the sender's domain name
- Check the righthand-side blacklists for A records matching the sender's domain name
If you have disabled all of the filters and spamdyke is still running slowly, you may have found a bug. Please report it!
Can spamdyke validate a recipient address before accepting a message?
No. It's something that would be very nice to have but it's not very easy to implement. Qmail doesn't make it easy to determine if an account is valid or not and vpopmail (if present) only complicates matters. Since it's better to accept spam than reject legitimate email, this feature doesn't exist (yet).
If you have a suggestion or method for checking recipient validity (that DOESN'T involve reusing portions of qmail code or recompiling qmail), I'd love to know about it.
Do I have to install the programs from the "utils" folder? Does spamdyke use them? Do they use spamdyke or each other?
No. domainsplit and domain2path are just small utilities for use in writing scripts. spamdyke doesn't use them or depend on them. Conversely, they don't use or depend on spamdyke.
passwordcheck is a small utility for checking the validity of an SMTP AUTH command for use with spamdyke. It runs the command and checks the output to determine what inputs the command supports.
dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt are just small, self-contained examples of how to make A, MX, NS, PTR, SOA and TXT DNS queries using libc. They're not really useful on their own. I made them available because I was looking for examples like them when I was writing the DNS code in spamdyke and I couldn't find any. Hopefully someone else will be able to learn from them. Aren't I just a swell guy?
Does spamdyke run its filters in any particular order?
Yes. spamdyke evaluates its filters in the following order (of course a filter is skipped if it's disabled):
- Check for an rDNS name
- Check for an IP address in a country code rDNS name
- Check for an rDNS whitelist entry
- Check for an rDNS blacklist entry
- Check for an IP whitelist entry
- Check for an IP blacklist entry
- Check for an IP address and keyword in the rDNS name
- Check if the rDNS name resolves
- Check DNS whitelists
- Check right-hand-side whitelists
- Check DNS RBLs
- Check right-hand-side blacklists
- Check for earlytalkers
- Check for an IP address in a country code rDNS name
The intent is to order the filters from least-to-most expensive, so connections will be rejected as quickly as possible. In a typical setup, DNS queries are more expensive than file searches, pattern matching is more expensive than simply checking for a file's existence, etc.
The remaining filters are all checked during the SMTP conversation.
- Limit the number of recipients
- Block unqualified recipient addresses
- Block relaying from unauthorized remote hosts
- Check for sender's domain MX record
- Graylisting
- Check sender whitelists
- Check sender blacklists
- Check right-hand-side whitelists for the sender's domain name
- Check right-hand-side blacklists for the sender's domain name
- Check recipient whitelists
- Check recipient blacklists
- Block unqualified recipient addresses
My graylist folders are getting huge -- many, many entries. Is this a problem? Can spamdyke automatically delete the old ones?
It's not really a problem, since most of the files contain no data. In other words, they don't take up any space and shouldn't cause a problem unless your filesystem runs low on inodes.
However, if they really bother you, you can run a simple command to remove the old entries:
find GRAYLIST_FOLDER -type f -mmin +$[SPAMDYKE_MAX_GRAYLIST_SECS/60] -print0 | xargs -0 rm -f
spamdyke doesn't automatically delete the old entries because it doesn't run as a daemon. In other words, if spamdyke suddenly decided to clean up the graylist folder, it would have to do that work while receiving an incoming email message. If your graylist folder is large, the cleanup could cause a large enough delay to bounce the message. Also, how is spamdyke to decide when to cleanup? Because a new spamdyke process is started for each incoming connection and those processes don't communicate with each other, there's a good chance multiple spamdyke instances would attempt to cleanup the folder at the same time. That would delay multiple incoming messages and place an unnecessary load on the server.
I use spamdyke to prevent relaying (because my qmail isn't patched to provide SMTP AUTH) but SpamAssassin has stopped scanning incoming messages. What gives?
You must be using qmail-scanner from qmail-scanner.sourceforge.net. That package has an interesting flaw: it assumes any time the environment variable RELAYCLIENT is set, no scanning should be performed. When spamdyke prevents relaying, it always sets that variable to keep qmail from interfering with its relaying decision.
To reenable scanning, modify your /etc/tcp.smtp file and add "QS_SPAMASSASSIN" to all of the connections you want scanned. For example:
127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue":allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue",QS_SPAMASSASSIN=""
I love spamdyke! Can I send you money? Can I have your children?
Well this is so sudden... we just met.
Thank you for your generosity, but I must decline. I wrote spamdyke to meet my own needs, not to make money. If you really feel that strongly, drop me an email to let me know you're using spamdyke and you like it. Ask questions, request features, report bugs if you find any. I love getting emails about spamdyke. :)