Adding a junk button to MUAs

Most web mail systems have a button a user can push to mark a message as junk, and report it back to their mail host. How would one add a similar junk button (JB) to a regular MUA? For the purposes of this discussion, the goal is to report the message back to the user's own mail system. They might further report it to the place the message came from, but that's a separate discussion. Also out of scope are other ways in which administrators respond to such reports, however it's worth noting that any response mechanism should be robust to false reports, whether they're malicious or accidental.

Any scheme requires an upgrade or at least a plugin to the MUA. Different schemes require different amounts of changes to mail delivery and retrieval software.

How many buttons?

Existing webmail all has a single junk button, or (gmail) one button that pops up 'do you want to unsub?' if the message looks like it's from a list.

Some people suggest multiple options, e.g., junk, unsub, don't want any more, etc. Examples cited include Twitter's use of "block" and "report for abuse". The difference is presumed to be the type of action that is requested. Note also that the labels will be language, and perhaps application specific.

ARF has many of these options, but as far as we can tell the only ones ever used are junk (which it calls abuse) and virus, which scripts can identify automatically from EXE and ZIP signatures.

Sending back reported messages

The two general approaches seem to be to send it back via the path it by which it was retrieved by the MUA, typically POP or IMAP, or to use a separate path, such as SMTP or SUBMIT. A Consensus Call has had 10 posts for normal posting , 2 for specific to the mechanism for retrieving the message , and 1 ambivalent .

Via the delivery path

Most but not all MUAs retrieve mail with IMAP or POP.

Overall pro: Reports are secure, since they're returned directly to the mail system. Configuration is likely to be automatic.

Overall con: Requires changes to multiple protocols and to server software. Changes to POP are severe.

Via IMAP

• Add an IMAP Junk flag that the MUA can set, or a message annotation.

Pro: Easy to implement with IMAP. Self-configuring with IMAP, MUA can tell automatically if JB is available. Relatively secure since client has already logged in to server.

Con: Requires changes to protocol and to server software.

• Move messages to a specific subfolder such as Spam.

Pro: Already implemented at AOL and in dovecot-antispam, requires no protocol changes. Relatively secure since client has already logged in to server.

Con: Requires IMAP server (or back end at least) changes, adds a reserved folder name. Arguably better to annotate the folder (IMAP METADATA extension required) than to reserve a name, since the name may be language specific. Overloads the "junk" folder, which is a place where filters automatically place suspected junk.

Via POP

Possibilities include:

• Add a command like XTND JUNK that the MUA can use to send a message back to the server.
• Modify the protocol so that UIDLs or message numbers persist beyond message deletion, and add a command to report the UIDL or session ID number.

Pro: Self-configuring, MUA can tell automatically if JB is available. Relatively secure since client has already logged in to server.

Con: Requires changes to protocol and server. XTND JUNK requires servers to accept arbitrarily large uploads, which they don't now. UIDL or ID report requires server to store potentially large amounts of additional mail.

Via other paths

Some MUAs read mailstore directly, or receive via UUCP or SMTP. Users would be out of luck.

Via a separate path

The MUA could build an ARF report from the reported message and send that report back to the server.

Overall pro: Server need not retain messages for potential report. Once configured, reporting system is independent of inbound path or server, not dependent on POP or IMAP.

Overall con: Less secure than returning directly to incoming server. Reports are larger compared to an IMAP flag change or POP UIDL report.

Via SMTP

• The MUA sends the ARF report to a reporting address at the mail system using standard SUBMIT or SMTP. Configuration only needs the target address.

Pro: MUAs already know how to create and send MIME messages, leverage existing software, simple configuration.

Con: SUBMIT or SMTP server might not be under the same management as inbound server, possibly leading to third party snooping or message loss. If systems accept reports other than from their own SUBMIT server, forged reports and DoS are possible.

Via HTTP PUT or POST

• The MUA uploads the ARF report using HTTP PUT or POST. Besides the URL, configuration may also need user/password for authentication.

Pro: Faster and and more secure than SMTP, since communication is directly from MUA to server. Another pro is that it may be more compatible with junk reporting used for non-SMTP messaging protocols, such as SMS, MMS, various IM, and SIP-based ones. Finally, it may be simpler to develop in JavaScript; for example, the Thunderbird plugin offered by blackhole.mx (abusix) uses XMLHttpRequest() to upload spam samples.

Con: Requires an HTTP server to accept reports, and an HTTP client in the MUA to do a PUT or POST. These may challenges.

Configuring the MUA junk button

The MUA needs to know both whether a message can be reported as junk, and how to report it. We seem to agree that automatic configuration is better than manual.

Per message approaches

At delivery time, the MDA adds a header that indicates that a JB is available, with configuration parameters such as the address to which to mail an ARF report. A likely candidate for the header would be an extended version of the Authentication-Results header.

Pro: No changes to retrieval protocol, independent of retrieval protocol. Can put different flags on different messages. Can potentially remain useful across forwarding or mail systems that consolidate mail by POP retrieval from others.

Con: Requires changes to MDA. Bad guys could add headers to hijack reports, or do low-level DoS against third parties. Doesn't work well with IMAP junk flag, POP UIDL report, etc.

Per server approaches

Since the MUA already knows how to find a POP or IMAP server, it can find the JB configuration directly or indirectly from that server.

Server queries

The POP CAPA command and IMAP CAPABILITY command allow MUAs to query what a server can do. For schemes that report directly to the retrieval server, the MUA can see if the reporting command(s) are available. For schemes that return the report via a different path, a simple command could return configuration info.

Pro: Secure since MUA already logs into server, protocol and server changes are relatively simple. Config can be per-account, which might be useful for servers that handle multiple resellers.

Con; Still requires protocol and server changes.

Metadata in the DNS

• Publish configuration info in the DNS using names based on server names, e.g.,

_report.pop.example.com TXT "r=arfreport@example.com"

Note that this record syntax is being considered for use by the mail host to send the report back to the relevant source.

Pro: No changes to MDA, retrieval protocol or software. Easy to configure in MUAs that can make DNS queries (most of them, presumably).

Con: New requirement to manage host names that refer to POP or IMAP server. Servers may have multiple names (thousands in some cases) via CNAME or multiple A records requiring a lot of DNS maintenance for any change. Servers may be accessed via tunnels or proxies that don't provide the server name. All users of the same server get the same metadata, unless they refer to it by different names in which case maybe they don't.

• If a server has many CNAMEs pointing at one A record, a variant would be for the MUA to resolve all the CNAMEs and look for the metadata near the name of the A record, e.g.

pop.example.com CNAME poppy.example.net
_report.poppy.example.net TXT "r=arfreport@example.net"

Pro: Addresses multiple CNAME problem.

Con: Only partial solution, doesn't address multiple A, tunnel, etc. Unusual DNS usage may be hard to implement or have subtle problems evident to DNS weeenies.

See also

• Abuse Reporting
• The marf (Messaging Abuse Reporting Format) IETF's WG. That charter comprises the issue of publishing somehow the address to which abuse reports should be sent.