1. System Aspects of Spam Control Architecture and Operations Issues IBM Academy 6 Apr 2005 Dave Crocker Brandenburg InternetWorking dcrocker@bbiw.net IBM Academy 6 Apr 2005 Dave Crocker Brandenburg InternetWorking dcrocker@bbiw.net

2. 2 2 D. CrockerIBM Academy / Spam Technical Issues Setting the Context © 1975(!) Datamation This? Oh, this is the display for my electronic junk mail.

3. 3 3 D. CrockerIBM Academy / Spam Technical Issues Approaching the Topic  Spam and email are complicated, global, human » We neeed a technical response to a social problem » Spammers are aggressive, bright, adaptive, well-organized » Nothing will “eliminate” spam! But we can control it.  Many points of control in the email architecture » We need a coherent framework for spam control » 1B users  Simplistic solutions will be damaging  Assess Proposals carefully » Risk, cost, scaling, efficacy and durability » Local, transient effects  spammers use different techniques, versus » Global, long-term effects that truly reduce spam at its core  Spam and email are complicated, global, human » We neeed a technical response to a social problem » Spammers are aggressive, bright, adaptive, well-organized » Nothing will “eliminate” spam! But we can control it.  Many points of control in the email architecture » We need a coherent framework for spam control » 1B users  Simplistic solutions will be damaging  Assess Proposals carefully » Risk, cost, scaling, efficacy and durability » Local, transient effects  spammers use different techniques, versus » Global, long-term effects that truly reduce spam at its core

4. 4 4 D. CrockerIBM Academy / Spam Technical Issues Dangerous Logic  “We have to do something now!” (Ignore any side-effects, or dismiss them as minor.)  “Maybe it’s not perfect… but at least we’re taking some action!”  “What have we got to lose?”  “ At least it reduces the problem… for now.”  “We must replace SMTP… even though we don’t know what we want to do  “We can do something in the interim…” Even though nothing on the Internet is ever interim  “We have to do something now!” (Ignore any side-effects, or dismiss them as minor.)  “Maybe it’s not perfect… but at least we’re taking some action!”  “What have we got to lose?”  “ At least it reduces the problem… for now.”  “We must replace SMTP… even though we don’t know what we want to do  “We can do something in the interim…” Even though nothing on the Internet is ever interim “…but this is urgent!!”

5. 5 5 D. CrockerIBM Academy / Spam Technical Issues Examples of Solution Challenges  Challenge/Response » Impose potentially large delay » Irritate legitimate senders & reduce spontaneous communications  False positives » Lose sales opportunities and purchase orders  Reputation mechanisms » Can be gamed  Challenge/Response » Impose potentially large delay » Irritate legitimate senders & reduce spontaneous communications  False positives » Lose sales opportunities and purchase orders  Reputation mechanisms » Can be gamed

6. 6 6 D. CrockerIBM Academy / Spam Technical Issues Taking the Long View Imagine that time has passed…  Different types of spam » Accountable business will behave acceptably (mostly) » Rogue (criminal) spammers will be worse than today  Email » Will it still be easy to reach everyone? » Will it be cumbersome, with fragmented communities? » Will legitimate forms of communication be blocked? Imagine that time has passed…  Different types of spam » Accountable business will behave acceptably (mostly) » Rogue (criminal) spammers will be worse than today  Email » Will it still be easy to reach everyone? » Will it be cumbersome, with fragmented communities? » Will legitimate forms of communication be blocked?

7. 7 7 D. CrockerIBM Academy / Spam Technical Issues What is Spam? (and isn’t it impressive we need to ask this question!) Challenges  No clear community consensus on definition » Strong on emotion » Weak on useful discussion  Minor, transient technical differences from good mail (!) » Internet mechanisms are expensive to implement » We must ensure they will quickly be effective for extended time » Danger of arms raceChallenges  No clear community consensus on definition » Strong on emotion » Weak on useful discussion  Minor, transient technical differences from good mail (!) » Internet mechanisms are expensive to implement » We must ensure they will quickly be effective for extended time » Danger of arms race Sample Definitions  Whatever recipient decides » This means we cannot provide institutional enforcement  Unsolicited Commercial » Religious, political, and “crazies” are just as problematic  Unsolicited Bulk » Focus on consent/permission » Focus on aggregate traffic

8. 8 8 D. CrockerIBM Academy / Spam Technical Issues Universal Spam Solution Rebuttal Checkbox form letter for responding to spam solutions proposals. See: Your post advocates a ( ) technical ( ) legislative ( ) market-based ( ) vigilante approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)… Checkbox form letter for responding to spam solutions proposals. See: Your post advocates a ( ) technical ( ) legislative ( ) market-based ( ) vigilante approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)…

9. 9 9 D. CrockerIBM Academy / Spam Technical Issues MHS MHS: Mail Handling Service AUAU AUAU AUAU AUAUAUAU Email Architecture MTAMTAMTAMTAMTAMTAMTAMTA MAIL: MUA= UserMTA = Transfer AGENTS: MSA = SubmissionMDA= Delivery Mlist= Mailing List AU = Administrative Unit oMUAoMUArMUArMUAActorsActors Originator SenderDestRelay RecipientMediator Actors MDAMDAMSAMSA BounceBounce MTAMTAMListMList

10. 10 D. CrockerIBM Academy / Spam Technical Issues More Than One “Sender” MTAMTAMTAMTAMTAMTAMTAMTA oMUAoMUArMUArMUA MDAMDAMSAMSA BounceBounce MTAMTA MListMList MTA IP HELO Domain Provider Network IP Sender From Mail From Received Sender

11. 11 D. CrockerIBM Academy / Spam Technical Issues Email Points of Control ReactiveFilteringReactiveFiltering Originator User Agent Origin Mail Transfer Agent External Mail Transfer Agent Receiver User Agent Receive Mail Transfer Agent External Mail Transfer Agent Proactive:PriceAccountabilityReactive:FilteringEnforcementProactive:PriceAccountabilityReactive:FilteringEnforcement

12. 12 D. CrockerIBM Academy / Spam Technical Issues A Spamming “Architecture” Spammer Zombie Controller Zombie Zombie Zombie Zombie Zombie Zombie Zombie Zombie Zombie Victim Initial Web Page Revenue Web Page

13. 13 D. CrockerIBM Academy / Spam Technical Issues Wheel of Spam (Mis)Fortune  Control of spam » Cannot be “surgically” precise » Must balance the wheel » Needs range of partial solutions » Different techniques for near- term vs. long-term, except that near-term never is  Heuristics » Long lists  complicated » Complicated  Be careful! Political Legal Social Human Administration Technical Management Deployment Many Facets

14. 14 D. CrockerIBM Academy / Spam Technical Issues Types of Control Proactive  Accountability » Sender/Author » Sending host  Enforcement » Laws and contracts » Acceptable Use Policy » Scope of control? » Sufficiently objective rules? » Avoids negative side-effects Proactive  Accountability » Sender/Author » Sending host  Enforcement » Laws and contracts » Acceptable Use Policy » Scope of control? » Sufficiently objective rules? » Avoids negative side-effects Reactive (filtering)  Detection » Source or destination » Content » Aggregate traffic » Accreditation/Reputation  Action » Divert or delete » Label » Notification » Delay

15. 15 D. CrockerIBM Academy / Spam Technical Issues Secondary Approaches  Charging – Sender pays fee » Some vs. all senders » How much? » Who gets the money?  Enforcement – Laws and contracts » Scope of control – national boundaries? » Precise, objective, narrow?  Administration » Exchange filtering rules » Exchange incident (abuse) reports » Coordination among Abuse desks  Charging – Sender pays fee » Some vs. all senders » How much? » Who gets the money?  Enforcement – Laws and contracts » Scope of control – national boundaries? » Precise, objective, narrow?  Administration » Exchange filtering rules » Exchange incident (abuse) reports » Coordination among Abuse desks

16. 16 D. CrockerIBM Academy / Spam Technical Issues Security Functions Make someone accountable TermFunctionIdentification Who does this purport to be? Authentication Is it really them? Authorization What are they allowed to do? Accreditation What do I think of the agency giving them that permission?

17. 17 D. CrockerIBM Academy / Spam Technical Issues Security Models Object Channel (Path) Secure Mail Secure Secure MTA MTA MTA MTA MTA MTA MTA Secure Secure Secure MTA Secure MTA MTA Secure MTA MTA MTA Secure MTA Secure

18. 18 D. CrockerIBM Academy / Spam Technical Issues SPF and Sender-ID: Path Registration oMUAMSA MTA 1 MTA 4 MDArMUA MTA 3 MTA 2 Peer MTA Assigns Sender & MailFrom Did MSA authorize MTA 1 to send this message ? Did MSA authorize MTA 2 to send this message ? Did MSA authorize MTA 3 to send this message ? 1.Authority and Accreditation of MSA and MSA domain administrators 2.MSA must pre- register and trust each MTA in path

19. 19 D. CrockerIBM Academy / Spam Technical Issues Beginning of Coherence  Validate content » DomainKeys, Identified Internet Mail (IIM) » Transit signature of msg  Validate operator » Client SMTP Validation (CSV) » Operator validates MTA  Validate Bounce » BATV  Validate content » DomainKeys, Identified Internet Mail (IIM) » Transit signature of msg  Validate operator » Client SMTP Validation (CSV) » Operator validates MTA  Validate Bounce » BATV  Reputation » CSA & DNA (CSV) » Still learning  Reporting » No candidates, yet  Enforcement » We are still learning

20. 20 D. CrockerIBM Academy / Spam Technical Issues Certified Server Validation (CSV) Assess Peer MTA MUAMSAMTA MTAMDAMUA MTA MTA Peer MTA Does a domain's operator authorize this MTA to be sending email? Do independent accreditation services consider that domain's policies and practices sufficient for controlling email abuse?

21. 21 D. CrockerIBM Academy / Spam Technical Issues Evaluating Proposals  Adoption » Effort to adopt proposal » Effort for ongoing use » Balance among participants » Threshold to benefit  Operations impact on » Adopters of proposal » Others  Internet scaling – What if… » Use by everyone » Much bigger Internet  Robustness » How easily circumvented  Adoption » Effort to adopt proposal » Effort for ongoing use » Balance among participants » Threshold to benefit  Operations impact on » Adopters of proposal » Others  Internet scaling – What if… » Use by everyone » Much bigger Internet  Robustness » How easily circumvented  System metrics » Cost » Efficiency » Reliability  Impact » Amount of Net affected » Amount of spam affected  Test scenarios » Personal post/Reply » Mailing List » Inter-Enterprise

22. 22 D. CrockerIBM Academy / Spam Technical Issues How to Choose the Future  Look at each proposal » Who must adopt it? When? » How much effort is need to administer it? » How much does it change email?  Where to look for documents » ietf.org Internet-Drafts » bbiw.net/current.html  Look at each proposal » Who must adopt it? When? » How much effort is need to administer it? » How much does it change email?  Where to look for documents » ietf.org Internet-Drafts » bbiw.net/current.html