This is primarily a political, not technical issue. Addressing it would require virtually every service provider to agree on a set of changes and implement them at the same time and the level of coordination (and authority) just doesn't exist. A central
issue in spam is spoofing, and even addressing this one aspect through SPF is unevenly and inconsistently applied. If we can't get mail admins to uniformly use this standard, then there's not much hope of doing anything more stringent and broadly based. Today,
people have pretty much just resigned themselves to the fact that spam is something they have to cope with (via filters, whitelists, graylisting and so on), rather than viewing it as an important issue that requires fundamental change.
Or to put it another way, we're so focused on addressing the symptoms of spam, there's no will to actually take the measures necessary to treat the underlying disease (i.e.: addressing inherent flaws in the protocol).
mstefan: exactly right, it's the willing ness or lack of to take the steps needed and to pony up to the cost also.
Dave: spammers are the problem in large part, the REAL way to combat the issue is not in filtering and testing.
the first real step to halt it is to make the replacement of SMTP be much more strict in the format of the email message - the headers esp.
and to setup the system to make it possiblt to track a message back to the sending server that put in on the network.
if you can see that users get traffic from a given host and that 99% of that traffic gets rejected by the users and the traffic is massivly inbound to your server then you can throttle them or block them with better tracking of why.
*IF* that model was used on the majority of mailservers then the bulk senders would find they would become less and less effective and would have to "Play fair"
while a good IT team can find the sources of the bulk inbound junk the current smtp syste make that to hard to really do and still be cost effective. so the alternative is the spam rating and blocking we see today.
it's cost effective and scales well.
but very far from perfect.
I would start with a simple upgrade to SMTP that would allow servers to "opt in" on a new message format based on a strict XML dialect.
that and some logic that says in short:
if a message comes in that you can not verify that it came from the server and identity that it claims to be from then you block it and report it as suspect.
just like we cut back on open relay servers a few years back... if a server in say north korea is sending messages that say they came from a hotmail account STOP right there and do not accept or deliver that message. also report that server to a server abuse
as more reports come in more servers can block that sender.
yes some of this is the same as the RBL / DNS blacklist systems but i start with the idea of halting the bogus message headers.
sending server and sending domain and sending account all need to be evaluated to halt the masking of the bulk sender.
expose them as what they are and make them take complaints for real.
and look at ways to account for the cost to the reciving server ....
i bet that at least 25% of the global cost of the internet is due to bulk / junk email
think what we could all do with 25% cash back on that scale....