Saturday approx 23:00 : A customer’s e-mail account becomes compromised, and a UK IP address starts to send out spammy messages.
Monday 04:00 : sending limits are reached, and our systems shut down the account from sending any more mail and log the event for attention during working hours.
Monday 06:30 : Krystal staff respond to the alert, the whole account is suspended, and the owner is contacted.
Monday 13:00 : Krystal staff work hard to remove/delist the server concerned from a number of RBLs (Real Time Blacklists). By lunchtime, all of the Blacklists monitored by http://mxtoolbox.com/ (the only ones that really matter) are clean.
This should really have been the end of it. Our servers send hundreds of thousands of messages every day, so it’s a given that from time to time someone’s site will be exploited, or their email login details will be compromised. Any rational hosting company knows this, and so they don’t penalise their customers unduly when these things happen.
Likewise, most Blacklists know this too, and they act quickly to remove servers from blacklists once notified that problems have been dealt with. There are exceptions of course, like SORBS, who try to charge companies for removing an IP address from their blacklist. Quite why spending money with a company should make your server appear less spammy to the world would appear to demonstrate the triumph of carpet-bagging over logic.
Anyway, I digress.
The only remaining fly in the ointment is Cisco’s Ironport Senderbase system which is still indicating our server as having a poor reputation, despite the problem being fixed.
Tuesday 14:00 : After waiting 24hrs for Cisco, I send them an email. It goes like this.
We operate dozens of servers from our UK datacentre, and one of them which houses several hundred customers has been suffering from poor reputation since Sunday night. Despite us being delisted by all other RBLs within 12hrs your URL
appears to be the primary reason for ongoing difficulties being experienced by our clients using ceres.uksrv.co.uk.
The customer who’s account was responsible has been deleted. The problem has been rectified.
I would be most grateful if you could expedite return of 22.214.171.124 to “GOOD” status without delay.
The reply comes back:
The most recent complaint we have on file for IP 126.96.36.199 is dated Saturday, October 06, 2012 4:37:08 PM -0700. If the spam problem is fixed as you believe it to be, then there should be no further complaints received. In general, once all issues have been addressed (fixed), reputation recovery can take anywhere from a few hours to just over one week to improve, depending on the specifics of the situation, and how much email volume the IP sends. Complaint ratios determine the amount of risk for receiving mail from an IP, so logically, reputation improves as the ratio of legitimate mails increases with respect to the number of complaints. Speeding up the process is not really possible. SenderBase Reputation is an automated system over which we have very little manual influence.
In the meantime, if there are recipients whom you cannot contact, we would recommend contacting the ISP involved to request temporary whitelisting or you can always arrange to contact the recipient via alternative means.
Now – blocking a server for sending spam is one thing – to continue blocking it for “just over one week” is far from efficient, and penalises otherwise genuine mail senders for no good reason. I replied:
How does Cisco determine the amount of “good” email we send from this particular server, or even the volume of email this server sends when this traffic will not even traverse your systems?
Forgive me impertinence, but the basis of your explanation appears false.
While I am still hopeful you can assist us, I doubt your system can tell the difference between a server which sends 100 spams amongst 1000 good messages and one that sends 100 spams amongst 1’000’000 good messages. We are in the latter situation, and are being treated with undue harshness by your system.
This devalues our service, and I would argue also devalues your service as your bona fide customers are unable to receive email from our bona fide customers.
Wednesday 15:00 : As of right now, Cisco have still not responded to us. As a result we have moved a number of customers to another server.
It is unlikely Cisco will listen to Krystal – but perhaps they will listen to their customers. If you use Cisco’s service and find your users are regularly unable to receive email from various sources, then perhaps now is the time to vote with your feet and quit using slow, un-responsive and over zealous Blacklist providers.