Security Audits and Plausible Deniability
On Tuesday, July 19th the Troy Record published a story regarding the results of a recent security audit of the Cohoes City School District’s information technology controls. The story, which you can find online by clicking here is short and sweet, as the school district decided not to publish the results of the audit. Naturally this left readers, and more importantly taxpayers and other vested parties of the Cohoes School District in a potentially undesirable position. After all, it’s their data that is being (un)secured, and with their tax dollars.
This issue was raised in a few security circles where it was suggested that the city of Cohoes was doing more damage by leaving the unknown unknown. People have vivid imaginations, and with the proliferation of databreaches, insider fraud and other security incidents that have been cropping up in Tech Valley, the mind runs wild with the possibilities. We conjure up images of “the Duanesburg incident“, Eastern European hackers and another $500,000 financial loss. Some suggested that they’d likely already been compromised.
While I too, can imagine the possibilities, I believe there are other factors at work.
Having been in the information security business for over a decade, I have been through tens, maybe hundreds of audits. HIPAA, SOX, SAS70, NERC CIP – you name it, I’ve lived it. Some of them have been succesful. Some of them have been horrible. Most of them have been somewhere in between. But in none of these cases was I tempted to share my results with outsiders. Why?
- Audit results have no context – Auditors are paid to identify weaknesses, called findings. If you get a good auditor they will remain objective, understand the security controls in question and interpret standards, regulations and other requirements in a balanced way. But an audit represents performance results at a single point in time. It doesn’t tell the full story. It won’t give you a break because you put a new application on the wire the week before the audit. It also won’t ding you for having the same audit items show up time after time after time. That’s somebody else’s job.
- Audit results are never current – Audits take time to properly prepare and document, and your infrastructure keeps changing. In nearly every one of my audits, I had remediated critical items before the audit was published. That left my audiences with an inaccurate picture of the current state. If you are effectively moving the security needle, your last audit will be out of date before anyone sees it.
- Audit results are implicitly sensitive – Good news or bad, audit results should not be made public. Instead, we should be developing metrics that demonstrate key security performance indicators, security goals and how well we’re progressing against them, which can then be shared with broader audiences. There’s no sense in broadcasting your weaknesses, we’ve got LulzSec for that.
In summary, I feel that we do owe our constituencies a sense of how well we’re protecting their assets, especially when they’re paying the bills, and writing our paycheck. I can’t imagine keeping my old bosses in the dark about our security posture (although I’ve had supervisors who wanted plausible deniability). That being said, there are ways to obfuscate the details and still give authorized parties enough information to make business decisions and do their jobs.
Lastly, I commend the Cohoes City School District for undergoing a security audit, whether it was self-selected or mandated. I hope they used a qualified security firm (they did not use us) that was thorough and objective. I hope that they are focused on addressing their risks and not just installing new hardware and software. I hope that the results of their next audit are better than the last.
I can recommend a good security company.