Tag Archive | Incident Response

Learning From Boston

The bombs that killed three people and wounded nearly 200 yesterday are a stark reminder that the odds are stacked against us when it comes to fighting crime.

While it appears that the response of the FBI, DHS, Boston Police Department, EMS and others was reasonably coordinated and effective, these situations inevitably raise recurring questions.

  • Were we prepared?
  • Who did this?
  • Why did this happen?
  • Could this have been prevented?

As the world puts its cities on high alert, many people revisit other dramatic and horrific crises, BostonMarathonBombreminded that it wasn’t long ago that we were in a situation just like this.

Governments, embassies, corporations and other entities have spent much time, money and energy in the hours since the Boston bombings reviewing (maybe panicking) and fortifying their protections.

And while this latest horror has caused all of us to ask these questions of ourselves, there is truly only one question that matters.

  • Is what we’re doing to protect ourselves really worth it?

At a time like this, when lives have been lost, Presidents are holding press conferences and emotions are high, this question seems callous.

This is not to suggest that we shouldn’t be putting protections in place – far from it. In fact, I’d argue that all too often we as human beings would rather “take our chances” than protect ourselves proactively. It’s exactly why we see businesses getting owned by hackers every day.

But oftentimes we see the knee-jerk reactions caused by these events distracting us from the real objective. If we had just stayed the course and done a decent job of understanding our risks all along we may not have been so vulnerable in the first place.

So we mourn our losses. These tragedies seem unavoidable, and perhaps they are.

But if we don’t learn from our mistakes it is all for nothing.

Advertisements

Rethinking Security – Part 1

Two weeks ago travelers in the Austin, TX Amtrak station got a big surprise – a squad of anti-terrorism forces armed with assault rifles and specialized inspection equipment. It was just one of hundreds of [probably not so] random appearances being made by the Transportation Security Administration’s (TSA) VIPR Team all across America.

The VIPR (Visible Intermodal Prevention and Response) team is not new, in fact it was launched in 2005 after the train bombings in Madrid. Its tactics, however have been changing over time. Random appearances are part of their “new strategy”.

Since September 11, law enforcement and counter-terrorism agencies have been focusing on the areas that, at the time, appeared to have the greatest exposure. Airlines, densely populated urban areas and critical infrastructure all made the list.

Unfortunately our enemies are smart enough to strike where we our defenses are least fortified.

Enter the VIPR Team.

TSA VIPR Team Inspects Amtrak Station

TSA VIPR Team Inspects Amtrak Station

The bombing in Madrid ushered in a new phase of terrorism, and subsequently a new phase of security. Our enemies began attacking softer targets, becoming more unpredictable. It was the definition of terror. We could take a few lessons from this new thinking.

During a half-day conference in Albany, NY recently we had the opportunity to speak to over one-hundred security professionals about the current state of information security. We discussed current trends, new threats and some recently targeted organizations. When it was over, we passed around a pocketknife and about a hundred audience members joined our wolfpack.

Perhaps most important of all the topics we discussed was the failure of the things we trust most in information security today. Cornerstones like defense in-depth, antivirus and least privilege. They all sound great, but the problem is, they’re not working.

Maybe it’s because we don’t have the resources. Maybe it’s because security still isn’t a priority for many organizations. Maybe it’s because we’re not measuring performance.

Or maybe, just maybe, these things are so predictable that our enemies know exactly how to get around them.

If I were an Internet criminal operating out of unsaid country in Eastern Europe, I would have a pretty good idea of where to start. I’d know which rootkits and payloads I’d need to deliver, and how to get them to their intended targets.

I’d know pretty much what to expect once my backdoor was operational, and I’d have a pretty good idea of how to pivot around my subject’s network. I’d know how to exfiltrate my objective and which tracks to cover.

And this goes for any organization.

How could this be? It’s not because I’m that smart or have intel on every company out there. It’s because most organizations [don’t] defend themselves in the same way.

So here’s an idea; the next time an uninvited intruder shows up on your network, surprise them. Utilize a control in a different way or implement it somewhere it normally isn’t found. Take a look at all of the things you’re doing, turn them 90 degrees, spin them once and give them a kick and see where they land. If they could be effective there in a different way, consider making the change.

Predictability is a vulnerability in itself. The VIPR Team has figured this out and so can we.

The Walking Dead

It seems that everybody loves a good zombie apocalypse.

The Walking Dead has become the highest rated cable series ever. And for good reason. The thought of free gas, unlimited travel and zombie target shooting is appealing to many.

And regardless of how you feel about Rick and company’s impending doom, there is one thing that is pretty clear – they weren’t exactly prepared.

That being said, they haven’t exactly screwed everything up, either.

Let’s take a critical look at the team’s security strengths and weaknesses, zombie-style:

Strengths:

  1. Leadership – Although at times challenged by Shane, Merle, his wife, zombies and the occasional deer, Rick quickly established himself as the Incident Response Lead, and a reasonably effective one. Nevermind that he had to kill his best friend to get there.
  2. Escape and Evasion – You can’t argue with success. Even the elderly, ladies and children have made it through hordes of feasting undead. And zombie meatsuits? Brilliant.
  3. Conservation – I’ve never seen a group of newbies shoot with such deadly accuracy. Ammo may be free in the post-zombie-apocalyptic world, but why take two shots if you can get the job done in one? High ratings here.

Weaknesses:

  1. Tactics – How many times is Rick going to wander off by himself in the middle of the night in a heavily zombie-occupied zone searching for someone who likely died two episodes ago? Isn’t this guy a trained Sheriff?
  2. Communication – Seriously, Rick, next time you’re in town grab a walkie-talkie or something. Or a flare. Anything. Must you all wander about wondering what everyone else is up to?
  3. Planning – Oh and while you’re at, grab a pencil. And WRITE SOMETHING DOWN. Like where the exit is. Or where you found the beans last time. Or maybe come up with a plan. Like what you’re going to do for the next 40 years.

I have to admit I’m a huge fan of the show, and I have been since it debuted in 2010. I’d be lying if I wasn’t a little jealous – living out a zombie-apocalypse is sort of a fantasy of mine. I often wonder how I would fare. Baked beans and all.

The real lesson here is that we can’t exactly plan for everything. Preparation is important, but adaptation is critical. The ability to survive – in business or otherwise – depends on our ability to recognize our threats, weaknesses and the most effective ways to counteract them.

Bullets and beans don’t equal survival. You need people who know how to use them. And a plan.

One way or another, we’re all going to end up a Rick or a zombie.

The choice is yours.

The Desmond Breach, and Why We Haven’t Learned Anything Yet

In May of 2011, the Desmond Hotel and Conference Center in Albany, NY was compromised by an as-yet-unnamed foreign entity. Very little has been made public about the incident, and it’s possible that we will never know the true extent of the damage.

What we do know is that the credit and debit card numbers of every hotel guest from May 2011 to March 2012 were potentially compromised. At least one patron had their bank account drained.

This story was noteworthy because it was local, because it affected countless individuals and because the Secret Service was involved.

Otherwise, it was just like the countless other breaches we’ve witnessed recently.

First, The Desmond had been compromised for nearly a year and didn’t know it. The Secret Service discovered evidence of the Desmond breach during routine investigations of foreign hackers and notified the hotel of their findings. We can only assume that the compromise would still be going on today if this stroke of luck hadn’t occurred.

Second, The Desmond didn’t have an Incident Response Plan. This is an assumption on my part, but one that I am confident in, given the post-event fallout. The incident, which could likely have been better contained, grew quickly and became a public relations nightmare that lasted for days.

Third, they didn’t think this could happen to them.

This is not a smear piece. The Desmond is my favorite hotel in the area, and one that we hope to make a client someday. Unfortunately, they became long-hanging fruit. They were simply the next target in a long line of victims, a queue that grows daily.

The Desmond made the news. 99% of breaches don’t. And it seems that until an organization experiences their own incident, there is little compelling them to protect themselves.

The industry, our peers, the media, the company where you work – all are providing us an education, but we are not learning from our mistakes. Psychology 101 teaches that human beings learn best when content is relevant, entertaining and interactive. It would seem that major public data breaches tick all of these boxes.

For now it seems the only thing that’s ticked is The Desmond’s customers.

 

Force Multipliers and Why You Need Your Own Seal Team 6

The United States Military has spent a lot of time and money developing its Special Operations forces. These elite teams of security operatives are highly trained to shoot, move and communicate under duress, with little or no advance notice of their impending scenario. Most missions involve dynamic entry, assessment and neutralization of threats, all while achieving a primary objective.

Seal Team 6 is the most famous of all the Special Operations teams. These are the operatives that led the capture and termination of Osama bin Laden, the maritime rescue of American sailors on the Maersk and the recent recovery of foreign journalists held hostage by Al-Shabaab terrorists. The best of the best has handled the worst of the worst.

Imagine what you could get done if you had your own Seal Team 6.

Think it sounds crazy?

On the surface, this legendary team appears to have accomplished near-mythical tasks. Once you start to break their missions and objectives down into their most basic milestones, actions and counteractions, however, their methods become much simpler.

Like Seal Team 6, cybersecurity has become a household word and responding to security incidents has become a common occurrence. Intellectual property, sensitive data and bank accounts have never been more at risk, and detecting, containing and correcting security incidents requires planning, practice and grace under pressure. Once developed, your Computer Incident Response Team (CIRT), Special Operations Team (SOT) or whatever you call it (just don’t call it Seal Team 7, that’s taken) will give your organization a “force multiplier”.

Force multipliers are assets that give your organization a strategic advantage and increase the effectiveness and efficiency of overall operations. These assets, or teams multiply the force of your organization because they carry out tasks that would normally require much greater resources to accomplish. This is possible due to the highly trained and focused nature of the team.

And yes, you can have your own.

Here’s what you need:

  1. Find the right operatives – Incident responders are born, not made. This isn’t completely true, but it takes a certain mindset and attitude to gain situational awareness, remain calm and lead when an organization is under attack. These skills are tough to teach. You probably already know who your operatives are. If you don’t, stop here and look for a qualified security provider that already delivers Incident Response services.
  2. Select the right weapons – A good shooter can make a bad gun shoot well. You don’t need the most expensive security tools, but you do need to make what you’ve got work effectively. If you’re about to embark on a digital forensics mission, you better have the right tool in your bag. And carry a backup. Your tools should meet their requirements and work consistently.
  3. Train, train, train – Training is the most important of all, and it should incorporate the following:
    1. The basics – Every Navy Seal is required to qualify with a rifle every year, regardless of his or her rank. It’s part of being a Seal. The ability to shoot, move and communicate makes our Military the greatest on Earth. Your CIRT resources should understand the basics of threat modeling, analysis, communication and tool manipulation. No exceptions.
    2. Scenario drills – Your training should incorporate regular attack and counterattack simulations – these should be as real as possible. Good penetration testing is critical, but tabletop exercises and other drills are important parts of the program. Training should incorporate “unplanned” scenarios, performance assessments and any analysis that will help the team perform during a real situation.
    3. Bugout – Training should include handling worst-case scenarios, or what the Military calls SHTF. When things really get sideways, you need to know what to do, when to do it and who’s going to do it. Crisis Management should integrate with your Incident Response Plan.

Regardless of what industry you’re in or what size your company is, at some point you will become a statistic (if you haven’t already). A Special Operations team specifically trained and ready to handle incidents could mean the difference between an achieved objective and a botched mission.

You may not be hunting terrorists or saving foreign diplomats, but your CEO might just give you a Medal of Honor.

What I Would Do if I Was Zappos

The Zappos hack this week made national headlines for a number of a reasons.

First, Zappos, a subsidiary of Amazon.com is a major brand recognized as a leading online footwear retailer. You don’t need to be female to know that Zappos sells just about every make and model of sandal, Skecher and pump known to man. And woman. And if you’re a woman there’s at least some chance that Zappos is your browser home page. I’ve seen it happen.

Second, the scale of the breach was massive. E-mail addresses, billing information, names and partial credit card numbers for an estimated 24 million individuals, making it one of the largest databreaches in recent history. The value of this data on the black market, using today’s cybercrime figures is in the tens of millions of dollars.

But in many ways the Zappos databreach isn’t unlike the countless other incidents we’ve witnessed lately. Which makes me wonder if we’re operationalizing information security this year any differently than we did last year. Or the year before that.

Of course history makes a great teacher, and this holds true for security, as well. And while they’re still grinding through the logs and other forensic evidence of this attack, there are some clear lessons to be learned here.

This is what I would do if I were Zappos:

  1. I would learn to be better at Public Relations – Because they expected a deluge of phone calls related to the hacking, Zappos said that they were temporarily turning off their phones, instead responding to inquiries by e-mail. “If 5% of our customers call, that would be over 1 million phone calls, most of which would not even make it into our phone system in the first place,” the company’s e-mail to employees said. Now I’m no PR expert, but if I just pissed off 24 million of my customers by exposing them to identity theft, disabling their preferred mode of communication might not sound like a bright idea. Perhaps they missed the beating that RSA and Sony took for their PR guffaws.
  2. I would learn to be better at helping my customers during a crisis – Have you ever tried getting your organization of 150 people to change their password? Exactly. Now multiply that times 160,000. It’s the equivalent of sucking an Olympic size swimming pool through a McDonald’s straw. In this day and age, there should be a way to programmatically reset customer passwords, provide them a means for securely accessing the new password, or simply leaving the account disabled until such time that the customer wants to use the account again. I’m betting that a significant percentage of those 24 million accounts are inactive in the first place.
  3. I would learn to better protect sensitive information – Zappos was warned daily – possibly more frequently – by The NY Times, The Washington Post, the GreyCastle Security blog and other global media outlets that they were going to be hacked, but they proceeded to store names, billing addresses, e-mail addresses and partial credit card numbers together, in one database, potentially on one server, packaged neatly for the next disgruntled employee, hacker and other miscreant. I’m guessing that Zappos didn’t have the budget, the time or the resources to secure this information appropriately. It wasn’t a priority. Until it was too late.
  4. I would learn to be a security evangelist – Now that I’ve been owned by hacker(s) unknown, exposed my customers to incalculable risk and started racking up unnecessary Incident Response bills, I would help other companies avoid what just happened to me. “We’ve spent over 12 years building our reputation, brand, and trust with our customers. It’s painful to see us take so many steps back due to a single incident,” the company’s CEO said. The biggest problem in our industry is not a technical one, it’s a psychological one. As long as Company XYZ believes that they have nothing worth protecting, and that this can’t happen to them, we’ll continue to experience these issues.

It’s not fair to single out Zappos. This blog could be written about thousands of other organizations. But so far security hindsight continues to be nearsighted.

Maybe Zappos should start selling eyewear.

Run for Your Lives

This past Saturday I woke up early and suddenly found myself running from bloody, muddy, brain-hungry zombies.

No, the world hadn’t suffered a raging viral infection. And no, I wasn’t a movie extra. It was the first annual Run for Your Lives Zombie 5K race near Baltimore, MD. There were thousands (OK maybe hundreds) of zombies to avoid, a dozen obstacles to overcome and endless fields of mud. There was blood. Whole pools of it. And there were several “teachable moments”.

Now in many ways, I feel like I’m better prepared than the next guy for the impending Zombie Apocalypse – my cardio level is above average, I prefer moving around at night and I love me some good baked beans. I also consider myself a bit of a survivalist, and I keep an ample supply of batteries, bleach and duct tape ready to go for when things get apocalyptic.

Yours Truly, Muddy and Bloody, but Alive

All that being said, this weekend’s events reminded me that there’s no way to prepare for everything. Despite the semi-lighthearted nature of the 3.1-mile obstacle course, I found myself surprised – even shocked – on several occasions. Midway through the race I found myself deciding between diving into a muddy lake filled with 55-degree water or being attacked by a crazed horde of killer undead. This particular teachable moment taught me that hypothermia may be for a while, but dead is for ever.

So what does all of this have to do with information security?

Just like the doomsday scenarios that scientists, religious zealots and Al Gore all predict for the human race, there is no way to prepare for everything in information security. In fact, the best preparation may be in preparing to be unprepared.

The harsh reality is, most businesses have already been compromised, whether they know it or not. Yesterday my company met with yet another organization who has been the victim of cybercrime. Not only did this business suffer major losses, but two months later they are still unsure if the money-stealing malware has been eradicated.

Having an Incident Response plan is an important part of running a successful business. Detection of malware and anomalies, containment of incidents and processes for forensics investigations and business resumption should be regular discussions for all management teams. If you haven’t already done so, add a chapter to your plan that accounts for the “unexpected”. Failing to plan is planning to fail.

Hindsight, as they say is 20/20. I’ve already thought of a few things that I’ll do differently to be better prepared for next year’s race. Luckily, it’ll only cost me $57 to learn from my mistakes.

If you’re a business without an Incident Response plan, it may be a little more expensive.

Drones, Phones and Automobiles

There’s been a lot of chatter over the past week regarding the alleged breach of U.S. Military unmanned aerial vehicles, or at least the networks that they use to transport video streams back to Operation Command Centers in Nevada, or wherever their 19-year old operators and joysticks are positioned.

The media have speculated that a virus, introduced from external media penetrated critical networks and was doing bad things. The Government has done its best to misinform and parry, suggesting (and confirming in some way?) that whatever malware did make its way onto its networks is just a nuisance. It was even suggested that the malware was the military’s own, a weapon that somehow escaped the labs.

Of course, the Department of Defense doesn’t comment on classified networks, so there’s a good chance we’ll never know the real story.

The real question we should be asking is, who cares?

I don’t mean to sound glib, but if Uncle Sam says it’s not a problem, maybe it’s not? After all, even if video streams from Unmanned Aerial Vehicles (UAVs) were intercepted, intended targets likely wouldn’t have time to escape before they were made into Afghani pottery anyway, so what’s the big deal? Perhaps, you say, the enemy is collecting intelligence on UAV flight patterns, so that it can predetermine and thusly avoid detection. Perhaps.

Or perhaps this story is more important to the media than it is to the masses? Not unlike the incessant droning (sorry) on about malware being delivered to Android-based phones these days. It’s a [nearly] proven fact that over 10% of Android applications do things we don’t want them to do, whether they’re harmlessly hijacking GPS coordinates and personal information to push personalized ads to your browser or they’re outright malware stealing online banking credentials. Here’s the thing – people don’t care. Androids – and their applications, stuffed with privacy-violating “features”, are flying off the shelves.

And when will it be time to start vulnerability scanning our cars? We’ve already seen Subaru Outbacks compromised using integrated Wi-Fi, and many vehicles braking systems are vulnerable to attack. And let’s face it – OnStar is a mass botnet just waiting to happen. Don’t look for the “Hardened Security Vehicle” checkbox at your local auto dealer – they don’t care either.

Perhaps the Department of Defense is giving us the cold shoulder because they’re a little embarrassed. Perhaps they’ve declassified this information because it’s helpful for the information security community. Or perhaps it’s because redirection and confusion is all part of their Computer Incident Response Team (CIRT) procedure.

Or perhaps they’re just teaching us a lesson. If we can care so much about a remote control airplane flying over a desert 7,000 miles away that we’ve never seen and will never have any effect whatsoever on us, why can’t we care about the stuff we use every day?

What Next, Hordes of Locusts?

It is said that any threat with enough resources or motivation will eventually find a vulnerability in a system. As I watched the overflowing Hudson River decimate the park, marina and restaurant behind my office, that theory became a staggering reality.

On Sunday, Troy, NY experienced its worst flooding since 1977. With record rainfall from Hurricane Irene, many area dams were at risk of failure and creeks and rivers were over their banks. Homes were flooded and vehicles were destroyed. Boats were lost from marinas, washed down the river along with tons of trees, barrels and other debris. The crowds of people who had gathered in front of the now-underwater Dinosaur BBQ added to the chaos.

Today however, just hours after the event, our city is already getting back to normal. Walking through the areas hardest hit by the flooding, it’s clear that recovery is well underway. The crowds have dissipated, the police tape is slowly disappearing, and businesses are getting back to normal operations. This recovery is occurring in large part because the first responders, law enforcement, FEMA and DHS personnel that responded to the disaster were prepared.

No one could have anticipated that Upstate New York was to be hit by both an earthquake and a hurricane in the same week. In fact, we were probably more likely to see a unicorn. But a good Incident Response plan assumes that we won’t necessarily have all of the intelligence, resources or time that we need to counteract a threat. A good Incident Response plan can also mean the difference between a business returning to normal operations, and a total disaster.

Security incidents come in all shapes and sizes. One day you may be responding to a malware outbreak, the next day you may be responding to the $250,000 that has been siphoned out of your company’s bank account. Irrespective of the type of organization, a good Incident Response plan should address the following:

  1. Containment – Whether isolating the latest worm or preserving evidence of a databreach for litigation, your containment strategy will vary depending on the incident. The most important considerations in this step are minimizing damage and neutralizing the threat without affecting your downstream mitigation options. It is important to understand your threat before enacting a containment strategy – an active shooter requires different counteractions than a perimeter attack.
  2. Mitigation – Once the threat is contained, it should be addressed. Again an understanding of the threat is important. In many instances, expertise in evidence preservation and chain of custody is critical, particularly in situations where legal proceedings are anticipated.
  3. Recovery – Rebuilding systems, restoring from backups or providing counseling for employees are all essential steps in the Incident Response process. Effective recovery requires advance planning and preparation, but it will provide significant returns if done properly.

Lastly your Incident Response plan should be governed by policy and handled by a team specially trained in response procedures. It’s not unusual to outsource some of your incident handling efforts. In fact, asking an internal team to perform technical forensics tasks or to understand the intricacies of evidence preservation could be like asking the Pakistani Army to capture bin Laden – it could get very messy and leave you without the desired outcome.

I had lunch in downtown Troy today, and if I hadn’t witnessed the flooding firsthand I would’ve never guessed that large parts of the city were underground 24 hours prior. Thanks to preparation, a trained team and a good Incident Response plan, today’s pizza tasted just like any other day.

%d bloggers like this: