SANS: Your Mileage will Vary

The top 20 security controls carried by SANS and the Center for Stategic and International Studies are sometimes mentioned to me by practitioners and managers of information security as a be-all and end-all for information security, as in “this is all we have to do to be safe and sound.”

The list is incomplete and inaccurate

I wish it were that simple, but it’s not. And anyone who’s convinced it is, is fooling themselves and those around them.

The list of the top 20 controls contains the following:

  • Inventory of Authorized and Unauthorized Devices
  • Inventory of Authorized and Unauthorized Software
  • Secure Configurations for Hardware and Software on Laptops, Workstations, and Servers
  • Secure Configurations for Network Devices such as Firewalls, Routers, and Switches
  • Boundary Defense
  • Maintenance, Monitoring, and Analysis of Security Audit Logs
  • Application Software Security
  • Controlled Use of Administrative Privileges
  • Controlled Access Based on Need to Know
  • Continuous Vulnerability Assessment and Remediation
  • Account Monitoring and Control
  • Malware Defenses
  • Limitation and Control of Network Ports, Protocols, and Services
  • Wireless Device Control
  • Data Loss Prevention
  • Secure Network Engineering
  • Penetration Tests and Red Team Exercises
  • Incident Response Capability
  • Data Recovery Capability
  • Security Skills Assessment and Appropriate Training to Fill Gaps


You can find more about these controls at both SANS and the Center for Strategic and International Studies at the following locations:

SANS: 2o Critical Security Controls

CSIS: Twenty Important Controls for Effective Cyber Defense and FISMA Compliance


Don’t get me wrong.

I think SANS and the Center for Strategic and International Studies have done everyone a favor by focusing on these controls. And, I actually think the list is a good one: with many of the controls on it being among those that are most automated by the best performing organizations among the 6,000 plus that have participated in benchmarks with the IT Policy Compliance Group.

But, the problems with the list are these:

1) it’s an incomplete list that’s primarily focused on  “technical controls”

2) there’s no assessment that one control may be better suited than another

3) the concept of managing risk is missing from the list

4) the focus and claims are somewhat over the top for one of the lists



The list happens to call attention to what’s also known as technical controls: those things you can place in the IT environment to manage risks directly related to technical procedures involving computer networks, systems, applications, data and software. What the list completely ignores is the people aspect of controls, or procedural controls, and policies that drive what the organization is trying to accomplish, direct behavior and which provide the ground rules defining what’s acceptable and what’s not.

A more complete listing of policies, procedures and controls can be found in the IT Policy Compliance benchmark research report entitled: Automation, Practice and Policy for Information Security.


Your mileage will vary

In addition to being incomplete, you’ll find the benefits of implementing any of these controls will vary: some controls will deliver far larger gains in reducing risks, while others provide less benefit in reducing risk. An example is the implementation of malware defenses versus account monitoring and control. Both are shown to reduce risk, but of the two, the use of malware controls is more important.

Differences in effectiveness for a wide range of controls, procedures and policies can be found in the same IT Policy Compliance report: Automation, Practice and Policy for Information Security.


Risk is missing

The entire concept of risk is completely missing from the lexicon. Why are controls implemented for IT? Ultimately it’s to manage the risks involved in using automated information systems and modern-day networking to fulfill your mission as an organization. But, the authors of this list present a list of primarily technical controls as universal and applicable to all organizations from small businesses to the largest of global multinationals, and from small town governments to the largest of nation-state agencies.

Focus and Claims

The focus and claims of the two are also very different between the two lists.

The report issued by the Center for the Strategic and Internal Studies is focused on Cyber-defense and Homeland Security among Federal agencies, while the one from SANS is focused on any business of any size and in any industry. It is very clear what the proposed focus and claims (FISMA compliance) are from this policy-think-tank.

But, the SANS prologue for its version of the report does not limit it to Federal agencies. Rather the language makes the list appear to be something for all organizations in every industry and size. In fact, the prologue for the SANS list states the automation of the Top 20 Controls will radically lower the cost of security while improving its effectiveness.  I have no doubt implementing more of these controls will increase the effectiveness if effectiveness  is measured by such things as reductions in severity-level 1 incidents, loss of theft of data, downtime due to IT failures, the loss or theft of data, or audit deficiencies found in IT.

But, the claim about radically lowering the cost of security is ludicrous. In fact you will spend more, not less, to implement and manage more controls. The benchmark results across more than 6,000 organizations proves you will spend more. And, the benchmark results also prove that your effectiveness will indeed “radically” increase through sharp reductions in risk measures.

So, how do the two proponents claims score on the old truth-o-meter?

To some extent they both do, but what is missing from the report issued by both organizations tends to obfuscate some rather critical controls – as well as totally ignore the concept of risk in its entirety. So I would rate the truth-o-meter for these aspects as somewhat true – due in large part to what is not covered or is missing. But the casual “it’s for everybody at all times in all circumstances” approach – along with the unjustifiable claim of saving money – from SANS rates this portion of the claims to be mostly untrue. In summary if you assume the SANS approach is gospel, you are likely to experience a letdown.

Let us know what your experience is.

Leave a Reply

You must be logged in to post a comment.