The Top 5 Key Performance Indicators

Benchmarks focused on performance results, actions to improve results and key performance indicators show the action most responsible for improving IT compliance results is the frequency of IT audit measurement, monitoring and reporting.

Almost all (97 percent) of the firms with the least number of compliance deficiencies and the smallest number of IT security events resulting in financial harm are conducting IT audit measurements at least monthly. But what are the IT compliance performance leaders measuring? Benchmarks focused on answering this question, conducted with 620 of 1,060 organizations, show the top 5 key performance indicators (KPIs) among industry leading organizations include:

1) Databases that are out of compliance with policy and configuration standards
2) Computing systems that are out of compliance with policy and configuration standards
3) Compliance deficiencies, no matter what the source, that must be corrected
4) IT security incidents and events, no matter what their result
5) Activities not in conformance with separation of duty policies

The sixth rank-ordered KPI being measured by the performance leaders is significant deficiencies in IT controls. In contrast, organizations performing at the norm for IT compliance are measuring too infrequently and are trying to measure up to 10 KPIs, some of which are the same as the leaders, others of which are not the same KPIs. Unlike the organizations performing at the norm, IT compliance laggards are measuring too few KPIs and are not measuring frequently enough (Table 1).

Table 1: Performance Factors and KPIs for IT Compliance

Performance factors	Industry laggards	Industry norm	Industry leaders
Frequency of IT audit and measurement	Once every nine months	Once every five months	Once every 21 days
Number of KPIs being measured between attestation of controls by auditors	4	10	6
Top 5 KPIs being measured	1. IT applications out of compliance 2. IT networks out of compliance 3. IT systems out of compliance 4. Compliance deficiencies that must be corrected 5. Attestation of controls by auditors	1. Deficiencies in IT controls 2. Deficiencies in documentation 3. IT systems out of compliance 4. Activities not conforming to separation of duty policies 5. Change management test results	1. IT databases out of compliance 2. IT systems out of compliance 3. Compliance deficiencies that must be corrected 4. IT security events and incidents 5. Activities not conforming to separation of duty policies

Source: ITPolicyCompliance.com, 2006

Frequency of Measurement, KPIs and Performance Results

The benchmarks clearly show a direct relationship between better performance results (lower compliance deficiencies and lower numbers of IT security events resulting in financial harm) and the frequency of measurements. But along with frequency of measurement it is also critical to know what to measure: the key performance indicators (KPIs) that are making a difference.

However, if KPIs are only being measured once per quarter or more, the benchmarks also show that organizations will end up suffering between three and four serious security events resulting in financial harm and as many as 20 annually. Thus, while it is important to focus on the KPIs to measure, better performance results for IT compliance start with frequent measurments: at least monthly.

Guidance Recommendations:

Guidance for all enterprises, based on fact-based benchmarks, include:

• Increase the frequency of IT audits, measurements and reporting to at least monthly if not more frequently.
• Use the key performance indicators (KPIs) that are shown to result in performance improvements for IT compliance
• Do not try to measure too little, nor everything equally, it will only result in too little useful information, or too much information leading to confusion and retarded results

© IT Policy Compliance Group, 2006