Recurring reporting made easy?

Recurring reporting made easy?
Most of us have to generate recurring reports for the state of security, system uptime or general performance at our respective work places. Solid, clear reports many not appear to be one of the foundations that security is built on, but many voices would strongly disagree and management is usually one of them. You have to clearly report on data, trends, issues and events from reasons ranging from simple best practice diligence to justifying the reason IT security is critical the your business and everything in between. Good reporting won’t get you fame and fortune, but can provide a historical context, trending and a clear value of your efforts. Developing these mystic skills comes down to defining the what, why, when and who these report are being read by, having conversations with those receiving the reports to tailor them to requirements and finally plenty of practice. Recently I've had to review a fair number of monthly reports from different sources and I'm always on the lookout for great ways to steal ideas from others and claim it as my own, er, learn and adapt from. I thought I'd share some observations on what I liked and what worked for me. One quick note; these were all from the private sector, so I haven't reviewed any government or public sector reports but the theory is hopefully the same. Rather than suggest some sort of universal template to fit every situation, I've observed applying the concepts of these four words to reporting. Using these concepts correctly just made being able to effectively understand and absorb the information so much easier: Clarity Consistency Concise Colourful Here’s my interpretation, from hours of wading through those reports, to avoid your next report being used to test the speed of the office shredder. Clarity Know you audience and write the report for them so apply the correct level of terminology, detail and as general groups think: your peer group, your boss, management or the general public. No weird or wacky fonts. I have no issue with standard bolding, italics or large cases but gothic or super fancy fonts are distracting and slow the flow of reading. Try to avoid jargon and the dreaded, unexplained three-letter acronym (TLA) as well. I spent an hour attempting to work out what ARE, ARM, POP and HIS meant and despite some pretty good guesses was completely wrong. The author had some team-only terminology of bespoke systems, so I may have well played on a lottery instead. Avoid complex language and stick to plain language and terms, if possible. I do enjoy attempting to slip defenestration in to certain reports, but if the reader has to look up the word it loses that “wit” Reports aren’t about showing off, so keep the language practical and uncomplicated. Consistency How can someone review previous or future reports without the same points of reference? Using the same template, with the same headings for the same recurring report makes it a breeze to see trends and the reader gets use to what to expect to see. If you are using tables or charts keep using the same data, not random facts and figures. Keeping people guess what might be on next month report is an interest approach, but one that much force them to form a “that doesn’t make any sense” and “but, but the last report does mention this…” lynch mobs. Concise As technical people we love the details, but those recurring reports many not want every bit of detail, so summarise. You can refer to data, but sixteen pages of eight point, densely compresses text of server alerts messages is hard to read, let alone understand. Keep away from using opening statements that could come from a novel “It was a dark and cold night. The winds were howling. Lightening forked wickedly across the skies, spearing the landscape surrounding the datacentre. With a final, silence choked gasp the UPS failed, mere seconds before the haggard night shift crew took their seats.” UPS failure at datacentre X 23:45 really covers that more effectively. Colourful This isn’t a reference to fruity language, although a smattering of four letter words will get the reader’s attention but for all the wrong reasons. I’m, of course, meaning those eye catching images stuffed in to reports to make assimilating complex data simple and quick. Have to be a bit careful with this, but good use of colour in tables or charts can draw in the reader. Well executed charts and tables can make absorbing complex data much easier and get across points very quickly. Use clashing colours or, just as bad, tones that blend in together making impossible to work out what’s happening are annoying and distract from the reader’s ability to understand the report or drain the will to continue through it. Wrapping up, take pity on those you’re writing a report for and try to avoid making it one of those government forms Ph.D’s struggle to comprehend. A book that I really enjoyed reading was from Jon Moon [1], an English author, who is determined to rid the world of confusing, poorly written paper work of all ilks. For those wondering well what should they could be writing these reports on have a look through SANS critical security controls [2], which my fellow handlers have spent this month expanding on and turn the points in to items to report back on about your environment. As always, if you have any suggestions, insights or tips please feel free to comment. [1] http://www.jmoon.co.uk/book.cfm [2] http://www.sans.org/critical-security-controls/ Chris Mohan --- Internet Storm Center Handler on Duty	Chris 105 Posts ISC Handler Oct 25th 2011
Thread locked Subscribe	Oct 25th 2011 9 years ago
Along this line, I'd love a post/discussion on what metrics are useful to publish as a report on a weekly or monthly basis. Malware infections? Time to remediate infected machines? There has been disagreement amongst peers on the most useful routine reports to provide and I'd be interested in what SANS folk have to say on the matter.	Anonymous
Quote	Oct 25th 2011 9 years ago
having to write many a report these are great insights and you've highlighted the problem with using acronyms, which is them not being defined before it is supplied/identified, but I wouldn't agree with trying to avoid their use. an acronym can make the writing and flow of a report more succinct, especially if you have to highlight a particular system/solution/issue numerous times in the same statement. no one likes to continually read the same words over and over again. the use of an acronym can help to break up the redundant rhythm of a particular paragraph or statement making for a better reading experience.	SonofSpiff 2 Posts
Quote	Oct 25th 2011 9 years ago
Which metrics you include in a report may also be influenced by whatever you deem as a particular risk to the organization. For instance, I've got net-snmp reporting the number of log messages matching certain regex patterns and then graph those with cacti so I can show the number of login failures on an ssh server (I show incorrect passwords and incorrect usernames as separate graph items). It's a way to remind management of the need for things like 3rd factor logins. I also graph the number of packets rejected by firewalls at various locations just to track trends (and I check them frequently just to look for anomalies - a big spike in rejected outbound traffic may mean a compromised or mis-configured system, for instance).	Brent 125 Posts
Quote	Oct 25th 2011 9 years ago
One important metric we track is number of machines with current definitions. It almost always gets overlooked, management thinks that if they have AV on all the systems, everything is good. Another metric is how many machines currently on the network have recently checked into the AV console. If the machines are on the network but not currently reporting in, again, they can't receive updates, can't report that they're (possibly) infected, and can't be remediated remotely.	Brent 1 Posts
Quote	Oct 26th 2011 9 years ago

Most of us have to generate recurring reports for the state of security, system uptime or general performance at our respective work places.
Solid, clear reports many not appear to be one of the foundations that security is built on, but many voices would strongly disagree and management is usually one of them. You have to clearly report on data, trends, issues and events from reasons ranging from simple best practice diligence to justifying the reason IT security is critical the your business and everything in between. Good reporting won’t get you fame and fortune, but can provide a historical context, trending and a clear value of your efforts. Developing these mystic skills comes down to defining the what, why, when and who these report are being read by, having conversations with those receiving the reports to tailor them to requirements and finally plenty of practice.

Recently I've had to review a fair number of monthly reports from different sources and I'm always on the lookout for great ways to steal ideas from others and claim it as my own, er, learn and adapt from. I thought I'd share some observations on what I liked and what worked for me.

One quick note; these were all from the private sector, so I haven't reviewed any government or public sector reports but the theory is hopefully the same.

Rather than suggest some sort of universal template to fit every situation, I've observed applying the concepts of these four words to reporting. Using these concepts correctly just made being able to effectively understand and absorb the information so much easier:

Clarity
Consistency
Concise
Colourful

Here’s my interpretation, from hours of wading through those reports, to avoid your next report being used to test the speed of the office shredder.

Clarity

Know you audience and write the report for them so apply the correct level of terminology, detail and as general groups think: your peer group, your boss, management or the general public.
No weird or wacky fonts. I have no issue with standard bolding, italics or large cases but gothic or super fancy fonts are distracting and slow the flow of reading.
Try to avoid jargon and the dreaded, unexplained three-letter acronym (TLA) as well. I spent an hour attempting to work out what ARE, ARM, POP and HIS meant and despite some pretty good guesses was completely wrong. The author had some team-only terminology of bespoke systems, so I may have well played on a lottery instead.
Avoid complex language and stick to plain language and terms, if possible. I do enjoy attempting to slip defenestration in to certain reports, but if the reader has to look up the word it loses that “wit” Reports aren’t about showing off, so keep the language practical and uncomplicated.

Consistency

How can someone review previous or future reports without the same points of reference? Using the same template, with the same headings for the same recurring report makes it a breeze to see trends and the reader gets use to what to expect to see.
If you are using tables or charts keep using the same data, not random facts and figures. Keeping people guess what might be on next month report is an interest approach, but one that much force them to form a “that doesn’t make any sense” and “but, but the last report does mention this…” lynch mobs.

Concise

As technical people we love the details, but those recurring reports many not want every bit of detail, so summarise. You can refer to data, but sixteen pages of eight point, densely compresses text of server alerts messages is hard to read, let alone understand.
Keep away from using opening statements that could come from a novel “It was a dark and cold night. The winds were howling. Lightening forked wickedly across the skies, spearing the landscape surrounding the datacentre. With a final, silence choked gasp the UPS failed, mere seconds before the haggard night shift crew took their seats.” UPS failure at datacentre X 23:45 really covers that more effectively.

Colourful

This isn’t a reference to fruity language, although a smattering of four letter words will get the reader’s attention but for all the wrong reasons. I’m, of course, meaning those eye catching images stuffed in to reports to make assimilating complex data simple and quick. Have to be a bit careful with this, but good use of colour in tables or charts can draw in the reader. Well executed charts and tables can make absorbing complex data much easier and get across points very quickly. Use clashing colours or, just as bad, tones that blend in together making impossible to work out what’s happening are annoying and distract from the reader’s ability to understand the report or drain the will to continue through it.

Wrapping up, take pity on those you’re writing a report for and try to avoid making it one of those government forms Ph.D’s struggle to comprehend. A book that I really enjoyed reading was from Jon Moon [1], an English author, who is determined to rid the world of confusing, poorly written paper work of all ilks.
For those wondering well what should they could be writing these reports on have a look through SANS critical security controls [2], which my fellow handlers have spent this month expanding on and turn the points in to items to report back on about your environment.

As always, if you have any suggestions, insights or tips please feel free to comment.

[1] http://www.jmoon.co.uk/book.cfm
[2] http://www.sans.org/critical-security-controls/

Chris Mohan --- Internet Storm Center Handler on Duty

Chris

105 Posts
ISC Handler

Oct 25th 2011

Along this line, I'd love a post/discussion on what metrics are useful to publish as a report on a weekly or monthly basis. Malware infections? Time to remediate infected machines? There has been disagreement amongst peers on the most useful routine reports to provide and I'd be interested in what SANS folk have to say on the matter.

Anonymous

having to write many a report these are great insights and you've highlighted the problem with using acronyms, which is them not being defined before it is supplied/identified, but I wouldn't agree with trying to avoid their use.

an acronym can make the writing and flow of a report more succinct, especially if you have to highlight a particular system/solution/issue numerous times in the same statement. no one likes to continually read the same words over and over again. the use of an acronym can help to break up the redundant rhythm of a particular paragraph or statement making for a better reading experience.

SonofSpiff

2 Posts

Which metrics you include in a report may also be influenced by whatever you deem as a particular risk to the organization. For instance, I've got net-snmp reporting the number of log messages matching certain regex patterns and then graph those with cacti so I can show the number of login failures on an ssh server (I show incorrect passwords and incorrect usernames as separate graph items). It's a way to remind management of the need for things like 3rd factor logins.

I also graph the number of packets rejected by firewalls at various locations just to track trends (and I check them frequently just to look for anomalies - a big spike in rejected outbound traffic may mean a compromised or mis-configured system, for instance).

Brent

125 Posts

One important metric we track is number of machines with current definitions. It almost always gets overlooked, management thinks that if they have AV on all the systems, everything is good.

Another metric is how many machines currently on the network have recently checked into the AV console. If the machines are on the network but not currently reporting in, again, they can't receive updates, can't report that they're (possibly) infected, and can't be remediated remotely.

Brent
1 Posts