Last Updated: 2012-06-22 18:37:59 UTC
by Kevin Liston (Version: 1)
This initially started off as a diary entry about creating final reports during the Lessons Learned phase of incident response, but I kept referring back to "the timeline" and realized that it needed an entry of its own.
Investigation is all about answering the who, what, where, when, why and how questions. One indispensable tool for organizing this process is the timeline. It can be as simple as a quick sketch in a notebook or as complex as an interactive infographic. It will start off as an un-sorted, un-structured collection of data and if curated properly it will become a tool that will unify your investigation efforts, help identify gaps, and enable you to communicate clearly to management.
What Makes up a Timeline?
The core element of a timeline is the "event." An event can be described as a set of:
Time-- either precise, or uncertain, (e.g. before Tuesday)
Place-- physical location, IP address, file-location, etc.
Person-- the actor, known or unknown
Action-- the "what happened" part of the event
Direct object-- if the "what happened" happened to someone or something, this is that someone or something.
Additionally, as events are processed you will want to enrich their entries with:
- Tags-- events will be tagged to help pull out the important events during different stages, as well as help create documents needed later in the investigation
- Evidence-- it's cumbersome and unwieldy to just drop a raw log entry into a timeline, but you will want to provide evidence that an event occurred.
Dealing with Uncertainty
Investigations are constantly dealing with uncertainty. At the beginning of an investigation you have very little information and a seemingly uncountable list of unanswered questions. It's okay to place empty squares on your time line that say things like "victim was web surfing" or leave a blank in one of the time/place/person fields of an event. This helps you call out what you don't know and will help refocus your resources or re-prioritize your efforts. Being able to inventory your "unknowns" is probably a better measure of the progress of your investigation than counting the Gigabytes you've acquired, the number of lines of log files you've analyzed, or other metrics of effort.
Where to Start
When confronted with the empty page, you can start with "when you were informed." This will form the basis of a response timeline. Next you can add events from the report that came in (e.g. the IDS alert, or escalation from your NOC, or 3rd party report.) Gathering events will come naturally as you ask questions about the incident and data comes in. For an example of semi-automated timeline creation I recommend a read of Rob Lee's "SUPER timeline" (http://computer-forensics.sans.org/blog/2011/12/07/digital-forensic-sifting-super-timeline-analysis-and-creation)
Using the Timeline for Coordination
Timeline creation can be spread across multiple teams. For example, your IDS team could build what they see, the firewall team does their part, while system administrators and digital forensic teams will perform their own version of an investigation and can provide you with a list of events. If you agree upon a common tool and standardized format ahead of time, this can allow you to tackle large, complex cases by distributing effort without losing too much context. A very simple format could involve spreadsheets that track: date, time, timezone, person, place, action, tags, evidence link. These could be collected from the different teams, merged, sorted and visualized.
Tagging and Aggregation
Now you've got what seems like a jumbled, insurmountable mess spread across multiple spreadsheets, and stuck in text files-- not to mention crucial details hiding visio diagrams, power points and emails. All of which has references to the memory and drive images that you've taken, the log files you've preserved and the pcap files that were captured. How is a timeline making this any easier?
The timeline is supposed to help you get organized, not clutter everything up. So if you're tasked with coordinating the timeline events from other groups, you will want to settle upon one tool for your own sake. As you're processing/reviewing events you'll want to tag them to note:
- critical phases in the incident response noting when the team determined entry into a new phase (e.g. containment commences, remediation complete)
- the interpreted phase of the attack (e.g recon, exfiltration)
- Control failure/opportunity (e.g. attack identified in system logs, but no IDS alert fired)
You will also need to perform quite a bit of data-reduction to turn a fully-populated timeline into something usable by management and other groups. All of the effort spent tagging events will pay off in this stage since you will be able to easily determine with a little bit of filtering on your spreadsheet (or whatever tool you're using) when recon was started, when the first successful attack struck, when you detected the breach, how long it took to resolve. This is what others are going to be interested in. A set of events can be aggregated and summarized into an overall event renamed using higher-level language, e.g. "Attacker #1 compromises DNS server.
Even at computer-speed, things rarely happen simultaneously, and one object can't do two things at once. While a lot of events are going to be happening in parallel, you should be able to chain events together into a series of actions. These chains should eventually appear untangled. Even in the case of an ugly internal worm outbreak you should be able to chain one infection to another. If you find that you've got a lot clouds of uncertainty on your timeline, that's just telling you that you have more work to do. It could also indicate that you can't answer those questions, for example the logs are missing or a monitor failed; in that case, flag that as a finding which you'll use later. You may also glean additional insight into the case by examining the blind-spots in the case, or struggling to untangle a set of event-chains. It could be that you're dealing with more than one attacker who happened to leverage the same vulnerability in your network and thus have overlapping incidents.
The Products of a Good Timeline
You should be able to walk through a chain of events and it should feel consistent, and if you've carefully linked to the evidence a compelling narrative of events will emerge. It should be easy to build after-action reports describing the series of events. Metrics for response should naturally come out of the timeline: each phase of the incident response process, time between event and detection, elapsed time from detection to remediation, etc. Preparing a case for law-enforcement should follow naturally from the timeline. The Lessons Learned document can be pre-populated by using the Control Failure/Opportunity tagged events (there are going to be other non-temporal issues like a lack of patches or weak separation of duties.)
You're Doing it Already
If timelines aren't a part of your standard investigative process, you're still very likely subconsciously going through the process. You're collecting the same amount of information was you try to solve the who, what, where, when, why and how questions and you're certainly organizing a chain of events in your head. Your raw case notes probably contains times, places, people, and actions, and you've got log files, and images, and pcaps just waiting to be turned into SUPER timelines. I bet your executive summary is written out as a series of aggregated events.
By keeping the timeline external, and using it to coordinate parts of an investigation, I hope this helps you tackle larger cases with less stress, and less sanity-loss.