Reports for People

4th September 2020

A key part of a technical assessment is summarising test findings for a non-technical audience. When I started consultancy, my understanding of this was solely from the perspective of explaining a finding in clear terms - how it arose, and how the systems enable the vulnerability, and what data or systems are affected.

Over time, however, it becomes clear that this is not the point of a non-technical description. It's not that the technical details aren't important - they are. But a non-technical description isn't simply a technical description couched in more approachable language - a non-technical description of a vulnerability or report should explain the technical issues and the non-technical ramifications. In other words, a non-technical summary should explain why your technical findings matter, and what their impact on the business would be.

The other item to consider is that penetration tests are rarely conducted in a vacuum. Within a business, if the expense of a penetration test could be avoided, it would be. Therefore, every project has a justification. I mentally group these into the following loose categories:

1. Regulatory
2. Political
3. Curiosity

The majority of the time, a test is commissioned because it has to be, mandated by an external force, be it compliance with a scheme such as PCI DSS or Cyber Essentials (Plus!), or because of the requirements of a regulatory authority such as the FCA.

Alternatively, the test is to evaluate or make a point within the client, as a company is rarely a unified force of will, but more a loose collection of interests. There is a tendency to see a company as an impassive monolith with singular intent, but this is rarely the case as everyone has their own priorities and budgetary expectations (whether time or money or people).

Finally, there are clients who are generally curious. Perhaps this is following a breach, or suspected breach, or the departure of an employee who'd raised concerns. Sometimes, it might even be after a regulatory engagement, which can suffer from restrictive scopes and reporting requirements. There are even clients who take pride in frustrating testers and gleefully providing copies of their near-empty report to their customer base (these are often the most fun).

No matter what the justification for the test is, they are commissioned by an agent of a client for a reason. Projects don't just appear out of thin air, and get delivered to a recycling bin (well, not always!) As with all things fun, projects often serve a greater purpose, and one which is important to consider when reporting.

Most companies, when reporting the results of a project, will include a high-level summary of the issues found, and some conclusions from the tests. This may include general trends from within a larger environment, or it may simply state the high-risk or immediate priorities, depending on how interventionist the consultant is feeling that day. Yet this remains a technical explanation, albeit one which a layperson can understand.

A non-technical explanation of issues raised should consider the justifications of the test. Why was the test commissioned? How would the exploitation of the findings impact the business? An SQL injection in a web application allows an attack to access and manipulate user data. This is the technical issue. But the non-technical summary is that anyone with access to the page could stop trading for the day, if the site was a storefront. Or cause lasting reputational damage, if the site requires financial regulation. Or affect their reputation while also inducing crippling GDPR fines, if the data is personally identifiable, or otherwise sensitive.

It's easy and understandable to get lured into describing the cool details of an exploitation, and for these details to creep their way northwards into the project conclusions and summaries. The business requirement from a penetration test (and thus the goal of reporting) is not to be wowed technically. It's to understand the risks that the company might face. And so, the ideal summary shouldn't really be considered in relation to the technical summary, but should instead be a plainly worded conclusion, clearly summarising the state of the company, and what the wider implications of successful exploitation would be.