Severity: Critical
I have both done pentests and received pentest reports. My observation is that the perceived severity often varies between the tester and the customer.
I have both done pentests and received pentest reports. My observation is that the perceived severity often varies between the tester and the customer.
I've found that using relative terminology seems to pierce the veil of ignorance.
When WiFi was new/newish and absolutely no one was securing it, I would bring with me a 300ft / 100m of CAT 5, string it out across the lawn out of a window (etc), and sit in a folding chair with my laptop to visually represent the threat. It never failed to get the point across.
These days as a last resort I will verbally liken an intruder or vulnerability with sexual predation. That gets the attention of someone in a position of power usually.
The problems I have encountered are mostly with hostile IT Depts / MIS / DevOps teams who think I'm there to point out thier mistakes. I'm there to help prevent costly mistakes, you guys figure out blame on your own time, because I literally don't give a shit who's to blame if anyone at all, and after this engagement, I'll disappear like a fart in the wind and on to the next client.
Even the potential threat wank they add to low severity stuff is ridiculous.
Finding: device responding to ping requests.
Severity: Low.
Threat: Using timing attacks and response analysis an attacker could derived the devices operating system.
This is a complicated topic, actually. If you know all of this stuff, disregard. I can just share my viewpoint from being in security for over 20 years which has slowly morphed from pure engineering work to more of an engineering/business/compliance hybrid skill set.
Context and actual risk matters. An easy exploit is bad and gives an adversary a place to pivot from in the org. However, where can the adversary pivot to after that? What resources are at risk then? ("Risk" is defined as the chance of "something" causing actual financial or reputational damage to an organization.)
Lets say that a customer has a site hosted externally on WP Engine and the admin page is compromised. There may be company contact information loss, possibility of employee password reuse to leverage and of course, one of their public facing pages could be defaced. There is more, but just keeping it simple for now.
Hopefully, WP Engine accounts and data is completely separate from the "meat" of the org: customer information, sensitive data, databases, etc.. If that is true, the easy exploit is still easy, but the actual risk to the org is much lower and from a business perspective, the finding gets bumped down in priority.
What I am saying is that a finding must be presented in full context. Is the finding easy to exploit but low risk or is it hard to exploit but has high risk? Is it easy to exploit and also is high risk?
What Jr. security staff almost always forget is that "risk" is something that is determined by the business, not by the third party pentesters. Part of the job of the security and compliance teams in the org is to take a finding and connect the dots from that finding to other parts of the org. Actual risk and priority can then be assigned.
Of all the security teams I have been part of, I can say that there are a million different ways to determine risk and a million more ways to prioritize a finding.
What is even more difficult to process is that "severity" may just be a summary score of risk and exploit difficulty. It depends on the company and what flavor of security frameworks they use. Severity could also include time to exploit, time to detect and remediate and if and exploit attempt could even be detected.
Good pentest reports will properly define all of its terms first. ie: What does "severity" and "risk" actually mean to the target organization? Security leadership needs to take that report and convert that into data that means something in terms of their budget. It's a sad reality of how businesses operate, unfortunately.
What I always see is that the business side of security is mostly ignored by jr engineers and pentesters. That isn't bad though! Real engineering work is the meat of security and the "business side" of things is a major distraction.
(My pet peeve is getting a pentest report with hypothetical issues where the tester couldn't even show step 1 to prove a vulnerability is even exploitable. I now have a report with a "high severity" call out with no proof attached to it, but still have to sit in meetings with my management telling them the finding is likely bullshit.)
Thanks for sharing your insight!
I really like this aspect where you say that the business determines the risk, not the tester. I think this is an easy pitfall, especially for people with less experience than you ;)
Sometimes the law determines the risk. Any critical/highs in PCI will get you speed bagged, so you sort those either way.
Now, sometimes the sorting is "turn if off for the retest" which is just the business ignoring risk in a complicated way, but it still gets addressed in some way.