Do not kill your pentester for little or no value-add

Disclaimer: This would be a long post (culmination of many old posts) with lot of different opinions, thoughts. If weaving is not right, please provide feedback on how it could be corrected.

I had the good fortune of reading couple of threads by gentlemen whom I respect for their grounded advices. Their posts triggered some thoughts in me that I had jotted in few old posts. So i tried to weave them into one post.

The thoughts are my own and hence, may read slightly off the mark. This is NOT a scientific post, but my observations. You have been warned.

I have divided my thoughts into 3 buckets. They are

  1. Technology
  2. People
  3. Process


Penetration testing (or pentesting) is evolving along with technology. Earlier days of nmap-metasploit-jackpot are over (although I have a nagging feeling that this may not be case for every organization). Defense is becoming smarter (EDR, better firewalls). Although more buzzwords have entered the market (red-team, breach and attack simulation, intelligence based penetration test, etc.), the implementation has not yet reached the mass.


Assuming there is certain body of knowledge related to pentesting, there are two ways to achieve it — in a top-down manner or bottoms-up. The BoK doesn’t change, only the method of attaining it changes. The method depends on the way how your brain is wired to learn things.

High level pentesting BoK

Top-down guys always look at things from high level and then take each step down. For example, a top down guy who is tasked with learning pentesting will almost always start from outer perimeter of the BoK (the process of pentesting, different methodologies, types of pentest, high level execution methodology, etc. This type of thinking suits a person at a middle level to higher levels of management, because this thought process supports an abstraction based thinking. These are also the people who will, usually, collect their thoughts in a way so as to publish a book.

Now, you are asking for disaster if you ask a top-down person to:-

  • Perform a technical penetration test (I am not talking your nmap-metasploit type);
  • Explain why a particular nmap switch has to be used;
  • Learn things by reading a book that doesn’t have any hands-on element in it;
  • Argue the differences between different firewall evasion strategies, etc.

A bottom-up guy will always start with getting their hands dirty (running the tool rather than going through the man-page). Then, slowly, this person will start picking different tools, running them, then combining them in a script. Understanding each tool’s output, executing a full-blown engagement will come later. These are the people who amass lot of knowledge & wisdom over a period of time, but organizing it into a book — sorry! These people don’t have patience to read a book from cover to cover, let alone design and publish one.

Now, hard-core penetration testers usually are of the bottoms-up type. Which means you are asking for disaster if you ask them to:-

  • Prepare an executive summary that can be consumed by an executive / decision maker
  • Align their findings / vulnerabilities with any particular risk management standard (e.g., FAIR, ISO 27001, etc.)
  • Perform risk assessment.
  • Explain differences between CVSS v2 & v3

Another question that needs thought — where to use bug-bounty hunters and pentesters?

TLDR — use bug bounty hunters after you reach “unknown unknowns”. See the sub-section “timing has to be right” below for more details.


This is where I think we need to work our things out. To that extent, I agree with most of the points that were made here. Here’s my commentary on Greg’s observations: –

Scoping it right, but it gets left all the time

Pentesting is both an offensive and individual engagement. Setting the rules of engagement is one of the toughest things to get right. Both pentester (am I doing more for less?) and the customer (am I getting less for more?) grapple with it. While customer don’t want to get hacked, they don’t want to expose their systems to harsh testing because they have a fear that it will break. They just want pentester to inform them of any vulnerability without exploiting it so that they can close it. However, pentester often fail to communicate it properly that without trying to break, they won’t know if a vulnerability exists! It is one of those catch-22 situations that lives long after the engagement is over.

This issue leads to ambiguous language in contracts that becomes cause of lot of heart-break and half-assed outcomes from a pentest. However, it is mostly because the engagement was not thought through.

If you want more vulnerabilities to be identified, provide information to your penetration tester and allow them to break it to test it.

Check inside, not just outside

I agree with Greg here. Testing outside perimeter alone won’t help in understanding a company’s security posture. Internal pentest need to happen as well. It will help understand the impact of what happens when an internal credentials / system is compromised. There is a lot more information that gets accessible to an adversary once a working AD credential is compromised.

Timing has to be right

I think every company, from a security posture point of view, goes through these phases (based on a loose interpretation of the Johari Window ( –

Known Knowns (they know they have issues, and others do as well)

The companies in this segment know that they have to improve on their security posture a lot and understand the major areas to focus (knowing fully well that vulnerabilities must be present in a plenty). Further, they have issues in their external perimeter (leaked info, keys, access, etc.). Pentester’s wet dream. These companies should start with regular scanning of their infrastructure (outside in, i.e., outside perimeter first, then inside) with a focus towards internalizing patch management (infra, apps, especially third-party apps like Java, Adobe etc.). Don’t dream of getting a red-team / purple-team engagement. You won’t get as much benefit as you should as you won’t stretch those red-teamers.

Known Unknowns (they know they have issues, but others don’t know that yet)

Most of companies fall in this bucket. They have an understanding of critical assets that they should protect and have implemented controls in / around those assets. However, they are not sure if that could be all. Also, these issues mostly exist in the internal environment. These companies should have 1 red-team engagement covering outside perimeter (without phishing in scope), coupled with round the year internal pentesting.

Unknown Unknowns (neither they know that they have issues, nor others yet)

It could be most dangerous (if the company doesn’t have a single decision maker who thinks security should be at-least an agenda point in decision making meetings) or most mature state in a company’s life, depending on various factors.

  • If it is former, ISO 27001 (led by a good consultant) should be a good choice, as it will start the institutionalization of security processes from top-down. Good consultant is imperative as it will THE deciding factor between an excellent ISMS that delivers value & a botched up one. ISMS implementation is a favorite topic of mine, more on that later.
  • If it is latter, they should hire red-teamers & bug-bounty hunters with focus on one area per quarter (e.g., phishing, external firewall bypass, physical security, chaining multiple vulnerabilities to access key company data, etc.).

Needs to align with high level controls to give a sense of state of security

I have been saying this in my previous articles as well. There is value in both top down approach (governance, assurance, audit) & bottoms-up approach (regular housekeeping activities related to security). One cannot exist without another. Top down and bottom-up people need to meet halfway before deciding on the security assessment part. Imagine this (sort of ideal scenario):-

  • The CISO consults involves internal auditors to set the scope for the RFP towards getting a vendor for their yearly penetration testing;
  • The Internal auditors design a set of technical controls, mapped to the company’s security policy / procedures.
  • Internal auditors plan/schedule their internal process audits along with these penetration testing exercises;
  • The technical controls are provided to pentesters to test, along with the normal pentesting workflow;
  • The output of the pentest exercise (aka findings) are then mapped to company’s security procedures & the internal audit findings, to provide a holistic view of security status of the company.

— End of post —

External Network VAPT: tools, information sources

Photo by Sean Musil on Unsplash

This is a live post; it will undergo changes, which are captured in change log, provided at the end of this post.

Being part of many external VAPT engagements, one thing was clear from outset — methodology is the key, but so are the information sources & tools for conducting the VAPT.

This series is my attempt to record all the tools and information sources that have been helpful to me over the last few years.

Traditionally, external VAPT can be divided into the following major phases:-

External VAPT, Process Flow

In this article, we will try to cover the first 2 phases — basic pre-engagement & reconnaissance.

Basic Pre-Engagement

It is very tempting to start an external penetration test as soon as customer asks us to. After all, we can perform this test from our current location (no need to travel to customer’s site to perform test) and,

Who cares about that silly thing called documentation? Turns out, except us, everyone.

Your customer wants it so that they can compare, at the end of test, whether the entire perimeter has been accessed or not (in other words, did we leave anything out);

Your boss / management (any other role who is a decision maker in your hierarchy) wants it so that they know we didn’t do more than we were asked to (scope creep is not mythical) & that we did all that we were supposed to;

So, here are the things that one should keep in mind before starting an external penetration test

Scope of work

Is entire perimeter to be tested, or few IP addresses?

Few customers will share the list of external IP addresses to test and won’t be happy if you stray from that list. Few other customers would want you to tell them the attack surface (in other words, what IP addresses of ours are exposed publically) before they give us permission (for full or partial test).

Type of test (black box, grey box, white box)

Black box — we won’t tell you anything about our network, the type of machine under that IP, etc. We want to see what an external attacker would do.

Here, you will have to make it clear that uptime of external IPs are not your responsibility. You will need their IT staff to be on standby during the test so that they can monitor the network and devices for any performance hit. You will be expected to stop when they ask you to.

Grey-box — we will give you some information (e.g., IP address space, type of devices, make / model, etc.).

The expectation here, from customer, usually boils down to one line — now that we told you the things that we have on our external network (and potentially saved you lot of time), what vulnerabilities are we exposed to?

Is physical security testing part of scope?

If it is, i suggest you get your get-out-of-jail card before moving a muscle. Also called “authorization letter”, this document (signed by an authorized signatory — may or may not be the CISO / CIO but the person who is legally authorized to sign agreements on behalf of the company) will help you when you get caught in a mis-understanding with local cops. The coalfire incident has shown how important is this document. So get this before starting the pentest of any service / infrastructure that is business / mission — critical for your customer. TrustedSec has made a host of legal documentation available for no cost (thank you guys!) to facilitate legal boundaries around your physical security pentest. I recommend you have a look at them.

Few more pointers:-

Report Deadline

Very important. End of testing is different from end of engagement. Testing ends when it ends (!). However, reporting is another, separate activity. Reporting starts after testing is over. You must schedule some time for this.

Preferred timings of test

Some customer would want you to initiate the test when no one is in office (out of working hours). Few customers would ask you to initiate the test during working hours. You have to document this.


Wikipedia defines it as

Credit to Wikipedia

The third definition relates to the objective of network recon. There are 2 major ways to perform recon — one that makes a lot of noise, letting everyone know that you are onto them (active) while the other is more sneaky, stealthy (passive).

Passive Recon

Many people have written lot about it. However, i have few points to mention here:-

Whether you have IP address or a domain name, you need to find ASN (Autonomous System Number) belonging to that company first. The benefit — it will help you find all the IP ranges that belong to the company. You can use below links to find ASN from a domain name or IP address:-

You would also want to look at sub-domain enumeration. Utilize the following tools

  • Amass
  • Censys
  • Shodan
  • Spiderfoot
  • Virustotal
  • VHostScan (
  • Google Transparency Report (
  • CertSpotter
  • CertDB

Active Recon

This is where we interact directly with the customer infrastructure. The tools that could be utilized in this activity include, but are not limited to, the following:-

So that’s all for now. We will expand on this series soon.