Do not kill your pentester for little or no value-add

Disclaimer: This would be a long post (culmination of many old posts) with lot of different opinions, thoughts. If weaving is not right, please provide feedback on how it could be corrected.

I had the good fortune of reading couple of threads by gentlemen whom I respect for their grounded advices. Their posts triggered some thoughts in me that I had jotted in few old posts. So i tried to weave them into one post.

The thoughts are my own and hence, may read slightly off the mark. This is NOT a scientific post, but my observations. You have been warned.

I have divided my thoughts into 3 buckets. They are

  1. Technology
  2. People
  3. Process


Penetration testing (or pentesting) is evolving along with technology. Earlier days of nmap-metasploit-jackpot are over (although I have a nagging feeling that this may not be case for every organization). Defense is becoming smarter (EDR, better firewalls). Although more buzzwords have entered the market (red-team, breach and attack simulation, intelligence based penetration test, etc.), the implementation has not yet reached the mass.


Assuming there is certain body of knowledge related to pentesting, there are two ways to achieve it — in a top-down manner or bottoms-up. The BoK doesn’t change, only the method of attaining it changes. The method depends on the way how your brain is wired to learn things.

High level pentesting BoK

Top-down guys always look at things from high level and then take each step down. For example, a top down guy who is tasked with learning pentesting will almost always start from outer perimeter of the BoK (the process of pentesting, different methodologies, types of pentest, high level execution methodology, etc. This type of thinking suits a person at a middle level to higher levels of management, because this thought process supports an abstraction based thinking. These are also the people who will, usually, collect their thoughts in a way so as to publish a book.

Now, you are asking for disaster if you ask a top-down person to:-

  • Perform a technical penetration test (I am not talking your nmap-metasploit type);
  • Explain why a particular nmap switch has to be used;
  • Learn things by reading a book that doesn’t have any hands-on element in it;
  • Argue the differences between different firewall evasion strategies, etc.

A bottom-up guy will always start with getting their hands dirty (running the tool rather than going through the man-page). Then, slowly, this person will start picking different tools, running them, then combining them in a script. Understanding each tool’s output, executing a full-blown engagement will come later. These are the people who amass lot of knowledge & wisdom over a period of time, but organizing it into a book — sorry! These people don’t have patience to read a book from cover to cover, let alone design and publish one.

Now, hard-core penetration testers usually are of the bottoms-up type. Which means you are asking for disaster if you ask them to:-

  • Prepare an executive summary that can be consumed by an executive / decision maker
  • Align their findings / vulnerabilities with any particular risk management standard (e.g., FAIR, ISO 27001, etc.)
  • Perform risk assessment.
  • Explain differences between CVSS v2 & v3

Another question that needs thought — where to use bug-bounty hunters and pentesters?

TLDR — use bug bounty hunters after you reach “unknown unknowns”. See the sub-section “timing has to be right” below for more details.


This is where I think we need to work our things out. To that extent, I agree with most of the points that were made here. Here’s my commentary on Greg’s observations: –

Scoping it right, but it gets left all the time

Pentesting is both an offensive and individual engagement. Setting the rules of engagement is one of the toughest things to get right. Both pentester (am I doing more for less?) and the customer (am I getting less for more?) grapple with it. While customer don’t want to get hacked, they don’t want to expose their systems to harsh testing because they have a fear that it will break. They just want pentester to inform them of any vulnerability without exploiting it so that they can close it. However, pentester often fail to communicate it properly that without trying to break, they won’t know if a vulnerability exists! It is one of those catch-22 situations that lives long after the engagement is over.

This issue leads to ambiguous language in contracts that becomes cause of lot of heart-break and half-assed outcomes from a pentest. However, it is mostly because the engagement was not thought through.

If you want more vulnerabilities to be identified, provide information to your penetration tester and allow them to break it to test it.

Check inside, not just outside

I agree with Greg here. Testing outside perimeter alone won’t help in understanding a company’s security posture. Internal pentest need to happen as well. It will help understand the impact of what happens when an internal credentials / system is compromised. There is a lot more information that gets accessible to an adversary once a working AD credential is compromised.

Timing has to be right

I think every company, from a security posture point of view, goes through these phases (based on a loose interpretation of the Johari Window ( –

Known Knowns (they know they have issues, and others do as well)

The companies in this segment know that they have to improve on their security posture a lot and understand the major areas to focus (knowing fully well that vulnerabilities must be present in a plenty). Further, they have issues in their external perimeter (leaked info, keys, access, etc.). Pentester’s wet dream. These companies should start with regular scanning of their infrastructure (outside in, i.e., outside perimeter first, then inside) with a focus towards internalizing patch management (infra, apps, especially third-party apps like Java, Adobe etc.). Don’t dream of getting a red-team / purple-team engagement. You won’t get as much benefit as you should as you won’t stretch those red-teamers.

Known Unknowns (they know they have issues, but others don’t know that yet)

Most of companies fall in this bucket. They have an understanding of critical assets that they should protect and have implemented controls in / around those assets. However, they are not sure if that could be all. Also, these issues mostly exist in the internal environment. These companies should have 1 red-team engagement covering outside perimeter (without phishing in scope), coupled with round the year internal pentesting.

Unknown Unknowns (neither they know that they have issues, nor others yet)

It could be most dangerous (if the company doesn’t have a single decision maker who thinks security should be at-least an agenda point in decision making meetings) or most mature state in a company’s life, depending on various factors.

  • If it is former, ISO 27001 (led by a good consultant) should be a good choice, as it will start the institutionalization of security processes from top-down. Good consultant is imperative as it will THE deciding factor between an excellent ISMS that delivers value & a botched up one. ISMS implementation is a favorite topic of mine, more on that later.
  • If it is latter, they should hire red-teamers & bug-bounty hunters with focus on one area per quarter (e.g., phishing, external firewall bypass, physical security, chaining multiple vulnerabilities to access key company data, etc.).

Needs to align with high level controls to give a sense of state of security

I have been saying this in my previous articles as well. There is value in both top down approach (governance, assurance, audit) & bottoms-up approach (regular housekeeping activities related to security). One cannot exist without another. Top down and bottom-up people need to meet halfway before deciding on the security assessment part. Imagine this (sort of ideal scenario):-

  • The CISO consults involves internal auditors to set the scope for the RFP towards getting a vendor for their yearly penetration testing;
  • The Internal auditors design a set of technical controls, mapped to the company’s security policy / procedures.
  • Internal auditors plan/schedule their internal process audits along with these penetration testing exercises;
  • The technical controls are provided to pentesters to test, along with the normal pentesting workflow;
  • The output of the pentest exercise (aka findings) are then mapped to company’s security procedures & the internal audit findings, to provide a holistic view of security status of the company.

— End of post —

How should a CISO deal with XSS?

I got many comments (thank you everyone, as i learnt a lot) for my article that i published some time back.

I realized that i need to explain my thoughts in a different way (as many people were of view that i am championing a person-with-no-technical-knowledge as CISO. In order to explain my thinking, let’s take a vulnerability like XSS and see how each role (from my previous article) should respond to it: –

  1. Security Analyst – He/she needs to showcase the PoC (Proof of Concept) of XSS, and not just share a text from OWASP website. In other words, the analyst needs to prove that the XSS (in this case) is exploitable, and how. We all know that not all XSS can be exploited, and market is already ripe with all those script kiddies who can just run some tools and email the report (after modifying the aesthetics).
  2. Security Manager– He/she needs to understand the impact to the application (e.g., what is the critical data here? Can it be lifted off because of this vulnerability? Can the system be taken over due to this vulnerability? Has it been proved here, with ample evidences so that i can take it to the CISO?)
  3. CISO — How important is this application to the company? Who are the customers for this application? Has the manager validated the vulnerability and the proof? Is it damaging our confidential data? What is being impacted here (C, I or A)? What else is affected here? Would it be possible to pivot to other important machines in our network after compromise of this system because of XSS? What are the other defense mechanisms in place to prevent a pivot? How much time will it take for this vulnerability to be patched? Would there be any downtime (during patch)? Can the customers of this application wait till the application is patched, tested, re-deployed? what can we do to expedite it (if it is so important to the company that downtime won’t be tolerated)?

After getting responses of all questions (in case of CISO performing role of manager, or vice-versa, he/she will have to ask both types of questions to the security analyst & to himself), he/she will have to decide the next course of action.

I, on my side, have seen critical application vulnerabilities & resulting risks being accepted during an engagement because no downtime could be afforded. Instead, in this case, the CISO (after discussing with the penetration tester) created an exploit video, demonstrating the impact. He then got a WAF (Web Application Firewall) deployed before the app (with new rules to handle the XSS, which were created by the penetration tester. He had gotten relevant clauses inserted in the PT engagement beforehand, predicting the shortage of resources and possible blockages by management) and used the video to increase management awareness. He got approval for the system upgrade, subject to the condition that the old copy will remain behind WAF till the new copy is developed and tested.

He was able to do all this because

1. He asked pertinent questions to understand the issue

2. He drove the penetration tester harder, asking for PoC, getting all relevant contractual clauses in place so that he gets the necessary support, etc.

3. He had a game plan in place to handle the vulnerabilities.

If i put understanding of XSS on a scale of 1–10, with

1 — knowledge of expansion of XSS

2 — understanding a visual representation of how XSS works (e.g., something on the lines of this URL

3 — Understanding the impact of XSS (either by pestering the penetration tester to prove it or going through online examples like this URL — OR–security-series-81)

4 — Ability to explain, with example of, XSS code (e.g., something like this URL —

5 — Ability to identify the vulnerability through automated tools (like burp or acunetix, etc.)

6 — Ability to identify the vulnerability without any tool (manually)

7–10 — Ability to understand XSS on a deeper technical level (ability to exploit XSS through various methods)

As you can see, steps 1–4 can be done by someone with basic knowledge of information security (and i am full advocate of a CISO having spent some time in this field. I don’t expect a newbie-in-infosec-but-a-veteran-in-management to succeed in this field). Script kiddies can handle the step #5, but you need a technical person to perform well between 6–10.

I am increasingly seeing this issue. One needs to understand how XSS works and its impact on that application and, as a consequence, the organization. One also need to understand the various defensive measures that are present in the industry. However, as a decision maker, your technical skills won’t save you if you do not have decision making skills. Decision making is a skill in itself that takes tremendous practice.

Please don’t kill your CISO for not knowing how a virus works

I came across this rant (with the usual don’t-kill-me-am-just-making-a-random-statement-and-fully-intend-to-get-away-with-it disclaimer) on LinkedIn about how CISO’s are clueless about how a virus works, even with CISA/CISM and a decade’s experience under their belt. It got me seething about how this statement is wrong on so many levels, but then I decided to marshal it in a better way and keyboard-and-mouse my thoughts (penning my thoughts would have sound a bit cliched, anyway) in a bit more structured way. I could be wrong but am willing to learn from feedback and remarks of my readers (if there are any. I am a realist).

A disclaimer first – I am not trying to promote anything here except my viewpoint.

CISO is 90% management, 10% technical post.

Let’s look at this line of thought.

Management, IMHO, is about making decisions without complete data at hand. This means that entire organization has to be put into an abstraction (nothing else will give a high level view that quickly). However, it results in a whole lot of jugglery and political skulduggery which has given a bad name to management. In an ideal world, every management guy has hands-on experience in everything that his company is doing. However, reality is different. Hence companies look for an ideal combination of practical know-how and management skills, with mixed results. Just like it is difficult for a management guy to learn technology, it is also difficult for a techie to learn management skills.

Companies seldom fail because of lack of technology. They fail due to lack of management / business sense and leadership (market disruptions notwithstanding) . It took a Steve Jobs to make Apple where it is now. It took Bill Gates’ decision making and management skills to make Microsoft where it is today.

Time for another disclaimer now.

I am not saying that Technology or techies are not needed. I am only saying that Management is also needed.

Time to look at an image that I created to illustrate my point. What I am trying to put here is the viewpoints of different roles in the information security food chain and an example of the viewpoint in action. See how abstraction builds up as roles get near to business.

InfoSec FoodChain

As per Rafeeq Rehman (someone whom I admire a lot), below are the different activities that a CISO has to perform these days ( You will notice that a lot of those activities are leadership activities and not technical. Although, technical knowledge is required, it is at an enabler level (in other words, understand what your team is saying so that you can make a decision), because CISO is essentially a decision maker / leader, sometimes a manager (if the company is small, CISO may also end up doing a bit of technical work himself). And, folks believe me when I say it – IT IS A FULL-TIME JOB.

Does a CISO need to know how a virus works? That is not the right question. The right question is –

Does a CISO need to know how a virus works in the same way as that of an incident responder under his team?

The answer is a resounding NO.

If you bring an OSCP (without the business experience) for CISO , you will suffer. And if you bring a CISA / CISM (without the necessary incident handling experience) for leading a team of incident handlers, my condolences.

CISA / CISM are technical certs, but not in the way you think.

Let me approach it in a different way.

I consider, anything that belongs to the know-how of the field, technical. So, detailed procedures on how to audit is technical stuff (related to audit). Its usage of software tools may be confined to MS Excel and Access, but it is still technical because it describes the innards of the field (e.g., how to sample, what is the meaning of audit, how is it different than assessment, how to do audit, etc.).

CISA (Certified Information System Auditors) is for auditors (present and wannabes) who audit information systems (security is a part of it). CISM (Certified Information Security Managers) is for managers (current and wannabes) who manage information security management systems. Together, they have a knowledge base that, if internalized, can help a person gain lot more insight into the field than without. That doesn’t mean everyone can stomach it. No so-called-techie can read CoBIT without falling asleep (that doesn’t mean that you claim yourself a techie just because you can’t stand CoBIT documentation), but a decision maker will immediately feel home with that documentation because it offers him a vantage point for the entire information security control landscape of an organization. Not just that, it will give him enough ammunition to oversee an existing information security system. Because the decision maker is used to see things in abstraction.

However, this also doesn’t mean that everyone who is a CISA / CISM has internalized all the knowledge that was part of the syllabus (you didn’t think I am vouching for incompetent guys, did you?). Which brings me to the next part.

Certifications are not knowledge pills, you still have to work your grey cells.

Having certifications only mean that you have cleared an exam. It doesn’t mean that you still retain, cross-reference, update, and internalize all that knowledge that was part of the syllabus, after the exam and in subsequent years. It also means that you still have to be interviewed for all those above verbs and your suitability to the role. I will stop here because this topic has been beaten so many times that it doesn’t make sense anymore to trample on the same. Just google “are information security certifications necessary” to know what I mean. Having a certification is no guarantee of job-material in this field.

Standards and Compliance are not bad things, neither are they cause of the current sad state of affairs. People are.

As far as I am concerned, all of our problems stem from increase in population. But that is for another post. Let’s focus on the current point.

Bruce Schneier made an interesting point in one of his books (I think it was “secrets and lies” but I am not sure) that firewalls got famous NOT because they were technically sound and made perfect sense, but because their absence was getting flagged as non-compliance by auditors! And you thought your CCIE made all the efforts!

With all the jokes running around compliance and ISO 27K1, I still think that the flaw is in the implementation, not in the spirit.

Security has so many facets these days that it is a full time job just to keep all the balls in air and make sure that they don’t crash. Add to it the daily pressure of selling security to top echelons so that you can keep your job lest a non-clued security director, or some other high title, decides that he doesn’t need you with all the flashy tools in place (SIEM / DLP / IDAM, once installed, must learn on its own) and you will sympathize with all the CISO’s since the term infosec was coined. Jokes apart, management needs someone who can translate the jargonic (wait, is that even a word?) aspects of security into a more manageable chunk so that they can decide where to direct the funding and oversight (in other words, tell me if it is working for business and show me how). Standards (like ISO 27001) and compliance / regulations are supposed to help a decision maker understand security in a language that they understand. But, as Ian Tibble points out so poignantly in his book “Security De-engineering”, excel masters took over and they took the whole focus away. And we have been cursing management ever since!

In other words (meaning I could have avoided all this rant and said these lines instead and still made my point): –

Security management and tech must co-exist to create a beautiful recipe that is tasteful to the business. Singular won’t work.

Bait for Your Identity

I overheard this interesting talk last sunday while harassing some poor developer to close an NC, have a dekko. But before that, a very short intro of the characters.

Character #1 — Baba Gyandev, aka if-google-had-a-body-this-would-be-it, BG in short

Character #2 — Baby Busy, aka this-will-never-happen-to-me, BB in short, BG’s follower#1

Character #3 — Paranoid Pandu, aka even-my-breadth-should-be-encrypted-to-save-it-from-sniffing, PP in short, another follower of BG

Context — BG & his disciples are in a very good mood. BG is happy because of planetary alignments, BB got a good appraisal recently, and PP just bought the latest encryption software for his laptop. More than that, they are happy because of their body has been treated to the best seafood meal that they had in recent times.

BB — This place is good, we should come here more often.

BG (after a big gurgling sound escaping the deepest corners of his intestine, making everyone else in the restaurant look for cover) — Yeah, fish is good.

BB — I don’t know why some people have devoted themselves to anti-fishing causes on Internet, it is not if we are trying to finish all the fishes!

PP — That was not this fish, BB, it is called Phishing, and it is very dangerous.

BB (with some alarm on her face) — Oh!

BG — PP, please do not terrorize her. BB, while it is true that phishing is a concern, it can be managed by some very easy-to-do things.

BB — Baba, please tell me more about this. What is this about?

PP — It is about stealing your identity.

BB — My identity? What identity?

BG — Bhaktjano, it’s not the identity that all of us are always looking for, inwardly (who am i? What am i on this earth for, stuff like that). I can talk about it more over seafood in Taj Banjara. The identity, that we are talking about now, is that of us on the information superhighway called Internet.

BB — Identity on Internet? What is my identity on the Internet?

PP (with some irritation) — Don’t you have a facebook account? Or yahoo/aol/hotmail/gmail ID? Or any other ID on any other website (irctc/icici/sbi/any-other-bank)?

BB — So what? Those are just login IDs, not my identity, Mr. know-it-all!

BG — Please don’t fight, kids. BB, in today’s online world, everything is connected to everything else on the Internet. You can share content of one website on another, e.g., share an online article or a review of latest movie that was put on some other news site, on your facebook account; you do a lot of financial transaction online. All of this requires that those sites know you. They give you login IDs so that they can recognize you, the next time you logon. So, all these IDs that we have online constitute our online identity. It is what we are and how people will recognize us when online.

BB — Yeah, i remember opening a recurring deposit account online in ICICI. They neither made me write a letter nor call me for an approval. I started it online and it automatically deducts money from my account every month.

PP — That was because they knew it were you, because they knew the login ID belonged to you.

BG — Correct. But now, the issue is — Crime always follows money. Bad people have realized that many (if not all) of the transactions are happening online now, it makes more sense if we can somehow get those IDs and passwords.

BB — Hmmm….. Baba, how do these people do it? Where does Phishing comes into picture?

BG — They will create copies of the well-known websites, with similar spellings, and put them online. Then they wait for you to land there.

PP — They do not always wait for your to come, they try to lure you to it. Remember that LinkedIn invitation that you said you had got from me? And the facebook invitation from your husband?

BB — Yeah, i do. I also remember that you had a look and then asked me to delete them and not to click on any link in that mail.

PP — Because it was a SPAM, meant for anyone who would believe and click on them, thereby landing on the fake site. The person will provide his actual user ID and password, and then, la-la land!

BB — How to stop it?

PP — These people are a reason why i am very skeptical while online. I don’t trust Internet!

BG — PP, in that case, stop buying house because land mafia may take it over, stop buying gold or silver ornaments because they can be stolen, stop carrying money in pocket because they can be , well, picked up. And while you are at it, stop living (PP looks at BG in shock) because there are criminals out there who murder for living.

BB starts laughing.

BG (with increased calmness) — Just because there are some issues with a technology or a facility, you don’t stop using it. Atleast not when you get so much benefits from it. More so, when you can save yourself using some common sensical tips.

BB — Please give me some tips so that i can save my identity online.

BG — the first step – don’t click on any link blindly. Check it before clicking on it. Is it pointing to what it says it would?

PP — A link to facebook should not go to some random site like

BG — True. In the lower left hand corner of most browsers users can preview and verify where the link is going to take them. Always check them before clicking.

PP — Also, look at the language of the mail.

BG — In other words, do not click on links within emails that ask for your personal information.

PP — True. Actually, no organization in its right mind would ask for it in mail. If it does, there is something ‘phishy’ there.

BG — Never enter your personal information in pop-up windows.

BB — What is wrong with pop-ups if it comes up after the original site has loaded? It means it has come from the site, right?

PP — Not necessarily. Sometimes a phisher will direct you to a real company’s, organization’s, or agency’s Web site, but then an unauthorized pop-up screen created by the scammer will appear, with blanks in which to provide your personal information. If you fill it in, your information will go to the phisher. Legitimate companies, agencies and organizations don’t ask for personal information via pop-up screens. Install pop-up blocking software to help prevent this type of phishing attack.

BB — Means, i should never give confidential information in pop-ups.

BG — Correct. Also, phishing doesn’t always need Internet.

BB — ?????

BG — You may get a call from someone pretending to be from a company or government agency, making the same kinds of false claims and asking for your personal information.

PP — If someone contacts you and says you’ve been a victim of fraud, verify the person’s identity before you provide any personal information.

BG — In other words, don’t give (or offer to give) your account ID and password to some guy over phone just because he claims to be from IT-Support. I know you did that yesterday.

BB (blushing) — that was because i needed some document very badly but was not able to logon to my machine. I had raised a ticket too.

PP — How do you know that this guy had called because of that ticket? I was there, too and you did not verify his identity.

BB (getting a little angry) — There is nothing interesting in my account, even if the user gets the password.

PP — yeah, true, but you re-use passwords, right? Which means one password of yours can open many accounts of yours !

BG — Actually, it is not just a matter of having something interesting in your account. Once your account is compromised, it will be used by bad people to lure your friends and contacts.

PP — For example, if i get your twitter / facebook / gmail ID, i can just ask your friends from little money (i can guess who are your friends by looking at your past activities), and if they are like you, they will transfer money first and then call. And that is just for starters.

BB is silent.

After some time, BB breaks the silence.

BB — So what should i do to stop it from happening?

BG — Be suspicious if someone contacts you unexpectedly and asks for your personal information. It could be in any format (online or offline), but ultimately, you have the responsibility over your information, Keep it secure!

PP — You can also keep changing your passwords regularly and use security features available with major sites (like two factor authentication of gmail, privacy features of facebook, etc.).

BG — Keep your browser and operating system updated and secure because many phishing attempts are hidden in viruses and other bad code.

BB — Baba, what if i accidentally gave some information? What should i do then?

BG — Contact related officials immediately and inform them.

PP — for example, if you accidentally gave your banking related information, then contact the bank immediately. In case of an online account, change the passwords immediately and notify the website.

BB — Thank you, BG and PP.

ISO 27001 : A Business View

Hi People,

I am back after a strong lethargic break. Before i go back to hibernation (i can promise that i will be regular from now onward, but people who know me will differ — and i don’t blame them, either — but i digress), let me share a presentation that i did for a NULL meeting (what? You don’t know NULL? Shame on you!, go back and Google; on second thoughts, read this please and then go back, coz i am not sure if you will come back!).

Please visit this Google Presentation and share the feedback. My take is:-

ISO 27001 is a standard which provides a structured and step-by-step approach in solving many security problems , most of which do not involve technology.

I have tried to take some examples to illustrate some events that technology will need some years to solve. However, using a methodology such as ISO 27001 helps us in securing, and maintaining the same, the information and infrastructure supporting it.