What are black box, grey box, and white box penetration testing? [Updated 2020]

Pentesters are apparently huge fans of colors. Different roles within pentesting assignments are designated as Red Team, Blue Team, Purple Team and others. Given this, it’s not surprising that different types of pentests are designated by color as well. You may have heard of white-box, black-box, and even gray-box pentesting but may be wondering what these terms mean.

Here, we’ll describe the three types of pentesting, how to choose the right type for a given assignment and how to become a pentester yourself.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

What are black, gray and white-box testing?

Pentesting assignments are classified based on the level of knowledge and access granted to the pentester at the beginning of the assignment. The spectrum runs from black-box testing, where the tester is given minimal knowledge of the target system, to white-box testing, where the tester is granted a high level of knowledge and access. This spectrum of knowledge makes different testing methodologies ideal for different situations.

Black-box testing

In a black-box testing assignment, the penetration tester is placed in the role of the average hacker, with no internal knowledge of the target system. Testers are not provided with any architecture diagrams or source code that is not publicly available. A black-box penetration test determines the vulnerabilities in a system that are exploitable from outside the network.

This means that black-box penetration testing relies on dynamic analysis of currently running programs and systems within the target network. A black-box penetration tester must be familiar with automated scanning tools and methodologies for manual penetration testing. Black-box penetration testers also need to be capable of creating their own map of a target network based on their observations, since no such diagram is provided to them.

The limited knowledge provided to the penetration tester makes black-box penetration tests the quickest to run, since the duration of the assignment largely depends on the tester’s ability to locate and exploit vulnerabilities in the target’s outward-facing services. The major downside of this approach is that if the testers cannot breach the perimeter, any vulnerabilities of internal services remain undiscovered and unpatched.

Gray-box testing

The next step up from black-box testing is gray-box testing. If a black-box tester is examining a system from an outsider’s perspective, a gray-box tester has the access and knowledge levels of a user, potentially with elevated privileges on a system. Gray-box pentesters typically have some knowledge of a network’s internals, potentially including design and architecture documentation and an account internal to the network.

The purpose of gray-box pentesting is to provide a more focused and efficient assessment of a network’s security than a black-box assessment. Using the design documentation for a network, pentesters can focus their assessment efforts on the systems with the greatest risk and value from the start, rather than spending time determining this information on their own. An internal account on the system also allows testing of security inside the hardened perimeter and simulates an attacker with longer-term access to the network.

White-box testing

White-box testing goes by several different names, including clear-box, open-box, auxiliary and logic-driven testing. It falls on the opposite end of the spectrum from black-box testing: penetration testers are given full access to source code, architecture documentation and so forth. The main challenge with white-box testing is sifting through the massive amount of data available to identify potential points of weakness, making it the most time-consuming type of penetration testing.

Unlike black-box and gray-box testing, white-box penetration testers are able to perform static code analysis, making familiarity with source code analyzers, debuggers and similar tools important for this type of testing. However, dynamic analysis tools and techniques are also important for white-box testers since static analysis can miss vulnerabilities introduced by misconfiguration of target systems.

White-box penetration testing provides a comprehensive assessment of both internal and external vulnerabilities, making it the best choice for calculation testing. The close relationship between white-box pentesters and developers provides a high level of system knowledge but may affect tester’s behaviors, since they operate based on knowledge not available to hackers.

Advantages and disadvantages of different testing methodologies

If all pentesting methodologies worked equally well, only one of them would be used. The main tradeoffs between black-box, gray-box and white-box penetration testing are the accuracy of the test and its speed, efficiency and coverage.

Engagement accuracy

The purpose of penetration testing is to identify and patch the vulnerabilities that would be exploited by an attacker. Therefore, the ideal form of penetration testing would be black-box, as the majority of attackers have no knowledge of the internal workings of their target network prior to launching their attack. However, the average attacker has much more time to devote to their process than the average pentester, so the other types of penetration tests have been developed to decrease engagement time by increasing the level of information provided to the tester.

At the other extreme from black-box testing is white-box testing, where testers are granted full information about the target system. The concern with this type of pentesting engagement is that the increased information will cause testers to act in a way different from black-box hackers, potentially leading them to miss vulnerabilities that a less-informed attacker would exploit.

Gray-box testing splits the difference between white-box and black-box testing. By providing a tester with limited information about the target system, gray-box tests simulate the level of knowledge that a hacker with long-term access to a system would achieve through research and system footprinting.

Speed, efficiency and coverage

The three penetration-testing methodologies make tradeoffs between speed, efficiency and coverage. In general, black-box penetration testing is the fastest type of penetration test. However, the limited information available to the testers increases the probability that vulnerabilities will be overlooked and decreases the efficiency of the test, since testers do not have the information necessary to target their attacks on the most high-value or likely vulnerable targets.

Gray-box testing makes a slight tradeoff in speed compared to black-box testing in exchange for increased efficiency and coverage. Access to design documentation allows testers to better focus their efforts and internal access to the network increases the coverage of the analysis. This is especially true when compared to black-box testing, where testers may never find a vulnerability that gives them access inside the network perimeter.

White-box testing is the slowest and most comprehensive form of pentesting. The large amount of data available to pentesters requires time to process; however, the high level of access improves the probability that both internal and outward-facing vulnerabilities will be identified and remediated.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

Becoming a penetration tester

Becoming an effective penetration tester requires a combination of knowledge and a good pentesting toolkit. This section describes how to build both of these.

Training and certification

Several certifications are available to the aspiring pentester who wants to be able to demonstrate their skills on a resume. The EC-Council offers both the Certified Ethical Hacker (CEH) and Licensed Penetration Tester Master certifications, while the Global Information Assurance Certification (GIAC®️) has both a Pentester (GPEN) and Exploit Researcher and Advanced Penetration Tester (GXPN) certification. Finally, Offensive Security offers the Offensive Security Certified Professional (OSCP) certification. For more information on pentesting certifications, see here.

Developing a penetration testing tool kit

Development of a penetration testing tool kit is an ongoing process. Penetration testers who are just starting out typically make use of existing tools created by other penetration testers and hackers. However, as they gain experience, it’s not uncommon for testers to build up a collection of self-written or team-written scripts and tools designed to automate common or complicated processes that come up in the course of their engagements. 

Development of simple tools only requires knowledge of a scripting language like Python or Ruby, but more complicated development may require a dedicated team and more sophisticated knowledge of target systems.

The tools and skill set required for penetration testing grows as you move along the continuum from black-box to white-box penetration testing. Black-box penetration testers primarily perform dynamic analysis and need the ability to build a network architecture diagram as they go. Gray-box penetration testers need the same tool kit as black-box testers but also need the ability to read architecture diagrams and design documentation and determine vulnerabilities at a system as well as local level. White-box testers require the same tools and capabilities as both of these, but also need the tools and experience required to perform static code analysis.

Dynamic analysis tools

Black-box and gray-box pentesters primarily perform dynamic analysis of running software. The following are some of the must-have tools in a pentester’s toolkit.

The Metasploit Exploitation Framework by Rapid7 is one of the most widely-known pentesting tools in existence. Armitage is a GUI front-end to Metasploit for users less familiar with CLIs.

When performing a penetration test, information is everything and a network traffic capture can be extremely valuable. Packet capture utilities like Wireshark and Kismet allow testers to capture Ethernet or wireless network traffic.

Passwords retrieved in a penetration test are rarely in plaintext. Password Crackers like John the Ripper, Aircrack-ng and Ophcrack take password hashes and discover the corresponding plaintext passwords.

The Nmap port scanner is a movie star, with appearances in “The Matrix Reloaded” and “Ocean’s 8,” among others. By allowing a pentester to see open ports and running services, a scan using Nmap or its GUI version, Zenmap, are a common first step in the recon stage of a penetration test.

Vulnerability scanners are tools designed to take the heavy lifting of pentesting off of the tester’s shoulders. Tools like Nikto, Nessus and OpenVAS scan targets for common vulnerabilities and generate a report for the user regarding potential attack vectors.

A web proxy is used to perform Man-in-the-Middle (MitM) attacks on web traffic, allowing data to be viewed and modified between a browser and the server. Fiddler and Burp Suite are examples of commonly-used web proxies.

Static analysis tools

In addition to the dynamic analysis performed in black-box and gray-box testing, white-box testers also are expected to perform static analysis of provided source code. This requires proficiency in the use of additional pentesting tools.

A debugger like Ollydbg or Windbg is commonly used in development and quality assurance to identify bugs within a program. Pentesters use these tools to understand the functionality of a program and explore different execution paths for exploitable code.

A compiler takes high-level code and translates it to machine code. A disassembler partially reverses this process by converting opcodes to human-readable code. A pentester will use IDA Pro, Hopper, gdb, radare2 and similar tools to convert an executable into a format where code can be analyzed for vulnerabilities and other useful data.

Source code analysis tools (SASTs) are used by white-box pentesters to detect vulnerabilities in source code. The OWASP project provides a great list of SASTs in their wiki.

FREE role-guided training plans

Get 12 cybersecurity training plans — one for each of the most common roles requested by employers.

Download Now

Conclusion

Black-, gray- and white-box pentests are all different approaches to simulating how a hacker would attack a network and identifying and patching the vulnerabilities discovered. Ideally, most penetration tests would be black-box, since it most closely resembles how a hacker approaches a network. However, time constraints and the desire to detect and remediate vulnerabilities inside the perimeter as well has led to the creation of gray-box and white-box penetration testing methodologies.

While black-box and gray-box use primarily dynamic analysis methodologies, white-box penetration testers must be proficient with static analysis techniques as well. Becoming a proficient penetration tester requires practice and familiarity with a variety of tools, techniques and targets.

Explore more articles about penetration testing

Penetration Testing Benefits

Top 6 Penetration Testing Certifications for Security Professionals

How to Land Your First Penetration Testing Job

Sources

Armitage

Wireshark

Kismet

John the Ripper

Aircrack-ng

Ophcrack

Nmap

Nikto

Nessus

Fiddler

Burp Suite