Summary
If you are a security, infrastructure, or project manager, then using whitebox testing methodology can give you greater assurance and better test coverage for the time allocated. This article sets out to explain why.
Assurance
There are different ways to gain assurance that a product or service is secure, and each method has different strengths and weaknesses. Threat modelling is a great way to choose a method that matches the threat.
Threat Modelling
Threat modelling is a process to identify the primary sources of attacks, their behaviour, and their corresponding possible points of entry into a system. It can help with gauging the likelihood of attacks being successful, and, most importantly, it can help in deciding where you can apply controls to most effectively manage risk.
CODA can support you through the process of threat modelling, or you can choose your own approach, such as by using an existing industry model. As an example of further reading, the following article discusses threat models in further detail:
https://insights.sei.cmu.edu/sei_blog/2018/12/threat-modeling-12-available-methods.html
Blackbox Testing
When testing from a blackbox perspective, the corresponding threat model, especially for public facing services is typically daily attacks from low-skilled, low-motivation, opportunistic attacks. The types of vulnerability will be historical vulnerabilities and well known misconfigurations.
Typically, this type of threat can be simulated effectively by vulnerability scanning and assessment. If penetration testing is used then the pentesting team gets very little information about the environment beyond confirming the scope.
Whitebox Testing
The threat model for whitebox testing is of an identified range of potential threats, including differing levels of skill, motivation and resources. Attacks may come from anywhere (customers, staff, from the internet or internal sources), but are generally based on readily available tooling and are focused on motivation specific to the context, such as information theft or financial gain.
Typically this can be simulated effectively by sharing information about the environment with a penetration testing team. This might include:
- An explanatory walk-through demonstration of the environment by a member of staff
- Source code for any in-scope web applications and APIs.
- Technical architecture diagrams for the environment.
- Any other relevant design or process documentation.
The tester wants as much information as possible so they can understand the environment and provide the most thorough testing in the time allocated. But testers with experience are realists — the tester knows that quite often no one person or organisation knows everything about an entire system.
An internal organisational objection might be that the information provided to testers is too sensitive, possibly from national security or defence perspectives, but more often from a commercial perspective. These risks can be controlled, such as by ensuring testers have the security clearance required, ensuring an NDA exists, and ensuring that the testing company has passed supplier onboarding security procedures such as background and reference checks, and assurance of appropriate security controls for shared data.
Greybox Testing
For the discussion in this article, greybox testing can be considered a constrained form of whitebox testing, with a reduction in the amount of information shared. Outside of specific scenarios, this could be viewed as a less effective version of whitebox testing. A test that requires greybox testing instead of whitebox may instead benefit from more context discussion to ensure the scope is appropriate.
Security Research
As well as the threats above, you may have identified a realistic threat from targeted, well-resourced groups. Alternatively, you want to make sure your system is robust and secure beyond the level required for compliance with typical good practice, or you’re operating in a constrained environment with devices that must remain operable for years, such as Telecoms, IoT and SCADA.
Targeted security research seeks to identify more profound or more insidious flaws within a system that a spot-check point-in-time testing might otherwise miss.
Effectiveness
Benefits of Whitebox Testing
Whitebox Penetration testing offers a sensible balance between effective assurance testing and value-for-money. Delivery of a practical whitebox assessment requires careful planning, however.
Increased Efficiency
It’s not possible to know everything, but the more we do know, the less time we spend on areas that are unlikely to yield meaningful results. This increased efficiency ultimately means that we’re saving you money.
It does not mean that blackbox testing is ineffective, merely that it is less efficient. The following fictional situation helps to give an illustration of how:
A blackbox assessment has identified that a web page is highly likely to be vulnerable to the very dangerous flaw; SQL injection. To determine how dangerous, we either need to exploit the bug, or precisely what the page (and more specifically the underlying database query) does.
Let’s imagine that the web page makes a database query like one of the following statements:
Something seemingly innocuous such as,
SELECT * FROM users WHERE id = '$USER_INPUT';
Or something more destructive like,
DELETE FROM users WHERE id = '$USER_INPUT';.
In the first instance, entering a typical attack string such as 'OR 1=1; -- might print the entire table of users into the response, and entering the same payload in the second query might wipe out all user records.
Through blackbox testing, the tester can infer which is more likely by the application behaviour but the tester would need to move carefully and iteratively to minimise potential harm. In contrast, through whitebox testing, the tester can look at the source code of the application and immediately determine which is correct without risk to data and without significant time taken.
Reduced Risk to Service Availability
Another benefit of whitebox testing, and one which relates to the above example, is the significantly reduced likelihood of causing system damage or outages. Blackbox testing relies heavily on inference and triggering of characteristic behaviour.
The most common approach is fuzzing of parameters and services to trigger unintended actions. These further investigated to refine the attacks, but this is not an exact science.
When attempting to identify flaws such as memory corruption or buffer overflow, there is a significant risk that a service outage or data corruption may occur. whitebox testing rarely has this problem as we can review the suspect system components directly.
Reduced Time
Closely related to the efficiency savings above, whitebox testing typically requires less time than blackbox testing, although these time-savings are dependent on the actual scope of the assessment.
An excellent example of time saved is when conducting a review of WiFi security, especially password policies. Typically, a WiFi penetration test begins with the arrival on-site, followed by capturing of a WPA2 handshake, followed by attempts to crack it. The tester might be successful, but the tester might not have full confidence that no weak passwords exists as it’s impossible to know without allowing an extremely long time-window. However, from a whitebox perspective, a review of your device and password configuration is typically all that’s required.
Specific Examples
Here are some real-life examples that could have benefited from a whitebox approach.
- A cryptographic system used the same key throughout. Initially the data appeared random (sufficiently encrypted) — primarily because of initialisation vectors. But a glance at the system configuration would have shown that all keys were hardcoded. It was tough for a tester to figure out from a blackbox perspective but a ex-employee would have had no issues which whitebox testing would uncover quickly.
- The blackbox tester spends significant time writing exploit code for a potentially vulnerable system to prove that it is an exploitable risk, but then discovers the system should not be on the network. From a whitebox perspective a cursory look at a network diagram would have achieved the same result and allowed more time for testing other systems.
- Testers often spend significant time brute forcing discovery of web directories, in case of an accidental “/backup”, or similar sensitive folder and data which results in the compromise of the application. Using the blackbox method, the volume of queries puts load on the web service and the test takes a long time whereas using the whitebox method a simple copy of the directory listing and server configuration would detail all folders available.
Conclusion
By becoming more open to sharing internals, primarily source code, network diagrams and credentials with the people assessing the security of their systems, companies gain a far higher level of assurance and more coverage for the test time allocated.
Rigorous use of blackbox testing may on the surface closely resemble an external attack, within the confines of the time allowed. However, a targeted attack won’t stick to a defined time-window, is unlikely to be concerned with causing system outages, and can afford to spend days, weeks, or months gathering sensitive information through other means.
The testing company has to work within your testing budget. So to ensure your testing has the maximum coverage and context for the time allowed, whitebox testing by providing the tester with full details of the environment, will give the better result.