Testing security controls and security components boils down to making sure that security controls behave as expected under chosen circumstances. Examples include active/passive attack on API calls wrapped in HTTPS, passing SQL injection patterns into user input, manipulating parameters to mount path traversal attacks, etc.

Some of the helpful open-source tools for such testing are:

BDD-security suite used as a testing framework for functional security testing, infrastructure security testing, and application security testing.

Gauntlt, a number of Ruby hooks to security tools to integrate into your CI infrastructure.

OWASP Zap and OWASP Zapper (Jenkins plugin) allow automating attack proxy to test some of the possible attacks.

Mittn, F-Secure’s security testing tooling for CI.

Testing memory behaviour

Buffer overflows and remote code execution are among the most dangerous and damaging security issues your code might create. Detecting memory problems, unidentified behaviour and other glitches that lead to attacks on execution flow can be automated through source code analysis. Most of the warnings to look at are memory leaks, buffer sizes, and such.

Fuzzing to find vulnerabilities

“Never trust your input” is one of the cardinal rules of computer programming. And fuzzing is an automated process in software testing that takes advantage of this rule and searches for exploitable bugs through feeding random, invalid, and unexpected inputs to the tested software.

The data provided by the fuzzer is just “ok” enough for the parser not to reject it, but is rubbish otherwise and the result of feeding it into the tested software can lead to quite unexpected outcomes. This helps surface the vulnerabilities that would be undetectable otherwise. Another advantage of fuzzing is that this kind of testing is almost fully devoid of false positives (which quite often take place with static analysers).

Security-wise, fuzzing is both about testing the security controls and testing memory behaviour, as bugs like famous Heartbleed (which combines both poor security controls and unexpected memory behaviour) could’ve been fuzzed easily.

For security testing, fuzzing is especially useful when it comes to feeding into the app such pseudo-valid inputs that cross the trust boundary. A trust boundary violation takes place when the tested app is made to trust the unvalidated data fed into it. This approach mimics an adversary trying to feed malicious content into the app in the hope of achieving privilege escalation or plain malfunction, crash, etc.

All fuzzing approaches will find vulnerabilities, and the more fuzzing is done — the better. However, to get fuzzing done right, you need to have at least a general idea of what you’re trying to accomplish. Fuzzing tests need to be well thought out, well planned, and well written. Fuzzing is also affected by the execution environment (the ripple effect), configuration, and capabilities of the test suite. It will not be a magic bullet for all your automated security testing, but it will take you far — but only as far as you’re willing to invest time and effort into preparation.

Testing larger entities and high-level behaviour

Apart from actually testing the code you write, it’s useful to test larger entities: whole services and infrastructural components. It is important when the product you’re developing consists of many high-level entities, micro-services, and holds many autonomous dependencies.

There are vulnerability scanners that also help with automating security testing through scanning your websites and/or network for a huge number of known risks. The result of such testing is usually a list of vulnerabilities detected in your infrastructure and recommendations on how they can be patched or otherwise secured. Sometimes the patching process can also be performed by the automatic vulnerability scanners.

This is specifically relevant for software that is being composed, rather than top-down written, which consists of many services, libraries, and chunks of code. Some of the popular free network vulnerability scanners include Open Vulnerability Assessment System (OpenVAS), Microsoft Baseline Security Analyzer, Nexpose Community Edition, Retina CS Community, SecureCheq.

It is worth noting that infrastructures should be checked when they are complete and functional (live or near-live) for the maximum impact and usefulness of the check-up.

Performance testing for security purposes

Performance testing is something that doesn’t really spring to mind in the context of “security testing”, but performance reliability is really the first step towards ensuring a safe and secure functioning of a system. One of the risks to consider is a denial of service caused by an overload or by (D)DoS-type attacks. Regardless of the possible cause, it is necessary to have an exact estimation of the future calculated load level and the point beyond which the denial of service happens, at the moment when the system is designed.

Despite the lack of a single yardstick for different systems, it is possible to get useful testing results. If they are recorded and published along with the characteristics of the testing platform, they will help the system architects. Running performance tests on the target equipment after the installation and configuration of the software will also yield more precise the threshold levels. This, in turn, will help to configure the load-limiting and alert systems accordingly.

686 cybersecurity breaches were reported in the first three months of 2018 alone, with unauthorized intrusion accounting for 38.9% of incidents. And with high-profile data breaches dominating headlines, it’s clear that while modern, complex software architecture might be more adaptable and data-intensive than ever, securing that software is proving a real challenge.

Penetration testing (or pentesting) is a vital component within the cybersecurity toolkit. In theory, it should be at the forefront of any robust security strategy. But it isn’t as simple as just rolling something out with a few emails and new software – it demands people with great skills, as well as a culture where stress testing and hacking your own system is viewed as a necessity, not an optional extra.

This is where artificial intelligence comes in – the automation that you can achieve through artificial intelligence could well help make pentesting much easier to do consistently and at scale. In turn, this would help organizations tackle both issues of skills and culture, and get serious about their cybersecurity strategies.

This is where artificial intelligence comes in – the automation that you can achieve through artificial intelligence could well help make pentesting much easier to do consistently and at scale. In turn, this would help organizations tackle both issues of skills and culture, and get serious about their cybersecurity strategies.

The shortcomings of established methods of pentesting

Typically, pentesting is carried out in 5 stages:

Every one of these stages, when carried out by humans, opens up the chance of error. Yes, software is important, but contextual awareness and decisions are required.

This process, then, provides plenty of opportunities for error. From misinterpreting data – like thinking a system is secure, when actually it isn’t – to taking care of evidence and thoroughly and clearly recording the results of pentests, even the most experienced pentester will get things wrong.

But even if you don’t make any mistakes, this whole process is hard to do well at scale. It requires a significant amount of time and energy to test a piece of software, which, given the pace of change created by modern processes, makes it much harder to maintain the levels of rigor you ultimately want from pentesting.

This is where artificial intelligence comes in.

The pentesting areas that artificial intelligence can impact

Let’s dive into the different stages of pentesting that AI can impact.

#1 Reconnaissance Stage

The most important stage in pentesting is the Reconnaissance or information gathering stage. As rightly said by many in cybersecurity, “The more information gathered, the higher the likelihood of success.” Therefore, a significant amount of time should be spent obtaining as much information as possible about the target.

Using AI to automate this stage would provide accurate results as well as save a lot of time invested. Using a combination of Natural Language Processing, Computer Vision, and Artificial Intelligence, experts can identify a wide variety of details that can be used to build a profile of the company, its employees, the security posture, and even the software/hardware components of the network and computers.

#2 Scanning Stage

Comprehensive coverage is needed in the scanning phase. Manually scanning through thousands of systems in an organization is not ideal. Nor is it ideal to interpret the results returned by scanning tools. AI can be used to tweak the code of the scanning tools to scan systems as well as interpret the results of the scan. It can help save pentesters time and help in the overall efficiency of the pentesting process.

AI can focus on test management and the creation of test cases automatically that will check if a particular program can be tagged having security flaw. They can also be used to check how a target system responds to an intrusion.

#3 Gaining and Maintaining access stage

Gaining access phase involves taking control of one or more network devices in order to either extract data from the target, or to use that device to then launch attacks on other targets. Once a system is scanned for vulnerabilities, the pentesters need to ensure that the system does not have any loopholes that attackers can exploit to get into the network devices. They need to check that the network devices are safely protected with strong passwords and other necessary credentials.

AI-based algorithms can try out different combinations of passwords to check if the system is susceptible for a break-in. The algorithms can be trained to observe user data, look for trends or patterns to make inferences about possible passwords used.

Maintaining access focuses on establishing other entry points to the target. This phase is expected to trigger mechanisms, to ensure that the penetration tester’s security when accessing the network. AI-based algorithms should be run at equal intervals to time to guarantee that the primary path to the device is closed. The algorithms should be able to discover backdoors, new administrator accounts, encrypted channels, new network access channels, and so on.

#4 Covering Tracks And Reporting

The last stage tests whether an attacker can actually remove all traces of his attack on the system. Evidence is most often stored in user logs, existing access channels, and in error messages caused by the infiltration process. AI-powered tools can assist in the discovery of hidden backdoors and multiple access points that haven’t been left open on the target network; all of these findings should be automatically stored in a report with a proper timeline associated with every attack done.

A great example of a tool that efficiently performs all these stages of pentesting is CloudSEK’s X-Vigil. This tool leverages AI to extract data, derive analysis and discover vulnerabilities in time to protect an organization from data breach.