Agile software development models have become the de-facto standard. They are taught at universities and implemented in practice as far as possible. Anyone who doesn’t develop software using agile processes is on the verge, and already tilting towards that. At least that is how it seems.
Consequently, the question is not whether the integration of penetration testing into the agile environment is feasible, but how we can integrate effective controls into the new, hip, rapid approaches.
Many features, many bugs
With classic software development processes such as waterfall or Rational Unified Process (RUP), penetration testers had to invest lots of time to sift through the numerous changes and understand them. That led, at worst, to another round in the development process to weed out the bugs.
In other words, another delay before the new functionalities finally reached the customer.
Agile FTW
The primary goal of agile methods is to provide benefits to the end user as soon as possible. Depending on the implementation of the process, new functionalities are made available within two to four weeks. To this end, work is divided into small tasks, prioritized according to the customer’s wishes, developed and built into production in a cycle. Such a cycle is typically called a sprint.
The advantage is clear: The customer is constantly involved and can adjust the priorities according to his requirements. He can also react relatively quickly to short-term changes in the market.
Trust is good, control is better
“Change” and “security” have always had a difficult relationship. The challenge is to be able to guarantee the security of the software and the data processed with it sprint after sprint. Hence, the question arises as to whether, how, where and when we can incorporate meaningful controls in the very short project cycles.
Healing through self-knowledge
Performing a full-blown penetration test every two weeks, we agree, is neither efficient nor financially feasible.
“Security is a process”, we hear evangelists in our industry say. We do assessments and audits. The results are a snapshot in time that becomes at least partially invalid within days or weeks. We can’t offer a service that could provide an up-to-date attestation at any time.
A true on-demand service would be a hotline for the developer, available at the time the expertise is needed.
Adventure is just bad planning
Finding the right time for a test is no big deal in agile approaches with defined release dates. However, if a team works according to a rolling concept such as Kanban, then releases are continuous. In that case, the tester must prove outstanding flexibility.
A team working on-call could handle the spontaneous requests. The duration of the work could be estimated and reserved in advance based on the planned features.
Let’s throw bug bounties at it!
Public bug bounties are complementary means to get a regular coverage for exposed services. Further, they provide a way to report vulnerabilities responsibly. However, they cannot offer guaranteed testing for critical features before they hit production.
Do we still need reports?
Software development is supported with tools. Tickets are pushed through workflows and these notify the team members of the progress of individual work packages. It would be logical that the results of a test would also be included in these tools.
So is a report still appropriate? A quarterly report or an annual report for the attention of the management, perhaps. In the form of a statement for end customers, maybe. But the developers do not need a traditional report, because it simply does not fit into their habits.
It is very difficult for an external tester to assign a vulnerability to a single ticket. Sometimes vulnerabilities arise in a constellation of innovations or only in the productive environment. Someone needs to bridge the gap here.
Reduce to the minimum
It would make sense that one person in the project team was responsible for explaining the features and providing up-to-date documentation.
Testers needs to know which features might really be security relevant. They also need an explanation of how the features should be used. This also includes defining what the necessary privileges or preconditions are and how to recognize misbehaviors.
Further, ideally the tester would not change all the time. Otherwise a lot of prior knowledge about the application and the business goals will lack in the attack scenarios of the penetration test.
Preparation is everything
It is important to set solid groundwork for testing. This includes making sure the tester understands what the final production environment will look like, but also how to get access to ticketing, documentation, or even source code. A set of test accounts and valid test data are a must. Sometimes, one or two configuration changes are needed to run a test efficiently. For instance, iOS or Android apps that are optimized for testing.
Our experience with various companies that follow agile models shows that the expression and maturity of the working methods varies enormously. For example, the automatic creation of a fully functional test environment, including provision of good test data, is rarely implemented.
Reality check
Automation is one of the keys to secure software. Deploying and populating test environments automatically, static source code analysis, vulnerability scanning, fuzzing, etc. This achieves basic coverage, gaining time to collect more critical features over several sprints and then have them tested in one run.
Training is another important aspect of software development. We see a trend in having one team member specializing in software and infrastructure security. The so-called security champion is the point of contact for the team, bears responsibility for the security of the developed components and collects or coordinates penetration tests with external suppliers. He or she is also responsible for assessing the findings from the tests and coordinating any remediation.
Conclusion
The condition for successful implementation of penetration tests for every software release is that hardly any effort be required for preparation and coordination. This means that an organization must demonstrate an above-average degree of automation.
In our experience, this is still rarely the case. As a result, the weight tips from penetration testing to preparation effort, and the tests become uneconomical.
The isolated analysis of individual innovations carries the risk that vulnerabilities caused by the overall construct of the software or infrastructure will not be detected. Consequently, in most cases we still advise against tying manual penetration testing to every release date of an agile project.
Ergo, most companies nowadays are better suited with quarterly or semi-annual “managed penetration testing”.
Leave a Reply