At the planning stage for a software test.
Following on from our previous articles we are now at the planning how the test should
be conducted is equally as important as doing the test itself. This is to ensure all the areas are covered in a methodical fashion and make best use of the resources available. This requires an appreciation of the underlying architecture, how the schema is designed, what modules are connected to what, what language as it is written etc. This determines the parameter of the test and gives some indication of expected results.
Threat modelling for software testing.
Threat modelling is the setting out of potential scenarios of what could go wrong. Is the database accessible from the net, could it be susceptible to a SQL injection attack? Are there controls on data input to limit the potential for a buffer overflow attack? Is there sufficient user authentication to limit the potential for collusion. All these potentials should be compared with business case to see what ramifications could take place if any area of the program was compromised.
How much should you test?
How fars should you go with your test? How long is a piece of string as programs are incredibly complicated with multiple uses and endless scenarios. The following are terms of reference of tests can be done within the time resource constraints.
- Statement coverage: Major components given a once over. This is not considered enough for release of commercial software.
- Branch coverage: goes through each multiple decision point to achieve a single expected response.
- Conditional coverage: Going through all possible scenarios (i.e. everything identified in the threat modelling stage) to check behaviour and responses.
- Loop, Path, data flow: Going beyoind the desicion ponts, we now follow each trail to it’s end conclusion depending upon program layout. Test inputs for expected results, then test volume, and different types to random to see what it can handle.
- Stress testing is pushing the program to the limit with volume and type of data input. See what happens when a user abuses their interface by putting in invalid or rubbish commands. Called negative/misuse testing. Like fuzz testing as mentioned in our previous article.
- Regression testing is done when ever the software goes through an update or change to see if any unexpected changes. As more software is increasing based on a relational database model, each module becomes multi-dependant on other sections then a minor change in one part can have unexpected ramifications in another section. I have seen a purchase ordering system cease to function due to an upgrade in the accounting module. Seemingly unrelated, but there was a link at the backend.
The tools used to test the program are dealt with in the following article.
Sometimes you can’t do everything.
Ultimately it is almost impossible to pick up on every bug or iteration. There needs to be a realistic view set on what is realistic and achievable to perform sufficient due diligence. Sometimes issues are not apparent until the solution is used in the field. In that eventuality there needs to be a prepared and ready response depending on the sensitivity and importance of the issue. As software changes and evolves then the requirement for constant testing will be part of the process of getting solutions to market.
Next in the series “What documentationdo we need”?
