Test cases are very important for ensuring quality in your ERP software testing process. They are the first steps in a test cycle and if test cases aren’t of sufficient quality, then the whole project will be ‘burdened’. Writing “great” test cases is a skill that gets form by simply doing it. But it’s very handy to have some insights that could help you. With this article I want to reach out to you and give suggestions to make it easier, more fun and better. The tips that will be given will in particularly relate with test cases at acceptance testing of ERP systems.
What are test cases?
Within the world of software testing, there are a lot of definitions of test cases. Because of this it is important to name our definition. In our philosophy test cases are (based on IEEE610): “A set of test inputs, execution conditions, and expected results developed for a particular objective.” Knowing how to write good test cases is extremely useful for anyone who wants to test. Whether it’s a functional test, a user acceptance test, testing a web application or a module of an ERP-system. In all situations described above the test cases determine to what extend the results give a verdict on the pre-set targets.
Why is writing test cases so hard?
As described in the definition, Test cases help guide testers through a sequence of test instructions to determine whether or not the the software meets the pre-defined requirements. The execution of test cases helps us gather and discover information to realize that specific goal or target. The first problem we directly encounter is the diversity of possible targets. And because there are different types of test and goals, there are different types of corresponding test cases.
A second problem is related to the content of the actual test instruction or steps. The level of these instructions is dependent on the type of tester that has to interpret this information and give an opinion. A professional tester will need different instructions as opposed to an end user that is involved with acceptance testing of an ERP-system
Tip 1: Determine your goal and what you want to report
Think about what you want to report, so that you can determine your derivative goal. Using that as a base, you’ve got the outline of the test case in view. There are many different goals, where we always have to ask ourselves what we are trying to learn or achieve when we are going to run the test. Here are some examples:
- Finding defects: This is the classic purpose of testing. A test is performed to expose defects.
- Maximize the number of bugs: The distinction between “maximize the number of bugs” and “finding defects” is that the total number of bugs is more important than the coverage.
- Block premature product releases: The goal of this test is to prematurely find as many severe defects (showstoppers) so that nobody takes the product into production.
- Support managers with their go/no-go decisions: Managers takes decisions based on risks. Risk indications as test coverage, impact of issues found, etc. Give them a better background on which to base their decisions.
- Assess conformance according to specification: the alleged specifications are checked for their operation. All matters that are not associated with the specifications are disregarded.
- Assess the quality: this is a difficult objective, because quality is multi-dimensional. The nature of quality is dependent on the nature of the product. To assess the quality clear criteria should be drawn up, that are defined in a way that they can actually be made measurable.
The test results as derived from the test cases provides direct relevant information about the goal.
Tip 2: reserve sufficient design time
Reserve enough time to design your test cases, so that they match your goals. Poor test cases will haunt you throughout the entire test process. Comparing test results, reporting on several test rounds, etc. Are essentially determined by the quality of your test cases.
If you are already running out of time to design, but still want to start testing, make sure you have at least defined the main risks. If 10 testers must review 5 test cases with 1 test step, this will result into 50 test results. No matter what these 50 results give more information about quality then doing nothing. This will probably not be exhaustive, but it’s a first step. From thereon, the level of detail of certain parts can be determined.
We prefer that you think well in advance about how the test should be designed, and that the results of the test give a real answer to the objective. But in reality this is sometimes unruly.
Tip 3: Name test case
Naming test cases is important. In an average ERP test you easily have more than 500 test cases. You will understand that a logical name structure enhances findability. In the literature is often referred to as a complete as possible name, in which the to be tested process, module, object etc. Are all included in the name. You can imagine that providing 500 test cases with such a complete title gives a chaotic administration. With a simple Excel sheet, you easily loose overview. Test management tools provide a structure to which you can relate test cases to reusable objects, without them “polluting” the name. You also have test registration tools that organize these relationships in a different way.
Within TestMonitor for example we invented a different solution. In the tool you can define labels or tags that you then can link to test cases. Within TestMonitor the test cases are linked to one or multiple business process-, risk-, requirement- or application labels. This allows test cases to be grouped and retrieved from different perspectives.
For the name of a test case within TestMonitor it suffices with a clear description of the purpose of the test case. To make it simple you describe an activity with an implicit expectation.
Example test case name:
“Termination of lease – independent home”
“Create customer” “Provisional booking receipt”
etc.
Example of test case “Provisional booking receipt” which is linked to several labels:
- business process ‘Receiving goods’
- requirement ‘Contract requirement’
- risk ‘Operational risk’
- application ‘ERP’
It’s important to describe the expected result per test case. The tester then knows in which direction the “answer” needs to be and directly gets an explicit testing framework.
Tip 4: Test step description
As described in the definition test cases are a collection of test instructions that help us discover information to achieve a particular goal.
A test case must have a clear beginning and end to determine whether the test case passed or failed. In addition, a test case is composed out of one or more test instructions or -steps, wherein there are multiple paths possible in order to achieve the desired result. Only testing the path of success is often insufficient. In certain situations following non-success paths can just make the difference.
It is important to define test steps as clearly as possible so that the end-user for a user acceptance test knows exactly what they have to do. Of course, there are pre-conditions to come up with, like functional level tester, knowledge of the new system, knowledge of the possible adjusted business processes, etc. But in essence everyone should understand all the test steps.
Suppose we further describe the test case “Lease termination – independent home” for a simple success path:
- Select a unit and start termination the lease. Control if there are prerequisites among which the termination of lease can/not be accepted and create a record of this.
- Schedule an appointment for the final inspection.
- Fill the data in the screen Termination of lease. Double check the tenant data in the lease termination card.
- Register the termination of lease and send a conformational letter. Check if the confirmation letter has come in the digital archive.
- Check in the contract register from the unit that the current contract is terminated, that the terminated contract is linked to the termination of lease and that there is a newly created a vacancy contract.
- Check the new (vacancy)contract based on the rental policy and element templates.
What do you notice?
- Each test step starts with a verb
- Followed by a topic
- Concluding with detailing and possibly control questions. You might choose to put the control questions in separate test steps, but the practice shows that the control question is a refinement of the action and then, logically, will be written down as a test step.
In the above mentioned example, the assumption is made that a specialist from a particular field will evaluate the test step. And because he or she is a specialist, no input conditions are given or explicit expectations outlined, because the specialist still has his own case studies up his sleeve and his expectations are crystal clear.
Should you currently not have a specialist, you can expand the test steps with input conditions and explicit expectations.
For example test step 1 at the test case “Termination of lease – independent home” with more details.
- Select the unit “Fleet street 677” and start the termination of lease. The conditions ensure that this unit cannot be terminated before the arrears have been paid.
- Etc.
Tip 5: Not more than 10 test instruction in 1 test case
We regularly come across test projects where >50 test step are assigned to one test case. This is too much for a few reasons:
- All test steps must be taken separately (or explicitly passed) before a test case gets a verdict.
- The final assessment of a test case is determined by the “worst” score. So it may well be that 49 test steps will be assessed as “good” and one as “wrong” what results in a “wrong” test case. The measurement of the test steps shall hereby be similar. By that I mean that virtually every test step must be equal to the effect of the evaluation. If you have 10 test steps you must follow, including 2 small test steps that they are disproportionate to the other 8 test steps, you need to rephrase them. The same applies the other way around with a tough test step.
- The tester quickly gets lost with too many test steps within a test case. We haven’t made a scientific study about it, but the practice shows that a test case shouldn’t include more than 10 test steps. One can think of many exceptions (conversion controls, etc), but in practice for a user acceptance test of an ERP system this works best.
- It is difficult for developers to reproduce the error found. With many test steps the developer will lose a lot of time trying to re-enact the situation.
- Reruns or retest of large test cases takes too much time. When a fault is detected by a tester, the corresponding test case will have to be retested. A retest requires that very step will be reassessed, you want to prevent regression errors obviously. By taking too many test steps, you’re most likely testing too much. This way the test process takes longer and can eventually overload your testers.
Beside that you also have test registration tools that can present the test cases in various forms to the tester. TestMonitor for example has two different view. TestMonitor has a display which shows all test steps separately per page and a display in which the test cases are presented per page including all test steps.The advantage of the first display is that each tester can obtain more information for each test step. The disadvantage in case the tester is more experienced he has to click on “next” every time he wants to proceed to the next test step. In the other view for each test case the advantages and disadvantages are vice versa.
Tip 6: Review by a non-designer/supplier
In practice, we regularly see test scripts prepared by the programmers of the software supplier. When reviewing these test scripts by the eventual testers, there usually are more questions than answers. Conversely, this works the same for test scripts prepared by their own employee’s. It gives a real added value reviewing these by the software supplier. They look at the completed test scripts with different eyes and always come up with meaningful additions or modifications.
With a little bit of brainstorming with the software supplier specialists and the client organization you quickly get focus on the essence of what needs to be tested. Then take the time to consider the test cases, non-success scenarios and you will see that your test model is rapidly becomes more extensive and detailed. Besides that, you get more information about the quality then you deemed possible.
Tip 7: TestMonitor
Beside that make use of a professional test platform and make quality really insightful. Request a free trial of TestMonitor today and see and experience the difference for yourself.
If you’re a newcomer in the world of testing, you easily become overwhelmed by all the testing terminology getting thrown around. In this article, we’ll try to explain some of the jargon you might encounter in your daily testing life, just to make it all a little bit more understandable.
Start testing with TestMonitor
