Why you should do testing

Someone once said "We are only going to do some testing once it is perfect". That kind of defeats the purpose of testing. Testing should be done to determine if it's perfect or not. And most of the times, especially a first delivery will be far from perfect.

One reason, building software is a human task and to err is human. Another reason, it might work perfect on the device of the developer but it might not work on someone else's device because of different settings or a different Windows or Browser version. And last but not least, the data. Usually during development, a database with much less data is used. Performance may not be an issue then, but it could be an issue on the real life data. Or settings were used in the production database that were different in the developer's database.

Enough reasons to do some serious testing. And here's how to do that. As an example I will be using the testing of an Order Entry Application.

Make Testcases

First thing you need to do is make some Testcases for common processes. One Testcase per process. And keep them as simple as possible, this doesn't have to be too extensive. Usually a bullet list of the steps necessary in the process will do. As long as you cover all the basic steps that are commonly used. Also, give some guidelines how data should be entered.

So for the Order Entry Application that could be something like:

Start entering a new Sales Order
Select a Customer using the pop-up
Enter the Order Date
Enter the Delivery Date
Fill out the Reference number using the current date and your initials
On the first Order Line, select an Item
Change the Quantity
Add another Order Line
Save and close the Sales Order

Be brief with your Testcases. Verify them with users that should be doing the process and add steps you might have forgotten. A Testcase is not a fixed text to be used over and over again. It should be updated as you go along and learn. Do some steps need extra attention? Update your Testcase. Are you going to the next version with new features? Update your Testcase.

Know what to expect

Once you have the basic process listed, make sure the user who will perform the test knows what to expect. In other words, let the user know what the application should or shouldn't do:

Start entering a new Sales Order
Select a Customer using the pop-up
Only active Customers can be selected. You cannot create a new Customer here.
Enter the Order Date
The current date is pre-filled
Enter the Delivery Date
The current date is pre-filled
Fill out the Reference number using the current date and your initials
On the first Order Line, select an Item
The Price and the Amount are automatically filled. The Quantity will be set to 1
Change the Quantity
The Amount is automatically updated when the Quantity changes
Fractions are not allowed
Add another Order Line
Save and close the Sales Order

Too often the users who will be doing the testing were not involved in the design process. So make sure the user who will do the testing knows what to expect and how the application should behave. It will save you a lot of false negatives in the test results.

Do some preparation

Before you start doing the testing, make sure you have enough data to do the testing. Also be sure that the data is recognizable. For example, the script says you should only be able to select active Customers. So create a new Customer called Active which is active, and another one called Inactive which is not active. The same goes for everything where you know that there should be a restriction when selecting data.

When it comes to prices, use round and easy to recognize prices. Yes, everybody can do a little math and there is always the calculator. But you will not be testing the math skills of your users. make it easy for them. 7 items with a price of 100 should give an amount of 700. And 14 items with a price of 283.26 using 5% discount? Hmm, good luck and better get that calculator ready.

So, never start testing randomly with existing data because it is difficult to check if the data matches the conditions or not.

Expect the Exceptions and Be Negative

One thing a lot of people do when testing software is test if it works as expected. And true, that is what the software should do. But even more important is to test what should not happen. And this can often be more tricky and more work than testing how it should work. So let's go back to our example:

Start entering a new Sales Order
Select a Customer using the pop-up
Do not use the pop-up but enter the Customer Number of the Inactive Customer directly into the field. The input should be rejected
Enter the Order Date
Use an invalid date format. It should switch back to the current date
Enter the Delivery Date
Use a date before today. It should switch back to the current date
Fill out the Reference number using the current date and your initials
Fill out a very long reference number (over 50 characters). Only 20 characters should be accepted
On the first Order Line, select an Item
Change the Quantity
Use a negative value. This should not be accepted and should switch back to 1
Use a quantity greater than 10, there should be a 5% discount
A value like 2.5 should not be accepted
Save and close the Sales Order

During testing, be negative and fill out forms they way you should not do it. Use invalid or incomplete dates. Use text in fields that should contain only numbers. In text fields, use special characters. Leave mandatory fields empty. In other words, be the worst user in history to ever work with this application.

Check the results

So you filled out everything correct and there were no errors. That looks good, but is everything as expected? Did you check that the discount was correctly applied for a Quantity greater than 10? And if so, if you change it back to 3, is the discount ignored? This is why good preparation in setting the data is important. It just makes it so much easier to check the results.

So everything looks ok? Then Save and Close the Sales Order. Then open it again. Everything looked ok, but was everything saved ok as well?

Report the issues

When the user reports back the issues found while testing, always ask for enough feedback so you are able to reproduce what happened. "It didn't work" does not provide you with enough feedback. What didn't work? What values were you using? What did you expect to happen and what actually happened?

Verify that something didn't work as expected. Always. Often users tend to report issues for things they would like to happen, not what should happen.

Prioritize the issues

All issues are important and need to be solved. No discussion there. Still, some things are more important than others. So it's good to let people know how they should prioritize the issue. A common way to that is like this:

Blocking Issues
If it happens in daily or critical processes and there is no workaround, then it is blocking
High Priority
If it happens in daily or critical processes but there is workaround, then it should be fixed a.s.a.p.
Medium Priority
If it doesn't happen in daily or critical processes and there is no workaround, then it should be fixed once the Blocking and High Priority issues are fixed
Low Priority
If it doesn't happen in daily or critical processes and there is a workaround, then it should be fixed once everything else is fixed
Suggestions
If it works as expected but can be improved, it is a suggestion and doesn't need fixing.

Especially if the application needs to go live on a specific date, it is important to prioritize the issues. For example when the existing application can not be used after that date. Time should not be wasted on Low Priority issues while there are still things that prevent you from going live. In an ideal world all issues should be fixed before the go-live, but in reality?

Validate the process after fixing

All too often it is only checked that the fix solved the problem as reported. But be sure to check if the fix didn't break anything that was working well. So there was a problem with calculating the VAT Amount when the Quantity changed and that is solved? Do check if the Discount is still calculated correctly because that also depends on changing the Quantity. Better said, validate the whole process. That's also the reason why you should keep your Testcases as simple as possible.

Automated testing

There are tools to do automated testing of web applications. Tools that will start a program, fill out forms and evaluate the results. But writing good scripts for automated testing is a job in itself. Such tools are usually used by software developers who work on a large application for a prolonged time. In that case, writing good automated tests will be worth the time since they will be used over and over again. For custom development however, writing such tests might take much more time than doing manual testing. Which kind of defeats the purpose of automation. But if you want, be sure to check such tools and see if it is applicable to use them.

Let's wrap it up

Testing is an integral part of software development. And testing is done on different levels. Of course, during the development stage a lot of technical testing is already done. So any delivery will have a certain quality. But then it is necessary to test on different environments, different data and by different people. Something might be perfect on a technical level, but does not deliver the results as expected. Be sure to test, always. And if you need any advice, just leave a comment below.

You can leave a comment or send a question to andre@andrespeek.com. Let me know if you got inspired and stay safe and healthy always!