Skip to main content

Why to do Software Testing?

Regardless of the limitations, testing is an integral part in software development. It is broadly deployed in every phase in the software development cycle. Typically, more than 50% percent of the development time is spent in testing. Testing is usually performed for the following purposes:

  • To improve quality.

As computers and software are used in critical applications, the outcome of a bug can be severe. Bugs can cause huge losses. Bugs in critical systems have caused airplane crashes, allowed space shuttle missions to go awry, halted trading on the stock market, and worse. Bugs can kill. Bugs can cause disasters. The so-called year 2000 (Y2K) bug has given birth to a cottage industry of consultants and programming tools dedicated to making sure the modern world doesn't come to a screeching halt on the first day of the next century. [Bugs] In a computerized embedded world, the quality and reliability of software is a matter of life and death.
Quality means the conformance to the specified design requirement. Being correct, the minimum requirement of quality, means performing as required under specified circumstances. Debugging, a narrow view of software testing, is performed heavily to find out design defects by the programmer. The imperfection of human nature makes it almost impossible to make a moderately complex program correct the first time. Finding the problems and get them fixed [Kaner93], is the purpose of debugging in programming phase.

  • For Verification & Validation (V&V)

Just as topic Verification and Validation indicated, another important purpose of testing is verification and validation (V&V). Testing can serve as metrics. It is heavily used as a tool in the V&V process. Testers can make claims based on interpretations of the testing results, which either the product works under certain situations, or it does not work. We can also compare the quality among different products under the same specification, based on results from the same test.
We can not test quality directly, but we can test related factors to make quality visible. Quality has three sets of factors -- functionality, engineering, and adaptability. These three sets of factors can be thought of as dimensions in the software quality space. Each dimension may be broken down into its component factors and considerations at successively lower levels of detail. Some of the most frequently cited quality considerations are:(Typical Software Quality Factors [Hetzel88] )

Functionality (exterior quality) -

  • Correctness
  • Reliability
  • Usability
  • Integrity

Engineering (interior quality)

  • Efficiency
  • Testability
  • Documentation
  • Structure

Adaptability (future quality)

  • Flexibility
  • Reusability
  • Maintainability

Good testing provides measures for all relevant factors. The importance of any particular factor varies from application to application. Any system where human lives are at stake must place extreme emphasis on reliability and integrity. In the typical business system usability and maintainability are the key factors, while for a one-time scientific program neither may be significant. Our testing, to be fully effective, must be geared to measuring each relevant factor and thus forcing quality to become tangible and visible. [Hetzel88]
Tests with the purpose of validating the product works are named clean tests, or positive tests. The drawbacks are that it can only validate that the software works for the specified test cases. A finite number of tests can not validate that the software works for all situations. On the contrary, only one failed test is sufficient enough to show that the software does not work. Dirty tests, or negative tests, refers to the tests aiming at breaking the software, or showing that it does not work. A piece of software must have sufficient exception handling capabilities to survive a significant level of dirty tests.
A testable design is a design that can be easily validated, falsified and maintained. Because testing is a rigorous effort and requires significant time and cost, design for testability is also an important design rule for software development.
For reliability estimation [Kaner93] [Lyu95]
Software reliability has important relations with many aspects of software, including the structure, and the amount of testing it has been subjected to. Based on an operational profile (an estimate of the relative frequency of use of various inputs to the program [Lyu95]), testing can serve as a statistical sampling method to gain failure data for reliability estimation.
Software testing is not mature. It still remains an art, because we still cannot make it a science. We are still using the same testing techniques invented 20-30 years ago, some of which are crafted methods or heuristics rather than good engineering methods. Software testing can be costly, but not testing software is even more expensive, especially in places that human lives are at stake. Solving the software-testing problem is no easier than solving the Turing halting problem. We can never be sure that a piece of software is correct. We can never be sure that the specifications are correct. No verification system can verify every correct program. We can never be certain that a verification system is correct either.

Comments

Wicky said…
" I want to share some insights about a recent survey results about software defects

.
1. Developers take 100 times less effort to find and fix a problem than one reported by a customer.
2. Half of software project work is wasted on unnecessary rework.
3. Twenty percent of the defects account for 80% of the rework.
4. Twenty percent of modules account for 80% of the defects and half the modules have no defects.
5. Ninety percent of the downtime comes from 10% of the defects.
6. Peer reviews catch 60% of the defects.
7. Directed reviews are 35% more effective than nondirected ones.
8. Discipline can reduce defects by 75%.
9. High-dependability modules cost twice as much to produce as low-dependability ones.
10. Half of all user programs contain nontrivial defects.

Software testing and analyzing source code using static tools will help developers to minimize the risks arising out of software defects. Companies like Symbian , Juniper networks ,Research in Motion(Blackberry),Cisco are using Coverity Prevent, a Static analysis code inspection tool for analyzing source code for fixing defects .
Coverity Prevent is also used by the Department of Homeland security to scan many open source projects. you can get more info at at http://www.Coverity.com
"

Popular posts from this blog

HP Quality Center - Best Practices

1.Introduction Quality Center is a test management tool which provides very good features for managing both your manual and automated test cases. This paper highlights the best practices for managing your test cases. When you open Quality center, depending on your rights it display the below mentioned option in the sidebar: 1. Requirements 2. Test Plan 3. Test Lab 4. Defects 5. Dashboard 2.Requirements When you have assigned with the responsibility of developing your test cases in the quality center then you must be wondering where to start with. I am going to share my experience to overcome such a situation. You need to find the solution of some question before you start writing your test cases. 1. Is your requirement developed and available? 2. Is your requirement organized in a logical sequence? If answer to both of the above question is Yes, then you can start with Requirement option in the side bar. In case your requirement is under development, then you keep your...

ABC of Unit Testing

What is Unit Testing? After writing any code, every programmer will do some kind of testing to make sure the code works as expected. This testing is called 'unit testing'. Unit testing is done in different ways. Some programmers write simple test applications to test their own code. Some others simply debug the code and change the values during debugging to make sure the code works fine for different cases. Some facts about unit testing: Unit testing is done by developers. Quality department does different tests, they are not unit tests. Every programmer must do unit testing after finishing development or during development itself. Unit testing can be done by writing separate test applications to call your classes and methods to make sure the classes work as expected. This is called manual unit testing. Unit testing can be automated by writing unit test scripts. This is called Automated Unit Testing. In automated Unit Testing, after writing each class/method, you will write sev...