Software Testing Tutorial
- Introduction to Software Testing
- Principles of Software Testing
- Software Development Life Cycle (SDLC)
- Software Testing Life Cycle (STLC)
- Levels of Testing
- Database Testing
- Ad-hoc Testing
- Testing Types
- Test Closure Report Preparation
- Test Report Preparation
- ISTQB Organization and Certification Overviews
A defects is a software bug that arises when the expected result don’t match with the actual results. It can also be error, flaw, failure, or fault in a computer program. Most bugs arise from mistakes and errors made by developers, architects.
Following are the methods for preventing programmers from introducing bugs during development:
- Programming Techniques adopted
- Software Development methodologies
- Peer Review
- Code Analysis
9.1 Types of Defects
Since, the primary purpose behind testing is to trace out the maximum defects, present in a software product, a tester needs to be aware about the different forms of the defects, which may prevail in a software product. The understanding of the multiple variants of defects helps a tester, to identify and locate good amount of defects, easily and quickly.
Defects are primarily classified into following two types:
Product Defects are the defects that are introduced and detected during the various stages of software development life cycle. While the defects get introduced during the various activities of the phase, the detection occurs during reviews and various types of testing efforts.
The various types of defects occuring during the SDLC activities can be categorized as follows:
- Functionality Errors:
- External Interface Errors
- Performance Problems
- Environment Errors
Examples of SDLC defects:
- Requirements: Performance Exceptions of the system not properly defined
- Design: Program Specifications did not specify of number of decimal places
- Design: Architecture was not properly planned
- Coding: Variable no initialized, Missing if condition
Process defects are caused during execution of project and usually detected during audits. While such defects get introduced during various project activities and get detected during various conducted during the project life cycle. All such defects are of the nature of deviations in process implementations of the organization or the project. Though it is difficult to draw a co-relation between process and product defects, however, the existence of process defects are indicative of larger number of product defects. Also, existence of such defects raise a question mark about the quality of the software product.
Examples of Process Defects:
- Risk Management Plan is not in place
- Change request impact on schedule and effort not reflected
- Project Management Plan not kept up to date with current status of the project
- Requirements tracking is incomplete
- Review Plan is not adequate
- Requirements gathering strategy not defined for the product
- Quality goals are not tracked
- Issue tracking mechanism is adhoc
Apart from these two , we can classify defects in following ways:
- Extra defectsA requirement incorporated into the product that wasn’t given by the end client. This is usually a discrepancy from the specification, but it also may be characteristic requested by the user of the product.
Nevertheless, it is supposed a defect because it’s a discrepancy from the existing requirements.
- Missing defectsThe requirement given by the client was not done. This also is a discrepancy from the specifications, an feature that a specification was not performed. It also may mean that requirement of the client wasn’t noted appropriately.
- Wrong defectsThe requirements have been performed falsely. Actually, this defect is a discrepancy from the given specification.
9.1.1 Software Quality Assurance
Software Quality Assurance (SQA) is a set of activities for ensuring quality in software engineering processes (that ultimately result in the quality of software products).It ensures that developed software meets and complies with the defined or standardized quality specifications. SQA is an ongoing process within the Software Development Life Cycle (SDLC) that routinely checks the developed software to ensure it meets the desired quality measures.
SQA practices are implemented in most types of software development, regardless of the underlying software development model being used. SQA incorporates and implements software testing methodologies to test the software. Rather than checking for quality after completion, SQA processes test for quality in each phase of development, until the software is complete. With SQA, the software development process moves into the next phase only once the current/previous phase complies with the required quality standards. SQA generally works on one or more industry standards that help in building software quality guidelines and implementation strategies.
SQA includes the following activities:
- Process definition and implementation
Processes could be:
- Software Development Methodology
- Project Management
- Configuration Management
- Requirements Development/Management
- Software Design
- Testing, etc.
Once the processes have been defined and implemented, Quality Assurance has the following responsibilities:
- Identify the weaknesses in the processes
- Correct those weaknesses to continually improve the process
The quality management system under which the software system is created is normally based on one or more of the following models/standards:
- Six Sigma
- ISO 9000
Software Quality Assurance encompasses the entire software development lifecycle and the goal is to ensure that the development and/or maintenance processes are continuously improved to produce products that meet specifications/requirements.
9.1.2 Software Quality Control
Software Quality Control (SQC) is a set of activities for ensuring quality in software products. Software Quality Control is limited to the Review/Testing phases of the Software Development Life Cycle and the goal is to ensure that the products meet specifications/requirements.
It includes the following activities:
- Requirement Review
- Design Review
- Code Review
- Deployment Plan Review
- Test Plan Review
- Test Cases Review
- Unit Testing
- Integration Testing
- System Testing
- Acceptance Testing
The process of Software Quality Control (SQC) is governed by Software Quality Assurance(SQA).
Differences between Software Quality Assurance and Software Quality Control
Differences between Software Quality Assurance (SQA) and Software Quality Control (SQC) listed here are meant to discourage people from wrongfully using the term Quality Assurance (QA) and Quality Control (QC) interchangeably.
The differences is given in the table below:
|Criteria||Software Quality Assurance (SQA)||Software Quality Control (SQC)|
|Definition||SQA is a set of activities for ensuring quality in software engineering processes (that ultimately result in quality in software products). The activities establish and evaluate the processes that produce products.||SQC is a set of activities for ensuring quality in software products. The activities focus on identifying defects in the actual products produced.|
|Focus||Process focused||Product focused|
|Orientation||Prevention oriented||Detection oriented|
|Breadth||Organization wide||Product/project specific|
|Scope||Relates to all products that will ever be created by a process||Relates to specific product|
|Scope||Relates to all products that will ever be created by a process||Relates to specific product|
9.2 Causes of Defects
Sometimes, software systems don’t work properly or as expected. There may be various reasons for the improper working of any software application including. The system user is making some mistake in using the system or software. Causes of software defects will be much more.
Software developers or system builders made some mistakes while designing and building the system, leaving some flaws in the software or system. And this is avoiding the system to work correctly.
- Error (Mistake): Any human action that can produce a wrong result.
- Defect (Bug, Fault): A flaw in a component or whole system that may cause the system or particular component to fail to perform its intended function.
- Failure: Deviation of any component or system from its expected function or result.
Defects and failures may arise due to various environmental conditions including radiation burst, pollution in the environment, strong magnetic field etc. Some human error in interacting with the software or someone knowingly trying to fail the system. As a result of previous error, intentional damage or previous defects in the system
Following are the different causes of defects in a system or a program:
- Errors in the specification and design of the softwareSpecification is basically a written document which describes the functional and non – functional aspects of the software by using prose and pictures. For testing specifications there is no need of having code. Without having code we can test the specifications. About 55% of all the bugs present in the product are because of the mistakes present in the specification. Hence testing the specifications can save lots of time and the cost in future or in later stages of the product.
- Errors in use of the systemErrors in use of the system or product or application may arise because of the following reasons:
- Inadequate knowledge of the product or the software to the tester. The tester may not be aware of the functionalities of the product and hence while testing the product there might be some defects or failures.
- Lack of the understanding of the functionalities by the developer. It may also happen that the developers may not have understood the functionalities of the product or application properly. Based on their understanding the feature they will develop may not match with the specifications. Hence this may result into the defect or failure.
- Environmental conditionsBecause of the wrong setup of the testing environment testers may report the defects or failures. As per the recent surveys it has been observed that about 40% of the tester’s time is consumed because of the environment issues and this has a great impact on quality and productivity. Hence proper test environments are required for quality and on time delivery of the product to the customers.
- Intentional damageThe defects and failures reported by the testers while testing the product or the application may arise because of the intentional damage.
- Potential consequences of earlier errorsErrors found in the earlier stages of the development reduce our cost of production. Hence it’s very important to find the error at the earlier stage. This could be done by reviewing the specification documents or by walkthrough. The downward flow of the defect will increase the cost of production.
Apart from these basic reasons there are some other reasons that may contribute to Defects and failures in software:
It is because human beings develop software.Human beings are not perfect. They are prone to make mistakes. As human beings develop software, it would be foolish to expect the software to be perfect and without any defects in it.Hence there are errors in software.
Unrealistic development timeframe
Let’s face it. More often than not software are developed under crazy release schedules, with limited/insufficient resources and with unrealistic project deadlines. So it is probable that compromises are made in requirement/design to meet delivery schedules. Sometimes the programmers are not given enough time to design, develop or test their code before handing it over to the testing team. Late design changes can require last minute code changes, which are likely to introduce errors.
Poor coding practices
Sometimes errors are slipped into the code due to simply bad coding. Bad coding practices such as inefficient or missing error/exception handling, lack of proper validations (datatypes, field ranges, boundary conditions, memory overflows etc.) may lead to introduction of errors in the code. In addition to this some programmers might be working with poor tools, faulty compilers, debuggers, profilers, validators etc. making it almost inevitable to invite errors and making it too difficult to debug them.
Buggy third-party tools
Quite often during software development we require many third-party tools, which in turn are software and may contain some bugs in them. These tools could be tools that aid in the programming (e.g. class libraries, shared DLLs, compilers, HTML editors, debuggers etc.) or some third-party shrink-wrapped plug-ins/add-ons used to save time (like a shopping cart plug-in, a map navigation API, a third party client for 24X7 tech support etc.). A bug in such tools may in turn cause bugs in the software that is being developed.
Lack of skilled testing
No tester would want to accept it but let’s face it; poor testing do take place across organizations. There can be shortcomings in the testing process that are followed. Lack of seriousness for testing, scarcity of skilled testing, testing activity conducted without much importance given to it etc. continues to remain major threats to the craft of
software testing. Give your team some time to introspect and I won’t be too surprised if you find it in your own testing team! While you might argue that poor testing do not
*introduce errors* in software, actually they do! Poor testing do leave the software in a buggy state. Moreover, in this era of agile software development poor unit tests (e.g. in TDD) may result in poor coding and hence escalate the risk of errors.
Last minute changes
Changes that are made to requirement, infrastructure, tools, platform can be dangerous, especially if are being made at the 11th hours of a project release. Actions like database migration, making your software compatible across a variety of OS/browsers can be complex things and if done in a hurry due to a last minute change in the requirement may cause errors in the application. These kind of late changes may result in last minute code changes, which are likely to introduce errors.
9.3 Classification of Defects
Defects are given a priority and severity level to help the development team understand the defect level and extent. Defect classification is essential and is absolutely needed as the development team cannot resolve all the defects simultaneously. This defect classification allows the development team to decide which defects need to be fixed on immediate basis and to what extent the functionality of the program is affected by the defects.
Let’s go through each of these types.
Severity defines the degree of impact. Thus, defect’s severity reflects the degree or intensity of a particular defect, to impact a software product or its working, adversely. Based on the severity metric, a defect may be further categorized into following:
- Critical: The defects termed as ‘critical’, needs immediate attention and treatment. A critical defect directly affects the critical and essential functionalities, which may affect a software product or its functionality on a large scale, such as failure of a feature/functionality or the whole system, system crash-down, etc.
- Major: Defects, which are responsible for affecting the core and major functionalities of a software product. Although, these defects does not results into complete failure of a system, but may bring several major functions of a software, to rest.
- Minor: These defects produces minor impact, and does not have any significant influence on a software product. The results of these defects may be seen in the product’s working, however, it does not stops users to execute it task, which may be carried out, using some other alternative.
- Trivial: These types of defects, have no impact on the working of a product, and sometimes, it is ignored and skipped, such as spelling or grammatical mistake.
One more angle to see a defect in a software application is on the basis of its probability to occur and getting encountered by the user. Based on the likelihood or the possibility of a defect in a software product in terms of percentage, it may be classified into following forms:
- High: Signifies the high probability of getting traced out by almost all the users of the application.
- Medium: Half of the users are able to trace out the defects presence in a software product.
- Low: Generally, not detected any user or may be detected only few numbers of user.
Defects could also be seen from the business perspective, as which defects needs to be corrected first and which may be fixed at a later stage, based on the current need and demand of the business. Similar to probability basis, it may also be categorized into following forms:
- High: High priority defines the foremost need of a business, to correct a defect, as soon as possible, in the same build.
- Medium: Medium priority defects comes next to that of high priority, and may be addressed in any next version or a release of a product.
- Low: These types of defects, does not needs to be individually corrected. It may or may not be considered or corrected, along with any other defect, which needs to be fixed.
Difference b/w Severity and Priority
|• Defect Priority is defined the order in which the developer should resolve a defect||• Defect Severity is defined as the degree of impact that a defect has on the operation of the product|
|Priority is categorized into three types
||• Severity are categorized into five types
|• Priority is associated with scheduling||• Severity is associated with functionality or standards|
|• Priority indicates how soon the bug should be fixed||• Severity indicates the seriousness of the defect on the product functionality|
|• Priority of defects is decided in consultation with the manager/client||• QA engineer determines the severity level of the defect|
|• Priority is driven by business value||• Severity is driven by functionality|
|• Its value is subjective and can change over a period of time depending on the change in the project situation||• Its value is objective and less likely to change|
|• High priority and low severity status indicates, defect have to be fixed on immediate bases but does not affect the application||• High severity and low priority status indicates defect have to be fixed but not on immediate bases|
|• Priority status is based on the customer requirements||• Severity status is based on the technical aspect of the product|
|• During UAT the development team fix defects based on priority||• During SIT, the development team will fix defects based on the severity and then priority|
Priority and Severity have some classifications amongst them that aid in determining how the defect must be handled. A lot of different organizations have different defect logging tools, so the levels might vary.
Let’s take a look at the different levels for both Priority and Severity.
- High priority, high severity
- High priority, low severity
- High Severity, low priority
- Low severity, low priority
The following figure depicts the classification of the categories in a single snippet.
High Severity and High Priority
Any Critical/major business case failure automatically gets promoted to this category.
Any defects due to which the testing cannot continue at any cost or causes a severe system failure fall into this category. Say for example clicking on a particular button doesn’t load the feature itself. Or performing a particular function brings down the server consistently and causes data loss. The red lines in the above figure indicate these kinds of defects.
The system crashes after you made the payment or when you are not able to add the items to the Cart, this defect is marked as High Severity and High Priority defect.
Another example would be ATM vending currency feature wherein after entering the correct username and the password, the machine does not dispense money but deducts the transferred from your account.
High Priority and Low severity
Any minor severity defects that could directly impact the user experience automatically gets promoted to this category.
Defects which have to be fixed but do not affect the application come under this category.
Say for example the feature is expected to display a particular error to the user with respect to its return code. In this case, functionally the code will throw an error, but the message will need to be more relevant to the return code generated. The blue lines in the figure indicate these kinds of defects.
The logo of the company in the front-page is wrong, it is considered to be High Priority and Low severity defect.
Example 1: In the Online shopping website, when the FrontPage logo is spelled wrong, for example instead of Flipkart it is spelled as Flipkrta.
Example 2: In the bank logo, instead of ICICI, it is written as ICCCI.
In terms of functionality, it is not affecting anything so we can mark as Low Severity, but it has an impact on user experience. This kind of defects needs to be fixed on high priority even though they have very less impact on the application side.
High Severity and Low Priority
Any defect that is functionally not meeting the requirements or have any functional implications on the system but sidelined to back seat by the stakeholders when it comes to business criticality automatically gets promoted to this category.
Defects which have to be fixed but not immediately. This can specifically occur during ad-hoc testing. It means that the functionality is affected to a large extent, but is observed only when certain uncommon input parameters are used.
Say for example a particular functionality can be used only on a later version of the firmware, so in order to verify this – the tester actually downgrades his system and performs the test and observes a serious functionality issue which is valid. In such a case the defects will be classified in this category denoted by pink lines, as normally end users will be expected to have a higher version of firmware.
In a social networking site, if a beta version of a new feature is released with not many active users using that facility as of today. Any defect found on this feature can be classified as a Low priority as the feature takes back seat due to business classification as not important.
Though this feature is having a functional defect, as it is not impacting the end customers directly, a business stakeholder can classify the defect under low priority though it has a severe functional impact on the application.
This is a high severity fault but can be prioritized to low priority as it can be fixed with next release as a change request. Business stakeholders also prioritize this feature as a rarely used feature and do not impact any other features that have a direct impact on user experience. This kind of defects can be classified under High severity but Low Priority category.
Low Severity and Low priority
Any spelling mistakes /font casing/ misalignment in the paragraph of the 3rd or 4th page of the application and not in the main or front page/ title.
These defects are classified in the green lines as shown in the figure and occur when there is no functionality impact, but still not meeting the standards to a small degree. Generally cosmetic errors or say dimensions of a cell in a table on UI are classified in here.
9.4 Defect Life Cycle
Defect life cycle, also known as Bug Life cycle is the journey of a defect cycle, which a defect goes through during its lifetime. It varies from organization to organization and also from project to project as it is governed by the software testing process and also depends upon the tools used.
Defect Life Cycle – Workflow
The bug has different states in the Life Cycle. The Life cycle of the bug can be shown diagrammatically as follows:
Bug or defect life cycle includes following steps or status:
- New: When a defect is logged and posted for the first time. It’s state is given as new.
- Assigned: After the tester has posted the bug, the lead of the tester approves that the bug is genuine and he assigns the bug to corresponding developer and the developer team. It’s state given as assigned.
- Open: At this state the developer has started analyzing and working on the defect fix.
- Fixed: When developer makes necessary code changes and verifies the changes then he/she can make bug status as ‘Fixed’ and the bug is passed to testing team.
- Pending retest: After fixing the defect the developer has given that particular code for retesting to the tester. Here the testing is pending on the testers end. Hence its status is pending retest.
- Retest: At this stage the tester do the retesting of the changed code which developer has given to him to check whether the defect got fixed or not.
- Verified: The tester tests the bug again after it got fixed by the developer. If the bug is not present in the software, he approves that the bug is fixed and changes the status to
- Reopen: If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “reopened”. The bug goes through the life cycle once again.
- Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “closed”. This state means that the bug is fixed, tested and approved.
- Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “duplicate“.
- Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “rejected”.
- Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software.
- Not a bug: The state given as “Not a bug” if there is no change in the functionality of the application. For an example: If customer asks for some change in the look and field of the application like change of colour of some text then it is not a bug but just some change in the looks of the application.
A bug life cycle refers to the stages a software bug undergoes from the moment it is reported till it is fixed and closed. The typical bug undergoes three phases in its life cycle:
- Bug Understanding: When a bug is discovered and reported, the first order of business is understanding its contents. This involves verifying that it actually is a bug with all its needed information — not a feature request or general feedback, and not a duplicate of an existing bug. Additionally, the reported severity level is verified to ensure there is no exaggeration.
- Bug Triage: Triage is originally a medical term that refers to prioritizing what needs to be treated first. In this stage, the “triager” determines the priority of the bug based on its severity and several other considerations specific to the project. The triager then determines the component having the bug and assigns it to the corresponding developer to work on a fix.
- Bug Resolution: Finally, this is when the developer works on identifying the cause of the error in the code and (hopefully) smooths out the kinks. However, the resolution does not always mean the bug is fixed; for instance, if the bug is too trivial (or too complex) it can be deferred for a later version or marked as WontFix.
These phases are broken down to several states or statuses, collectively known as the bug life cycle or workflow.
9.5 Create Defect Report/ Reporting
Creating defect report is needed in case of removing defects. It may help in next round to pre- analyse the upcoming defects. Finding defects is only part of the job, because if developers can’t
reproduce the bugs you find, they’re going to have a very tough time fixing them. The bug tracking software should include a number of mandatory fields to ensure that testers give a complete account of the defect they encountered. And testers should hone their descriptive skills.
Good bug reports contains following features:
Everything starts with a title. It must be clear and descriptive, so that you get an idea of what it is about at a glance.
Try to remain clear and concise in your description. Describing something accurately and concisely is a skill that develops with practice. Remember that the person reading your description has not seen the bug and try not to make assumptions.
You have to make it clear what you expected to happen and how the defect diverged from that expectation. This field helps to prevent any misunderstandings, and it gives the developer a clear idea of what went wrong.
Details About The Project And Version
It’s not unusual for testers to work on multiple projects at the same time, and sometimes they’ll share a database, so ensuring that your bug is listed in the correct project, and section within that project, is vital. You also need to get the software version right. Sometimes a bug will be fixed when a different defect is addressed, or simply by some change in the newest version of the software. If the version is wrong, developers may be embarking on a wild goose chase.
Any background information you can provide about your environment will help developers track that bug down. What device were you using? What operating system was it running? What model was it? What browser did you use? Every detail you can give about the platform will help.
For Example : Device Type, Model,Browser Version and OS Version.
Defect Type And Severity
These fields go hand-in-hand. A functional bug will generally be treated more seriously than a suggestion. Developers also need to know how severe the bug is. No product ships with zero defects, so having bugs categorized correctly in terms of type and severity really helps the decision-making process with regards to what gets fixed and what doesn’t. So it is always better to mention the Priority and Severity of the Bugs.
Steps To Reproduce (This Is Very Important!)
You want to give a step-by-step account of exactly what you did to find any defect. Sometimes you can use software tools that catch your key strokes, or record screenshots or video files as you test, other times you will be writing from memory, so take notes as you go. Make sure that you test your own steps again before submitting the bug.
Give an indication of reproducibility as well. Sometimes you’ll be confident about your steps, but upon repeating them the bug won’t necessarily occur. If it happens every time, at random, or only once, then the developer needs to know that.
Supporting material is always gratefully received by those tasked with fixing defects. Usually this will be screenshots, but it can include audio and video files. Keep in mind this is often what developers look at first, before reading the text you provided. If you can convey the issue in a single screenshot, you’re helping save precious time!
Tags & Links
It can be a good idea to use descriptive tags that enable you to filter the database and find groups of related bugs. Sometimes you’ll want to include another bug ID or a link within your defect report to something that you feel is strongly related or similar, but not similar enough to be a duplicate.
It’s usually going to be up to the lead developer or management to assign bugs, but on occasion you might receive instructions about who to assign to. Leaving unassigned or wrongly assigned bugs in the database is a dangerous thing because they can potentially slip through the cracks. Make sure you’re clear on the expectations for assigning.Eclipse software development kit (SDK) is free and open-source software, released under the terms of the Eclipse Public License, although it is incompatible with the GNU General Public License. It was one of the first IDEs to run under GNU Classpath and it runs without problems under IcedTea.
9.6 Defect Tracking and Test Management Tools
Defect tracking is the process of tracking the logged defects in a product from beginning to closure (by inspection, testing, or recording feedback from customers), and making new versions of the product that fix the defects. Defect tracking is important in software engineering as complex software systems typically have tens or hundreds or thousands of defects: managing, evaluating and prioritizing these defects is a difficult task. When the numbers of defects gets quite large, and the defects need to be tracked over extended periods of time, use of a defect tracking system can make the management task much easier.
There are several defect tracking software available which provide powerful features to effectively manage defect life cycle, such as:
- Store and relay the information of bugs in a hassle free manner.
- Provides organized and efficient way of defect management.
- Defect tracking tool provides a centralized overview of the bugs with current information.
- Maintains the history of bugs.
- Entire team can report and view progress of bugs at one place.
- Generate email notifications to the concerned person when an activity takes place on a bug.
- Enables the team to specify the priority, severity, estimated time and assignment details of defect.
- Easy navigation through bugs as users can filter bugs on the basis of assignment, priority, iteration etc.
- Enables the manager to see open bugs in the project. It gives him clarity about the overall progress of the project, work completed successfully and the remaining work.
Below are the list of various defects tracking software available in the market.
Bugzilla is a web-based general-purpose bug tracker and testing tool originally developed and used by the Mozilla project, and licensed under the Mozilla Public License.
Bugzilla has been a leading bug tracking tools widely used by many organizations for quite some time now. It is very simple to use, web-based interface. It has all the features of the essence, convenience, and assurance. It is completely open sourced and is free to use.
Key Features of Bugzilla
- Bugzilla has many keys as well as advanced features, which makes it unique. Following is a list of some of Bugzilla’s most significant features:
- Bugzilla is powerful and it has advanced searching capabilities.
- Bugzilla supports user configurable email notifications whenever the bug status changes.
- Bugzilla displays the complete bug change history.
- Bugzilla provides inter bug dependency track and graphic representation.
- Bugzilla allows users to attach Bug supportive files and manage it.
- Bugzilla has integrated, product-based, granular security schema that makes it more secure.
- It has complete security audit and runs under the Perl’s taint mode.
- Bugzilla supports a robust, stable RDBMS (Rational Data Base Management System) back end.
- It supports Web, XML, E-Mail and console interfaces.
- Bugzilla has a wide range of customized, user preferences features.
- It supports localized web user interface.
- Extensive configurability as it allows to be configured with other test management tools for a better user experience.
- Bugzilla has a smooth upgrade pathway among different versions.
9.6.2 Lifecycle of Bugzilla
The defect life cycle in Bugzilla is given below:
- Step 1: A new defect or bug is created or opened.
- Step 2: A developer fixes the defect.
- Step 3: A quality assurance or QA representative checks the defect to verify it has been fixed. If not, then it fails verification and goes back to development for action. If the defect does pass QA, then it is verified closed.
- Step 4: If the reported defect is assessed as not being a bug, it is closed after validation by QA.
- Step 5: If the defect comes back or resurfaces after being closed, then its status changes to re-opened.
In details it can be looked as
The Defect Life Cycle in Details
When the defect life cycle is applied in industry, all bugs are not necessarily closed out as fixed. They may be postponed, deferred, or rejected depending on how the project team has setup the project team to handle defects. Common statuses assigned to defects once they are identified are: New, Postpone, Retest, Open, Pending Reject, Pending Retest, Closed, and Deferred. The following are some examples of paths that defects may take during the bug life cycle:
- Step 1 – A software tester discovers a defect and reports to the Test Team Lead via a defect tracking system.
- Step 2 – The Test Team Lead or designated QA representative assess the validity of the defect.
- Step 3 – QA determines the defect is not valid and the defect status is changed to “Rejected.”
Rejected Defect in Detail
- Step 1 – A new defect is passed to the QA or Test Team leader.
- Step 2 – QA verifies the defect is valid and changes the status to “New.” The bug is then reported to the development team.
- Step 3 – The project development team leader and designated personnel assess the validity of the bug. The bug is determined to be invalid and status changed to “Pending Reject” and goes back to the testing team.
- Step 4 – The test team leader reads the feedback from the development team and agrees that the bug is invalid. The status is then changed to “Rejected.”
Defect from Discovery to Closed Status
- Step 1: A new defect or bug is discovered and reported to QA or the Test Team Lead.
- Step 2: QA determines that the bug is valid and marks the status as “New.” The defect is then reported to the development team.
- Step 3: The development team determines that the bug is valid. The team leader assigns the defect to a team member. Status of the defect changes to “Assigned.”
- Step 4: The development team member fixes the bug. Status is changed to “Fixed” and is passed back to the team leader.
- Step 5: The development team leader changes the bug status to “Pending Retest” and sends to the testing team.
- Step 6: The test team leader changes the defect status to “Retest” and sends to a tester.
- Step 7: The tester validates the defect is fixed and changes the status to “Closed.” If the tester finds that the same problem remains, he or she will consult with the test team leader and change the bug status to “Reopen.” The defect will then be passed to the development team to fix.
A Rejected Bug Example
- Step 1: A new bug is reported to the Test Team Lead (or QA).
- Step 2: QA validates the bug and changes the status to “New.”
- Step 3: The development team attempts to validate the bug but is unable to replicate the conditions that resulted in the defect being discovered.
- Step 4: The test team is unable to recreate the situation that resulted in the original bug report and status is changed to “Rejected.”
Postponing Defect Fixes
If testing a defect requires on functionality or data that is not available, the defect correction and retesting is postponed. When this occurs, the status is changed to “Postponed.” If the defect is not considered critical and can be postponed indefinitely, then the project team may change the status to “Deferred.”
Defect Life Cycle Final Status Entries
In the expanded example, all defects will have a final status of Postponed, Deferred, Closed, or Rejected.
Jira is a proprietary issue tracking product used for bug tracking, issue tracking, and project management functions.
Thousands of software professionals use JIRA as a bug-tracking tool because of its easy to use framework. JIRA is a commercial product and helps to capture and organize the team issues, prioritizing the issue and updating them with the project. It is a tool that directly integrates with the code development environments making it a perfect fit for developers as well. Due to its capability to track any kind of issues it is not just restricted to the software industry. It supports agile projects. It comes with many add-ons that make this tool more powerful than other tools.
Use of JIRA
Following are some of the most significant uses of JIRA.
- JIRA is used in Bugs, Issues and Change Request Tracking.
- JIRA can be used in Help desk, Support and Customer Services to create tickets and track the resolution and status of the created tickets.
- JIRA is useful in Project Management, Task Tracking and Requirement Management.
- JIRA is very useful in Workflow and Process management.
A Project contains issues; a JIRA project can be called as a collection of issues. A JIRA Project can be of several types as given below:
- Software Development Project
- Marketing Project
- Migration to other platform project
- Help Desk Tracking Project
- Leave Request Management System
- Employee Performance System
- Website Enhancement
Mantis is a Bug Reporting tool, widely used as an issue tracking tool for all types of testing. This is a brief tutorial that introduces the readers to the basic features and usage of Mantis. The various chapters of this tutorial will guide the readers on how to utilize the tool in reporting and maintain an issue.
Mantis Bug Tracker is an open source web-based application tool. It is used for Bug Tracking throughout Defect Lifecycle. Along with bug tracking, Mantis supports Release Features to manage various releases of a project or a software.
Mantis is also known as MantisBT, which stands for Mantis Bug Tracker. The name and logo of Mantis refers to Mantidae family of insects. In software, it is referred to as a bug.
Mantis provides Demo, Download (to set up your own Mantis) and Hosting version of the tool. It supports various features for Issue Tracking and life cycle along with Release Management.
The development of Mantis started in 2000 by Kenzaburo Ito. Later in 2002, other team members (JeroenLatour, Victor Boctor and Julian Fitzell) joined. The basic version 1.0.0 of Mantis was released in February 2006.
In 2012, GitHub became the official repository for the Mantis project source code.
Usage of Mantis
Mantis is used to do following things:
- To track all bugs/issue throughout Defect lifecycle.
- To manage at Release level as well.
- Supports Analytic and Reporting features.
- Supports plug-in with other softwares and IDE to make it more strong and effective.
9.6.5 Cost of Defects
The cost of fixing defects is dependent on resources need to fix a defect. The cost of defects rises considerably across the software life cycle. To find and fix defects is cheap and efficient in early stages of development. The defect found in the design phase can be corrected by redesigning the architecture with a little expense.
Once the system is implemented, then it is most expensive to find and fix the defects in the system as rework is needed in all previous phases to spot the errors in the system. Further whole testing work also needs to be repeated to gain the required confidence level in the system.
The cost of defects can be measured by the impact of the defects and when we find them. Earlier the defect is found lesser is the cost of defect. For example, if error is found in the requirement specifications during requirements gathering and analysis, then it is somewhat cheap to fix it. The correction to the requirement specification can be done and then it can be re-issued. In the same way when defect or error is found in the design during design review then the design can be corrected, and it can be re-issued. But if the error is not caught in the specifications and is not found till the user acceptance then the cost to fix those errors or defects will be way too expensive.
If the error is made and the consequent defect is detected in the requirements phase, then it is relatively cheap to fix it.
Similarly, if a requirement specification error is made and the consequent defect is found in the design phase then the design can be corrected and reissued with relatively little expense.
The points below summarize the topics as discussed:
- A defect can be error, flaw, failure, or fault that arises when the expected result doesn’t match with the actual results.
- Defects are primarily of two kinds; Product Defects and Process Defects.
- Software Quality Assurance (SQA) is a set of activities for ensuring quality in software engineering processes.
- Software Quality Control (SQC) is a set of activities for ensuring quality in software products.
- Different cause of defects leads to unexpected working of the software systems.
- Defects are given a priority and severity level to help the development team understand the defect level and extent.
- Defect life cycle, also known as Bug Life cycle is the journey of a defect cycle, which a defect goes through during its lifetime.
- Creating defect report is the process of creating a report based upon the defects occurred in the software system.
- Bugzilla is a web-based general-purpose bug tracker and testing tool which has been used worldwide.