Uses of software metrics in the area of Software Testing
Report Prepared By:
Report Submitted To:
I express my thanks to Ma’am Momina
of University of Lahore Pakpattan Campus for having
provided me the facilities to do the Software Testing report successfully.
TABLE OF CONTENTS
Conclusions & Recommendations
In the current trend most of the
software is required to be evaluated for two reasons. One the customer wants to
get satisfied from the quality of the product which he will be going to use
because he has invested a lot of amount to get the product. Two the developers
want to quantify the quality so that his effort in further stages and in
maintenance must not increased due to low quality. The testing is required to
be fast and with less effort that is through the automated testing tools. The
setback is confusion of way of selection of tool. The positive side of the
paper is that it helps the developer in deciding the best testing tool as per
his project by calculating the metrics value for the available tools in hand.
The weakness of the proposed work is that it required a bit of time and effort
in doing calculations for evaluating the tools and some data may also be needed
in calculation of metrics.
As per the experience we have
during the work, we would like to mention our view for the further extension of
this work. The upper and lower bound of the tools metrics must be investigated.
The more number of tools must be exposed to metrics to have the further
empirical analysis. The categorization of the tools on the basis of
functionality is required and the specification of particular metric for that
type may be suggested. An algorithmic approach may be generated to have the
fast automated evaluation of metrics which may reduce the effort to calculate
the metrics and automated suggestion for the better tools may also be a work
Software Metrics is a measurement
based technique which is applied to processes, products and services to supply engineering
and management information and working on the information supplied to improve
processes, products and services, if required. Software Testing Metrics are experimental
measures that could be used to measure different characteristics of a software
system or the software development process. In recent years software testing technologies
have appeared as a leading software engineering practice which helps in
effective cost control, quality improvements, time and risk reduction etc.
We need metrics because it helps us
in understanding the type of improvement required and helps in taking decisions
on process or technology change. The importance of metrics
is to improve the quality and productivity of products and services through
customer satisfaction. Metrics provides enhancement for current processes.
Metrics is used to improve the
quality and productivity of products and services thus achieving Customer
Easy for management to digest one
number and drill down, if required.
Different Metric(s) trend act as
monitor when the process is going out-of-control.
Metrics provides improvement for
What is Software
testing, Metric is a quantitative measure of the degree to which
a system, system component, or process possesses a given attribute.
In other words, metrics helps
estimating the progress, quality and health of a software testing effort. The
ideal example to understand metrics would be a weekly mileage of a car compared
to its ideal mileage recommended by the manufacturer.
There is a need to establish a
software testing process that is cost effective and efficient to
meet the market pressures of delivering low cost and quality software.
Measurement is a key element of an effective and efficient software testing
process as it evaluates the quality and
effectiveness of the process. Moreover, it assesses the productivity of the
in testing activities and helps improving the software testing procedures,
methods, tools and
activities 21. Gathering of software testing related measurement data and
provides an opportunity for the organizations to learn from their past history
and grow in
software testing process maturity.
Measurement is a tool through which the management identifies important events
trends, thus enabling them to make informed decisions. Moreover, measurements
predicting outcomes and evaluation of risks, which in turn decreases the
unanticipated surprises in different processes. In order to face the challenges
posed by the
rapidly changing and innovative software industry, the organizations that can
software testing processes are able to predict costs and schedules and increase
effectiveness, efficiency and profitability of their business 22.Therefore,
measuring what is being done is important for an effective testing effort 9.
Type of Software Testing Metrics
Based on the types of testing performed,
following are the types of software testing metrics: –
Manual Testing Metrics
Performance Testing Metrics
Automation Testing Metrics
Manual Testing Metrics
Test Case Productivity (TCP)
This metric gives the test case writing productivity based
on which one can have a conclusive remark.
Efforts took for writing 183 steps
is 8 hours.
Test case productivity = 23
Defect Acceptance (DA)
This metric determine the number of
valid defects that testing team has identified during execution.
The value of this metric can be
compared with previous release for getting bet picture
Defect Rejection (DR)
This metric determine the number of
defects rejected during execution.
Bad Fix Defect (B)
Defect whose resolution give rise
to new defect(s) are bad fix defect.
This metric determine the
effectiveness of defect resolution process.
Test Efficiency (TE)
This metric determine the efficiency of the testing team in
identifying the defects.
It also indicated the defects missed out during testing
phase which migrated to the next phase.
DT = Number of valid defects identified during testing.
DU = Number of valid defects
identified by user
of this thesis is to offer visibility in to the software testing process using
measurements. The focus of this thesis is at the possible measurements in the
and Test Design process of the system testing level.
Why do Test Metrics?
“We cannot improve what we
cannot measure” and Test Metrics helps us to do exactly the same.
Take decision for next phase of
Evidence of the claim or prediction
Understand the type of improvement
Take decision or process or
It can be used to improve the
process efficiency of the SDLC ( Software Development Life Cycle)
It deals with the quality of the
It can be used to measure the
efficiency of a project team or any testing tools being used by the team
A detailed and comprehensive
literature study will be carried out to gather material related to
software testing lifecycle in general and for software test planning, test
design and metrics in
particular. The literature study will encompass the material as written down in
and web references.
The literature study is conducted with the purpose of finding key areas of
to software testing and metrics support for test planning and test design processes.
literature study is divided into two steps. The first step creates a relevant
analyzing key phases in the
software testing lifecycle, inputs required for starting the
software test planning and test design processes and metrics indicating the end
of these two
processes. The second step identifies the attributes of software test planning
and test design
processes along with metric support for each of the identified attributes.
While performing the literature study, it is important to define a search
main search engines used in the study were IEEE Xplore and ACM digital library.
search was performed with combination of various key terms that we were
interested in. By
using of the above mentioned search engines and using combination of terms for
we expect to have minimized the chances of overlooking papers and literature.
manual search was performed on the relevant conference proceedings. The most
these is the International Software Metrics Symposium proceedings. In addition
Xplore and ACM digital library, web search engines Google and Yahoo were used
occasionally with the purpose of having a diverse set of literature sources.
For each paper of interest, relevant material was extracted and compared
other relevant literature. Specifically; the abstract, introduction and the
conclusion of each of
the articles were studied to assess the degree of relevance of each article.
The references at
the end of each research paper were also quickly scanned to find more relevant
a summary of the different views of authors was written to give reader
and commonalities on the topic.
It is also worth mentioning the use of books written by well-known authors in
of software metrics and software testing. It is to complement the
summarizations done from
the research articles found from above mentioned search strategy. A total of
books were referred. The reference lists given at the end of relevant chapters
in these books
also proved a valuable source of information
Today, the majority of the metrics
in software testing are based on test execution phase and
on the basis of number of faults found in test execution. There is an apparent
gap when we
are interested in metrics for test planning and test design processes.
Therefore, by focusing
on the metric support in the test planning and test design processes, this
contributed in filling part of this gap.
The measurable attributes of
software test planning and test design are not mentioned in
literature in a consolidated form. By identifying seventeen attributes for
planning and thirty one attributes for software test design processes, the
thesis serves to
consolidate the attributes that are useful for measurement. The thesis
attributes in different categories, for software test planning there are four
there are five for software test design process. Such a categorization is a
towards assigning classes to the identified attributes, which may entice
within each category.
The thesis discussed metric support for the identified attributes for the test
test design processes. Again, there were different studies contributing to the
each attribute. The thesis presented the different ways to measure each of the
the intention to provide a variety of methods for the reader to think of and
to the context of the situation at hand.
An interesting aspect resulting from this thesis is that although measurements
informed decision making, but a degree of subjectivity, expert judgment and
plays an important role in the final decision relating to software test
planning and test design.
Therefore, reaching a decision must interplay between knowledgeable judgment
An organization can build a metrics program for its software test planning and
design processes on the foundations of attributes identified in the thesis. It
is expected that
such an effort will lead to informed decision making about different software
activities, in addition to the formulation of a baseline for measuring
Metric is the cornerstone in
assessment and foundation for any business improvement. It is a Measurement
Based Technique which is applied to processes, products and services to supply
engineering and management information and working on the information supplied
to improve processes, products and services, if required. It indicates level of
Customer satisfaction, easy for management to digest number and drill down,
whenever required and act as monitor when the process is going out-of-control.