Generating Integration Test Orders for Aspect-Oriented Software with
Multi-objective Algorithms
Thelma Elita Colanzi1,2 , Wesley Klewerton Guez Assunção1 , Silvia Regina Vergilio1 , Aurora Trinidad Ramirez Pozo1
1
DInf - Federal University of Paraná (UFPR), CP: 19081, CEP: 19031-970, Curitiba, Brazil
2
DIN - State University of Maringá (UEM), CEP: 87.020-900, Maringá, Brazil
Email: {thelmae, wesleyk, silvia, aurora}@inf.ufpr.br
Abstract—During the integration test of aspect-oriented software, it is necessary to determine an order to integrate and test
classes and aspects, which should be associated to a minimal
possible stubbing cost. To determine such orders, strategies
based on graphs were proposed. However, the problem is not
trivial and in most cases sub-optimal solutions are obtained.
Furthermore, the strategies do not consider different factors
that influence on the stubbing process related to different
coupling measures, contractual issues, etc. Often, these factors
are in conflict and diverse good solutions are possible. To
deal properly with this problem, this paper introduces a
multi-objective optimization approach to generate a set of
good orders that achieve a balanced compromise between
different measures. The approach is evaluated by using two
multi-objective evolutionary algorithms and two real AspectJ
systems. The results point out that the approach can solve the
problem efficiently. Moreover, it is shown how the tester can
use the found solutions, according to the test goals.
Keywords-Integration testing; aspect-oriented
multi-objective evolutionary algorithms
software;
I. I NTRODUCTION
Testing is a fundamental activity in the aspect-oriented
(AO) software development related to quality assurance.
Similarly to object-oriented (OO) software testing, different
phases should be conducted [1]. The integration testing
phase is necessary to test the interactions among classes and
aspects. However, such phase contributes to increase testing
costs, because many times stubs need to be constructed.
In most systems, there is a dependency relation between
two modules (either a class or aspect). When dependency
cycles exist, it is necessary to break the dependency and
to construct a stub to emulate the behavior of the required
module. A recent study [2] shows that is very common
to find complex dependency cycles in Java programs. In
the AO context is very usual to find crosscutting concerns
that are dependent of other crosscutting concerns, implying
in dependency between aspects, and between classes and
aspects [3]. Hence, to reduce stubbing costs, it is important
to determine a sequence for integration and test of classes
and aspects. Determine such sequence is not a trivial task,
because different factors influence on the stub creation.
In the OO context this problem is known as CITO (Class
Integration and Test Order) problem [4], which has been
intensively investigated by several authors [4], [5], [6]. These
works have been recently extended to the AO context [3],
[7], where the problem is called CAITO (Class and Aspects
Integration Test Order [8]). The extensions propose graphbased strategies. These strategies, however, do not present
satisfactory solutions. Frequently, they produce local optimal
solution because they do not analyze the consequences of
breaking a dependency. Other disadvantage is that they need
some extension to be used with other factors related to the
stubbing process. For example, number of class attributes,
number of calls or methods invoked, constraints related to
organizational or contractual reasons, etc.
To overcome these limitations Genetic Algorithms (GA)
have been applied in the OO [9], [10] and AO [8] contexts.
In both contexts, these strategies obtained better results than
the traditional ones based on graphs. They allow the use
of different factors to establish the test orders by using
a fitness function based on an aggregation (a weighted
average) of number of operations and number of attributes
to be minimized. However, this fitness function requires that
the tester adjusts the weight of each objective which is a
labor intensive task for complex cases.
To reduce these efforts and make the evolutionary strategy
more practical, multi-objective optimization algorithms have
been used to solve the problem in the OO context [11], [12].
These works present promising results when compared with
a simple GA strategy. The algorithms achieve more adequate
solutions considering real constraints and diverse factors that
may influence the CITO problem. In experiments conducted
with different algorithms [12], the Multi-Objective Evolutionary Algorithms (MOEAs) presented the best results.
Motivated by the results obtained with MOEAs in the
OO context, we propose, in this paper, a multi-objective
approach for the CAITO problem. We used two well known
and widely applied MOEAs [13], which presented satisfactory results in the CITO context [14], to evaluate the
approach: NSGA-II [15] and SPEA2 [16]. The goal is to
find a test order set that satisfies constraints and optimizes
different factors. NSGA-II and SPEA2 were used with two
real AspectJ systems.
The paper is organized as follows. Section II reviews related works. Section III presents main concepts about multiobjective optimization. Section IV introduces our multiobjective approach and Section V describes the evaluation
conducted. Section VI presents the results and a comparison
of the MOEAs. Section VII contains some usage examples
and Section VIII concludes the paper.
II. C LASS AND A SPECT I NTEGRATION AND T EST O RDER
P ROBLEM
In literature, many works address the CITO problem. The
solutions proposed by these works are based on directed
graphs, named Object Relation Diagrams (ORDs) [17]. In
such graphs the vertexes are the classes, and their relations
are the edges. When there are no dependency cycles in the
graph, the solution is found with a simple reverse topological
ordering of classes considering their dependencies. In most
cases, this ordering cannot be applied, because most programs contains cycles [2]. This motivated studies to establish
strategies that break the dependency cycles and produce an
order that minimizes stubbing costs [4], [5], [17], [6].
However, Briand et al [9] mention a disadvantage of all
the graph-based solutions. They are very hard to be adapted
to consider many factors involved in the stub creation. Other
limitation of these solutions is that they have as goal the
reduction of broken cycles. However, there are cases where
breaking two dependencies has a lower cost than breaking
only one, and the solutions can be sub-optimal.
To overcome these limitations, in another work, Briand et
al [9], [10] explore the use of a GA and use fitness functions
based on two coupling measures besides the dependency
factor: the number of called methods and used attributes.
The multi-objective approach was introduced in the OO
context by Cabral et al [11]. They use Pareto concepts [18] to
deal with the CITO problem. Results reported in [11] show
that the multi-objective approach presents better solutions
than the function based on aggregation used by Briand
et al. Furthermore, this approach does not need weights
adjustments and generates a set of good solutions to be used
by the tester. In [12] the authors evaluate the multi-objective
approach with three different algorithms, and NSGA-II, the
evolutionary one, obtained the best results.
In the AO context, the CAITO problem was inherited from
the OO context. Hence, the existing strategies have been
adapted. Most works propose graph-based strategies. In the
strategy of Ceccato et al [19] (a) the classes are first tested
without aspects, (b) the aspects are integrated and tested with
the classes, and (c) the classes are tested in the presence of
the aspects. The work of Ré et al [7] propose an extended
ORD to consider dependency relations between classes and
aspects. In the extended ORD the vertexes represent both
classes and aspects, and considering the aspects mechanisms,
new relationships between vertexes are possible. They are:
Crosscutting Association (C) is the association generated by a pointcut with a class method or other advice;
• Dependency (U) is generated by a relation between
advices and pointcuts, and between pointcuts;
• Association Dependency (As) occurs between objects
involved in pointcuts;
• Intertype Declaration Dependency (It) occurs when
there are intertype relationships between aspects and
the base class;
• Inheritance Dependency (I) represents inheritance relationships among aspects or among classes and aspects.
The proposed ORD extension was evaluated in [3], in
which four strategies were studied. They are: Combined,
Incremental+, Reverse, and Random. As a result of the study,
the strategies Combined and Incremental+ performed better
than the others, producing a lower number of stubs.
The use of evolutionary algorithms in the AO context is
recent. Galvan et al [8] described a mono-objective GA that
uses an aggregation of functions for the problem. The GA
presents better solutions, considering attribute and operation
complexities, than the strategies based on graphs proposed
by Ré et al. However the simple GA strategy presents the
same limitations that in the OO context. In spite of the
promising results obtained by the MOEAs to solve the CITO
problem, in the AO context, these algorithms have not been
explored yet. Because of this, in Section IV, we describe
an approach based on multi-objective optimization to solve
the CAITO problem. The approach is based on the extended
ORD proposed in [7] and can be used with either strategy
studied by Ré et al. However, to evaluate the approach
(Section V), we use the Combined strategy [3] because
it seems to be more practical, since classes and aspects
probably are tested together if both are under development.
To implement the approach we use MOEAs. These choices
have their fundamentals based in [3] and [12].
•
III. M ULTI - OBJECTIVE O PTIMIZATION
Optimization problems with two or more objective functions are called multi-objective. In such problems, the objectives to be optimized are usually in conflict, which means
that they do not have a single solution. In this way, the goal
is to find a good trade-off between the objectives.
Multi-objective optimization algorithms have been widely
applied to solve problems with multiple interdependent
interests (objectives) that must be optimized simultaneously.
Variants of GA adapted to multi-objective problems were
proposed. A GA is a heuristic inspired by the theory
of natural selection and genetic evolution [20]. From an
initial population, basic operators are applied. Through the
selection operator, more copies of those individuals with
the best values of the objective function are selected to
be parent. So the best individuals (candidate solutions)
will survive in the next population. The crossover operator
combines parts of two parent solutions to create a new one.
Evaluate Objectives Values;
Assign Rank (level) based on Pareto - sort;
5 Generate Child Population;
6
Binary Tournament Selection;
7
Recombination and Mutation;
8 for i = 1 to g do
9
for each Parent and Child in Population do
10
Assign Rank (level) based on Pareto - sort;
11
Generate sets of nondominated vectors;
12
Determine Crowding distance;
binary
tournament
selects
individuals
lower front;
case
13
Loop (inside) by
adding solutions
to nextof
generation
starting in
from
f irst the
frontsolution
until N 0 individuals;
of same the
fronts,
with greater crowding distance is
14
end
chosen.
New
populations
arewith
generated
withdistance;
recombination
15
Select
points
on the lower front
high crowding
16
Create next(line
generation;
and
mutation
18 of Figure 1).
17
Binary Tournament Selection;
Recombination
Mutation;
B.18 Strength
Paretoand
Evolutionary
Algorithm (SPEA2)
19 end
3
4
The mutation operator randomly modifies a solution. These
operators evolve the population, generation by generation.
The general multi-objective minimization problem with
no restrictions can be stated as to minimize Equation 1,
−
−
subjected to →
x ∈ Π, where: →
x is a vector of decision
variables and Π is a finite set of feasible solutions.
−
→ −
f (→
x)
→
→
= (f1 (−
x ), ..., fB (−
x ))
(1)
−
−
Let →
x ∈ Π and →
y ∈ Π be two solutions. For a
−
−
minimization problem, the solution →
x dominates →
y if
→
−
Equations 2 and 3 are satisfied; x is a non-dominated
−
−
solution if there is no solution →
y that dominates →
x.
→
−
−
−
∀fi ∈ f , i = 1...B, fi (→
x ) ≤ fi (→
y)
→
−
→
−
→
−
∃fi ∈ f , fi ( x ) < fi ( y )
(2)
(3)
The two MOEAs used in our work, NSGA-II (Nondominated Sorting Genetic Algorithm) [15] and SPEA2
(Strength Pareto Evolutionary Algorithm) [16], are briefly
described below.
A. Non-dominated Sorting Genetic Algorithm (NSGA-II)
NSGA-II [15] (see Figure 1) uses a strong elitism strategy. For each generation, NSGA-II sorts the individuals
from parent and offspring populations, considering the nondominance, creating several fronts (lines 10-11 of Figure 1).
The first front is composed by all non-dominated solutions.
The second one has the solutions dominated by only one
solution, and the fronts are created until all solutions are
classified. For the solutions of the same front, another
sort is performed using the crowding distance to maintain
the diversity of solutions (line 12). The crowding distance
calculates how far away the neighbors of a given solution are
and, after calculation, the solutions are decreasingly sorted.
Procedure NSGA-II
Input: N 0 , g, fk (X) . N 0 members evolved g generations to solve fk (X)
0
1 Initialize Population P ;
0
2 Generate random population - size N ;
3 Evaluate Objectives Values;
4 Assign Rank (level) based on Pareto - sort;
5 Generate Child Population;
6
Binary Tournament Selection;
7
Recombination and Mutation;
8 for i = 1 to g do
9
for each Parent and Child in Population do
10
Assign Rank (level) based on Pareto - sort;
11
Generate sets of nondominated solutions;
12
Determine Crowding distance;
13
Loop (inside) by adding solutions to next generation starting from
the f irst front until N 0 individuals;
14
end
15
Select points on the lower front with high crowding distance;
16
Create next generation;
17
Binary Tournament Selection;
18
Recombination and Mutation;
19 end
Figure 1.
Pseudocode of NSGA-II (adapted from [13])
Both sorting procedures, front and crowding distance, are
used by the selection operator (line 17 of the Figure 1). The
Procedure SPEA2
Input: N 0 , g, fk (X) . N 0 members evolved g generations to solve fk (X)
0
1 Initialize Population P ;
0
2 Create empty external set E ;
The main distinction of SPEA2 [16] (see Figure 2) is
the use of an external archive, which stores non-dominated
solutions found at each generation. In each generation a
strength value for all solutions is calculated and used by
the selection operator. The strength value of a solution i
corresponds to the number of j individuals, belonging to the
archive and the population, dominated by i.
Procedure SPEA2
Input: N 0 , g, fk (X) . N 0 members evolved g generations to solve fk (X)
0
1 Initialize Population P ;
0
2 Create empty external set E ;
3 for i = 1 to g do
4
Compute fitness of each individual in P0 and E0 ;
5
Copy all nondominated individual from P0 and E0 to E0 ;
6
Use the truncation operator to remove elements from E0 when the
capacity of the file has been exceeded;
7
If the capacity of E0 has not been exceeded then use dominated
individuals in P0 to fill E0 ;
8
Perform binary tournament selection;
9
Apply crossover and mutation;
10 end
Figure 2.
Pseudocode of SPEA2 (adapted from [13])
1
The fitness of a solution is the
sum of the strength values
of all its dominators, from archive and population (line 4
of Figure 2). Value equal to 0 indicates a non-dominated
individual, and, high values point out that the individual is
dominated by many others. After the selection, new populations are generated with recombination and mutation (lines
8-9). In the evolutionary procedure, the external archive that
is used in the next generation is filled with non-dominated
solutions of the current archive and population (line 5).
IV. T HE M ULTI - OBJECTIVE A PPROACH
The first step to apply the MOEAs to CAITO is to
find a good representation for the problem. In this case,
since the CAITO problem is related to permutations of
modules (classes and aspects), which form test orders, the
chromosome is represented by a vector of integer numbers
that represents the modules. The size of this vector is equal
to the number of modules of each system.
The output produced by the approach (and algorithms) is
a set of good (non-dominated) solutions considering all the
objectives. The approach has the following entries:
1) A dependency model: specifies the kind of dependencies that should be considered. In our work we are using the
extended ORD proposed by Ré et al [7]. We consider that
Inheritance and Inter-type declaration dependencies cannot
be broken. The dependencies that cannot be broken are
an input for the algorithms, called dependency matrix1 .
1 Our algorithm do not break inheritance relationships, as it occurs in the
experiment of Briand et al [9]
These constraints are checked during the generation of initial
population and in the application of mutation and crossover
operators. When a precedence constraint is broken, this
module is placed at the end of the chromosome and all
modules after the position are decremented by one;
2) An integration strategy: this strategy specifies the way
that the testing can be performed. In our work, the strategy
Combined is used;
3) A set of objectives to be minimized (or maximized):
these objectives are related to a set of collected measures
(or a cost model) to be evaluated by the fitness function. We
used the same coupling measures adopted by most related
works [3], [8], [9], [11], [12], the dependencies between
the server and client modules in terms of the number of
attributes used and the number of distinct methods called.
So, considering that: (i) mi and mj are two coupled
modules (mi depends on mj ), (ii) modules are either classes
or aspects, and (iii) the “operation” term represents class
methods, aspect methods and aspect advices. We define:
a) Attribute Coupling: the number of attributes locally
declared in mj when references or pointers to instances of
mj appear in the argument list of some operations in mi , as
the type of their return value, in the list of attributes (data
members) of mi , or as local parameters of operations of mi ;
b) Operation Coupling: the number of operations (including constructors) locally declared in mj which are invoked
by operations of mi .
The stubbing complexity of an order t is based on its
attribute and operation coupling. Two complexities are then
calculated in the following way:
A(t) (attribute complexity): counts the maximum number of attributes that would have to be handled in the stub if
the dependency were broken (attribute coupling measure).
This information is an input for the algorithms and is
represented by a matrix A(i, j), where rows and columns
refer to modules and i depends on j. Then, for a given test
order t and a set of d dependencies to be broken, the attribute
complexity A is calculated according to Equation 4, where n
is the total number of modules and k is any module included
before the module i in t.
A(t) =
n
XX
A(i, j); j 6= k
(4)
i=1,n j=1
O(t) (operation complexity): counts the number of operations that would have to be emulated in the stub if the
dependency were broken (operation coupling measure). This
information is an input for the algorithms and is represented
by a matrix O(i, j), where rows and columns refer to
modules and i depends on j. Then, for a given test order
t and a set of d dependencies to be broken, the operation
complexity O is computed as defined by Equation 5, where n
is the total number of modules and k is any module included
before the module i in t.
O(t) =
n
XX
O(i, j); j 6= k
(5)
i=1,n j=1
Based on these constraints and measures presented above,
the problem is the search for an order that minimizes two
objectives: the attribute and operation complexities.
V. E VALUATION D ESCRIPTION
In order to evaluate the proposed approach, we used two
MOEAs: NSGA-II and SPEA2. We do not compare our
approach with graph-based strategies since they are worst
than GA to solve CAITO problem [8], [13]. Furthermore, we
do not compare the two MOEAs with GA because MOEAs
can obtain best results than GA [12], [13].
Differently of the Ré et al’s work [3], we used two
real systems (written in AspectJ), which contain more than
one thousand dependencies, in our evaluation: AJHotDraw
and AJHSQLDB. The first is an AO refactoring of the
JHotDraw two-dimensional graphics framework. The second
is an AO refactoring of HSQLDB, which is a database
manager. Table I contains information of both systems, such
as number of classes, aspects, LOC, etc.
In practice the coupling measures are generally calculated
during the software design, however it is difficult to obtain
architectural design documentation of complex systems. So,
reverse engineering was performed to identify the existing
dependencies between modules from programs code. A
parser based on AJATO2 (AspectJ and Java Assessment
Tool) was developed to do this. It uses the Java/AspectJ
code as entry and returns the syntactic tree code. From
this tree, all dependencies were identified. At the end, the
parser generated as output three matrices (dependency and
complexities) that were used as input to the MOEAs.
NSGA-II and SPEA2 are available at the JMetal framework [21]. The parameters of these algorithms were set in
a strategy consisting of three steps: (i) the population size
and the number of maximum fitness evaluations; (ii) the
mutation rate; and (iii) the crossover rate. In all steps, the
algorithms were executed 4 times with the same parameters.
During this process the dependency matrices obtained from
BCEL3 system were used. This system was previously used
by [9], [11] in the OO context, providing a reference of
minimal values desired. The values adopted for both MOEAs
are: Population Size = 300, Fitness Evaluation = 20000,
Mutation Rate = 0.02, Crossover Rate = 0.95, and, only for
SPEA2, Archive Size = 250. In spite of each algorithm being
individually calibrated, both found better results with same
parameters values. Each MOEA was executed 30 times for
each AO system. Both algorithms executed the same number
of fitness evaluation in order to analyze whether they can
2 http://www.teccomm.les.inf.puc-rio.br/emagno/ajato/
3 http://archive.apache.org/dist/jakarta/bcel/old/v5.0/
Table I
U SED S YSTEMS
System
Version
AJHotDraw
AJHSQLDB
0.4
18
# Dependencies
#
#
Classes Aspects I
U
As It PointCuts Advices Total
http://sourceforge.net/projects/ajhotdraw/
18586
290
31
234 1177 140 40
0
1
1592
http://sourceforge.net/projects/ajhsqldb/files/ 68550
276
25
107 960 271 0
0
0
1338
Available at
LOC
produce similar solutions when they are restricted to the
same resources.
VI. R ESULTS AND A NALYSIS
This section first presents the results achieved by NSGA-II
and SPEA2, and after the results are compared using the
quality indicator Euclidean Distance from the Ideal Solution
(ED). ED is used to find the closest solution to the best
objectives. An ideal solution has the minimum value of
each objective, considering a minimization problem [22].
These minimum values are obtained from all non-dominated
solutions found by the two algorithms in all runs. The
cost of the ideal solution is: (A=73,O=27) for AJHotDraw,
and (A=1761,O=454) for AJHSQLDB. Note that the ideal
solution is not a feasible solution.
The last column of Table II presents the number of solutions found by each MOEA. Apparently there is not a direct
relationship between the number of modules/dependencies
and number of solutions, since AJHotDraw has more modules and more dependencies than AJHSQLDB, but the
algorithms found a lower number of solutions to the former.
This table also presents, respectively in the third and fourth
columns, the lowest ED and the fitness of the solution
associated to this ED, for each algorithm. The MOEA that
achieved the lowest ED is typed in boldface. We can note
that NSGA-II achieved the solutions that have the lowest ED
from the ideal solution for AJHotDraw and AJHSQLDB.
Table III
R ESULTS FOR GD INDICATOR
NSGA-II
SPEA2
Standard
Standard p-value
Average
Average
Deviation
Deviation
AJHotDraw 0.521715 0.636712 0.665711 1.063900 0.2025
AJHSQLDB 0.103939 0.066915 0.143139 0.065957 0.0055
System
GD indicator. NSGA-II also has good solutions near to the
ideal solution, as the ED indicator shows. Often, decision
makers prefer solutions near to the ideal solution. So, in
this case the NSGA-II should be chosen.
VII. U SAGE E XAMPLES
From the obtained solutions, the tester must choose which
objective(s) to prioritize and sort the solutions according to
the priorities. The decision on the prioritization of objectives
can be influenced by several factors, such as restrictions on
business, economics, etc. To illustrate this, consider some
NSGA-II solutions presented in Table IV. The solution a has
the lowest ED and b is an alternative since it has the second
lowest ED. But, how may the tester choose among them? If
the solution a were chosen for testing AJHotDraw, it would
be necessary to create stubs for emulating 79 attributes and
33 operations. On the other hand, if the solution b of the
same system were chosen it would be necessary to create
stubs for emulating 77 attributes and 38 operations.
Table IV
S OME S OLUTIONS ACHIEVED BY NSGA-II
Table II
M AIN RESULTS
Software
AJHotDraw
AJHSQLDB
MOEA
NSGA-II
SPEA2
NSGA-II
SPEA2
Lowest
ED
8.485
14.422
132.969
136.821
Fitness of the
Solutions
(79, 33)
(85, 35)
(1826, 570)
(1845, 562)
Number of
Solutions
9
11
22
21
The results are also compared using the quality indicator
Generational Distance (GD) [23]. GD is an error measure
used to examine the convergence of an algorithm to the
P F true. P F true encompasses all non-dominated solutions
found by all algorithms in all runs. The values obtained
for this indicator are very similar (Table III), but NSGA-II
reached better average. However, according to the Wilcoxon
test there is statistical difference only for AJHSQLDB where
NSGA-II is better. p − values are presented in the last
column of Table III and they were obtained considering
α = 0.05.
From these results, it is possible to affirm that NSGA-II
has a slight better convergence than SPEA2 as presented by
a
b
A
79
77
AJHotDraw
O
ED
33 8.485
38 11.70
A
1826
1858
AJHSQLDB
O
ED
570 132.969
564
146.65
So, in the first case, the prioritized objective would be the
measure O and, in the second case, the measure A. Solutions
a and b have the best trade-off between these objectives and
because of this they have lower ED than the solutions that
contain the lowest value to measures A and O.
In our study, the Combined strategy was used. Hence,
all solutions returned by the algorithms contain orders to
integration testing of classes and aspects together. However,
another strategy could be used, for instance, the integration
of only classes and, after this, aspects. Then the tester could
use the solutions in a different way.
VIII. C ONCLUDING R EMARKS
This paper presented an approach to solve the CAITO
problem by using MOEAs. To evaluate such approach, the
MOEAs NSGA-II and SPEA2 were executed with two AO
systems: AJHotDraw and AJHSQLDB. In this evaluation
the MOEAs aimed at minimizing two objectives related to
coupling measures: attribute and operation complexities.
The results provide evidences that MOEAs can be efficiently used to solve the CAITO problem by achieving
a good set of solutions. From the presented results, it is
possible to affirm that NSGA-II has good solutions near the
ideal solution, as the ED indicator shows.
The algorithms find a set of different solutions containing
different alternatives of compromise among the objectives. It
is important to observe that these systems are complex with
a lot of dependencies and consequently have many cycles.
In such case an automatic support to determine the orders is
fundamental to reduce the test effort and consequently costs.
One limitation of our study is concerned to the used
systems. So, we intend to perform experiments with other
AO systems and other strategies for integrating classes
and aspects to confirm the evidences found in this work.
Furthermore, different configurations of the approach entries
can be explored, and we plan to analyze the efficiency of
MOEAs to solve the same problem but now using a greater
number of objectives. In this case, other MOEAs also can
be used, such as PAES and MOCell algorithms.
IX. ACKNOWLEDGMENTS
We would like to thank Edison K. Fillus for his contribution. This work was supported by CAPES/Reuni and CNPq.
[8] R. Galvan, A. Pozo, and S. Vergilio, “Establishing Integration
Test Orders for Aspect-Oriented Programs with an Evolutionary Strategy,” in LA-WASP, 2010.
[9] L. C. Briand, J. Feng, and Y. Labiche, “Using genetic algorithms and coupling measures to devise optimal integration
test orders,” in 14th International Conference on Software
Engineering and Knowledge Engineering, July 2002.
[10] ——, Experimenting with Genetic Algorithms and Coupling
Measures to Devise Optimal Integration Test Orders. Carleton University, Technical Report SCE-02-03, October 2002.
[11] R. Cabral, A. Pozo, and S. Vergilio, “A Pareto Ant Colony
Algorithm Applied to the Class Integration and Test Order
Problem,” in 22nd IFIP International Conference on Testing
Software and Systems (ICTSS’10). Springer, 2010.
[12] A. Pozo, G. Bertoldi, J. Árias, R. Cabral, and S. Vergilio,
“Multi-objective optimization algorithms applied to the class
integration and test order problem,” Software Tools for Technology Transfer, 2011, submitted.
[13] C. A. C. Coello, G. B. Lamont, and D. A. V. Veldhuizen, Evolutionary Algorithms for Solving Multi-Objective Problems
(Genetic and Evolutionary Computation), 2nd ed. Secaucus,
NJ, USA: Springer-Verlag New York, Inc., 2007.
[14] W. Assunção, T. Colanzi, A. Pozo, and S. Vergilio, “Establishing integration test orders of classes with several coupling
measures,” in 13th Annual Genetic and Evolutionary Computation Conference - GECCO 2011, 2011, pp. 1867–1874.
R EFERENCES
[15] K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan, “A fast
elitist non-dominated sorting genetic algorithm for multiobjective optimization: NSGA-II,” Lecture Notes in Computer
Science, pp. 849–858, 2000.
[1] R. T. Alexander, J. M. Bieman, and A. A. Andrews, Towards
the Systematic Testing of Aspect-Oriented Programs. Colorado State University, Technical Report, 2004.
[16] E. Zitzler, M. Laumanns, and L. Thiele, “SPEA2: Improving
the Strength Pareto Evolutionary Algorithm,” Gloriastrasse
35, CH-8092 Zurich, Switzerland, Tech. Rep. 103, 2001.
[2] H. Melton and E. Tempero, “An empirical study of cycles
among classes in Java,” Empirical Software Engineering,
vol. 12, pp. 389–415, 2007.
[17] D. C. Kung, J. Gao, P. Hsia, J. Lin, and Y. Toyoshima, “Class
firewal, test order and regression testing of object-oriented
programs,” J. of Object-Oriented Progr., vol. 8, no. 2, 1995.
[3] R. Ré and P. C. Masiero, “Integration testing of aspectoriented programs: a characterization study to evaluate how
to minimize the number of stubs,” in Brazilian Symposium
on Software Engineering, October 2007, pp. 411–426.
[18] V. Pareto, Manuel D’Economie Politique. Paris: Ams Press,
1927.
[4] A. Abdurazik and J. Offutt, “Coupling-based class integration
and test order,” in International Workshop on Automation of
Software Test. Shanghai, China: ACM, May 2006.
[19] M. Ceccato, P. Tonella, and F. Ricca, “Is AOP code easier or
harder to test than OOP code,” in First Workshop on Testing
Aspect-Oriented Program (WTAOP), Chicago, Illinois, 2005.
[20] D. E. Goldberg, Genetic Algorithms in Search, Optimization,
and Machine Learning. Addison-Wesley, January 1989.
[5] L. C. Briand and Y. Labiche, “An investigation of graph-based
class integration test order strategies,” IEEE Transactions on
Software Engineering, vol. 29, no. 7, pp. 594–607, July 2003.
[21] J. Durillo, A. Nebro, and E. Alba, “The jMetal framework
for multi-objective optimization: Design and architecture,” in
CEC 2010, Barcelona, Spain, July 2010, pp. 4138–4325.
[6] Y. L. Traon, T. Jéron, J.-M. Jézéquel, and P. Morel, “Efficient object-oriented integration and regression testing,” IEEE
Transactions on Reliability, pp. 12–25, 2000.
[22] J. Cochrane and M. Zeleny, Multiple Criteria Decision Making. University of South Carolina Press, Columbia, 1973.
[7] R. Ré, O. A. L. Lemos, and P. C. Masiero, “Minimizing stub
creation during integration test of aspect-oriented programs,”
in 3rd Workshop on Testing Aspect-Oriented Programs, Vancouver, British Columbia, Canada, March 2007, pp. 1–6.
[23] D. A. van Veldhuizen and G. B. Lamont, “Multiobjective
evolutionary algorithm test suites,” in Proceedings of the
1999 ACM symposium on Applied computing, ser. SAC ’99.
New York, NY, USA: ACM, 1999, pp. 351–357. [Online].
Available: http://doi.acm.org/10.1145/298151.298382