Experiments of a New Generation Points Strategy in a Multilocal Method

: Nonlinear programming problems appear frequently in industrial/real problems. It is important to obtain its solution in the lowest time possible, since the company could beneﬁt from this. Taking this into account, a derivative free method (MCSFilter method) is addressed with a different strategy to generate the initial points to start each local search. The idea is to spread more the points so that the code execution will require a shortest amount of time when compared with the MCSFilter method execution. Some experiments were performed with simple bounds problems, equality and inequality constraint problems, chosen from a set of well known nonlinear problems. The results obtained were encouraging and with the new strategy the method needs less time to obtain the global solution.


INTRODUCTION
Problems coming from different parts in engineering or economics, among other areas, are very complex and can be modeled as nonlinear programming problems (Floudas et al., 1999) or (Hendrix and G.-Tóth, 2010).
Sometimes the derivatives are not known, thus it is important to use a method that allows us to solve these problems without this knowledge.The MCSFilter method is a derivative free method that is able to treat discontinuous or non-differentiable functions (Fernandes et al., 2013) -a kind of functions that usually appear in these areas.This method is a multilocal method meaning that it finds all the minimizers, local and global.The MCSFilter method was already used to solve small engineering problems (Amador et al., 2017) or (Amador et al., 2018).
The results obtained with the MCSFilter method are quite satisfactory, nevertheless, the generation of initial points is performed randomly.This means that two or more points can be generated closely to each other and converge to the same minimizer without adding new valuable information to the method.The idea is now to obtain a new strategy, to generate this points, in such a way that they will be more spread a https://orcid.org/0000-0002-8611-7706b https://orcid.org/0000-0001-9542-4460around all the search space, leading to a small number of functions evaluation (or less time) to obtain the global minimizer.The paper is organized as follows: in Section 2 a brief description of MCSFilter method is given; in Section 3 the new strategy to generate the initial points is presented; in Section 4 the numerical results are shown and in Section 5 some conclusions are performed.

MCSFilter METHOD
The MCSFilter method is a derivative free method able to treat nonlinear and constraint problems based on a multistart strategy coupled with a filter coordinate search.For more details see (Kolda et al., 2003) and (Fernandes et al., 2013).The exploration part of the method is related with the multistart strategy, meanwhile the exploitation is related with the derivative free local search procedure CSFilter, in order to obtain all the solutions (local and global solutions).The problem can be formulated in the usual way: where, at least one of the functions f , g j : R n −→ R is nonlinear and F = {x ∈ R n : g(x) ≤ 0 , l ≤ x ≤ u} is the feasible region.Problems with general equality constraints can be reformulated in the above form by introducing h(x) = 0 as an inequality constraint |h(x)| − τ ≤ 0, where τ is a small positive relaxation parameter.
The problem is rewritten as a bi-objective problem aiming to minimize both, the objective function f (x) and a nonnegative continuous aggregate constraint violation function θ(x) defined by where v + = max{0, v}.Therefore, a minimizer will be computed by the local search CSFilter to the biobjective optimization problem min When a multistart strategy is applied to obtain all the solutions, some or all of the minimizers may be found over and over again.To avoid convergence to a previously computed solution, a clustering technique based on the regions of attraction computation associated with the local search procedure of previously identified minimizers, y i , is defined as: where CSFilter(x) is the minimizer obtained when the local search procedure CSFilter starts at point x.
The regions are built and the next local search is performed, depending (in a probabilistic way) if the initial point belongs to the region of attraction of some minimizer found until the moment.The objective of these regions of attraction is to avoid starting the local procedure if, for a given point, the local search has a big probability of converging to some previous determined minimizers.
Figure 1 illustrates the region of attraction of each minimizer.
It is possible to see 4 minimizers and the initial points used for each local search.The red line identifies the first local search which converged first to each minimizer; the magenta lines identify all the local searches that converged to minimizers already found; the dashed white lines identify all the initial points that were discarded using the regions of attraction.
To be easier to understand which part is different from the original version, it is presented the MCSFilter algorithm with the new strategy, related with the multistart part, Algorithm 1.In this algorithm lines 1, 2 and 5 substitute the random generation of points of the original MCSFilter algorithm.
As stopping condition, the original MCSFilter algorithm uses the estimate of the fraction of uncovered space where n min is the number of recovered minimizers after having performed n loc local search procedures.
The multistart algorithm then stops if P(n min ) ≤ ε, for a small ε > 0. The CSFilter algorithm is not presented since it is the same as previously.
All the details about all the parameters inside the original Algorithm 1 can be seen in (Fernandes et al., 2013).

GENERATION OF INITIAL POINTS
Taking into account the knowledge about the MCSFilter algorithm it is possible to state that, with a big frequency, there are many points generated close to each other and, on the other side, there are parts in the search space that do not have any point there.
The idea is to obtain a new way to generate the points more spread around the search space.
According with (Hedar and Fukushima, 2006) the interval between the bounds, for each coordinate, is divided into 4 sub-intervals.In each sub-interval a value will be generated for each coordinate based on a probability that depends on which intervals the coordinate has already been generated, and on a dispersion parameter α ≥ 0, as it can be seen in Algorithm 2.
The number of points to be generated are defined using 4 different rules.These rules are based in the search space, given by the bounds and, also, based in the fact that each interval is divided into 4 subintervals and are defined as follows: points generated (T) is given by: with ⌈•⌉ being the ceil function.
ND2A 2020 -Special Session on Nonlinear Data Analysis and Applications Algorithm 1 : MCSFilter algorithm with a new strategy of generation of initial points.

Require: Parameter values
1: Generate T points using Algorithm 2; 2: x is the first point generated from the T points; x is the next point generated from the T points;  ♮y − y l ≤ γ * A min .
• RGP2: Since, for each variable, the interval is divided into 4 equal parts then: • RGP3: In this proposed rule the objective is to combine both, RGP1 and RGP2: • RGP4: This rule, the simpler one, is based only in the number on variables: It is also considered that: • if T ≥ 1500 then T = 1500; Algorithm 2: Initial Points Generation.
1: Initialization: the number of points T is defined according with RGP j , j = 1, • • • , 4. 2: repeat 3: ∆ i = ub i − lb i is divided into 4 equal adjacent intervals -basic intervals.A counter of number of occurrences is initialised for these basic intervals.
4: a starting point is randomly generated within ∆ i and its corresponding basic interval number of occurrences is updated.the current maximum number of points defined in a basic interval, maxOcc, accounted using all basic intervals is determined. 7: an auxiliar value is computed, by basic interval, according with this rule: i) , with i = 1, ..., 4. In this equation, Occ(i) represents the number of points in basic interval i.

8:
Exception: if α = 0, for the basic intervals that have a number of occurrences equal to the maximum number of occurrences value, the auxiliar value attributed is 1.

9:
a new probability distribution is determined using the auxiliary values: , with i = 1, ..., 4. 10: the cumulative probability function of this probability distribution is determined and is used as threshold, in conjunction with a new uniform random number generator that outputs values between 0 and 1, to define the basic interval of the next point. 11: a random number is generated in the basic interval identified in the previous step to generate the point.Its corresponding number of occurrences is updated. 12: until all values are defined in a single variable 13: until all values are defined for all the variables 14: assemble final points by using the coordinates previously obtained.
In order to visualize the influence of the value of the dispersion parameter α and how it controls the way the points are generated some remarks about the algorithm should be made: • if α = 0, the starting basic interval will always get a probability of 1 and all others a probability of 0. This implies that the starting basic interval will have all points -case of greater concentration of points generation, as depicted in Figure 2. • if α = 1, all basic intervals will be affected with the same probability -case of equiprobability between basic intervals to generate the next point.
One example is presented in Figure 3. • if 0 < α < 1, since aux(i) is a power of α related to the number of points in the basic interval, intervals with higher number of occurrences will get a lower power of α, providing them with higher aux values.A simulation of this scenario can be seen in Figure 4.This also means that the probability of achieving higher concentrated points increases inversely with the value of α.
• if α > 1, given the relations used, intervals with lower number of occurrences get higher aux values.A higher value of α increases more quickly the value of the exponential distribution used for the auxiliary values, thus, it implies: -basic intervals with the maximum number of occurrences have lower probability of being used in the point generation: higher value of α ⇒ (more extreme) lower basic interval usage probability.-higher value of α, given the higher difference between different powers of α tends to more rapidly uniformize the number of occurrences through all basic intervals.This will render a situation similar to the case of α = 1 in terms of points distribution as depicted in Figure 5, however, there is one important difference between these two scenario: when α = 1 all intervals are equiprobable, despite the present occurrences distribution; when α > 1 intervals will have different probability values, being equiprobable, related to the current number of occurrences.Furthermore, the intervals with the maximum number of occurrences will have a much lower probability and this value decreases with the increase of the α value.

NUMERICAL RESULTS
To analyze the performance of the MCSFilter algorithm, a set of 16 test problems is used (see Table 6 and the reference therein for more details about each problem).The set contains bound constrained problems, inequality and equality constrained problems, multimodal objective functions, with one global and some local, more than one global, and a unimodal optimization problem.In order to compare the time needed by the original version of the MCSFilter algorithm and the new one, the original MCSFilter algorithm was run 10 times in a laptop Intel(R) Core i5-7200U CPU and 2.5GHz, using the same values presented in (Fernandes et al., 2013).In this version the points are randomly generated and to stop the MCSFilter Algorithm Equation 5 was used, with ε = 0, 1.
These results are listed in Table 1.The first column shows the problems, the second shows the average number of minimizers found and the last column shows the average time needed (in seconds).
Tables 2 -5 list the same results as previously but now to each new Rule inside Algorithm 2, such that a comparison can be performed between the original and the new strategy.Therefore, for each new 4 ways to limit the generation of the initial points, the average number of minimizers and the average time needed to obtain the solutions is presented.
For the proposed MCSFilter algorithm version, the stop condition is given by Equation 5or if all T points are used.All the other parameters remain equal to the original algorithm.Related with Algorithm 2 and the α parameter, after some preliminary experiments it was chosen α = 10.If a comparison is made between Table 1 and Table 2 it is possible to state that using Rule RGP1: • there are 12 problems out of 16 for which the algorithm takes less time to obtain an equal or larger number of minimizers; • there are 2 problems that take more time but the average number of minimizers found is larger than the original; • for problem P16 it takes less time but the average number of minimizers (3) is smaller than the average number (3,7) from the original MCSFilter, nevertheless the global is always obtained in all 10 runs.
• there is 1 problem in which the results obtained are worst than the original ones (problem P7).
Considering now Table 3 and Rule RGP2, it is possible to observe that: • there are 10 problems out of 16 for which the algorithm takes less time to obtain an equal or larger number of minimizers; • there are some problems that take more time but the average number of minimizers found is larger than the original.
• for problem P16 the global solution was obtained only in six runs (out of 10).
To compare the original results with the ones obtained using Rule RGP3, Table 4 lists the average values to all the problems.
As it is possible to observe:  • there are 8 problems out of 16 for which the algorithm takes less time to obtain an equal or larger number of minimizers; • there are three problems that take more time but the average number of minimizers found is larger than the original.• for problem P16 the global minimizer was found only in 5 runs (out of 10).• for problem P14, the algorithm (with this rule) found a small number of minimizers, being the worst result till now, considering all rules.
Finally, Table 5 shows the results obtained with Rule RGP4.
Comparing these results with Table 1: • there are 10 problems out of 16 for which the algorithm takes less time to obtain an equal or larger number of minimizers; • for the remaining problems the results are worst than the ones obtained with the original algorithm.• for problem P16 the global minimizer was found only in 8 runs (out of 10).
Problem P15 has the same behaviour with almost all rules: finds a number of minimizers greater than 1.This happen because the local search stops before reaching the minimizer.
Globally, the MCSFiter algorithm with the mew points generation strategy needs less time than the original version to find the minimizers, regardless of the used rule to control the number of points to be generated, which is promising.

CONCLUSIONS
MCSFilter ia a multilocal method able to find local and global solutions of a nonlinear problem.The original method is based on a multistart strategy coupled with a coordinate search filter methodology.Inside the multistart part and in the original method, the initial points were randomly generated.Sometimes, the points generated were too close of each other leading to more calls of the local procedure without adding more information to the solution.The objective of the new strategy is to spread as much as possible the points so that it will be possible to avoid some calls of the local procedure and, hence, taking less time to obtain the solution.In order to test the efficiency of this approach (Algorithm 2), four different rules were used to limit the number of points to be generated.The dispersion or the concentration of the points around the search space depends on the α parameter.
The new algorithm was tested with all the rules using, always, a benchmark of test problems -a set of 16 problems (bound, equality and inequality constraint problems).
This preliminary work shows that Algorithm 2 with rules RGP1, RGP2 and RGP4 presents better results than with RGP3.For bounded problems the results are better than the original method but the problems with equality or inequality constraints have a different behaviour.
Currently, the method generates all points first and uses these generated points in MCSFilter algorithm.That can cause time inefficiency for large dimension problems.Taking this issue into account, individual point generation, using a similar generation strategy, should be investigated.

Figure 1 :
Figure 1: Regions of attraction of four minimizers.

Figure 3 :
Figure 3: α = 1 and the points spread around the search space.

Figure 5 :
Figure 5: α = 10 and the points spread around the search space.

Table 1 :
Results obtained with the original MCSFilter.

Table 2 :
Results with the RGP1 rule in Algorithm 2.

Table 3 :
Results with the RGP2 rule in Algorithm 2.

Table 4 :
Results with the RGP3 rule in Algorithm 2.

Table 5 :
Results with the RGP4 rule in Algorithm 2.