NSGA-III for solving dynamic flexible job shop scheduling problem considering deterioration effect

: The production process of manufacturing systems is usually not static and always interrupted by stochastic events, such as the deterioration effect of cutting tools. This study focuses on the dynamic flexible job shop-scheduling problem considering the deterioration effect (DFJSP-DE). Two types of disturbances are considered, i.e. the implicit disturbance caused by the deterioration effect and the explicit disturbance caused by the reprocessing of unqualified jobs. Firstly, a step-deterioration effect model is proposed, with which the actual processing time of each operation can be predicted more accurately. A multi-objective optimisation model is formulated for the DFJSP-DE. The makespan, energy consumption, and stability of rescheduling solutions are three objectives to be optimised simultaneously. For this non-deterministic polynominal (NP)-hard problem, the non-dominated sorting genetic algorithm III is employed to search the Pareto solutions for the DFJSP-DE. Finally, the results of three numerical experiments show that the proposed approach can solve DFJSP-DE effectively and efficiently.

idle power of machine k T k1 duration needed to turn on machine k T k2 duration needed to turn off machine k ST k1 time to turn on machine k at the beginning of a schedule ET k2 time to turn off machine k at the ending of a schedule β penalty factor if the selected machine for an operation in the rescheduling solution is different from the one in the initial scheduling solution γ penalty factor for rescheduling to start processing at an earlier time

Introduction
Scheduling plays an important role in manufacturing systems. It deals with the resource allocation for tasks within a given time period, aiming at optimising one or more objectives [1]. Since the 1950s, many studies have been carried out on scheduling problems. employed in many factories. In the FMS, machines are usually multi-function machines and can process different operations. As the extension of the JSP, the scheduling problem in FMS has been modelled as the flexible job shop-scheduling problem (FJSP). In practical production, the processing environment is not static and often interrupted by stochastic events. These disturbances can be divided into explicit disturbances and implicit disturbances. A single explicit disturbance such as machine failure, the random arrival of a new job, and the reprocessing of an unqualified job, will have a significant impact on the production process. Different from the explicit disturbance, a single implicit disturbance may not affect the production process significantly. However, when the implicit disturbance increases with time and frequency, its accumulation will also affect the processing significantly, e.g. the processing time of jobs will be lengthened due to the deterioration effect of cutting tools. Furthermore, in some manufacturing systems, machining precision directly influences the quality of products, but cannot always be qualified. Also, the unqualified operations are required to be reprocessed immediately, thus avoid delivering the defective jobs to the subsequent processing stage and causing more repairing costs. Dynamic scheduling is necessary to respond to these disturbances, which is of great importance for the successful execution of the practical production process [2].
Up to now, three dynamic scheduling strategies have been developed, i.e. completely reactive scheduling, predictive-reactive scheduling, and pro-active scheduling [3,4].
Completely reactive scheduling is also named as online scheduling. When a disturbance occurs in the process of online scheduling, a scheduling strategy is needed to be made according to the information obtained at the decision time [2]. For example, Lee et al. [5] considered both online scheduling and offline scheduling in a flow shop with the objective of minimising the makespan and they designed a greedy algorithm. Hopf et al. [6] studied an online flow shop-scheduling problem and developed a competitive deterministic online algorithm and a matching lower bound for online scheduling. Wang et al. [7] considered the online scheduling problem of minimising the makespan on unbounded parallel-batch machines. Lan et al. [8] studied the online scheduling of parallel machines with buffers and online deadline scheduling. Wu et al. [9] studied a framework of a data-driven scheduling method for the semiconductor production line and applied a machine-learning algorithm to obtain the dynamic scheduling model.
Predictive-reactive scheduling first generates an initial scheduling solution without considering any dynamic events. With time, once a disturbance occurs, the unexecuted operations up to that time are rescheduled. predictive-reactive scheduling is the most widely used [10]. For example, Adibi et al. [11] proposed an adaptive variable neighbourhood search algorithm in response to a dynamic event. Gao et al. [12] designed a two-stage artificial bee colony algorithm to solve the dynamic FJSP (DFJSP) problem considering new job insertion. Kundakcı and Kulak [13] proposed a hybrid genetic algorithm for minimising makespan in a dynamic job shop-scheduling problem. Liu and Zhou [14] studied the dynamic scheduling for the unexpected arrival of a rush job in an open shop. They considered two different rescheduling timedomain determination methods during exploring the rescheduling implementation method and developed two problem-specific heuristics.
Pro-active scheduling focuses on improving the robustness of the initial scheduling solution by predicting the uncertain influence of the disturbance events. For example, Wang et al. [15] designed a knowledge-based evolutionary proactive scheduling approach and used a support vector regression to evaluate the robustness of the solution. Cui et al. [16] studied the integration of production scheduling and maintenance planning in order to optimise the biobjective of quality robustness and solution robustness for a flow shop with uncertain quality failure. Wu et al. [17] proposed a novel and comprehensive measure for schedule risk evaluation based on the internal relation among the total slack time, the probability and downtime of random machine breakdowns, and the makespan delay.
In all, it can be seen that reactive scheduling has good practicability in the manufacturing system with frequent dynamic events and high uncertainty. However, since the decision information is derived from local real-time information, it is difficult to predict the whole system's performance. Predictivereactive scheduling is also suitable for online dynamic optimisation and is most widely used. It is a scheduling/rescheduling process that modifies the scheduling solution according to the dynamic events has good global scheduling performance. Pro-active scheduling has high robustness and can absorb dynamic disturbance to some extent. In order to improve robustness, proactive scheduling can reduce machine utilisation. In summary, for the implicit disturbance caused by the deterioration effect and the explicit disturbance caused by the reprocessing of unqualified jobs considered simultaneously, the predictive-reactive scheduling strategy is adopted in this study.
However, most existed literature mainly optimise the efficiency of rescheduling solutions, which may result in the instability of scheduling schemes and the waste of many setup works. Therefore, it is necessary to consider both efficiency and stability in a predictive-reactive scheduling strategy [18]. Besides, the energysaving and green development has been becoming the national development strategy in China, the study on scheduling needs to consider energy consumption as well as the traditional optimisation objectives such as the makespan, delivery time, and so on [19]. Inspired by this, the paper studies the DFJSP considering deterioration effect (DFJSP-DE) and energy consumption and the non-dominated sorting genetic algorithm III (NSGA-III) is designed to deal with both the implicit and explicit disturbance. To the best of our knowledge, this is the first study on the DFJSP-DE. The main contributions of this study are as follows: (i) both the implicit disturbance and the explicit disturbance are considered in the DFJSP; (ii) the makespan, the energy consumption, and the stability are optimised simultaneously when generating a rescheduling solution with an improved NSGA-III. The rest of the paper is structured as follows. Section 2 introduces the DFJSP-DE. Section 3 formulates the DFJSP-DE. Section 4 proposes the NSGA-III for the DFJSP-DE. Section 5 reports the case study. Section 6 concludes the paper.

DFJSP-DE description
The DFJSP-DE can be described as follows. In those fine processing manufacturing system, there are n jobs to be processed, indexed by j = 1, 2,…, n and m machines available, indexed by k = 1, 2, …, m. Operations follow a pre-defined precedence constraint. The operation O i j can be processed on any machine among a set of available machines M i j ⊂ 1, 2, …, m and occupies a different amount of the processing time on different machines. Besides, the actual processing time of the operation is variable with the influence of the deterioration effect. It consists of two parts: the fixed basic processing time and the variable penalty time [20]. Given a deterioration duration threshold, if the job cannot be processed before the given lower bound of the threshold, then the period of penalty time will be required. If the job is processed after the given upper bound of the threshold, the penalty time will be fixed. Within the threshold, the penalty time is the product of the deterioration rate and the starting time of O ij , which means that the later the starting time is, the worse the deterioration effect will be. The energy consumption per time unit of each machine varies with different speed levels. The unqualified jobs dynamically occur and need to be reprocessed immediately. The task for solving the DFJSP-DE is to assign a machine for each operation and sequence the operations on each machine in order to optimise the makespan, the energy consumption, and the stability of the rescheduling solution simultaneously.

Notations
Nomenclature lists all the notations.

Assumptions
To simplify the model, some assumptions are as follows.
(i) Jobs are independent and pre-emption is not allowed. (ii) Each machine can process only one job at a time.
(iii) All jobs and machines are available at the beginning, i.e. t = 0. (iv) Once a job finishes processing on a machine, it is immediately delivered to the next machine and the delivery time is negligible.
(v) The ideal processing time of all operations on their available machines is known. (vi) The setup time is considered in the processing time and is negligible when a scheduling solution is generated. (vii) The reprocessing event for each job occurs once at most.

Formulation of DFJSP-DE
Before the DFJSP-DE at the rescheduling point t l is formulated, the energy consumption model and the deterioration effect model should be built. Both can be referred to as one of our previous studies [20]. At the rescheduling time point t l , the processed operations, the processing operations, the unprocessed operations, and the operations to be reprocessed should be marked first, and then the unprocessed operations and those in need of reprocessing will be rescheduled to optimise the makespan, the energy consumption, and the stability of the rescheduling solutions (see (8)) where (1) indicates the optimisation objectives, the makespan C max , the total energy consumption E, and the stability of the rescheduling solution W that should be optimised simultaneously. Equation (2) formulates the makespan objective. Equation (3) formulates the total energy consumption objective. Equation (4) formulates the stability objective. Equation (5) is the formula to calculate the actual processing time of O ij . Equation (6) indicates the precedence relation. Equation (7) indicates the precedence relation of operations. Equation (8) indicates that one machine cannot process more than one job at a time. Equation (9) indicates that an operation can only be processed by one of the available machines. Equation (10) indicates that the processing duration of operation O i jk consists of three parts: the duration for tool positioning and cutter's approach to job j, the cutting duration, and the duration for cutter's departure from job j. Equation (11) calculates the energy consumption of machine k. Equations (12)- (15) are to calculate the energy consumption referred to Wu et al. [20]. Equations (16) and (17) define the range of decision variables.

Framework of predictive-reactive rescheduling approach
The framework of the predictive-reactive rescheduling approach is shown in Fig. 1. Before a dynamic event occurs, a predictive schedule considering the objectives of makespan and energy consumption is carried out. After a dynamic event occurs at the rescheduling time point, the NSGA-III is triggered to generate a set of non-dominated scheduling solutions by optimising the makespan, energy consumption, and stability simultaneously. The decision maker could select a reactive scheduling solution according to his/her preference. Then the selected reactive schedule will be executed until the next reprocessing event occurs.

Framework of NSGA-III:
The NSGA-III is employed due to its outstanding performance [21]. The basic framework of the NSGA-III is similar to that of the non-dominated sorting genetic algorithm II (NSGA-II). The main difference between them lies in the selection mechanism. The details of the NSGA-III for DFJSP-DE are discussed as follows. Among them, please refer to Wu et al. [20] for the representation and decoding algorithm.

Crossover operator:
Crossover operator is applied to generate offsprings. Among the various genetic algorithms for solving FJSP [22], the position-based crossover (PBX) and the liner order crossover (LOX) are most frequently employed. Therefore, considering the representation characteristics of the chromosomes, the LOX and PBX are randomly selected to perform the crossover operations.
(i) The PBX operator Step 1: Select some positions randomly and exchange the genes of the two parent individuals at the selected positions.
Step 2: Delete the same gene and insert in the remaining genes sequentially at the unselected gene positions (Fig. 2).
(ii) The LOX operator Step 1: Select two positions randomly and exchange the fragments of the two parent individuals between the two positions.
Step 2: Delete the same gene and insert in the remaining genes sequentially outside the two intersections (Fig. 3).

Mutation operator:
The two-point swap mutation operator is applied. The steps are as follows: Step 1: Select two positions randomly.
Step 2: Swap the genes at the two positions (Fig. 4).

Rescheduling strategies
Three rescheduling strategies are employed, namely the full rescheduling, the local rescheduling, and the improved right shift rescheduling. When a dynamic event occurs, the operations to be reprocessed and the unprocessed operations need to be rescheduled, wherein the former has higher priority.
(i) The full rescheduling strategy: the full rescheduling strategy is to reschedule the operations to be reprocessed and unprocessed operations. Both the machine arrangement and processing sequence can be destroyed. The details are as follows.
Step 1: Generate a predictive schedule to optimise both the makespan and the energy consumption, and record the machine arrangement and the processing sequence of operations according to the predictive scheduling solution; Step 2: Execute the predictive schedule until a reprocessing event occurs.
Step 3: Divide the tasks into the processed operations, the processing operations, the unprocessed operations, and the operations to be reprocessed.
Step 4: Reschedule the operations to be reprocessed and unprocessed operations to optimise the stability besides the makespan and the energy consumption, where the reprocessing operations are prioritised, return to step 2.
(ii) The local rescheduling strategy: The local rescheduling strategy is to keep the machines assigned to the unprocessed operations unchanged so that the stability can be optimised. The other steps are similar to those in the full rescheduling strategy. (iii) The improved right shift rescheduling strategy: The improved right shift rescheduling strategy is to keep the machine arrangement and the processing sequence of the unprocessed operations unchanged and then reschedule the reprocessing and unprocessed operations. Adjust operation sequence according to the energy-saving scheduling heuristic considering the deterioration effect [20].
In summary, the three rescheduling strategies have their own advantages and disadvantages. The full rescheduling strategy has the largest solution space but spends long searching time. The solution space and searching time of the local rescheduling strategy are second, but the rescheduling solutions have high stability. The operation of the improved right shift rescheduling strategy is the easiest.

Data source:
The instances from Brandimarte [23] are taken as the experiment data. Referring to Wang et al. [24], the deterioration rate of machine k is generated randomly using the uniform distribution U [0.005, 0.150]. The values of the cutting power, the air cutting power, the idle power, and the turning-on/off energy consumption are set as those in [25], please see Table 1.

Parameters setting:
The parameters are set as Table 2.

Aims:
The aims of the experiments are as follows: (i) to verify the effectiveness of the NSGA-III algorithm for solving multi-objective optimisation problems; (ii) to compare the advantages and the disadvantages of three rescheduling strategies; (iii) to analyse the rationality of the energy-saving scheduling algorithm considering the deterioration effect.

Performance analysis of NSGA-III:
In order to study the performance of NSGA-III for solving the many objectives scheduling problem, we test the DFJSP-DE. The NSGA-II is employed to compare with the NSGA-III. The parameters setting for the NSGA-II are referred to as Wu and Sun [25]. Test the Brandimarte [23] instances and run the NSGA-III and NSGA-II, respectively. The results are reported in Fig. 5  be seen that for the instances MK01-MK04, there is no significant difference between the NSGA-III and the NSGA-II; for the instances MK05-MK10, the NSGA-III outperforms the NSGA-II significantly. In order to compare the NSGA-III and the NSGA-II intuitively, the Pareto solutions obtained with the NSGA-III and NSGA-II are first normalised and then summed. Equation (18) is for normalising and (19) is for summing where f jmin represents the minimum value of the jth objective in the Pareto solutions obtained with the NSGA-II and NSGA-III, f jmax represents the maximum value of the jth objective in the Pareto solutions obtained with the NSGA-II and NSGA-III, f ji represents the value of the ith Pareto solution for the jth objective, f ji ′ represents the normalised value of the ith Pareto solution for the jth objective, and F represents the sum of the normalised values.
The data are shown in Table 3 and presented in a line chart as shown in Fig. 15. It can be seen that for the instances, MK01-MK02, the NSGA-II outperforms the NSGA-III slightly; for the instances, MK03-MK10, the NSGA-III outperforms the NSGA-II significantly. Hence, for the many objectives of DFJSP, the NSGA-III has better performance than the NSGA-II.

5.2.2
Comparison among three rescheduling strategies: Three rescheduling strategies are designed, i.e. the full rescheduling strategy, the local rescheduling strategy, and the improved right shift rescheduling strategy. In order to verify the effects of the three rescheduling strategies, MK07 is employed as the test instance. The rescheduling time point and rescheduling tasks are the same for the three rescheduling strategies. Run the algorithms ten times, respectively. The experimental results are shown in Figs. [16][17][18][19][20][21][22][23][24][25]. It can be seen that, except for the fifth experimental result, the remaining nine results show the local rescheduling strategy performs best, followed by the full rescheduling strategy and the improved right shift rescheduling strategy.
In order to further analyse the performance of the three rescheduling strategies, the typical Pareto solutions for each strategy are selected for comparison. A typical Pareto solution from the Pareto set obtained with the full rescheduling strategy is selected with the stability priority, followed by makespan and energy consumption. With the selected typical Pareto solution as   the standard, a Pareto solution, which is better than the selected typical Pareto solution in terms of makespan, energy consumption, and stability is tried to be found from the Pareto set obtained with the local rescheduling strategy and the improved right shift rescheduling strategy. The objectives of the typical Pareto solutions for the three strategies are shown in Table 4 and are shown by line graphs, as shown in Figs. 26-28. It can be seen from Table 4 and Figs. 26-28 that, for the stability objective, the results of ten rounds show the local rescheduling strategy performs best, followed by the full rescheduling strategy and the improved right shift rescheduling strategy. For the makespan objective, nine out of ten rounds of results show the local rescheduling strategy performs best, followed by the full rescheduling strategy and the improved right shift rescheduling strategy, and only in one round the full rescheduling strategy performs best, followed by the local rescheduling strategy and the improved right shift rescheduling strategy. For the energy consumption objective, in eight rounds the local rescheduling strategy performs best, followed by the full rescheduling strategy and the improved right shift rescheduling strategy, in one round the full rescheduling strategy performs best, followed by the local rescheduling strategy and the improved right shift rescheduling strategy, and in one round the local rescheduling strategy performs best, followed by the improved right shift rescheduling strategy and the full rescheduling strategy. Hence, for the DFJSP-DE, experiment results show the local rescheduling strategy performs best, followed by the full rescheduling strategy and the improved right shift rescheduling strategy. The Gantt chart for the initial scheduling solution is shown in Fig. 29. Also, the Gantt chart for a local rescheduling solution is shown in Fig. 30, where the coloured rectangles represent the reprocessing operation of the unqualified jobs.

Analysis of the influence of deterioration effect:
The core of the approach for solving the DFJSP-DE is the energy-saving scheduling algorithm considering the deterioration effect. In order to study its performance, we carry out an experiment to compare the difference between the energy-saving scheduling algorithm considering the deterioration effect and that without considering the deterioration effect. Fig. 31 is the Pareto solutions for instance MK07 obtained with the two algorithms and it can be seen that the energy-saving scheduling algorithm considering the deterioration effect is superior to that without considering the deterioration effect. For a more intuitive representation, the experiment results obtained with the two scheduling algorithms are shown in box plots, as shown in Figs. 32-34. It can be seen from Figs. 32-34 that for all the three objectives, i.e. makespan, energy consumption, and stability, the scheduling results obtained with the energy-saving scheduling algorithm considering the deterioration effect are superior to those obtained with the energy-saving scheduling algorithm without considering the deterioration effect. Two typical Pareto solutions are selected from the Pareto sets obtained with the two decoding algorithms, respectively, according to the stability   that the rescheduling solution obtained with the energy-saving scheduling algorithm considering the deterioration effect has a shorter makespan and less machine idle time.
This study solves the DFJSP problems considering two types of disturbances, which are caused by the deterioration effect and the reprocessing of unqualified jobs and can easily destroy the stability of the scheduling solution. We take these two disturbances into   account when modelling, and design an effective energy-saving decoding algorithm considering the deterioration effect according to the problem characteristics. Therefore, the stability of the rescheduling solution is improved obviously.

Conclusions
In this paper, we study the DFJSP-DE. An optimisation model is formulated to optimise the makespan, energy consumption, and stability simultaneously. The NSGA-III is designed to solve the problem. Three numerical experiments are carried out to verify the performance of the proposed algorithm. The main conclusions are as follows.
(i) For the many objectives optimisation problems, the performance of the NSGA-III is better than the NSGA-II. (ii) For the DFJSP-DE, the results show that the local rescheduling strategy performs best, followed by the full rescheduling strategy and the improved right shift rescheduling strategy. (iii) The deterioration effect is considered in the optimisation model, which reduces the impact of the deterioration effect on practical production.
To the best of our knowledge, this is the first study on the DFJSP-DE. However, there are still some limitations to our study. For example, only one kind of implicit disturbance and one kind of explicit disturbance are considered. In future research, it is necessary to further improve the deterioration effect model and the energy consumption model, and consider more kinds of disturbance events.