Citation: | Qian-Long Dang, Wei Xu, Yang-Fei Yuan. A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization. Machine Intelligence Research, vol. 19, no. 2, pp.138-152, 2022. https://doi.org/10.1007/s11633-022-1314-7 |
[1] |
S. L. Karri, L. C. De Silva, D. T. C. Lai, S. Y. Yong. Identification and classification of driving behaviour at signalized intersections using support vector machine. International Journal of Automation and Computing, vol. 18, no. 3, pp. 480–491, 2021. DOI: 10.1007/s11633-021-1295-y.
|
[2] |
H. T. Ye, Z. Q. Li. PID neural network decoupling control based on hybrid particle swarm optimization and differential evolution. International Journal of Automation and Computing, vol. 17, no. 6, pp. 867–872, 2020. DOI: 10.1007/s11633-015-0917-7.
|
[3] |
W. Jia, W. Xia, Y. Zhao, H. Min, Y. X. Chen. 2D and 3D palmprint and palm vein recognition based on neural architecture search. International Journal of Automation and Computing, vol. 18, no. 3, pp. 377–409, 2021. DOI: 10.1007/s11633-021-1292-1.
|
[4] |
A. Jaszkiewicz. On the performance of multiple-objective genetic local search on the 0/1 knapsack problem-a comparative experiment. IEEE Transactions on Evolutionary Computation, vol. 6, no. 4, pp. 402–412, 2002. DOI: 10.1109/TEVC.2002.802873.
|
[5] |
K. Deb. Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation, vol. 7, no. 3, pp. 205–230, 1999. DOI: 10.1162/evco.1999.7.3.205.
|
[6] |
Q. F. Zhang, H. Li. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, vol. 11, no. 6, pp. 712–731, 2007. DOI: 10.1109/TEVC.2007.892759.
|
[7] |
K. Deb, A. Pratap, S. Agarwal, T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182–197, 2002. DOI: 10.1109/4235.996017.
|
[8] |
S. Lu, Y. M. Li, B. X. Ding. Multi-objective dimensional optimization of a 3-DOF translational PKM considering transmission properties. International Journal of Automation and Computing, vol. 16, no. 6, pp. 748–760, 2019. DOI: 10.1007/s11633-019-1184-9.
|
[9] |
P. S. Oliveto, D. Sudholt, C. Zarges. On the benefits and risks of using fitness sharing for multimodal optimisation. Theoretical Computer Science, vol. 773, pp. 53–70, 2019. DOI: 10.1016/j.tcs.2018.07.007.
|
[10] |
C. Y. Lin, W. H. Wu. Niche identification techniques in multimodal genetic search with sharing scheme. Advances in Engineering Software, vol. 33, no. 11−12, pp. 779–791, 2002. DOI: 10.1016/S0965-9978(02)00045-5.
|
[11] |
R. Thomsen. Multimodal optimization using crowding-based differential evolution. In Proceedings of Congress on Evolutionary Computation, IEEE, Portland, USA, pp. 1382−1389, 2004.
|
[12] |
M. Q. Li, D. Lin, J. S. Kou. A hybrid niching PSO enhanced with recombination-replacement crowding strategy for multimodal function optimization. Applied Soft Computing, vol. 12, no. 3, pp. 975–987, 2012. DOI: 10.1016/j.asoc.2011.11.032.
|
[13] |
J. P. Li, M. E. Balazs, G. T. Parks, P. J. Clarkson. A species conserving genetic algorithm for multimodal function optimization. Evolutionary Computation, vol. 10, no. 3, pp. 207–234, 2002. DOI: 10.1162/106365602760234081.
|
[14] |
Q. Q. Fan, X. F. Yan. Solving multimodal multiobjective problems through zoning search. IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 51, no. 8, pp. 4836–4847, 2021. DOI: 10.1109/TSMC.2019.2944338.
|
[15] |
K. Miettinen. Nonlinear Multiobjective Optimization, Boston, USA: Kluwer Academic Publishers, 1999.
|
[16] |
J. J. Liang, C. T. Yue, B. Y. Qu. Multimodal multi-objective optimization: A preliminary study. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Vancouver, Canada, pp. 2454−2461, 2016.
|
[17] |
S. C. Maree, T. Alderliesten, P. A. N. Bosman. Real-valued evolutionary multi-modal multi-objective optimization by hill-valley clustering. In Proceedings of the Genetic and Evolutionary Computation Conference, Association for Computing Machinery, Prague, Czech Republic, pp. 568−576, 2019.
|
[18] |
C. T. Yue, B. Y. Qu, J. Liang. A multi-objective particle swarm optimizer using ring topology for solving multimodal multi-objective problems. IEEE Transactions on Evolutionary Computation, vol. 22, no. 5, pp. 805–817, 2018. DOI: 10.1109/TEVC.2017.2754271.
|
[19] |
R. Tanabe, H. Ishibuchi. A decomposition-based evolutionary algorithm for multi-modal multi-objective optimization. In Proceedings of the 15th International Conference on Parallel Problem Solving from Nature, Springer, Coimbra, Portugal, pp. 249−261, 2018.
|
[20] |
R. Tanabe, H. Ishibuchi. A framework to handle multimodal multiobjective optimization in decomposition-based evolutionary algorithms. IEEE Transactions on Evolutionary Computation, vol. 24, no. 4, pp. 720–734, 2020. DOI: 10.1109/TEVC.2019.2949841.
|
[21] |
Y. M. Peng, H. Ishibuchi. A decomposition-based large-scale multi-modal multi-objective optimization algorithm. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Glasgow, UK, pp. 1−8, 2020.
|
[22] |
R. Tanabe, H. Ishibuchi. A niching indicator-based multi-modal many-objective optimizer. Swarm and Evolutionary Computation, vol. 49, pp. 134–146, 2019. DOI: 10.1016/j.swevo.2019.06.001.
|
[23] |
Q. F. Zhang, W. D. Liu, H. Li. The performance of a new version of MOEA/D on CEC09 unconstrained MOP test instances. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Trondheim, Norway, pp. 203−208, 2009.
|
[24] |
Q. Kang, X. Y. Song, M. C. Zhou, L. Li. A collaborative resource allocation strategy for decomposition-based multiobjective evolutionary algorithms. IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 49, no. 12, pp. 2416–2423, 2019. DOI: 10.1109/TSMC.2018.2818175.
|
[25] |
A. M. Zhou, Q. F. Zhang. Are all the subproblems equally important? Resource allocation in decomposition-based multiobjective evolutionary algorithms IEEE Transactions on Evolutionary Computation, vol. 20, no. 1, pp. 52–64, 2016. DOI: 10.1109/TEVC.2015.2424251.
|
[26] |
Q. Z. Lin, G. M. Jin, Y. P. Ma, K. C. Wong, C. A. Coello Coello, J. Q. Li, J. Y. Chen, J. Zhang. A diversity-enhanced resource allocation strategy for decomposition-based multiobjective evolutionary algorithm. IEEE Transactions on Cybernetics, vol. 48, no. 8, pp. 2388–2401, 2018. DOI: 10.1109/TCYB.2017.2739185.
|
[27] |
X. Y. Cai, Y. X. Li, Z. Fan, Q. F. Zhang. An external archive guided multiobjective evolutionary algorithm based on decomposition for combinatorial optimization. IEEE Transactions on Evolutionary Computation, vol. 19, no. 4, pp. 508–523, 2015. DOI: 10.1109/TEVC.2014.2350995.
|
[28] |
Y. Xiang, Y. R. Zhou, L. P. Tang, Z. F. Chen. A decomposition-based many-objective artificial bee colony algorithm. IEEE Transactions on Cybernetics, vol. 49, no. 1, pp. 287–300, 2019. DOI: 10.1109/TCYB.2017.2772250.
|
[29] |
K. Li, A. Fialho, S. Kwong, Q. F. Zhang. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, vol. 18, no. 1, pp. 114–130, 2014. DOI: 10.1109/TEVC.2013.2239648.
|
[30] |
H. L. Liu, L. Chen, Q. F. Zhang, K. Deb. Adaptively allocating search effort in challenging many-objective optimization problems. IEEE Transactions on Evolutionary Computation, vol. 22, no. 3, pp. 433–448, 2018. DOI: 10.1109/TEVC.2017.2725902.
|
[31] |
H. K. Chen, G. H. Wu, W. Pedrycz, P. N. Suganthan, L. N. Xing, X. M. Zhu. An adaptive resource allocation strategy for objective space partition-based multiobjective optimization. IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 51, no. 3, pp. 1507–1522, 2021. DOI: 10.1109/TSMC.2019.2898456.
|
[32] |
J. J. Zhou, L. Gao, X. Y. Li. Ensemble of dynamic resource allocation strategies for decomposition-based multi-objective optimization. IEEE Transactions on Evolutionary Computation, vol. 25, no. 4, pp. 710–723, 2021. DOI: 10.1109/TEVC.2021.3060899.
|
[33] |
F. van den Bergh, A. P. Engelbrecht. A cooperative approach to particle swarm optimization. IEEE Transactions on Evolutionary Computation, vol. 8, no. 3, pp. 225–239, 2004. DOI: 10.1109/TEVC.2004.826069.
|
[34] |
Y. J. Shi, H. F. Teng, Z. Q. Li. Cooperative co-evolutionary differential evolution for function optimization. In Proceedings of the 1st International Conference on Advances in Natural Computation, Springer, Changsha, China, pp. 1080−1088, 2005.
|
[35] |
J. J. Liang, A. K. Qin, P. N. Suganthan, S. Baskar. Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Transactions on Evolutionary Computation, vol. 10, no. 3, pp. 281–295, 2006. DOI: 10.1109/TEVC.2005.857610.
|
[36] |
Z. Y. Yang, K. Tang, X. Yao. Large scale evolutionary optimization using cooperative coevolution. Information Sciences, vol. 178, no. 15, pp. 2985–2999, 2008. DOI: 10.1016/j.ins.2008.02.017.
|
[37] |
M. Yang, M. N. Omidvar, C. H. Li, X. D. Li, Z. H. Cai, B. Kazimipour, X. Yao. Efficient resource allocation in cooperative co-evolution for large-scale global optimization. IEEE Transactions on Evolutionary Computation, vol. 21, no. 4, pp. 493–505, 2017. DOI: 10.1109/TEVC.2016.2627581.
|
[38] |
Y. H. Jia, W. N. Chen, T. L. Gu, H. X. Zhang, H. Q. Yuan, S. Kwong, J. Zhang. Distributed cooperative co-evolution with adaptive computing resource allocation for large scale optimization. IEEE Transactions on Evolutionary Computation, vol. 23, no. 2, pp. 188–202, 2019. DOI: 10.1109/TEVC.2018.2817889.
|
[39] |
X. N. Shen, Y. N. Guo, A. M. Li. Cooperative coevolution with an improved resource allocation for large-scale multi-objective software project scheduling. Applied Soft Computing, vol. 88, Article number 106059, 2020. DOI: 10.1016/j.asoc.2019.106059.
|
[40] |
Y. H. Jia, Y. Mei, M. J. Zhang. Contribution-based cooperative co-evolution for nonseparable large-scale problems with overlapping subcomponents. IEEE Transactions on Cybernetics, to be published.
|
[41] |
M. G. Gong, Z. D. Tang, H. Li, J. Zhang. Evolutionary multitasking with dynamic resource allocating strategy. IEEE Transactions on Evolutionary Computation, vol. 23, no. 5, pp. 858–869, 2019. DOI: 10.1109/TEVC.2019.2893614.
|
[42] |
Y. Wang, H. C. Tan, Y. K. Wu, J. K. Peng. Hybrid electric vehicle energy management with computer vision and deep reinforcement learning. IEEE Transactions on Industrial Informatics, vol. 17, no. 6, pp. 3857–3868, 2021. DOI: 10.1109/TII.2020.3015748.
|
[43] |
Y. Deng, F. Bao, Y. Y. Kong, Z. Q. Ren, Q. H. Dai. Deep direct reinforcement learning for financial signal representation and trading. IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 3, pp. 653–664, 2017. DOI: 10.1109/TNNLS.2016.2522401.
|
[44] |
Y. X. Wang, K. Wang, H. W. Huang, T. M. Miyazaki, S. Guo. Traffic and computation co-offloading with reinforcement learning in fog computing for industrial applications. IEEE Transactions on Industrial Informatics, vol. 15, no. 2, pp. 976–986, 2019. DOI: 10.1109/TII.2018.2883991.
|
[45] |
W. Y. Wang, J. W. Li, X. D. He. Deep reinforcement learning for NLP. in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, Association for Computational Linguistics, Melbourne, Australia, pp. 19−21, 2018.
|
[46] |
Z. Q. Wan, C. Jiang, M. Fahad, Z. Ni, Y. Guo, H. B. He. Robot-assisted pedestrian regulation based on deep reinforcement learning. IEEE Transactions on Cybernetics, vol. 50, no. 4, pp. 1669–1682, 2020. DOI: 10.1109/TCYB.2018.2878977.
|
[47] |
E. Mocanu, D. C. Mocanu, P. H. Nguyen, A. Liotta, M. E. Webber, M. Gibescu, J. G. Slootweg. On-line building energy optimization using deep reinforcement learning. IEEE Transactions on Smart Grid, vol. 10, no. 4, pp. 3698–3708, 2019. DOI: 10.1109/TSG.2018.2834219.
|
[48] |
X. Y. Zhang, Y. Tian, Y. C. Jin. A knee point-driven evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, vol. 19, no. 6, pp. 761–776, 2015. DOI: 10.1109/TEVC.2014.2378512.
|
[49] |
K. X. Wei, A. Aviles-Rivero, J. W. Liang, Y. Fu, C. B. Schonlieb, H. Huang. Tuning-free plug-and-play proximal algorithm for inverse imaging problems. In Proceedings of the 37th International Conference on Machine Learning, ICML, Vienna, Austria, pp. 10158−10169, 2020.
|
[50] |
J. Liang, B. Y. Qu, D. W. Gong, C. T. Yue. Problem definitions and evaluation criteria for the CEC 2019 special session on multimodal multiobjective optimization. Technical Report, Zhengzhou University, China, 2019.
|
[51] |
E. Zitzler, L. Thiele. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, vol. 3, no. 4, pp. 257–271, 1999. DOI: 10.1109/4235.797969.
|
[52] |
S. Bandyopadhyay, S. K. Pal, B. Aruna. Multiobjective GAs, quantitative indices, and pattern classification. IEEE Transactions on Systems,Man,and Cybernetics − Part B:Cybernetics, vol. 34, no. 5, pp. 2088–2099, 2004. DOI: 10.1109/TSMCB.2004.834438.
|
[53] |
B. Y. Qu, C. Li, J. Liang, L. Yan, K. J. Yu, Y. S. Zhu. A self-organized speciation based multi-objective particle swarm optimizer for multimodal multi-objective problems. Applied Soft Computing, vol. 86, Article number 105886, 2020. DOI: 10.1016/j.asoc.2019.105886.
|
[54] |
M. Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937. DOI: 10.1080/01621459.1937.10503522.
|
[55] |
Y. Liu, H. Ishibuchi, G. G. Yen, Y. Nojima, N. Masuyama. Handling imbalance between convergence and diversity in the decision space in evolutionary multi-modal multi-objective optimization. IEEE Transactions on Evolutionary Computation, vol. 24, no. 3, pp. 551–565, 2020. DOI: 10.1109/TEVC.2019.2938557.
|
[56] |
Y. P. Liu, H. Ishibuchi, Y. Nojima, N. Masuyama, K. Shang. A double-niched evolutionary algorithm and its behavior on polygon-based problems. In Proceedings of the 15th International Conference on Parallel Problem Solving from Nature, Springer, Coimbra, Portugal, pp. 262−273, 2018.
|
[57] |
Y. P. Liu, G. G. Yen, D. W. Gong. A multimodal multiobjective evolutionary algorithm using two-archive and recombination strategies. IEEE Transactions on Evolutionary Computation, vol. 23, no. 4, pp. 660–674, 2019. DOI: 10.1109/TEVC.2018.2879406.
|
[58] |
K. Deb, S. Tiwari. Omni-optimizer: A generic evolutionary algorithm for single and multi-objective optimization. European Journal of Operational Research, vol. 185, no. 3, pp. 1062–1087, 2008. DOI: 10.1016/j.ejor.2006.06.042.
|
[59] |
A. M. Zhou, Q. F. Zhang, Y. C. Jin. Approximating the set of Pareto-optimal solutions in both the decision and objective spaces by an estimation of distribution algorithm. IEEE Transactions on Evolutionary Computation, vol. 13, no. 5, pp. 1167–1189, 2009. DOI: 10.1109/TEVC.2009.2021467.
|