advanced
Volume 19 Issue 2
April  2022
Turn off MathJax
Article Contents
Relative Articles
Qian-Long Dang, Wei Xu, Yang-Fei Yuan. A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization. Machine Intelligence Research, vol. 19, no. 2, pp.138-152, 2022. https://doi.org/10.1007/s11633-022-1314-7
Citation: Qian-Long Dang, Wei Xu, Yang-Fei Yuan. A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization. Machine Intelligence Research, vol. 19, no. 2, pp.138-152, 2022. https://doi.org/10.1007/s11633-022-1314-7
shu

A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization

doi: 10.1007/s11633-022-1314-7

  • School of Mathematics and Statistics, Xidian University, Xi′an 710126, China

More Information
  • Author Bio:

    Qian-Long Dang received the B. Eng. degree in applied mathematics from Henan University of Technology, China in 2016. He is currently a Ph. D. degree candidate in School of Mathematics and Statistics, Xidian University, China. His research interests include computational intelligence, swarm intelligence, evolution algorithm, and their applications.E-mail: xidianqldang@163.com (Corresponding author) ORCID iD: 0000-0001-9295-1361

    Wei Xu received the B. Eng. degree in applied mathematics from Inner Mongolia University of Science and Technology, China in 2018. He is currently a master student in School of Mathematics and Statistics, Xidian University, China.His research interests include computational intelligence, swarm intelligence, evolution algorithm, and their applications.E-mail: xuwei18782149972@163.com

    Yang-Fei Yuan received the B. Eng. degree in applied mathematics from Inner Mongolia University of Science and Technology, China in 2018. He is currently a master student in School of Mathematics and Statistics, Xidian University, China.His research interests include computational intelligence, swarm intelligence, evolution algorithm, and their applications.E-mail: st123456yyf@163.com

  • Received Date: 2021-04-15
  • Accepted Date: 2021-08-19
  • Publish Online: 2022-01-07
  • Publish Date: 2022-04-01
    Abstract
  • Many isolation approaches, such as zoning search, have been proposed to preserve the diversity in the decision space of multimodal multi-objective optimization (MMO). However, these approaches allocate the same computing resources for subspaces with different difficulties and evolution states. In order to solve this issue, this paper proposes a dynamic resource allocation strategy (DRAS) with reinforcement learning for multimodal multi-objective optimization problems (MMOPs). In DRAS, relative contribution and improvement are utilized to define the aptitude of subspaces, which can capture the potentials of subspaces accurately. Moreover, the reinforcement learning method is used to dynamically allocate computing resources for each subspace. In addition, the proposed DRAS is applied to zoning searches. Experimental results demonstrate that DRAS can effectively assist zoning search in finding more and better distributed equivalent Pareto optimal solutions in the decision space.

     

    • FullText(HTML)
  • loading
    • References(59)
  • [1]
    S. L. Karri, L. C. De Silva, D. T. C. Lai, S. Y. Yong. Identification and classification of driving behaviour at signalized intersections using support vector machine. International Journal of Automation and Computing, vol. 18, no. 3, pp. 480–491, 2021. DOI: 10.1007/s11633-021-1295-y.
    [2]
    H. T. Ye, Z. Q. Li. PID neural network decoupling control based on hybrid particle swarm optimization and differential evolution. International Journal of Automation and Computing, vol. 17, no. 6, pp. 867–872, 2020. DOI: 10.1007/s11633-015-0917-7.
    [3]
    W. Jia, W. Xia, Y. Zhao, H. Min, Y. X. Chen. 2D and 3D palmprint and palm vein recognition based on neural architecture search. International Journal of Automation and Computing, vol. 18, no. 3, pp. 377–409, 2021. DOI: 10.1007/s11633-021-1292-1.
    [4]
    A. Jaszkiewicz. On the performance of multiple-objective genetic local search on the 0/1 knapsack problem-a comparative experiment. IEEE Transactions on Evolutionary Computation, vol. 6, no. 4, pp. 402–412, 2002. DOI: 10.1109/TEVC.2002.802873.
    [5]
    K. Deb. Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation, vol. 7, no. 3, pp. 205–230, 1999. DOI: 10.1162/evco.1999.7.3.205.
    [6]
    Q. F. Zhang, H. Li. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, vol. 11, no. 6, pp. 712–731, 2007. DOI: 10.1109/TEVC.2007.892759.
    [7]
    K. Deb, A. Pratap, S. Agarwal, T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, vol. 6, no. 2, pp. 182–197, 2002. DOI: 10.1109/4235.996017.
    [8]
    S. Lu, Y. M. Li, B. X. Ding. Multi-objective dimensional optimization of a 3-DOF translational PKM considering transmission properties. International Journal of Automation and Computing, vol. 16, no. 6, pp. 748–760, 2019. DOI: 10.1007/s11633-019-1184-9.
    [9]
    P. S. Oliveto, D. Sudholt, C. Zarges. On the benefits and risks of using fitness sharing for multimodal optimisation. Theoretical Computer Science, vol. 773, pp. 53–70, 2019. DOI: 10.1016/j.tcs.2018.07.007.
    [10]
    C. Y. Lin, W. H. Wu. Niche identification techniques in multimodal genetic search with sharing scheme. Advances in Engineering Software, vol. 33, no. 11−12, pp. 779–791, 2002. DOI: 10.1016/S0965-9978(02)00045-5.
    [11]
    R. Thomsen. Multimodal optimization using crowding-based differential evolution. In Proceedings of Congress on Evolutionary Computation, IEEE, Portland, USA, pp. 1382−1389, 2004.
    [12]
    M. Q. Li, D. Lin, J. S. Kou. A hybrid niching PSO enhanced with recombination-replacement crowding strategy for multimodal function optimization. Applied Soft Computing, vol. 12, no. 3, pp. 975–987, 2012. DOI: 10.1016/j.asoc.2011.11.032.
    [13]
    J. P. Li, M. E. Balazs, G. T. Parks, P. J. Clarkson. A species conserving genetic algorithm for multimodal function optimization. Evolutionary Computation, vol. 10, no. 3, pp. 207–234, 2002. DOI: 10.1162/106365602760234081.
    [14]
    Q. Q. Fan, X. F. Yan. Solving multimodal multiobjective problems through zoning search. IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 51, no. 8, pp. 4836–4847, 2021. DOI: 10.1109/TSMC.2019.2944338.
    [15]
    K. Miettinen. Nonlinear Multiobjective Optimization, Boston, USA: Kluwer Academic Publishers, 1999.
    [16]
    J. J. Liang, C. T. Yue, B. Y. Qu. Multimodal multi-objective optimization: A preliminary study. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Vancouver, Canada, pp. 2454−2461, 2016.
    [17]
    S. C. Maree, T. Alderliesten, P. A. N. Bosman. Real-valued evolutionary multi-modal multi-objective optimization by hill-valley clustering. In Proceedings of the Genetic and Evolutionary Computation Conference, Association for Computing Machinery, Prague, Czech Republic, pp. 568−576, 2019.
    [18]
    C. T. Yue, B. Y. Qu, J. Liang. A multi-objective particle swarm optimizer using ring topology for solving multimodal multi-objective problems. IEEE Transactions on Evolutionary Computation, vol. 22, no. 5, pp. 805–817, 2018. DOI: 10.1109/TEVC.2017.2754271.
    [19]
    R. Tanabe, H. Ishibuchi. A decomposition-based evolutionary algorithm for multi-modal multi-objective optimization. In Proceedings of the 15th International Conference on Parallel Problem Solving from Nature, Springer, Coimbra, Portugal, pp. 249−261, 2018.
    [20]
    R. Tanabe, H. Ishibuchi. A framework to handle multimodal multiobjective optimization in decomposition-based evolutionary algorithms. IEEE Transactions on Evolutionary Computation, vol. 24, no. 4, pp. 720–734, 2020. DOI: 10.1109/TEVC.2019.2949841.
    [21]
    Y. M. Peng, H. Ishibuchi. A decomposition-based large-scale multi-modal multi-objective optimization algorithm. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Glasgow, UK, pp. 1−8, 2020.
    [22]
    R. Tanabe, H. Ishibuchi. A niching indicator-based multi-modal many-objective optimizer. Swarm and Evolutionary Computation, vol. 49, pp. 134–146, 2019. DOI: 10.1016/j.swevo.2019.06.001.
    [23]
    Q. F. Zhang, W. D. Liu, H. Li. The performance of a new version of MOEA/D on CEC09 unconstrained MOP test instances. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Trondheim, Norway, pp. 203−208, 2009.
    [24]
    Q. Kang, X. Y. Song, M. C. Zhou, L. Li. A collaborative resource allocation strategy for decomposition-based multiobjective evolutionary algorithms. IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 49, no. 12, pp. 2416–2423, 2019. DOI: 10.1109/TSMC.2018.2818175.
    [25]
    A. M. Zhou, Q. F. Zhang. Are all the subproblems equally important? Resource allocation in decomposition-based multiobjective evolutionary algorithms IEEE Transactions on Evolutionary Computation, vol. 20, no. 1, pp. 52–64, 2016. DOI: 10.1109/TEVC.2015.2424251.
    [26]
    Q. Z. Lin, G. M. Jin, Y. P. Ma, K. C. Wong, C. A. Coello Coello, J. Q. Li, J. Y. Chen, J. Zhang. A diversity-enhanced resource allocation strategy for decomposition-based multiobjective evolutionary algorithm. IEEE Transactions on Cybernetics, vol. 48, no. 8, pp. 2388–2401, 2018. DOI: 10.1109/TCYB.2017.2739185.
    [27]
    X. Y. Cai, Y. X. Li, Z. Fan, Q. F. Zhang. An external archive guided multiobjective evolutionary algorithm based on decomposition for combinatorial optimization. IEEE Transactions on Evolutionary Computation, vol. 19, no. 4, pp. 508–523, 2015. DOI: 10.1109/TEVC.2014.2350995.
    [28]
    Y. Xiang, Y. R. Zhou, L. P. Tang, Z. F. Chen. A decomposition-based many-objective artificial bee colony algorithm. IEEE Transactions on Cybernetics, vol. 49, no. 1, pp. 287–300, 2019. DOI: 10.1109/TCYB.2017.2772250.
    [29]
    K. Li, A. Fialho, S. Kwong, Q. F. Zhang. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, vol. 18, no. 1, pp. 114–130, 2014. DOI: 10.1109/TEVC.2013.2239648.
    [30]
    H. L. Liu, L. Chen, Q. F. Zhang, K. Deb. Adaptively allocating search effort in challenging many-objective optimization problems. IEEE Transactions on Evolutionary Computation, vol. 22, no. 3, pp. 433–448, 2018. DOI: 10.1109/TEVC.2017.2725902.
    [31]
    H. K. Chen, G. H. Wu, W. Pedrycz, P. N. Suganthan, L. N. Xing, X. M. Zhu. An adaptive resource allocation strategy for objective space partition-based multiobjective optimization. IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 51, no. 3, pp. 1507–1522, 2021. DOI: 10.1109/TSMC.2019.2898456.
    [32]
    J. J. Zhou, L. Gao, X. Y. Li. Ensemble of dynamic resource allocation strategies for decomposition-based multi-objective optimization. IEEE Transactions on Evolutionary Computation, vol. 25, no. 4, pp. 710–723, 2021. DOI: 10.1109/TEVC.2021.3060899.
    [33]
    F. van den Bergh, A. P. Engelbrecht. A cooperative approach to particle swarm optimization. IEEE Transactions on Evolutionary Computation, vol. 8, no. 3, pp. 225–239, 2004. DOI: 10.1109/TEVC.2004.826069.
    [34]
    Y. J. Shi, H. F. Teng, Z. Q. Li. Cooperative co-evolutionary differential evolution for function optimization. In Proceedings of the 1st International Conference on Advances in Natural Computation, Springer, Changsha, China, pp. 1080−1088, 2005.
    [35]
    J. J. Liang, A. K. Qin, P. N. Suganthan, S. Baskar. Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Transactions on Evolutionary Computation, vol. 10, no. 3, pp. 281–295, 2006. DOI: 10.1109/TEVC.2005.857610.
    [36]
    Z. Y. Yang, K. Tang, X. Yao. Large scale evolutionary optimization using cooperative coevolution. Information Sciences, vol. 178, no. 15, pp. 2985–2999, 2008. DOI: 10.1016/j.ins.2008.02.017.
    [37]
    M. Yang, M. N. Omidvar, C. H. Li, X. D. Li, Z. H. Cai, B. Kazimipour, X. Yao. Efficient resource allocation in cooperative co-evolution for large-scale global optimization. IEEE Transactions on Evolutionary Computation, vol. 21, no. 4, pp. 493–505, 2017. DOI: 10.1109/TEVC.2016.2627581.
    [38]
    Y. H. Jia, W. N. Chen, T. L. Gu, H. X. Zhang, H. Q. Yuan, S. Kwong, J. Zhang. Distributed cooperative co-evolution with adaptive computing resource allocation for large scale optimization. IEEE Transactions on Evolutionary Computation, vol. 23, no. 2, pp. 188–202, 2019. DOI: 10.1109/TEVC.2018.2817889.
    [39]
    X. N. Shen, Y. N. Guo, A. M. Li. Cooperative coevolution with an improved resource allocation for large-scale multi-objective software project scheduling. Applied Soft Computing, vol. 88, Article number 106059, 2020. DOI: 10.1016/j.asoc.2019.106059.
    [40]
    Y. H. Jia, Y. Mei, M. J. Zhang. Contribution-based cooperative co-evolution for nonseparable large-scale problems with overlapping subcomponents. IEEE Transactions on Cybernetics, to be published.
    [41]
    M. G. Gong, Z. D. Tang, H. Li, J. Zhang. Evolutionary multitasking with dynamic resource allocating strategy. IEEE Transactions on Evolutionary Computation, vol. 23, no. 5, pp. 858–869, 2019. DOI: 10.1109/TEVC.2019.2893614.
    [42]
    Y. Wang, H. C. Tan, Y. K. Wu, J. K. Peng. Hybrid electric vehicle energy management with computer vision and deep reinforcement learning. IEEE Transactions on Industrial Informatics, vol. 17, no. 6, pp. 3857–3868, 2021. DOI: 10.1109/TII.2020.3015748.
    [43]
    Y. Deng, F. Bao, Y. Y. Kong, Z. Q. Ren, Q. H. Dai. Deep direct reinforcement learning for financial signal representation and trading. IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 3, pp. 653–664, 2017. DOI: 10.1109/TNNLS.2016.2522401.
    [44]
    Y. X. Wang, K. Wang, H. W. Huang, T. M. Miyazaki, S. Guo. Traffic and computation co-offloading with reinforcement learning in fog computing for industrial applications. IEEE Transactions on Industrial Informatics, vol. 15, no. 2, pp. 976–986, 2019. DOI: 10.1109/TII.2018.2883991.
    [45]
    W. Y. Wang, J. W. Li, X. D. He. Deep reinforcement learning for NLP. in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, Association for Computational Linguistics, Melbourne, Australia, pp. 19−21, 2018.
    [46]
    Z. Q. Wan, C. Jiang, M. Fahad, Z. Ni, Y. Guo, H. B. He. Robot-assisted pedestrian regulation based on deep reinforcement learning. IEEE Transactions on Cybernetics, vol. 50, no. 4, pp. 1669–1682, 2020. DOI: 10.1109/TCYB.2018.2878977.
    [47]
    E. Mocanu, D. C. Mocanu, P. H. Nguyen, A. Liotta, M. E. Webber, M. Gibescu, J. G. Slootweg. On-line building energy optimization using deep reinforcement learning. IEEE Transactions on Smart Grid, vol. 10, no. 4, pp. 3698–3708, 2019. DOI: 10.1109/TSG.2018.2834219.
    [48]
    X. Y. Zhang, Y. Tian, Y. C. Jin. A knee point-driven evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, vol. 19, no. 6, pp. 761–776, 2015. DOI: 10.1109/TEVC.2014.2378512.
    [49]
    K. X. Wei, A. Aviles-Rivero, J. W. Liang, Y. Fu, C. B. Schonlieb, H. Huang. Tuning-free plug-and-play proximal algorithm for inverse imaging problems. In Proceedings of the 37th International Conference on Machine Learning, ICML, Vienna, Austria, pp. 10158−10169, 2020.
    [50]
    J. Liang, B. Y. Qu, D. W. Gong, C. T. Yue. Problem definitions and evaluation criteria for the CEC 2019 special session on multimodal multiobjective optimization. Technical Report, Zhengzhou University, China, 2019.
    [51]
    E. Zitzler, L. Thiele. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, vol. 3, no. 4, pp. 257–271, 1999. DOI: 10.1109/4235.797969.
    [52]
    S. Bandyopadhyay, S. K. Pal, B. Aruna. Multiobjective GAs, quantitative indices, and pattern classification. IEEE Transactions on Systems,Man,and Cybernetics − Part B:Cybernetics, vol. 34, no. 5, pp. 2088–2099, 2004. DOI: 10.1109/TSMCB.2004.834438.
    [53]
    B. Y. Qu, C. Li, J. Liang, L. Yan, K. J. Yu, Y. S. Zhu. A self-organized speciation based multi-objective particle swarm optimizer for multimodal multi-objective problems. Applied Soft Computing, vol. 86, Article number 105886, 2020. DOI: 10.1016/j.asoc.2019.105886.
    [54]
    M. Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701, 1937. DOI: 10.1080/01621459.1937.10503522.
    [55]
    Y. Liu, H. Ishibuchi, G. G. Yen, Y. Nojima, N. Masuyama. Handling imbalance between convergence and diversity in the decision space in evolutionary multi-modal multi-objective optimization. IEEE Transactions on Evolutionary Computation, vol. 24, no. 3, pp. 551–565, 2020. DOI: 10.1109/TEVC.2019.2938557.
    [56]
    Y. P. Liu, H. Ishibuchi, Y. Nojima, N. Masuyama, K. Shang. A double-niched evolutionary algorithm and its behavior on polygon-based problems. In Proceedings of the 15th International Conference on Parallel Problem Solving from Nature, Springer, Coimbra, Portugal, pp. 262−273, 2018.
    [57]
    Y. P. Liu, G. G. Yen, D. W. Gong. A multimodal multiobjective evolutionary algorithm using two-archive and recombination strategies. IEEE Transactions on Evolutionary Computation, vol. 23, no. 4, pp. 660–674, 2019. DOI: 10.1109/TEVC.2018.2879406.
    [58]
    K. Deb, S. Tiwari. Omni-optimizer: A generic evolutionary algorithm for single and multi-objective optimization. European Journal of Operational Research, vol. 185, no. 3, pp. 1062–1087, 2008. DOI: 10.1016/j.ejor.2006.06.042.
    [59]
    A. M. Zhou, Q. F. Zhang, Y. C. Jin. Approximating the set of Pareto-optimal solutions in both the decision and objective spaces by an estimation of distribution algorithm. IEEE Transactions on Evolutionary Computation, vol. 13, no. 5, pp. 1167–1189, 2009. DOI: 10.1109/TEVC.2009.2021467.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(8)

    Get Citation
    PDF
    XML
    Entire Issue PDF

    用微信扫码二维码

    分享至好友和朋友圈

    Article Metrics

    Article views (596) PDF downloads(104) Cited by()
    Proportional views
    Related
    Browse MIR
    Published Online Current Issue Special Issue Archive Early Access
    About MIR
    Aims and Scopes Index information Editorial Board Contact Us Springer Page

    For Authors
    Guide for Authors Submission Template Copyright Agreement
    For Referees
    Guide for Referees Referees Acknowledgement

    MIR Event
    MIR News MIR Awards
    WeChat: 机器智能研究MIR
    Twitter: MIR_Journal

    © Institute of Automation, Chinese Academy of Sciences. Published by Springer Nature and Science Press. All rights reserved. 京ICP备14019135号-25

    Supported by Beijing Renhe Information Technology Co. Ltd  support: info@rhhz.net  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return