Submit problems and corrections on GitHub with pull requests / issues, through the Google form, or by email: koen.van.der.blom@cwi.nl
| name | textual description | suite/generator/single | objectives | dimensionality | variable type | constraints | dynamic | noise | multi-fidelity | source (real-world/artificial) | reference | implementation |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BBOB | suite | 1 | scalable | continuous | no | no | no | no | https://doi.org/10.1080/10556788.2020.1808977 | https://github.com/numbbo/coco | ||
| BBOB-biobj | suite | 2 | 2-40 | continuous | no | no | no | no | https://doi.org/10.48550/arXiv.1604.00359 | https://github.com/numbbo/coco | ||
| BBOB-noisy | suite | 1 | scalable | continuous | no | no | yes | no | https://hal.inria.fr/inria-00369466 | https://web.archive.org/web/20210416065610/https://coco.gforge.inria.fr/doku.php?id=downloads | ||
| BBOB-largescale | suite | 1 | 20-640 | continuous | no | no | no | no | https://doi.org/10.48550/arXiv.1903.06396 | https://github.com/numbbo/coco | ||
| BBOB-mixint | suite | 1 | 5-160 | integer;continuous;mixed | no | no | no | no | https://doi.org/10.1145/3321707.3321868 | https://github.com/numbbo/coco | ||
| BBOB-biobj-mixint | suite | 2 | 5-160 | integer;continuous;mixed | no | no | no | no | https://doi.org/10.1145/3321707.3321868 | https://github.com/numbbo/coco | ||
| BBOB-constrained | suite | 1 | 2-40 | continuous | yes | no | no | no | http://numbbo.github.io/coco-doc/bbob-constrained/ | https://github.com/numbbo/coco | ||
| MOrepo | suite | 2 | ? | combinatorial | ? | ? | ? | no | https://github.com/MCDMSociety/MOrepo | |||
| ZDT | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1162/106365600568202 | https://github.com/anyoptimization/pymoo | ||
| DTLZ | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/CEC.2002.1007032 | https://pymoo.org/problems/many/dtlz.html | ||
| WFG | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2005.861417 | https://pymoo.org/problems/many/wfg.html | ||
| CDMP | suite | 2+ | scalable | continuous | yes | ? | ? | no | https://doi.org/10.1145/3321707.3321878 | ? | ||
| SDP | suite | 2+ | scalable | continuous | no | yes | ? | no | https://doi.org/10.1109/TCYB.2019.2896021 | ? | ||
| MaOP | suite | 2+ | scalable | continuous | no | no | ? | no | https://doi.org/10.1016/j.swevo.2019.02.003 | ? | ||
| BP | suite | 2+ | scalable | continuous | no | no | ? | no | https://doi.org/10.1109/CEC.2019.8790277 | ? | ||
| GPD | generator | 2+ | scalable | continuous | optional | no | optional | no | https://doi.org/10.1016/j.asoc.2020.106139 | ? | ||
| ETMOF | suite | 2-50 | 25-10000 | continuous | no | yes | no | no | https://doi.org/10.48550/arXiv.2110.08033 | https://github.com/songbai-liu/etmo | ||
| MMOPP | suite | 2-7 | ? | ? | yes | no | no | no | http://www5.zzu.edu.cn/system/_content/download.jsp?urltype=news.DownloadAttachUrl&owner=1327567121&wbfileid=4764412 | http://www5.zzu.edu.cn/ecilab/info/1036/1251.htm | ||
| CFD | expensive evaluations 30s-15m | suite | 1-2 | scalable | ? | yes | no | no | no | real world | https://doi.org/10.1007/978-3-319-99259-4_24 | https://bitbucket.org/arahat/cfd-test-problem-suite |
| GBEA | expensive evaluations 5s-35s | suite | 1-2 | scalable | continuous | no | no | yes | no | real world | https://doi.org/10.1145/3321707.3321805 | ? |
| Car structure | 54 constraints | suite | 2 | 144-222 | discrete | yes | no | no | no | real world | https://doi.org/10.1145/3205651.3205702 | http://ladse.eng.isas.jaxa.jp/benchmark/ |
| EMO2017 | suite | 2 | 4-24 | continuous | no | no | no | no | real world | https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/ | https://www.ini.rub.de/PEOPLE/glasmtbl/projects/bbcomp/downloads/realworld-problems-bbcomp-EMO-2017.zip | |
| JSEC2019 | expensive evaluations 3s; 22 constraints | single | 1-5 | 32 | continuous | yes | no | no | no | real world | http://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html | http://www.jpnsec.org/files/competition2019/EC-Symposium-2019-Competition-English.html |
| RE | suite | 2-9 | 2-7 | continuous;integer;mixed | no | no | no | no | real world like | https://doi.org/10.1016/j.asoc.2020.106078 | https://github.com/ryojitanabe/reproblems | |
| CRE | suite | 2-5 | 3-7 | continuous;integer;mixed | yes | no | no | no | real world like | https://doi.org/10.1016/j.asoc.2020.106078 | https://github.com/ryojitanabe/reproblems | |
| Radar waveform | single | 9 | 4-12 | integer | yes | no | no | no | real world | https://doi.org/10.1007/978-3-540-70928-2_53 | http://code.evanhughes.org/ | |
| MF2 | suite | 1 | 1-n | continuous | no | no | no | yes | https://doi.org/10.21105/joss.02049 | https://github.com/sjvrijn/mf2 | ||
| AMVOP | suite | 1 | scalable | mixed continuous+ordinal+categorical+both | no | no | no | no | https://doi.org/10.1109/TEVC.2013.2281531 | ? | ||
| RWMVOP | suite | 1 | scalable | continuous;mixed continuous+ordinal+categorical+both | yes | no | no | no | real world | https://doi.org/10.1109/TEVC.2013.2281531 | ? | |
| SBOX-COST | problems from BBOB but allows instances with the optimum close to the boundary | suite | 1 | scalable | continuous | no | no | no | no | https://doi.org/10.48550/arXiv.2305.12221 | https://github.com/IOHprofiler/IOHexperimenter/ | |
| ρMNK-Landscapes | tunable variable and objective dimensions; tunable multimodality and correlation between objectives | generator | scalable | scalable | binary | no | no | no | no | https://doi.org/10.1016/j.ejor.2012.12.019 | https://gitlab.com/aliefooghe/mocobench/ | |
| mUBQP | tunable variable and objective dimensions; tunable density and correlation between objectives | generator | scalable | scalable | binary | no | no | no | no | https://doi.org/10.1016/j.asoc.2013.11.008 | https://gitlab.com/aliefooghe/mocobench/ | |
| ρmTSP | tunable variable and objective dimensions; tunable instance type (euclidian/random); tunable correlation between objectives | generator | scalable | scalable | permutations | no (apart from being permutations) | no | no | no | https://doi.org/10.1007/978-3-319-45823-6_40 | https://gitlab.com/aliefooghe/mocobench/ | |
| CEC2015-DMOO | suite | 2-3 | ? | continuous | ? | yes | no | no | Benchmark Functions for CEC 2015 Special Session and Competition on Dynamic Multi-objective Optimization | |||
| Ealain | Real-world-like, easily extensible to increase complexity | generator | 1+ | scalable | continuous,binary,integer | optional | optional | no | optional | Real-world-like | https://doi.org/10.1145/3638530.3654299 | https://github.com/qrenau/Ealain |
| MA-BBOB | Generator that creates affine combinations of BBOB functions | generator | 1 | scalable | continuous | no | no | no | no | artificial | https://doi.org/10.1145/3673908 | https://github.com/IOHprofiler/IOHexperimenter/blob/master/example/Competitions/MA-BBOB/Example_MABBOB.ipynb |
| MPM2 | nonlinear nonseparable nonsymmetric; scalable in terms of time to evaluate the objective function | generator | 1 | scalable | continuous | no | no | no | no | https://ls11-www.cs.tu-dortmund.de/_media/techreports/tr15-01.pdf | https://github.com/jakobbossek/smoof/blob/master/inst/mpm2.py | |
| Convex DTLZ2 | Variant of DTLZ2 with a convex Pareto front (instead of concave) | single | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2013.2281535 | ? | |
| Inverted DTLZ1 | Variant of DTLZ1 with an inverted Pareto front | single | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2013.2281534 | ? | |
| Minus DTLZ | Variant of DTLZ that minimises the inverse of the base DTLZ functions | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2016.2587749 | ? | |
| Minus WFG | Variant of WFG that minimises the inverse of the base WFG functions | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1109/TEVC.2016.2587749 | ? | |
| L1-ZDT | Variant of ZDT with linkages between variables within one of two groups but not between variables in a different group; Linear recombination operators can potentially take advantage of the problem structure | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L2-ZDT | Variant of ZDT with linkages between all variables; Linear recombination operators can potentially take advantage of the problem structure | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L3-ZDT | Variant of L2-ZDT using a mapping to prevent linear recombination operators from potentially taking advantage of the problem structure | suite | 2 | scalable | continuous;binary | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L2-DTLZ | Variant of DTLZ2 and DTLZ3 with linkages between all variables; Linear recombination operators can potentially take advantage of the problem structure | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| L3-DTLZ | Variant of L2-DTLZ using a mapping to prevent linear recombination operators from potentially taking advantage of the problem structure | suite | 2+ | scalable | continuous | no | no | no | no | https://doi.org/10.1145/1143997.1144179 | ? | |
| CEC2018 DT - CEC2018 Competition on Dynamic Multiobjective Optimisation | 14 problems. Time-dependent: Pareto front/Pareto set geometry; irregular Pareto front shapes; variable-linkage; number of disconnected Pareto front segments; etc. | suite | 2 or 3 | scalable? | ? | no | yes | no | no | artificial | https://www.academia.edu/download/94499025/TR-CEC2018-DMOP-Competition.pdf | https://pymoo.org/problems/dynamic/df.html |
| MODAct - multiobjective design of actuators | Realistic Constrained Multi-Objective Optimization Benchmark Problems from Design. Need the https://github.com/epfl-lamd/modact package installed; evaluation times around 20ms | suite | 2 3 4 or 5 | 20 | mixed; integer and continuous | yes | no | no | no | real-world | https://doi.org/10.1109/TEVC.2020.3020046 | https://pymoo.org/problems/constrained/modact.html |
| IOHClustering | Set of benchmark problems from clustering: optimization task is selecting cluster centers for a given set of data, with the number of clusters defining problem dimensionality. Includes both a suite and a generator. Based on ML clustering datasets | suite; generator | 1 | scalable | continuous | no | no | no | no | artificial, but based on real data | https://arxiv.org/pdf/2505.09233 | https://github.com/IOHprofiler/IOHClustering |
| GNBG-II | Generalized Numerical Benchmark Generator (version 2). Also in IOH https://github.com/IOHprofiler/IOHGNBG | suite; generator | 1 | scalable | continuous | no | no | no | no | artificial | https://dl.acm.org/doi/pdf/10.1145/3712255.3734271 | https://github.com/rohitsalgotra/GNBG-II |
| GNBG | Generalized Numerical Benchmark Generator | suite; generator | 1 | scalable | continuous | no | no | no | no | artificial | https://arxiv.org/abs/2312.07083 | https://github.com/Danial-Yazdani/GNBG-Generator |
| DynamicBinVal | Four versions of the dynamic binary value problem | suite | 1 | scalable | binary | no | yes | no | no | artificial | https://arxiv.org/pdf/2404.15837 | https://github.com/IOHprofiler/IOHexperimenter |
| PBO | Suite of 25 binary optimization problems | suite | 1 | scalable | binary | no | no | no | no | artificial | https://dl.acm.org/doi/pdf/10.1145/3319619.3326810 | https://github.com/IOHprofiler/IOHexperimenter |
| W-model | Tunable generator for binary optimization based on several difficulty features | generator | 1 | scalable | binary | no | no | no | no | artificial | https://dl.acm.org/doi/abs/10.1145/3205651.3208240?casa_token=S4U_Pi9f6MwAAAAA:U9ztNTPwmupT8K3GamWZfBL7-8fqjxPtr_kprv51vdwA-REsp0EyOFGa99BtbANb0XbqyrVg795hIw | https://github.com/thomasWeise/BBDOB_W_Model |
| Submodular Optimitzation | set of graph-based submodular optimization problems from 4 problem types | suite | 1 | scalable | binary | no | no | no | no | artificial | https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10254181 | https://github.com/IOHprofiler/IOHexperimenter |
| CEC2013 | suite used for cec2013 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimenter | suite | 1 | scalable | continuous | no | no | no | no | artificial | https://peerj.com/articles/cs-2671/CEC2013.pdf | https://github.com/P-N-Suganthan/CEC2013 |
| CEC2022 | suite used for cec2022 competition. Also in IOH https://github.com/IOHprofiler/IOHexperimenter | suite | 1 | scalable | continuous | no | no | no | no | artificial | https://github.com/P-N-Suganthan/2022-SO-BO/blob/main/CEC2022%20TR.pdf | https://github.com/P-N-Suganthan/2022-SO-BO |
| Onemax+Sphere / Zeromax+Sphere | single | 2 | scalable | binary and continuous;mixed; | no | no | no | no | artificial | https://doi.org/10.1145/3449726.3459521 | None | |
| Onemax+Sphere / DeceptiveTrap+RotatedEllipsoid | single | 2 | scalable | binary and continuous;mixed; | no | no | no | no | artificial | https://doi.org/10.1145/3449726.3459521 | None | |
| InverseDeceptiveTrap+RotatedEllipsoid / DeceptiveTrap+RotatedEllipsoid | single | 2 | scalable | binary and continuous;mixed; | no | no | no | no | artificial | https://doi.org/10.1145/3449726.3459521 | None | |
| PorkchopPlotInterplanetaryTrajectory | suite | 1 | 2 | continuous | no | no | no | no | real-world | https://doi.org/10.1109/CEC65147.2025.11042973 | https://github.com/ShuaiqunPan/Transfer_Random_forests_BBOB_Real_world | |
| KinematicsRobotArm | suite | 1 | 21 | continuous | no | no | no | no | real-world | https://doi.org/10.1023/A:1013258808932 | https://github.com/ShuaiqunPan/Transfer_Random_forests_BBOB_Real_world | |
| VehicleDynamics | suite | 1 | 2 | continuous | no | no | no | no | real-world | https://www.scitepress.org/Papers/2023/121580/121580.pdf | https://zenodo.org/records/8307853 | |
| MECHBench | This is a set of problems with inspiration from Structural Mechanics Design Optimization. The suite comprises three physical models, from which the user may define different kind of problems which impact the final design output. | Problem Suite | 1 | scalable' | Continuous | yes | no | no | no | Real-World Application | https://arxiv.org/abs/2511.10821 | https://github.com/BayesOptApp/MECHBench |
| EXPObench | Wind farm layout optimization, gas filter design, pipe shape optimization, hyperparameter tuning, and hospital simulation | Problem Suite | 1 | 10 to 135 | Continuous, Integer, Categorical, Conditional | yes | no | yes | no | Real-World Application | https://doi.org/10.1016/j.asoc.2023.110744 | https://github.com/AlgTUDelft/ExpensiveOptimBenchmark |
| Gasoline direct injection engine design | A multi-objective optimization problem seeking to minimize fuel consumption and NOx emissions over a two-minute dynamic duty cycle, subject to five constraints (turbine inlet temperature, number of knock occurrences, peak cylinder pressure, peak cylinder pressure rise, total work). Seven decision variables are defined: four define the hardware choices of cylinder compression ratio, turbo machinery and EGR cooler sizing; three relate to control variables that parameterise the engine control logic. | Single Problem | 2 | 7 | Continuous, Ordinal | yes | no | no | yes | Real-World Application | https://doi.org/10.1016/j.ejor.2022.08.032 | |
| BEACON | Generator for bi-objective benchmark problems with explicitly controlled correlations in continuous spaces. | Generator | 2 | scalable | Continuous | no | no | no | no | Artificially Generated | https://dl.acm.org/doi/10.1145/3712255.3734303 | https://github.com/Stebbet/BEACON/ |
| TulipaEnergy | Determine the optimal investment and operation decisions for different types of assets in the energy system (production, consumption, conversion, storage, and transport), while minimizing loss of load. | Problem Suite | 1 | scalable | Continuous | yes | no | yes | yes | Real-World Application | https://tulipaenergy.github.io/TulipaEnergyModel.jl/stable/40-scientific-foundation/45-scientific-references | https://tulipaenergy.github.io/TulipaEnergyModel.jl/stable/ |
| ATO | Parameters of the Modules of the Automatic Train Operation should be optimized. The parameters are continuous with different ranges. There are two objectives (minimizing energy consumption, minimizing driving duration. | Single Problem | 2 | 10 | Continuous | no | no | no | no | Real-World Application | - | |
| Brachytherapy treatment planning | Treatment planning for internal radiation therapy | Problem Suite | 2-3 | 100-500 | Continuous | yes | no | no | yes | Real-World Application | https://www.sciencedirect.com/science/article/pii/S1538472123016781 | |
| FleetOpt | Healthcare organisation in the UK provided data about their current fleet of vehicles to conduct non-emergency heathcare trips in the Argyll and Bute region of Scotland, UK. They also provided historical data about the trips the vehicles took and about the bases which the vehicles return to. The aim is to reduce the existing fleet of vehicles while still ensuring all trips can be covered. Moving a vehicle from one base to another to help cover trips is OK as long as the original base can still cover its trips. Link to paper with more details: https://dl.acm.org/doi/abs/10.1145/3638530.3664137 | Single Problem | 1 | Upper level: 54; lower level: 13208 | Integer | yes | no | no | no | Real-World Application | https://dl.acm.org/doi/abs/10.1145/3638530.3664137 | Not public: was done for real client with their private data |
| Building spatial design | Optimise the spatial layout of a building to: minimise energy consumption for climate control, and minimise the strain on the structure | Single Problem | 2 | scalable depending on problem size (e.g. 90 for) | Continuous, Boolean | yes | no | no | no | Real-World Application | https://hdl.handle.net/1887/81789 | https://github.com/TUe-excellent-buildings/BSO-toolbox |
| Electric Motor Design Optimization | The goal is to find a design of a synchronous electric motor for power steering systems that minimizes costs and satisfies all constraints. | Single Problem | 1 | 13 | Continuous, Integer | yes | no | yes | no | Real-World Application | https://dis.ijs.si/tea/Publications/Tusar23Multistep.pdf (paper in Slovene) | Implementation not freely available |
| name | textual description | suite/generator/single | objectives | dimensionality | variable type | constraints | dynamic | noise | multi-fidelity | source (real-world/artificial) | reference | implementation |