SBArchOpt Tutorial¶
Try it out yourself:
SBArchOpt (es-bee-ARK-opt) provides a set of classes and interfaces for applying Surrogate-Based Optimization (SBO) for system architecture optimization problems:
- Expensive black-box problems: evaluating one candidate architecture might computationally expensive
- Mixed-discrete design variables: categorical architectural decisions mixed with continuous sizing variables
- Hierarchical design variables: decisions can deactivate/activate (parts of) downstream decisions
- Multi-objective: stemming from conflicting stakeholder needs
- Subject to hidden constraints: simulation tools might not converge for all design points
This tutorial shows how to use SBArchOpt:
- Solve a simple analytical problem using NSGA2 (an evolutionary optimization algorithm)
- Use advanced features for dealing with architecture optimization: sampling, restart
- Run the ArchSBO algorithm to more efficiently solve an analytical problem
- Implement your own optimization problem
Note: a "seed" argument is used to get reproducible results when randomization is involved. This is useful for tutorials and testing, however should be avoided when using in production (i.e. when solving real problems)!
Simple Example: Evolutionary Algorithm (NSGA2)¶
The code below shows a simple example of solving an architecture optimization problem using NSGA2, a multi-objective evolutionary algorithm. Continue with the rest of the tutorial for more background information.
In [1]:
Copied!
from sb_arch_opt.algo.pymoo_interface import plot
from pymoo.optimize import minimize
from sb_arch_opt.problems.hierarchical import HierZDT1Small
from sb_arch_opt.algo.pymoo_interface import get_nsga2
# Create the problem
problem = HierZDT1Small()
# Create the algorithm (with a population size of 100)
nsga2 = get_nsga2(pop_size=100)
# Run the optimization: 20 generations (20 x 100 = 2000 evaluations)
# NOTE: remove "seed" argument when using in production!
result_nsga2 = minimize(problem, nsga2, termination=('n_gen', 20), seed=42)
# Plot results
plot(result_nsga2.opt.get('F'), labels=['NSGA2 (2000 evaluations)'])
from sb_arch_opt.algo.pymoo_interface import plot
from pymoo.optimize import minimize
from sb_arch_opt.problems.hierarchical import HierZDT1Small
from sb_arch_opt.algo.pymoo_interface import get_nsga2
# Create the problem
problem = HierZDT1Small()
# Create the algorithm (with a population size of 100)
nsga2 = get_nsga2(pop_size=100)
# Run the optimization: 20 generations (20 x 100 = 2000 evaluations)
# NOTE: remove "seed" argument when using in production!
result_nsga2 = minimize(problem, nsga2, termination=('n_gen', 20), seed=42)
# Plot results
plot(result_nsga2.opt.get('F'), labels=['NSGA2 (2000 evaluations)'])
Simple Example: Evolutionary Algorithm (SBO)¶
The code below shows a simple example of solving an architecture optimization problem using ArchSBO, the built-in Surrogate-Based Optimization (SBO) algorithm. Compared to evolutionary algorithms (e.g. NSGA2), surrogate-based optimization algorithms converge in less iterations.
The SBO algorith implemented here has been specifically configured to deal with architecture optimization, and can deal with mixed-discrete, multi-objective optimization problems including hidden constraints. For more information, refer to the documentation.
Note: make sure you have installed the arch_sbo dependencies:
pip install sb-arch-opt[arch_sbo]
In [2]:
Copied!
from sb_arch_opt.algo.pymoo_interface import plot
from pymoo.optimize import minimize
from sb_arch_opt.problems.hierarchical import HierZDT1Small
from sb_arch_opt.algo.arch_sbo.api import get_arch_sbo_gp
# Create the problem
problem = HierZDT1Small()
# Create the algorithm (with an initial database size of 10*n_var = 10*6 = 60)
n_init = problem.n_var*10
sbo = get_arch_sbo_gp(problem, n_parallel=4, init_size=n_init)
# Run the optimization: 100 infill points (60 + 100 = 160 evaluations)
# NOTE: remove "seed" argument when using in production!
n_infill = 100
result_sbo = minimize(problem, sbo, termination=('n_eval', n_init+n_infill), seed=42)
# Plot results
plot(result_nsga2.opt.get('F'), result_sbo.opt.get('F'), labels=['NSGA2 (2000 evaluations)', 'SBO (160 evaluations)'])
from sb_arch_opt.algo.pymoo_interface import plot
from pymoo.optimize import minimize
from sb_arch_opt.problems.hierarchical import HierZDT1Small
from sb_arch_opt.algo.arch_sbo.api import get_arch_sbo_gp
# Create the problem
problem = HierZDT1Small()
# Create the algorithm (with an initial database size of 10*n_var = 10*6 = 60)
n_init = problem.n_var*10
sbo = get_arch_sbo_gp(problem, n_parallel=4, init_size=n_init)
# Run the optimization: 100 infill points (60 + 100 = 160 evaluations)
# NOTE: remove "seed" argument when using in production!
n_infill = 100
result_sbo = minimize(problem, sbo, termination=('n_eval', n_init+n_infill), seed=42)
# Plot results
plot(result_nsga2.opt.get('F'), result_sbo.opt.get('F'), labels=['NSGA2 (2000 evaluations)', 'SBO (160 evaluations)'])
INFO 2025-12-01 16:21:49,852 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (60 real unique, 60 eval) INFO 2025-12-01 16:21:50,753 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (60 real unique, 60 eval) INFO 2025-12-01 16:21:51,303 sb_arch_opt.sbo : Infill: 4 new (eval 60 real unique, 60 eval) INFO 2025-12-01 16:21:54,980 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (64 real unique, 64 eval) INFO 2025-12-01 16:21:55,480 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (64 real unique, 64 eval) INFO 2025-12-01 16:21:56,244 sb_arch_opt.sbo : Infill: 4 new (eval 64 real unique, 64 eval) INFO 2025-12-01 16:21:59,199 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (68 real unique, 68 eval) INFO 2025-12-01 16:21:59,953 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (68 real unique, 68 eval) INFO 2025-12-01 16:22:00,772 sb_arch_opt.sbo : Infill: 4 new (eval 68 real unique, 68 eval) INFO 2025-12-01 16:22:04,920 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (72 real unique, 72 eval) INFO 2025-12-01 16:22:05,594 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (72 real unique, 72 eval) INFO 2025-12-01 16:22:06,265 sb_arch_opt.sbo : Infill: 4 new (eval 72 real unique, 72 eval) INFO 2025-12-01 16:22:11,267 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (76 real unique, 76 eval) INFO 2025-12-01 16:22:12,169 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (76 real unique, 76 eval) INFO 2025-12-01 16:22:12,432 sb_arch_opt.sbo : Infill: 4 new (eval 76 real unique, 76 eval) INFO 2025-12-01 16:22:17,473 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (80 real unique, 80 eval) INFO 2025-12-01 16:22:18,531 sb_arch_opt.sbo : Infill: 4 new (eval 80 real unique, 80 eval) INFO 2025-12-01 16:22:23,021 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (84 real unique, 84 eval) INFO 2025-12-01 16:22:24,036 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (84 real unique, 84 eval) INFO 2025-12-01 16:22:25,104 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (84 real unique, 84 eval) INFO 2025-12-01 16:22:25,936 sb_arch_opt.sbo : Surrogate infill gen 40 @ 4000 points evaluated (84 real unique, 84 eval) INFO 2025-12-01 16:22:27,032 sb_arch_opt.sbo : Surrogate infill gen 50 @ 5000 points evaluated (84 real unique, 84 eval) INFO 2025-12-01 16:22:27,543 sb_arch_opt.sbo : Infill: 4 new (eval 84 real unique, 84 eval) INFO 2025-12-01 16:22:31,462 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (88 real unique, 88 eval) INFO 2025-12-01 16:22:31,921 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (88 real unique, 88 eval) INFO 2025-12-01 16:22:32,214 sb_arch_opt.sbo : Infill: 4 new (eval 88 real unique, 88 eval) INFO 2025-12-01 16:22:36,962 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (92 real unique, 92 eval) INFO 2025-12-01 16:22:37,602 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (92 real unique, 92 eval) INFO 2025-12-01 16:22:38,062 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (92 real unique, 92 eval) INFO 2025-12-01 16:22:38,380 sb_arch_opt.sbo : Infill: 4 new (eval 92 real unique, 92 eval) INFO 2025-12-01 16:22:42,193 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (96 real unique, 96 eval) INFO 2025-12-01 16:22:42,803 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (96 real unique, 96 eval) INFO 2025-12-01 16:22:43,482 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (96 real unique, 96 eval) INFO 2025-12-01 16:22:43,887 sb_arch_opt.sbo : Infill: 4 new (eval 96 real unique, 96 eval) INFO 2025-12-01 16:22:48,340 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (100 real unique, 100 eval) INFO 2025-12-01 16:22:48,937 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (100 real unique, 100 eval) INFO 2025-12-01 16:22:49,407 sb_arch_opt.sbo : Infill: 4 new (eval 100 real unique, 100 eval) INFO 2025-12-01 16:22:54,034 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (104 real unique, 104 eval) INFO 2025-12-01 16:22:54,670 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (104 real unique, 104 eval) INFO 2025-12-01 16:22:55,255 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (104 real unique, 104 eval) INFO 2025-12-01 16:22:56,154 sb_arch_opt.sbo : Infill: 4 new (eval 104 real unique, 104 eval) INFO 2025-12-01 16:23:00,076 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (108 real unique, 108 eval) INFO 2025-12-01 16:23:01,032 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (108 real unique, 108 eval) INFO 2025-12-01 16:23:01,714 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (108 real unique, 108 eval) INFO 2025-12-01 16:23:02,076 sb_arch_opt.sbo : Infill: 4 new (eval 108 real unique, 108 eval) INFO 2025-12-01 16:23:05,901 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (112 real unique, 112 eval) INFO 2025-12-01 16:23:06,545 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (112 real unique, 112 eval) INFO 2025-12-01 16:23:07,020 sb_arch_opt.sbo : Infill: 4 new (eval 112 real unique, 112 eval) INFO 2025-12-01 16:23:10,859 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (116 real unique, 116 eval) INFO 2025-12-01 16:23:11,302 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (116 real unique, 116 eval) INFO 2025-12-01 16:23:11,720 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (116 real unique, 116 eval) INFO 2025-12-01 16:23:12,176 sb_arch_opt.sbo : Surrogate infill gen 40 @ 4000 points evaluated (116 real unique, 116 eval) INFO 2025-12-01 16:23:12,338 sb_arch_opt.sbo : Infill: 4 new (eval 116 real unique, 116 eval) INFO 2025-12-01 16:23:16,489 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (120 real unique, 120 eval) INFO 2025-12-01 16:23:17,132 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (120 real unique, 120 eval) INFO 2025-12-01 16:23:17,586 sb_arch_opt.sbo : Infill: 4 new (eval 120 real unique, 120 eval) INFO 2025-12-01 16:23:21,986 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (124 real unique, 124 eval) INFO 2025-12-01 16:23:22,587 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (124 real unique, 124 eval) INFO 2025-12-01 16:23:23,366 sb_arch_opt.sbo : Infill: 4 new (eval 124 real unique, 124 eval) INFO 2025-12-01 16:23:27,821 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (128 real unique, 128 eval) INFO 2025-12-01 16:23:28,354 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (128 real unique, 128 eval) INFO 2025-12-01 16:23:29,013 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (128 real unique, 128 eval) INFO 2025-12-01 16:23:29,599 sb_arch_opt.sbo : Infill: 4 new (eval 128 real unique, 128 eval) INFO 2025-12-01 16:23:33,932 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (132 real unique, 132 eval) INFO 2025-12-01 16:23:34,073 sb_arch_opt.sbo : Infill: 4 new (eval 132 real unique, 132 eval) INFO 2025-12-01 16:23:38,157 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (136 real unique, 136 eval) INFO 2025-12-01 16:23:38,536 sb_arch_opt.sbo : Infill: 4 new (eval 136 real unique, 136 eval) INFO 2025-12-01 16:23:43,324 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (140 real unique, 140 eval) INFO 2025-12-01 16:23:44,000 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (140 real unique, 140 eval) INFO 2025-12-01 16:23:44,366 sb_arch_opt.sbo : Infill: 4 new (eval 140 real unique, 140 eval) INFO 2025-12-01 16:23:50,269 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (144 real unique, 144 eval) INFO 2025-12-01 16:23:50,981 sb_arch_opt.sbo : Infill: 4 new (eval 144 real unique, 144 eval) INFO 2025-12-01 16:23:56,230 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (148 real unique, 148 eval) INFO 2025-12-01 16:23:56,480 sb_arch_opt.sbo : Infill: 4 new (eval 148 real unique, 148 eval) INFO 2025-12-01 16:24:01,470 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (152 real unique, 152 eval) INFO 2025-12-01 16:24:02,082 sb_arch_opt.sbo : Infill: 4 new (eval 152 real unique, 152 eval) INFO 2025-12-01 16:24:07,729 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (156 real unique, 156 eval) INFO 2025-12-01 16:24:08,423 sb_arch_opt.sbo : Infill: 4 new (eval 156 real unique, 156 eval)
The Architecture Problem Class¶
Architecture optimization problems in SBArchOpt are defined by extending the sb_arch_opt.problem.ArchOptProblemBase class.
This class provides all information needed to optimization algorithms for solving optimization problems:
- Design variable definitions (continuous, integer, categorical)
- Mechanisms for automatically correcting design vectors to ensure they are valid
- Some metrics for describing the design space
Correcting design vectors is important, because due to design variable hierarchy design variables might be inactive for some design vectors. Ignoring this effect could lead to the optimizer evaluating different design vectors that represent the same underlying architecture, thereby wasting computational resources. The degree of hierarchy is quantified using the imputation ratio: the ratio between the number of declared design points to the number of valid design points. A value of 1 indicates no hierarchy, higher values indicate higher levels of hierarchy. For example, a value of 100 means that only 1% (1/100) of randomly generated design vectors actually represent a valid design vector.
Another characteristic of architecture optimization is that often some options of design variables occur much more often in the complete valid design space than others. For example, one option might deactivate many other design variables and therefore yield a "smaller" subproblem. This characteristic is quantified using rate diversity and can be determined by analyzing all possible (discrete) design vectors of the problem.
For more information, refer to Implementing an Architecture Optimization Problem.
In [3]:
Copied!
from sb_arch_opt.problems.hierarchical import HierZDT1Small
# Instantiate the problem class and print some statistics
problem = HierZDT1Small()
problem.print_stats()
from sb_arch_opt.problems.hierarchical import HierZDT1Small
# Instantiate the problem class and print some statistics
problem = HierZDT1Small()
problem.print_stats()
problem: HierZDT1Small(NoHierarchyWrappedProblem(), imp_ratio=2.0, n_subproblem=10, n_opts=3, diversity_range=0.25, cont_ratio=1)
n_discr: 3
n_cont : 3
n_obj : 2
n_con : 0
MD : True
MO : True
HIER : True
n_valid_discr: 10
imp_ratio : 1.80 (discr.: 1.80; cont.: 1.00)
corr_ratio : 1.12 (discr.: 1.12; cont.: 1.00; fraction of imp_ratio: 20.0%)
x0 x1 x2 max
inactive 0.6
opt 0 0.5 0.4 0.5
opt 1 0.2 0.3 0.5
opt 2 0.3 0.3
diversity 0.3 0.1 0.4 0.4
active-diversity 0.3 0.1 0.0 0.3
x_type cat cat cat
is_cond False False True
In [4]:
Copied!
from sb_arch_opt.algo.pymoo_interface import plot
# Plot the Pareto front
# Normally only test problems have access to the "real" Pareto front!
plot(problem.pareto_front())
from sb_arch_opt.algo.pymoo_interface import plot
# Plot the Pareto front
# Normally only test problems have access to the "real" Pareto front!
plot(problem.pareto_front())
Solving an Analytical Problem with NSGA2¶
NSGA2 is a powerful evolutionary optimization algorithm. The algorithm as implemented in pymoo works well, however SBArchOpt adds several features on top to help with solving architecture optimization problems:
- Intermediate results storage and restart
- A repair operator that uses the architecture problem definition to automatically correct design points
To get a preconfigured NSGA2 algorithm, use the get_nsga2 function. It can then be used like a normal pymoo algorithm to run an optimization.
In [5]:
Copied!
from pymoo.optimize import minimize
from sb_arch_opt.algo.pymoo_interface import get_nsga2
# Create the pre-configured NSGA2 algorithm
nsga2 = get_nsga2(pop_size=100)
# Run the optimization
result = minimize(problem, nsga2, termination=('n_gen', 20), seed=42)
# Compare results
plot(problem.pareto_front(), result.opt.get('F'), labels=['Known Pareto front', 'NSGA2'])
from pymoo.optimize import minimize
from sb_arch_opt.algo.pymoo_interface import get_nsga2
# Create the pre-configured NSGA2 algorithm
nsga2 = get_nsga2(pop_size=100)
# Run the optimization
result = minimize(problem, nsga2, termination=('n_gen', 20), seed=42)
# Compare results
plot(problem.pareto_front(), result.opt.get('F'), labels=['Known Pareto front', 'NSGA2'])
Restart and Intermediate Result Storage¶
Restart is very important when running expensive optimization studies, because if any problem happens you do not want to start from scratch (if the problem was not due to any bug in the evaluation code of course). Intermediate results storage can be useful for checking if generated results are correct and realistic while the algorithm is running.
To enable restart, we can use the initialize_from_previous_results folder. Restart also requires intermediate results storage, which can be activated by passing a results_folder when instantiating the algorithm.
In [6]:
Copied!
import warnings
warnings.filterwarnings('ignore')
import warnings
warnings.filterwarnings('ignore')
In [7]:
Copied!
import tempfile
# We create a temporary folder for demonstration purposes
td = tempfile.TemporaryDirectory()
results_folder = td.name
# Create the algorithm with the results folder
nsga2 = get_nsga2(pop_size=100, results_folder=results_folder)
# Run for a few generations
result5 = minimize(problem, nsga2, termination=('n_gen', 5), copy_algorithm=False, seed=42)
assert abs(nsga2.evaluator.n_eval - 500) < 10 # 5*100
plot(problem.pareto_front(), result5.opt.get('F'),
labels=['Known Pareto front', 'NSGA2 (5 gen)'])
import tempfile
# We create a temporary folder for demonstration purposes
td = tempfile.TemporaryDirectory()
results_folder = td.name
# Create the algorithm with the results folder
nsga2 = get_nsga2(pop_size=100, results_folder=results_folder)
# Run for a few generations
result5 = minimize(problem, nsga2, termination=('n_gen', 5), copy_algorithm=False, seed=42)
assert abs(nsga2.evaluator.n_eval - 500) < 10 # 5*100
plot(problem.pareto_front(), result5.opt.get('F'),
labels=['Known Pareto front', 'NSGA2 (5 gen)'])
The results folder now contains some files that you can inspect:
In [8]:
Copied!
import os
print(os.listdir(results_folder))
import os
print(os.listdir(results_folder))
['pymoo_population.csv', 'pymoo_population.pkl', 'pymoo_population_cumulative.csv', 'pymoo_population_cumulative.pkl', 'pymoo_results.pkl']
In [9]:
Copied!
from sb_arch_opt.algo.pymoo_interface import initialize_from_previous_results
# Now create the algorithm again and initialize it from previously-generated results
nsga2 = get_nsga2(pop_size=100)
initialize_from_previous_results(nsga2, problem, results_folder)
# NOTE: normally you'd of course also pass the results folder here,
# but we omit it to prevent updating previous results in the notebook
# nsga2 = get_nsga2(pop_size=100, results_folder=results_folder)
# Previously-generated results count towards the evaluation budget,
# however not to the iteration ("generations") budget
result10 = minimize(problem, nsga2, termination=('n_gen', 6), copy_algorithm=False, seed=42)
assert abs(nsga2.evaluator.n_eval - 1000) < 10 # 500 from previous + 5*100 continued optimization
plot(problem.pareto_front(), result5.opt.get('F'), result10.opt.get('F'),
labels=['Known Pareto front', 'NSGA2 (5 gen)', 'NSGA2 (10 gen)'])
from sb_arch_opt.algo.pymoo_interface import initialize_from_previous_results
# Now create the algorithm again and initialize it from previously-generated results
nsga2 = get_nsga2(pop_size=100)
initialize_from_previous_results(nsga2, problem, results_folder)
# NOTE: normally you'd of course also pass the results folder here,
# but we omit it to prevent updating previous results in the notebook
# nsga2 = get_nsga2(pop_size=100, results_folder=results_folder)
# Previously-generated results count towards the evaluation budget,
# however not to the iteration ("generations") budget
result10 = minimize(problem, nsga2, termination=('n_gen', 6), copy_algorithm=False, seed=42)
assert abs(nsga2.evaluator.n_eval - 1000) < 10 # 500 from previous + 5*100 continued optimization
plot(problem.pareto_front(), result5.opt.get('F'), result10.opt.get('F'),
labels=['Known Pareto front', 'NSGA2 (5 gen)', 'NSGA2 (10 gen)'])
INFO 2025-12-01 16:24:22,186 sb_arch_opt.pymoo : Previous results loaded from pymoo results: 500 design points INFO 2025-12-01 16:24:22,186 sb_arch_opt.pymoo : Evaluation status: 500 of 500 (100.0%) are already evaluated
Sampling an Architecture Design Space¶
If you only need to sample an architecture design space (i.e. not perform optimization), you can either use the HierarchicalSampler directly or use the DoE algorithm to also evaluate them, including restart and intermediate storage capabilities.
In [10]:
Copied!
from sb_arch_opt.sampling import HierarchicalSampling
# Generate samples in the hierarchical architecture design space
pop = HierarchicalSampling(seed=42).do(problem, n_samples=100)
x = pop.get('X')
x.shape
from sb_arch_opt.sampling import HierarchicalSampling
# Generate samples in the hierarchical architecture design space
pop = HierarchicalSampling(seed=42).do(problem, n_samples=100)
x = pop.get('X')
x.shape
Out[10]:
(100, 6)
In [11]:
Copied!
from sb_arch_opt.algo.pymoo_interface import get_doe_algo
# Create a new results folder
td.cleanup()
td = tempfile.TemporaryDirectory()
results_folder = td.name
# Create and run the DoE algorithm
doe_algo = get_doe_algo(doe_size=200, results_folder=results_folder)
result = minimize(problem, doe_algo, seed=42)
plot(problem.pareto_front(), result.pop.get('F'), labels=['Known Pareto front', 'Sampled population'])
from sb_arch_opt.algo.pymoo_interface import get_doe_algo
# Create a new results folder
td.cleanup()
td = tempfile.TemporaryDirectory()
results_folder = td.name
# Create and run the DoE algorithm
doe_algo = get_doe_algo(doe_size=200, results_folder=results_folder)
result = minimize(problem, doe_algo, seed=42)
plot(problem.pareto_front(), result.pop.get('F'), labels=['Known Pareto front', 'Sampled population'])
In [12]:
Copied!
# Continue with an optimization from previous DoE results
nsga2 = get_nsga2(pop_size=100)
initialize_from_previous_results(nsga2, problem, results_folder)
result = minimize(problem, nsga2, termination=('n_gen', 10), copy_algorithm=False, seed=42)
assert abs(nsga2.evaluator.n_eval - 1100) < 10 # 200 from DoE + 9*100 from optimization
plot(problem.pareto_front(), result.opt.get('F'), labels=['Known Pareto front', 'NSGA2 (10 gen)'])
# Continue with an optimization from previous DoE results
nsga2 = get_nsga2(pop_size=100)
initialize_from_previous_results(nsga2, problem, results_folder)
result = minimize(problem, nsga2, termination=('n_gen', 10), copy_algorithm=False, seed=42)
assert abs(nsga2.evaluator.n_eval - 1100) < 10 # 200 from DoE + 9*100 from optimization
plot(problem.pareto_front(), result.opt.get('F'), labels=['Known Pareto front', 'NSGA2 (10 gen)'])
INFO 2025-12-01 16:24:26,393 sb_arch_opt.pymoo : Previous results loaded from pymoo results: 200 design points INFO 2025-12-01 16:24:26,393 sb_arch_opt.pymoo : Evaluation status: 200 of 200 (100.0%) are already evaluated
In [13]:
Copied!
td.cleanup()
td.cleanup()
Surrogate-Based Architecture Optimization¶
Surrogate-Based Optimization (SBO) is more useful for expensive problems as generally it requires less function evaluations to find a Pareto front. SBArchOpt contains integrations for several SBO frameworks, however also comes with its own basic SBO algorithm that should be fine for most architecture optimization problems.
Note: make sure you have installed the arch_sbo dependencies:
pip install sb-arch-opt[arch_sbo]
In [14]:
Copied!
td_sbo = tempfile.TemporaryDirectory()
results_folder = td_sbo.name
td_sbo = tempfile.TemporaryDirectory()
results_folder = td_sbo.name
In [15]:
Copied!
from sb_arch_opt.algo.arch_sbo import get_arch_sbo_gp
n_doe = 10
n_infill = 20
# Get SBO with a Gaussian Process model
sbo_algo = get_arch_sbo_gp(problem, init_size=n_doe, results_folder=results_folder)
result = minimize(problem, sbo_algo, termination=('n_eval', n_doe+n_infill), seed=1)
from sb_arch_opt.algo.arch_sbo import get_arch_sbo_gp
n_doe = 10
n_infill = 20
# Get SBO with a Gaussian Process model
sbo_algo = get_arch_sbo_gp(problem, init_size=n_doe, results_folder=results_folder)
result = minimize(problem, sbo_algo, termination=('n_eval', n_doe+n_infill), seed=1)
INFO 2025-12-01 16:24:28,763 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (10 real unique, 10 eval) INFO 2025-12-01 16:24:29,077 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (10 real unique, 10 eval) INFO 2025-12-01 16:24:29,312 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (10 real unique, 10 eval) INFO 2025-12-01 16:24:29,390 sb_arch_opt.sbo : Infill: 1 new (eval 10 real unique, 10 eval) INFO 2025-12-01 16:24:32,168 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (11 real unique, 11 eval) INFO 2025-12-01 16:24:32,450 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (11 real unique, 11 eval) INFO 2025-12-01 16:24:32,702 sb_arch_opt.sbo : Infill: 1 new (eval 11 real unique, 11 eval) INFO 2025-12-01 16:24:34,944 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (12 real unique, 12 eval) INFO 2025-12-01 16:24:35,271 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (12 real unique, 12 eval) INFO 2025-12-01 16:24:35,525 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (12 real unique, 12 eval) INFO 2025-12-01 16:24:35,572 sb_arch_opt.sbo : Infill: 1 new (eval 12 real unique, 12 eval) INFO 2025-12-01 16:24:37,720 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (13 real unique, 13 eval) INFO 2025-12-01 16:24:37,987 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (13 real unique, 13 eval) INFO 2025-12-01 16:24:38,316 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (13 real unique, 13 eval) INFO 2025-12-01 16:24:38,426 sb_arch_opt.sbo : Infill: 1 new (eval 13 real unique, 13 eval) INFO 2025-12-01 16:24:40,293 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (14 real unique, 14 eval) INFO 2025-12-01 16:24:40,465 sb_arch_opt.sbo : Infill: 1 new (eval 14 real unique, 14 eval) INFO 2025-12-01 16:24:42,849 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (15 real unique, 15 eval) INFO 2025-12-01 16:24:43,147 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (15 real unique, 15 eval) INFO 2025-12-01 16:24:43,272 sb_arch_opt.sbo : Infill: 1 new (eval 15 real unique, 15 eval) INFO 2025-12-01 16:24:45,291 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (16 real unique, 16 eval) INFO 2025-12-01 16:24:45,547 sb_arch_opt.sbo : Infill: 1 new (eval 16 real unique, 16 eval) INFO 2025-12-01 16:24:47,868 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (17 real unique, 17 eval) INFO 2025-12-01 16:24:48,104 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (17 real unique, 17 eval) INFO 2025-12-01 16:24:48,370 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (17 real unique, 17 eval) INFO 2025-12-01 16:24:48,402 sb_arch_opt.sbo : Infill: 1 new (eval 17 real unique, 17 eval) INFO 2025-12-01 16:24:50,441 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (18 real unique, 18 eval) INFO 2025-12-01 16:24:50,629 sb_arch_opt.sbo : Infill: 1 new (eval 18 real unique, 18 eval) INFO 2025-12-01 16:24:52,982 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (19 real unique, 19 eval) INFO 2025-12-01 16:24:53,390 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (19 real unique, 19 eval) INFO 2025-12-01 16:24:53,751 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (19 real unique, 19 eval) INFO 2025-12-01 16:24:53,908 sb_arch_opt.sbo : Infill: 1 new (eval 19 real unique, 19 eval) INFO 2025-12-01 16:24:55,916 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (20 real unique, 20 eval) INFO 2025-12-01 16:24:56,324 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (20 real unique, 20 eval) INFO 2025-12-01 16:24:56,701 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (20 real unique, 20 eval) INFO 2025-12-01 16:24:56,733 sb_arch_opt.sbo : Infill: 1 new (eval 20 real unique, 20 eval) INFO 2025-12-01 16:24:58,788 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (21 real unique, 21 eval) INFO 2025-12-01 16:24:59,164 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (21 real unique, 21 eval) INFO 2025-12-01 16:24:59,519 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (21 real unique, 21 eval) INFO 2025-12-01 16:24:59,871 sb_arch_opt.sbo : Infill: 1 new (eval 21 real unique, 21 eval) INFO 2025-12-01 16:25:02,317 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (22 real unique, 22 eval) INFO 2025-12-01 16:25:02,710 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (22 real unique, 22 eval) INFO 2025-12-01 16:25:02,898 sb_arch_opt.sbo : Infill: 1 new (eval 22 real unique, 22 eval) INFO 2025-12-01 16:25:05,235 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (23 real unique, 23 eval) INFO 2025-12-01 16:25:05,486 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (23 real unique, 23 eval) INFO 2025-12-01 16:25:05,784 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (23 real unique, 23 eval) INFO 2025-12-01 16:25:05,863 sb_arch_opt.sbo : Infill: 1 new (eval 23 real unique, 23 eval) INFO 2025-12-01 16:25:07,792 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (24 real unique, 24 eval) INFO 2025-12-01 16:25:08,043 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (24 real unique, 24 eval) INFO 2025-12-01 16:25:08,184 sb_arch_opt.sbo : Infill: 1 new (eval 24 real unique, 24 eval) INFO 2025-12-01 16:25:10,207 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (25 real unique, 25 eval) INFO 2025-12-01 16:25:10,474 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (25 real unique, 25 eval) INFO 2025-12-01 16:25:10,742 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (25 real unique, 25 eval) INFO 2025-12-01 16:25:11,007 sb_arch_opt.sbo : Surrogate infill gen 40 @ 4000 points evaluated (25 real unique, 25 eval) INFO 2025-12-01 16:25:11,196 sb_arch_opt.sbo : Infill: 1 new (eval 25 real unique, 25 eval) INFO 2025-12-01 16:25:13,109 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (26 real unique, 26 eval) INFO 2025-12-01 16:25:13,361 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (26 real unique, 26 eval) INFO 2025-12-01 16:25:13,659 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (26 real unique, 26 eval) INFO 2025-12-01 16:25:13,753 sb_arch_opt.sbo : Infill: 1 new (eval 26 real unique, 26 eval) INFO 2025-12-01 16:25:15,874 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (27 real unique, 27 eval) INFO 2025-12-01 16:25:16,203 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (27 real unique, 27 eval) INFO 2025-12-01 16:25:16,407 sb_arch_opt.sbo : Infill: 1 new (eval 27 real unique, 27 eval) INFO 2025-12-01 16:25:18,384 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (28 real unique, 28 eval) INFO 2025-12-01 16:25:18,776 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (28 real unique, 28 eval) INFO 2025-12-01 16:25:19,043 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (28 real unique, 28 eval) INFO 2025-12-01 16:25:19,341 sb_arch_opt.sbo : Surrogate infill gen 40 @ 4000 points evaluated (28 real unique, 28 eval) INFO 2025-12-01 16:25:19,372 sb_arch_opt.sbo : Infill: 1 new (eval 28 real unique, 28 eval) INFO 2025-12-01 16:25:21,427 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (29 real unique, 29 eval) INFO 2025-12-01 16:25:21,914 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (29 real unique, 29 eval) INFO 2025-12-01 16:25:22,244 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (29 real unique, 29 eval) INFO 2025-12-01 16:25:22,338 sb_arch_opt.sbo : Infill: 1 new (eval 29 real unique, 29 eval)
In [16]:
Copied!
plot(problem.pareto_front(), result.opt.get('F'), labels=['Known Pareto front', 'SBO (10+20)'])
plot(problem.pareto_front(), result.opt.get('F'), labels=['Known Pareto front', 'SBO (10+20)'])
As can be seen, the results are not quite near the Pareto front yet. Let's add some more infill points.
In [17]:
Copied!
n_infill = 50
sbo_algo = get_arch_sbo_gp(problem, init_size=n_doe, results_folder=results_folder)
sbo_algo.initialize_from_previous_results(problem, results_folder)
result2 = minimize(problem, sbo_algo, termination=('n_eval', n_doe+n_infill), seed=1)
n_infill = 50
sbo_algo = get_arch_sbo_gp(problem, init_size=n_doe, results_folder=results_folder)
sbo_algo.initialize_from_previous_results(problem, results_folder)
result2 = minimize(problem, sbo_algo, termination=('n_eval', n_doe+n_infill), seed=1)
INFO 2025-12-01 16:25:22,448 sb_arch_opt.pymoo : Previous results loaded from pymoo results: 30 design points INFO 2025-12-01 16:25:22,448 sb_arch_opt.pymoo : Evaluation status: 30 of 30 (100.0%) are already evaluated INFO 2025-12-01 16:25:24,440 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (30 real unique, 30 eval) INFO 2025-12-01 16:25:24,848 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (30 real unique, 30 eval) INFO 2025-12-01 16:25:25,116 sb_arch_opt.sbo : Infill: 1 new (eval 30 real unique, 30 eval) INFO 2025-12-01 16:25:27,299 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (31 real unique, 31 eval) INFO 2025-12-01 16:25:27,644 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (31 real unique, 31 eval) INFO 2025-12-01 16:25:27,911 sb_arch_opt.sbo : Infill: 1 new (eval 31 real unique, 31 eval) INFO 2025-12-01 16:25:29,840 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (32 real unique, 32 eval) INFO 2025-12-01 16:25:30,186 sb_arch_opt.sbo : Infill: 1 new (eval 32 real unique, 32 eval) INFO 2025-12-01 16:25:32,088 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (33 real unique, 33 eval) INFO 2025-12-01 16:25:32,433 sb_arch_opt.sbo : Infill: 1 new (eval 33 real unique, 33 eval) INFO 2025-12-01 16:25:34,472 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (34 real unique, 34 eval) INFO 2025-12-01 16:25:34,849 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (34 real unique, 34 eval) INFO 2025-12-01 16:25:35,115 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (34 real unique, 34 eval) INFO 2025-12-01 16:25:35,194 sb_arch_opt.sbo : Infill: 1 new (eval 34 real unique, 34 eval) INFO 2025-12-01 16:25:37,374 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (35 real unique, 35 eval) INFO 2025-12-01 16:25:37,672 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (35 real unique, 35 eval) INFO 2025-12-01 16:25:38,237 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (35 real unique, 35 eval) INFO 2025-12-01 16:25:38,496 sb_arch_opt.sbo : Infill: 1 new (eval 35 real unique, 35 eval) INFO 2025-12-01 16:25:40,543 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (36 real unique, 36 eval) INFO 2025-12-01 16:25:41,124 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (36 real unique, 36 eval) INFO 2025-12-01 16:25:41,250 sb_arch_opt.sbo : Infill: 1 new (eval 36 real unique, 36 eval) INFO 2025-12-01 16:25:43,211 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (37 real unique, 37 eval) INFO 2025-12-01 16:25:43,666 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (37 real unique, 37 eval) INFO 2025-12-01 16:25:43,854 sb_arch_opt.sbo : Infill: 1 new (eval 37 real unique, 37 eval) INFO 2025-12-01 16:25:45,987 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (38 real unique, 38 eval) INFO 2025-12-01 16:25:46,505 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (38 real unique, 38 eval) INFO 2025-12-01 16:25:46,976 sb_arch_opt.sbo : Infill: 1 new (eval 38 real unique, 38 eval) INFO 2025-12-01 16:25:49,063 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (39 real unique, 39 eval) INFO 2025-12-01 16:25:49,267 sb_arch_opt.sbo : Infill: 1 new (eval 39 real unique, 39 eval) INFO 2025-12-01 16:25:51,280 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (40 real unique, 40 eval) INFO 2025-12-01 16:25:51,595 sb_arch_opt.sbo : Infill: 1 new (eval 40 real unique, 40 eval) INFO 2025-12-01 16:25:53,707 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (41 real unique, 41 eval) INFO 2025-12-01 16:25:54,336 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (41 real unique, 41 eval) INFO 2025-12-01 16:25:54,508 sb_arch_opt.sbo : Infill: 1 new (eval 41 real unique, 41 eval) INFO 2025-12-01 16:25:56,791 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (42 real unique, 42 eval) INFO 2025-12-01 16:25:57,230 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (42 real unique, 42 eval) INFO 2025-12-01 16:25:57,387 sb_arch_opt.sbo : Infill: 1 new (eval 42 real unique, 42 eval) INFO 2025-12-01 16:25:59,612 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (43 real unique, 43 eval) INFO 2025-12-01 16:25:59,973 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (43 real unique, 43 eval) INFO 2025-12-01 16:26:00,334 sb_arch_opt.sbo : Infill: 1 new (eval 43 real unique, 43 eval) INFO 2025-12-01 16:26:02,605 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (44 real unique, 44 eval) INFO 2025-12-01 16:26:02,936 sb_arch_opt.sbo : Infill: 1 new (eval 44 real unique, 44 eval) INFO 2025-12-01 16:26:05,287 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (45 real unique, 45 eval) INFO 2025-12-01 16:26:05,697 sb_arch_opt.sbo : Infill: 1 new (eval 45 real unique, 45 eval) INFO 2025-12-01 16:26:08,039 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (46 real unique, 46 eval) INFO 2025-12-01 16:26:08,622 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (46 real unique, 46 eval) INFO 2025-12-01 16:26:08,810 sb_arch_opt.sbo : Infill: 1 new (eval 46 real unique, 46 eval) INFO 2025-12-01 16:26:11,039 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (47 real unique, 47 eval) INFO 2025-12-01 16:26:11,416 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (47 real unique, 47 eval) INFO 2025-12-01 16:26:11,794 sb_arch_opt.sbo : Infill: 1 new (eval 47 real unique, 47 eval) INFO 2025-12-01 16:26:14,007 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (48 real unique, 48 eval) INFO 2025-12-01 16:26:14,478 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (48 real unique, 48 eval) INFO 2025-12-01 16:26:14,778 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (48 real unique, 48 eval) INFO 2025-12-01 16:26:15,123 sb_arch_opt.sbo : Surrogate infill gen 40 @ 4000 points evaluated (48 real unique, 48 eval) INFO 2025-12-01 16:26:15,297 sb_arch_opt.sbo : Infill: 1 new (eval 48 real unique, 48 eval) INFO 2025-12-01 16:26:17,417 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (49 real unique, 49 eval) INFO 2025-12-01 16:26:17,795 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (49 real unique, 49 eval) INFO 2025-12-01 16:26:18,078 sb_arch_opt.sbo : Infill: 1 new (eval 49 real unique, 49 eval) INFO 2025-12-01 16:26:20,357 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (50 real unique, 50 eval) INFO 2025-12-01 16:26:20,776 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (50 real unique, 50 eval) INFO 2025-12-01 16:26:21,183 sb_arch_opt.sbo : Surrogate infill gen 30 @ 3000 points evaluated (50 real unique, 50 eval) INFO 2025-12-01 16:26:21,230 sb_arch_opt.sbo : Infill: 1 new (eval 50 real unique, 50 eval) INFO 2025-12-01 16:26:23,740 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (51 real unique, 51 eval) INFO 2025-12-01 16:26:24,179 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (51 real unique, 51 eval) INFO 2025-12-01 16:26:24,351 sb_arch_opt.sbo : Infill: 1 new (eval 51 real unique, 51 eval) INFO 2025-12-01 16:26:27,035 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (52 real unique, 52 eval) INFO 2025-12-01 16:26:27,427 sb_arch_opt.sbo : Infill: 1 new (eval 52 real unique, 52 eval) INFO 2025-12-01 16:26:30,408 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (53 real unique, 53 eval) INFO 2025-12-01 16:26:30,878 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (53 real unique, 53 eval) INFO 2025-12-01 16:26:31,176 sb_arch_opt.sbo : Infill: 1 new (eval 53 real unique, 53 eval) INFO 2025-12-01 16:26:33,779 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (54 real unique, 54 eval) INFO 2025-12-01 16:26:34,202 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (54 real unique, 54 eval) INFO 2025-12-01 16:26:34,531 sb_arch_opt.sbo : Infill: 1 new (eval 54 real unique, 54 eval) INFO 2025-12-01 16:26:37,276 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (55 real unique, 55 eval) INFO 2025-12-01 16:26:37,574 sb_arch_opt.sbo : Infill: 1 new (eval 55 real unique, 55 eval) INFO 2025-12-01 16:26:40,398 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (56 real unique, 56 eval) INFO 2025-12-01 16:26:40,712 sb_arch_opt.sbo : Infill: 1 new (eval 56 real unique, 56 eval) INFO 2025-12-01 16:26:43,474 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (57 real unique, 57 eval) INFO 2025-12-01 16:26:43,913 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (57 real unique, 57 eval) INFO 2025-12-01 16:26:44,148 sb_arch_opt.sbo : Infill: 1 new (eval 57 real unique, 57 eval) INFO 2025-12-01 16:26:46,894 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (58 real unique, 58 eval) INFO 2025-12-01 16:26:47,537 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (58 real unique, 58 eval) INFO 2025-12-01 16:26:47,772 sb_arch_opt.sbo : Infill: 1 new (eval 58 real unique, 58 eval) INFO 2025-12-01 16:26:50,549 sb_arch_opt.sbo : Surrogate infill gen 10 @ 1000 points evaluated (59 real unique, 59 eval) INFO 2025-12-01 16:26:51,113 sb_arch_opt.sbo : Surrogate infill gen 20 @ 2000 points evaluated (59 real unique, 59 eval) INFO 2025-12-01 16:26:51,567 sb_arch_opt.sbo : Infill: 1 new (eval 59 real unique, 59 eval)
In [18]:
Copied!
plot(problem.pareto_front(), result.opt.get('F'), result2.opt.get('F'), labels=['Known Pareto front', 'SBO (10+20)', 'SBO (10+50)'])
plot(problem.pareto_front(), result.opt.get('F'), result2.opt.get('F'), labels=['Known Pareto front', 'SBO (10+20)', 'SBO (10+50)'])
As you can see, already a good approximation of the Pareto front is achieved after only 60 evaluations, compared to 1000 evaluations for NSGA2 for a similar performance.
In [19]:
Copied!
td_sbo.cleanup()
td_sbo.cleanup()
Implementing an Architecture Optimization Problem¶
Implementing a new architecture optimization problem is done by extending the ArchOptProblemBase class.
The hierarchical structure of the design space can be specified in two ways:
- Implicitly by implementing correction functions yourself
- Explicitly using a model of the hierarchical structure
Both are good options depending on what you want to achieve. If you already have some other code that implements design variable correcting then you can use the implicit method. If you are starting from scratch then the explicit method is recommended.
We demonstrate the different using the Jenatton test problem:
- 3 categorical variables: 0 or 1
- 6 continuous variables
- The categorical variables select which continuous variables are used for the calculation
We base our implementation on the one found in Ax (BoTorch).
In [20]:
Copied!
import numpy as np
from pymoo.core.variable import Real, Choice
from sb_arch_opt.problem import ArchOptProblemBase
# Define the evaluation function
# You can clearly see the hierarchical structure
def jenatton_evaluate(x):
if x[0] == 0:
if x[1] == 0:
return x[3]**3 + .1 + x[7]
else:
return x[4]**3 + .1 + x[7]
else:
if x[2] == 0:
return x[5]**3 + .1 + x[8]
else:
return x[6]**3 + .1 + x[8]
class JenattonImplicit(ArchOptProblemBase):
def __init__(self):
# Define the design variables using pymoo variable classes
des_vars = [
Choice(options=[0, 1]), Choice(options=[0, 1]), Choice(options=[0, 1]), # x1,2,3
Real(bounds=[0, 1]), Real(bounds=[0, 1]), Real(bounds=[0, 1]), # x4,5,6
Real(bounds=[0, 1]), Real(bounds=[0, 1]), Real(bounds=[0, 1]), # x7,8,9
]
super().__init__(des_vars=des_vars)
def _arch_evaluate(self, x: np.ndarray, is_active_out: np.ndarray, f_out: np.ndarray, g_out: np.ndarray,
h_out: np.ndarray, *args, **kwargs):
"""
Implement evaluation and write results in the provided output matrices:
- x (design vectors): discrete variables have integer values, imputed design vectors can be output here
- is_active (activeness): vector specifying for each design variable whether it was active or not
- f (objectives): written as a minimization
- g (inequality constraints): written as "<= 0"
- h (equality constraints): written as "= 0"
"""
# Call the correction function to output which variables are inactive
self._correct_x_impute(x, is_active_out)
# Evaluate the design vectors
for i, xi in enumerate(x):
f_out[i, 0] = jenatton_evaluate(xi)
def _correct_x(self, x: np.ndarray, is_active: np.ndarray):
"""Fill the activeness matrix and (if needed) impute any design variables that are partially inactive.
Imputation of inactive design variables is always applied after this function."""
# Start from everything inactive except the first variable
is_active[:, 1:] = False
# First branch
branch1_1 = x[:, 0] == 0
branch1_2 = x[:, 0] == 1
is_active[branch1_1, 1] = True # x2 and x8 are active in branch 1,1
is_active[branch1_1, 7] = True
is_active[branch1_2, 2] = True # x3 and x9 are active in branch 1,2
is_active[branch1_2, 8] = True
# x1 and x2 branch
branch2_1 = branch1_1 & (x[:, 1] == 0)
branch2_2 = branch1_1 & (x[:, 1] == 1)
branch2_3 = branch1_2 & (x[:, 2] == 0)
branch2_4 = branch1_2 & (x[:, 2] == 1)
is_active[branch2_1, 3] = True
is_active[branch2_2, 4] = True
is_active[branch2_3, 5] = True
is_active[branch2_4, 6] = True
def _is_conditionally_active(self):
"""Return for each design variable whether it is conditionally active (i.e. might become inactive). Not needed
if an explicit design space is provided."""
# Only the first variable never becomes inactive
return [False if i == 0 else True for i in range(self.n_var)]
def _get_n_active_cont_mean(self) -> int:
"""Get the mean number of active continuous dimensions, as seen over all discrete design vectors."""
# Each of the four discrete branches has 2 active continuous variables
return 2
def _get_n_valid_discrete(self) -> int:
"""Return the number of valid discrete design points (ignoring continuous dimensions); enables calculation of
the imputation ratio. Not needed if an explicit design space is provided."""
return 4
def __repr__(self):
return f'{self.__class__.__name__}()'
JenattonImplicit().print_stats()
import numpy as np
from pymoo.core.variable import Real, Choice
from sb_arch_opt.problem import ArchOptProblemBase
# Define the evaluation function
# You can clearly see the hierarchical structure
def jenatton_evaluate(x):
if x[0] == 0:
if x[1] == 0:
return x[3]**3 + .1 + x[7]
else:
return x[4]**3 + .1 + x[7]
else:
if x[2] == 0:
return x[5]**3 + .1 + x[8]
else:
return x[6]**3 + .1 + x[8]
class JenattonImplicit(ArchOptProblemBase):
def __init__(self):
# Define the design variables using pymoo variable classes
des_vars = [
Choice(options=[0, 1]), Choice(options=[0, 1]), Choice(options=[0, 1]), # x1,2,3
Real(bounds=[0, 1]), Real(bounds=[0, 1]), Real(bounds=[0, 1]), # x4,5,6
Real(bounds=[0, 1]), Real(bounds=[0, 1]), Real(bounds=[0, 1]), # x7,8,9
]
super().__init__(des_vars=des_vars)
def _arch_evaluate(self, x: np.ndarray, is_active_out: np.ndarray, f_out: np.ndarray, g_out: np.ndarray,
h_out: np.ndarray, *args, **kwargs):
"""
Implement evaluation and write results in the provided output matrices:
- x (design vectors): discrete variables have integer values, imputed design vectors can be output here
- is_active (activeness): vector specifying for each design variable whether it was active or not
- f (objectives): written as a minimization
- g (inequality constraints): written as "<= 0"
- h (equality constraints): written as "= 0"
"""
# Call the correction function to output which variables are inactive
self._correct_x_impute(x, is_active_out)
# Evaluate the design vectors
for i, xi in enumerate(x):
f_out[i, 0] = jenatton_evaluate(xi)
def _correct_x(self, x: np.ndarray, is_active: np.ndarray):
"""Fill the activeness matrix and (if needed) impute any design variables that are partially inactive.
Imputation of inactive design variables is always applied after this function."""
# Start from everything inactive except the first variable
is_active[:, 1:] = False
# First branch
branch1_1 = x[:, 0] == 0
branch1_2 = x[:, 0] == 1
is_active[branch1_1, 1] = True # x2 and x8 are active in branch 1,1
is_active[branch1_1, 7] = True
is_active[branch1_2, 2] = True # x3 and x9 are active in branch 1,2
is_active[branch1_2, 8] = True
# x1 and x2 branch
branch2_1 = branch1_1 & (x[:, 1] == 0)
branch2_2 = branch1_1 & (x[:, 1] == 1)
branch2_3 = branch1_2 & (x[:, 2] == 0)
branch2_4 = branch1_2 & (x[:, 2] == 1)
is_active[branch2_1, 3] = True
is_active[branch2_2, 4] = True
is_active[branch2_3, 5] = True
is_active[branch2_4, 6] = True
def _is_conditionally_active(self):
"""Return for each design variable whether it is conditionally active (i.e. might become inactive). Not needed
if an explicit design space is provided."""
# Only the first variable never becomes inactive
return [False if i == 0 else True for i in range(self.n_var)]
def _get_n_active_cont_mean(self) -> int:
"""Get the mean number of active continuous dimensions, as seen over all discrete design vectors."""
# Each of the four discrete branches has 2 active continuous variables
return 2
def _get_n_valid_discrete(self) -> int:
"""Return the number of valid discrete design points (ignoring continuous dimensions); enables calculation of
the imputation ratio. Not needed if an explicit design space is provided."""
return 4
def __repr__(self):
return f'{self.__class__.__name__}()'
JenattonImplicit().print_stats()
problem: JenattonImplicit() n_discr: 3 n_cont : 6 n_obj : 1 n_con : 0 MD : True MO : False HIER : True n_valid_discr: 4 imp_ratio : 6.00 (discr.: 2.00; cont.: 3.00)
In [21]:
Copied!
result_impl = minimize(JenattonImplicit(), get_nsga2(pop_size=50), termination=('n_gen', 20), seed=42)
print(f'Optimum: {result_impl.opt.get("F")}')
print(f'At X = {result_impl.opt.get("X")}')
result_impl = minimize(JenattonImplicit(), get_nsga2(pop_size=50), termination=('n_gen', 20), seed=42)
print(f'Optimum: {result_impl.opt.get("F")}')
print(f'At X = {result_impl.opt.get("X")}')
Optimum: [[0.10115816]] At X = [[1. 0. 1. 0.5 0.5 0.5 0.01855634 0.5 0.00115177]]
The explicit design space definition uses the ExplicitArchDesignSpace class and a dedicated set of design variable definition classes.
Under the hood, it uses ConfigSpace, so all its powerful condition and forbidden-clause modeling functionalities can be used.
In [22]:
Copied!
from sb_arch_opt.design_space_explicit import *
class JenattonExplicit(ArchOptProblemBase):
def __init__(self):
# We define the hierarchical structure using the explicit design space class
# The design variables are defined by
ds = ExplicitArchDesignSpace([
CategoricalParam('x1', [0, 1]),
CategoricalParam('x2', [0, 1]),
CategoricalParam('x3', [0, 1]),
ContinuousParam('x4', 0, 1),
ContinuousParam('x5', 0, 1),
ContinuousParam('x6', 0, 1),
ContinuousParam('x7', 0, 1),
ContinuousParam('r8', 0, 1),
ContinuousParam('r9', 0, 1),
])
# Define activation conditions
ds.add_conditions([
# x2 and r8 are activated if x1 == 0
EqualsCondition(ds['x2'], ds['x1'], 0),
EqualsCondition(ds['r8'], ds['x1'], 0),
# x4 and x5 are additionally only activated if x2 == 0 or 1, respectively
EqualsCondition(ds['x4'], ds['x2'], 0),
EqualsCondition(ds['x5'], ds['x2'], 1),
# x1 == 1 activates x3, r9
EqualsCondition(ds['x3'], ds['x1'], 1),
EqualsCondition(ds['r9'], ds['x1'], 1),
# x6 and x7 are additionally only activated if x3 == 0 or 1, respectively
EqualsCondition(ds['x6'], ds['x3'], 0),
EqualsCondition(ds['x7'], ds['x3'], 1),
])
super().__init__(ds)
def _arch_evaluate(self, x: np.ndarray, is_active_out: np.ndarray, f_out: np.ndarray, g_out: np.ndarray,
h_out: np.ndarray, *args, **kwargs):
# No need to call the correction function, because the input is automatically
# corrected according to the design space definition
# Evaluate the design vectors
for i, xi in enumerate(x):
f_out[i, 0] = jenatton_evaluate(xi)
def __repr__(self):
return f'{self.__class__.__name__}()'
# Note how the problem definition is exactly the same as the implicit definition
JenattonExplicit().print_stats()
from sb_arch_opt.design_space_explicit import *
class JenattonExplicit(ArchOptProblemBase):
def __init__(self):
# We define the hierarchical structure using the explicit design space class
# The design variables are defined by
ds = ExplicitArchDesignSpace([
CategoricalParam('x1', [0, 1]),
CategoricalParam('x2', [0, 1]),
CategoricalParam('x3', [0, 1]),
ContinuousParam('x4', 0, 1),
ContinuousParam('x5', 0, 1),
ContinuousParam('x6', 0, 1),
ContinuousParam('x7', 0, 1),
ContinuousParam('r8', 0, 1),
ContinuousParam('r9', 0, 1),
])
# Define activation conditions
ds.add_conditions([
# x2 and r8 are activated if x1 == 0
EqualsCondition(ds['x2'], ds['x1'], 0),
EqualsCondition(ds['r8'], ds['x1'], 0),
# x4 and x5 are additionally only activated if x2 == 0 or 1, respectively
EqualsCondition(ds['x4'], ds['x2'], 0),
EqualsCondition(ds['x5'], ds['x2'], 1),
# x1 == 1 activates x3, r9
EqualsCondition(ds['x3'], ds['x1'], 1),
EqualsCondition(ds['r9'], ds['x1'], 1),
# x6 and x7 are additionally only activated if x3 == 0 or 1, respectively
EqualsCondition(ds['x6'], ds['x3'], 0),
EqualsCondition(ds['x7'], ds['x3'], 1),
])
super().__init__(ds)
def _arch_evaluate(self, x: np.ndarray, is_active_out: np.ndarray, f_out: np.ndarray, g_out: np.ndarray,
h_out: np.ndarray, *args, **kwargs):
# No need to call the correction function, because the input is automatically
# corrected according to the design space definition
# Evaluate the design vectors
for i, xi in enumerate(x):
f_out[i, 0] = jenatton_evaluate(xi)
def __repr__(self):
return f'{self.__class__.__name__}()'
# Note how the problem definition is exactly the same as the implicit definition
JenattonExplicit().print_stats()
problem: JenattonExplicit() n_discr: 3 n_cont : 6 n_obj : 1 n_con : 0 MD : True MO : False HIER : True n_valid_discr: 4 imp_ratio : 6.00 (discr.: 2.00; cont.: 3.00) corr_ratio : 3.00 (discr.: 1.00; cont.: 3.00; fraction of imp_ratio: 61.3%)
In [23]:
Copied!
result_expl = minimize(JenattonExplicit(), get_nsga2(pop_size=50), termination=('n_gen', 20), seed=42)
print(f'Optimum: {result_expl.opt.get("F")}; implicit: {result_impl.opt.get("F")}; difference = {result_expl.opt.get("F")-result_impl.opt.get("F")}')
print(f'At X = {result_expl.opt.get("X")}')
result_expl = minimize(JenattonExplicit(), get_nsga2(pop_size=50), termination=('n_gen', 20), seed=42)
print(f'Optimum: {result_expl.opt.get("F")}; implicit: {result_impl.opt.get("F")}; difference = {result_expl.opt.get("F")-result_impl.opt.get("F")}')
print(f'At X = {result_expl.opt.get("X")}')
Optimum: [[0.10115816]]; implicit: [[0.10115816]]; difference = [[0.]] At X = [[1. 0. 1. 0.5 0.5 0.5 0.01855634 0.5 0.00115177]]