If I try to use HyperNOMAD I have the following problem
caterina99@pc0425:/mnt/c/Users/calessi/Desktop/HyperNomad_linux/examples$ hypernomad.exe mnist_x0.txt
WARNING: hyperparameters explicitely set by name are superseded
by settings done using X0, LOWER_BOUND and UPPER_BOUND.
BLOCKS OF HYPERPARAMETERS
Each block has a head hyperparameters and possibly
several groups of associated hyperameters.
Convolutionnal layers {
Head of block NUM_CON_LAYERS -> x0=2, lb=0, ub=100, is VARIABLE
Multiple times associated hyperparameters: 2 groups
Group #0
NUM_OUTPUT_LAYERS -> x0=6, lb=1, ub=1000, is VARIABLE
KERNELS -> x0=5, lb=1, ub=20, is FIXED
STRIDES -> x0=1, lb=1, ub=3, is VARIABLE
PADDINGS -> x0=0, lb=0, ub=2, is VARIABLE
POOLING_SIZE -> x0=1, lb=1, ub=5, is VARIABLE
Group #1
NUM_OUTPUT_LAYERS -> x0=16, lb=1, ub=1000, is VARIABLE
KERNELS -> x0=5, lb=1, ub=20, is FIXED
STRIDES -> x0=1, lb=1, ub=3, is VARIABLE
PADDINGS -> x0=0, lb=0, ub=2, is VARIABLE
POOLING_SIZE -> x0=1, lb=1, ub=5, is VARIABLE
}
Full layers {
Head of block NUM_FC_LAYERS -> x0=2, lb=0, ub=500, is VARIABLE
Multiple times associated hyperparameters: 2 groups
Group #0
SIZE_FC_LAYER -> x0=128, lb=1, ub=1000, is VARIABLE
Group #1
SIZE_FC_LAYER -> x0=84, lb=1, ub=1000, is VARIABLE
}
Batch size {
Head of block BATCH_SIZE -> x0=128, lb=1, ub=400, is VARIABLE
No associated hyperparameters
}
Optimizer {
Head of block OPTIMIZER_CHOICE -> x0=3, lb=1, ub=4, is VARIABLE
One time associated hyperparameters (always 1 group)
Group #0
OPT_PARAM_1 -> x0=0.1, lb=0, ub=1, is VARIABLE
OPT_PARAM_2 -> x0=0.9, lb=0, ub=1, is VARIABLE
OPT_PARAM_3 -> x0=0.0005, lb=0, ub=1, is VARIABLE
OPT_PARAM_4 -> x0=0, lb=0, ub=1, is VARIABLE
}
Dropout rate {
Head of block DROPOUT_RATE -> x0=0.2, lb=0, ub=0.95, is FIXED
No associated hyperparameters
}
Activation function {
Head of block ACTIVATION_FUNCTION -> x0=1, lb=1, ub=3, is VARIABLE
No associated hyperparameters
}
Warning: {
Model use is disabled for problem with categorical variables.
}
Warning: {
Setting granularity different than 0 is disabled for problem with categorical variables.
}
Warning: {
Default anisotropic mesh is disabled with categorical and binary variables.
}
HyperNomad - version 1.0
Using Nomad version 3.9.1 - www.gerad.ca/nomad
Nomad parameters {
dimension : n=22
lower bounds : ( NaN 1 1 1 0 1 1 1 1 0 1 NaN 1 1 1 NaN 0 0 0 0 0 1 )
upper bounds : ( NaN 1000 20 3 2 5 1000 20 3 2 5 NaN 1000 1000 400 NaN 1 1 1 1 0.95 3 )
fixed variables : ( NaN NaN 5 NaN NaN NaN NaN 5 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 0.2 NaN )
blackbox input types : ( cat(C) int(I) int(I) int(I) int(I) int(I) int(I) int(I) int(I) int(I) int(I) cat(C) int(I) int(I) int(I) cat(C) cont(R) cont(R) cont(R) cont(R) cont(R) int(I) )
extended poll trigger: 10
variable groups {
group #0 {
indexes: { 14 }
directions {
n : 1
types : { [Ortho-MADS n+1 NEG] }
sec poll types: { }
int poll types: { }
seed : 0
}
}
group #1 {
indexes: { 21 }
directions {
n : 1
types : { [Ortho-MADS n+1 NEG] }
sec poll types: { }
int poll types: { }
seed : 0
}
}
group #2 {
indexes: { 12 13 }
directions {
n : 2
types : { [Ortho-MADS n+1 NEG] }
sec poll types: { }
int poll types: { }
seed : 0
}
}
group #3 {
indexes: { 0 11 15 }
no directions (categorical variables)
}
group #4 {
indexes: { 16 17 18 19 }
directions {
n : 4
types : { [Ortho-MADS n+1 NEG] }
sec poll types: { }
int poll types: { }
seed : 0
}
}
group #5 {
indexes: { 1 3 4 5 6 8 9 10 }
directions {
n : 8
types : { [Ortho-MADS n+1 NEG] }
sec poll types: { }
int poll types: { }
seed : 0
}
}
}
blackbox outputs (m=1) {
#0 OBJ $python /mnt/c/Users/calessi/Desktop/HyperNomad_linux/src/blackbox/pytorch_bb.py MNIST $python /mnt/c/Users/calessi/Desktop/HyperNomad_linux/src/blackbox/pytorch_sgte.py MNIST
}
signature : standard
has surrogate : yes
sort trial points with surrogate: yes
surrogate cost : none
sort trial points randomly : no
add seed to output file names : yes
solution file : none
history file : history.0.txt
stats file : (stats.0.txt) BBE ( SOL ) OBJ
cache file : none
x0 : ( 2 6 5 1 0 1 16 5 1 0 1 2 128 84 128 3 0.1 0.9 0.0005 0 0.2 1 )
directions : Ortho-MADS n+1 NEG
gmesh (isotropic) {
granularity : 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
coarsening exponent : 1
refining exponent : -1
initial mesh index : 0
initial mesh size : ( 1 10 1 1 1 1 10 1 1 1 1 1 10 10 10 1 1 1 1 1 1 1 )
initial poll size : ( 1 50 5 5 5 5 50 5 5 5 5 1 50 50 50 1 1 1 1 1 1 5 )
}
snap to bounds : yes
opportunistic evaluations : yes
use models (search and sort) : no
speculative search : yes
VNS search : no
NelderMead (NM) search {
max_trial_pts_nfactor: 80
gamma (shrink):0.5
delta_oc (outside contraction):0.5
delta_ic (inside contraction):-0.5
delta_e (expansion):2
intensive: no
opportunistic: no
use_only_Y: no
init_Y_best_von: no
use_short_Y0: no
include_factor: 8 }
Latin-Hypercube (LH) search : no
cache search : no
random seed / run id : 0
epsilon : 1e-13
undefined string : NaN
infinity string : inf
display degrees {
general : full (3)
search : full (3)
poll : full (3)
iterative: full (3)
}
display stats : BBE ( SOL ) OBJ
display all evaluations : no
point display limit : no limit
max number of blackbox eval. : 100
max cache memory : 2000 MB
}
MADS run {
starting point evaluation {
x0 eval point: ( 2 6 5 1 0 1 16 5 1 0 1 2 128 84 128 3 0.1 0.9 0.0005 0 0.2 1 )
list of points evaluation (x0 evaluation) {
submitted evaluation 1/1 {
point #0 ( 2 6 5 1 0 1 16 5 1 0 1 2 128 84 128 3 0.1 0.9 0.0005 0 0.2 1 )
mesh indices : ( 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 )
not in cache
}
sh: 1: python: not found
evaluation 1/1 {
point #0
x0 evaluation unsuccessful point #0: evaluation failed (you may need to check the source of the problem).
}
} end of evaluations
terminate MADS : yes
termination cause : problem with starting point evaluation
iteration status : unsuccessful
new feas. incumbent : none
new infeas. incumbent: none
} end of starting point evaluation
} end of run (problem with starting point evaluation)
NOMAD final display {
cache {
number of cache points: 1
size in memory : 619 B
cache file : -
}
stats {
MADS iterations : 0
blackbox evaluations : 1
simulated blackbox evaluations : 0
evaluations : 1
failed evaluations : 1 (all evaluations failed)
interrupted sequences of eval. : 0
cache hits : 0
number of poll searches : 0
dyn. direction successes : 0
number of speculative searches : 0
number of user searches : 0
number of LH searches : 0
number of NM searches : 0
number of TM line searches : 0
number of cache searches : 0
number of VNS searches : 0
no model has been constructed
wall-clock time : 0s
}
miscellaneous {
mesh indices : min= ( 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ), max = ( 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ), last= ( 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 )
best feasible solution : no feasible solution has been found
}
}
The starting point cannot be evaluated. Please verify that the Pytorch script is available and runs correctly. The default setting for bbExe is $python /mnt/c/Users/calessi/Desktop/HyperNomad_linux/src/blackbox/pytorch_bb.py MNIST. Make sure it works correctly on its own.
If I try to use HyperNOMAD I have the following problem
caterina99@pc0425:/mnt/c/Users/calessi/Desktop/HyperNomad_linux/examples$ hypernomad.exe mnist_x0.txt
WARNING: hyperparameters explicitely set by name are superseded
by settings done using X0, LOWER_BOUND and UPPER_BOUND.
BLOCKS OF HYPERPARAMETERS
Each block has a head hyperparameters and possibly
several groups of associated hyperameters.
Convolutionnal layers {
Head of block NUM_CON_LAYERS -> x0=2, lb=0, ub=100, is VARIABLE
Multiple times associated hyperparameters: 2 groups
Group #0
NUM_OUTPUT_LAYERS -> x0=6, lb=1, ub=1000, is VARIABLE
KERNELS -> x0=5, lb=1, ub=20, is FIXED
STRIDES -> x0=1, lb=1, ub=3, is VARIABLE
PADDINGS -> x0=0, lb=0, ub=2, is VARIABLE
POOLING_SIZE -> x0=1, lb=1, ub=5, is VARIABLE
Group #1
NUM_OUTPUT_LAYERS -> x0=16, lb=1, ub=1000, is VARIABLE
KERNELS -> x0=5, lb=1, ub=20, is FIXED
STRIDES -> x0=1, lb=1, ub=3, is VARIABLE
PADDINGS -> x0=0, lb=0, ub=2, is VARIABLE
POOLING_SIZE -> x0=1, lb=1, ub=5, is VARIABLE
}
Full layers {
Head of block NUM_FC_LAYERS -> x0=2, lb=0, ub=500, is VARIABLE
Multiple times associated hyperparameters: 2 groups
Group #0
SIZE_FC_LAYER -> x0=128, lb=1, ub=1000, is VARIABLE
Group #1
SIZE_FC_LAYER -> x0=84, lb=1, ub=1000, is VARIABLE
}
Batch size {
Head of block BATCH_SIZE -> x0=128, lb=1, ub=400, is VARIABLE
No associated hyperparameters
}
Optimizer {
Head of block OPTIMIZER_CHOICE -> x0=3, lb=1, ub=4, is VARIABLE
One time associated hyperparameters (always 1 group)
Group #0
OPT_PARAM_1 -> x0=0.1, lb=0, ub=1, is VARIABLE
OPT_PARAM_2 -> x0=0.9, lb=0, ub=1, is VARIABLE
OPT_PARAM_3 -> x0=0.0005, lb=0, ub=1, is VARIABLE
OPT_PARAM_4 -> x0=0, lb=0, ub=1, is VARIABLE
}
Dropout rate {
Head of block DROPOUT_RATE -> x0=0.2, lb=0, ub=0.95, is FIXED
No associated hyperparameters
}
Activation function {
Head of block ACTIVATION_FUNCTION -> x0=1, lb=1, ub=3, is VARIABLE
No associated hyperparameters
}
Warning: {
Model use is disabled for problem with categorical variables.
}
Warning: {
Setting granularity different than 0 is disabled for problem with categorical variables.
}
Warning: {
Default anisotropic mesh is disabled with categorical and binary variables.
}
HyperNomad - version 1.0
Using Nomad version 3.9.1 - www.gerad.ca/nomad
Nomad parameters {
}
MADS run {
sh: 1: python: not found
} end of run (problem with starting point evaluation)
NOMAD final display {
}
The starting point cannot be evaluated. Please verify that the Pytorch script is available and runs correctly. The default setting for bbExe is $python /mnt/c/Users/calessi/Desktop/HyperNomad_linux/src/blackbox/pytorch_bb.py MNIST. Make sure it works correctly on its own.