在Optuna研究期间动态添加/删除参数时会发生什么情况？

Question

在研究过程中动态调整参数范围时，Optuna的FAQ有一个clear answer：因为每个采样器都是单独定义的，所以没有问题

但是如何添加和/或删除参数呢？Optuna是否能够处理此类调整

在执行此操作时，我注意到的一点是，在结果数据框中，这些参数为其他试验获取nan项。如果能够将这些nan设置为它们在未采样时的（默认）值，会有什么好处吗？这项研究在所有这些未知值下仍然可靠吗

Answer 1

问题得到答复here：

Thanks for the question. Optuna internally supports two types of sampling: optuna.samplers.BaseSampler.sample_independent and optuna.samplers.BaseSampler.sample_relative.

The former optuna.samplers.BaseSampler.sample_independent is a method that samples independently on each parameter, and is not affected by the addition or removal of parameters. The added parameters are taken into account from the timing when they are added.

The latter optuna.samplers.BaseSampler.sample_relative is a method that samples by considering the correlation of parameters and is affected by the addition or removal of parameters. Optuna's default search space for correlation is the product set of the domains of the parameters that exist from the beginning of the hyperparameter tuning to the present. Developers who implement samplers can implement their own search space calculation method optuna.samplers.BaseSampler.infer_relative_search_space. This may allow correlations to be considered for hyperparameters that have been added or removed, but this depends on the sampling algorithm, so there is no API for normal users to modify.