Deprecate Skopt Suggestion Service

[tenzen-y]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]
Unfortunately, the scikit-optimize was closed on February 29, 2024.

https://github.com/scikit-optimize/scikit-optimize

So we need to stop supporting the skopt suggestion service as well.
I think that we can follow similar deprecation steps as the MXNet.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

---

Love this feature? Give it a 👍 We prioritize the features with the most 👍

[tenzen-y]

@kubeflow/wg-automl-leads WDYT?

[PeterWrighten]

May I work for this issue?
/assign

[tenzen-y]

/unassign @PeterWrighten

We need to follow the deprecation flow step by step.

[PeterWrighten]

/unassign @PeterWrighten

We need to follow the deprecation flow step by step.

Got it. Thanks!

[andreyvelich]

That is sad to hear @tenzen-y.

Do we have any alternative to use Bayesian Optimization in Hyperopt or Optuna ?

[tenzen-y]

That is sad to hear @tenzen-y.

Do we have any alternative to use Bayesian Optimization in Hyperopt or Optuna ?

Unfortunately, both optuna and Hyperopt don't provide Bayesian sampler.

• https://optuna.readthedocs.io/en/stable/reference/samplers/index.html
• https://hyperopt.github.io/hyperopt/#algorithms

[tenzen-y]

If we want to provide Bayesian sample, we need to implement it ourselves.

[andreyvelich]

Maybe we could check some Open Source BO implementations.
For example: https://github.com/bayesian-optimization/BayesianOptimization

Also, quite recently @contramundum53 added GPSampler support in Optuna: optuna/optuna#5185

Maybe we can re-use it.
cc @c-bata @a9p for your ideas here.

[tenzen-y]

Also, quite recently @contramundum53 added GPSampler support in Optuna: optuna/optuna#5185

Oh, I didn't find that.

[tenzen-y]

So, setting optuna as a default Bayesian sampler might be better.

[contramundum53]

@andreyvelich @tenzen-y I'm glad that you are considering integration of Optuna's GPSampler into katib!
GPSampler is planned to be released in Optuna v3.6, our next minor release. It is still experimental and future breaking changes/bugs are possible (the code is only a month old!), but if it's okay for you, we'll be happy to hear that.