Skip to content

Fix legacy optimizer handling in compile_from_config(). #18492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 25, 2023
Merged

Fix legacy optimizer handling in compile_from_config(). #18492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dfeaf41
Fix legacy optimizer handling in `compile_from_config()`.
nkovela1 Sep 25, 2023
4c87b8b
Add associated test
nkovela1 Sep 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion keras/engine/training.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3680,7 +3680,12 @@ def compile_from_config(self, config):
return
config = saving_lib.deserialize_keras_object(config)
self.compile(**config)
if hasattr(self, "optimizer") and self.built:
if (
hasattr(self, "optimizer")
# Exempt legacy optimizers.
and isinstance(self.optimizer, optimizer.Optimizer)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it every possible to pass an optimizer that isn't an Optimizer instance? Wouldn't that blow up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All custom optimizers that need to be built on deserialization must subclass from Optimizer afaik, and legacy optimizers (which do not) are exempted here because they do not have a corresponding build method.

Is there any edge case in particular you are referring to? The corresponding CL did not cause any TGP issues.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legacy optimizers simply don't exist in Keras 3, do they?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is actually for the Keras 2.15 release, so legacy optimizers still exist in this revision actually (merging into the cherrypick branch).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, I had not noticed this was in the 2.15 branch.

and self.built
):
# Create optimizer variables.
self.optimizer.build(self.trainable_variables)

Expand Down
28 changes: 28 additions & 0 deletions keras/optimizers/legacy/optimizer_v2_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"""Functional test for OptimizerV2."""

import collections
import os
from copy import deepcopy

import numpy as np
Expand Down Expand Up @@ -568,6 +569,33 @@ def testOptimizerWithKerasModel(self):
batch_size=5,
)

@test_combinations.generate(test_combinations.combine(mode=["eager"]))
def testOptimizerSaving(self):
np.random.seed(1331)
input_np = np.random.random((10, 3))
output_np = np.random.random((10, 4))
a = input_layer.Input(shape=(3,), name="input_a")
model = sequential.Sequential()
model.add(core.Dense(4, kernel_initializer="zeros", name="dense"))
model.add(regularization.Dropout(0.5, name="dropout"))
model(a)
optimizer = gradient_descent.SGD(learning_rate=0.1)
model.compile(optimizer, loss="mse", metrics=["mae"])

model.fit(
input_np,
output_np,
batch_size=10,
validation_data=(input_np, output_np),
epochs=2,
verbose=0,
)

temp_filepath = os.path.join(self.get_temp_dir(), "optv2_model.keras")
model.save(temp_filepath)
loaded_model = keras.models.load_model(temp_filepath)
self.assertAllClose(model(input_np), loaded_model(input_np), atol=1e-6)

@test_combinations.generate(
test_combinations.combine(mode=["graph", "eager"])
)
Expand Down