Basic usage¶

SciKeras is designed to maximize interoperability between sklearn and Keras/TensorFlow. The aim is to keep 99% of the flexibility of Keras while being able to leverage most features of sklearn. Below, we show the basic usage of SciKeras and how it can be combined with sklearn.

This notebook shows you how to use the basic functionality of SciKeras.

Table of contents¶

1. Setup
2. Training a classifier and making predictions
- 2.1 A toy binary classification task
- 2.2 Definition of the Keras classification Model
- 2.3 Defining and training the neural net classifier
- 2.4 Making predictions, classification
3 Training a regressor
- 3.1 A toy regression task
- 3.2 Definition of the Keras regression Model
- 3.3 Defining and training the neural net regressor
- 3.4 Making predictions, regression
4. Saving and loading a model
- 4.1 Saving the whole model
- 4.2 Saving using Keras’ saving methods
5. Usage with an sklearn Pipeline
6. Callbacks
7. Usage with sklearn GridSearchCV
- 7.1 Special prefixes
- 7.2 Performing a grid search

1. Setup¶

[1]:

try:
    import scikeras
except ImportError:
    !python -m pip install scikeras

Silence TensorFlow logging to keep output succinct.

[2]:

import warnings
from tensorflow import get_logger
get_logger().setLevel('ERROR')
warnings.filterwarnings("ignore", message="Setting the random state for TF")

[3]:

import numpy as np
from scikeras.wrappers import KerasClassifier, KerasRegressor
import keras

2. Training a classifier and making predictions¶

2.1 A toy binary classification task¶

We load a toy classification task from sklearn.

[4]:

import numpy as np
from sklearn.datasets import make_classification

X, y = make_classification(1000, 20, n_informative=10, random_state=0)

X.shape, y.shape, y.mean()

[4]:

((1000, 20), (1000,), 0.5)

2.2 Definition of the Keras classification Model¶

We define a vanilla neural network with.

Because we are dealing with 2 classes, the output layer can be constructed in two different ways:

Single unit with a "sigmoid" nonlinearity. The loss must be "binary_crossentropy".
Two units (one for each class) and a "softmax" nonlinearity. The loss must be "sparse_categorical_crossentropy".

In this example, we choose the first option, which is what you would usually do for binary classification. The second option is usually reserved for when you have >2 classes.

[5]:

import keras


def get_clf(meta, hidden_layer_sizes, dropout):
    n_features_in_ = meta["n_features_in_"]
    n_classes_ = meta["n_classes_"]
    model = keras.models.Sequential()
    model.add(keras.layers.Input(shape=(n_features_in_,)))
    for hidden_layer_size in hidden_layer_sizes:
        model.add(keras.layers.Dense(hidden_layer_size, activation="relu"))
        model.add(keras.layers.Dropout(dropout))
    model.add(keras.layers.Dense(1, activation="sigmoid"))
    return model

2.3 Defining and training the neural net classifier¶

We use KerasClassifier because we’re dealing with a classifcation task. The first argument should be a callable returning a Keras.Model, in this case, get_clf. As additional arguments, we pass the number of loss function (required) and the optimizer, but the later is optional. We must also pass all of the arguments to get_clf as keyword arguments to KerasClassifier if they don’t have a default value in get_clf. Note that if you do not pass an argument to KerasClassifier, it will not be avilable for hyperparameter tuning. Finally, we also pass random_state=0 for reproducible results.

[6]:

from scikeras.wrappers import KerasClassifier


clf = KerasClassifier(
    model=get_clf,
    loss="binary_crossentropy",
    hidden_layer_sizes=(100,),
    dropout=0.5,
)

As in sklearn, we call fit passing the input data X and the targets y.

[7]:

clf.fit(X, y);

 1/32 ━━━━━━━━━━━━━━━━━━━━ 14s 479ms/step - loss: 0.6073: </pre>
textbf{ 1/32} textcolor{ansi-white}{━━━━━━━━━━━━━━━━━━━━} textbf{14s} 479ms/step - loss: 0.6073: end{sphinxVerbatim}

[1m 1/32[0m [37m━━━━━━━━━━━━━━━━━━━━[0m [1m14s[0m 479ms/step - loss: 0.6073

32/32 ━━━━━━━━━━━━━━━━━━━━ 1s 901us/step - loss: 0.6832: </pre>
textbf{32/32} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{1s} 901us/step - loss: 0.6832: end{sphinxVerbatim}

[1m32/32[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m1s[0m 901us/step - loss: 0.6832

Also, as in sklearn, you may call predict or predict_proba on the fitted model.

2.4 Making predictions, classification¶

[8]:

y_pred = clf.predict(X[:5])
y_pred

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 28ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 28ms/step

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 28ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 28ms/step

[8]:

array([1, 0, 0, 0, 0])

[9]:

y_proba = clf.predict_proba(X[:5])
y_proba

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 11ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 11ms/step

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 12ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 12ms/step

[9]:

array([[0.41824633, 0.5817537 ],
       [0.83141077, 0.1685892 ],
       [0.82014996, 0.17985004],
       [0.9229777 , 0.07702232],
       [0.8843903 , 0.11560971]], dtype=float32)

3 Training a regressor¶

3.1 A toy regression task¶

[10]:

from sklearn.datasets import make_regression

X_regr, y_regr = make_regression(1000, 20, n_informative=10, random_state=0)

X_regr.shape, y_regr.shape, y_regr.min(), y_regr.max()

[10]:

((1000, 20), (1000,), -649.0148244404172, 615.4505181286091)

3.2 Definition of the Keras regression Model¶

Again, define a vanilla neural network. The main difference is that the output layer always has a single unit and does not apply any nonlinearity.

[11]:

def get_reg(meta, hidden_layer_sizes, dropout):
    n_features_in_ = meta["n_features_in_"]
    model = keras.models.Sequential()
    model.add(keras.layers.Input(shape=(n_features_in_,)))
    for hidden_layer_size in hidden_layer_sizes:
        model.add(keras.layers.Dense(hidden_layer_size, activation="relu"))
        model.add(keras.layers.Dropout(dropout))
    model.add(keras.layers.Dense(1))
    return model

3.3 Defining and training the neural net regressor¶

Training a regressor has nearly the same data flow as training a classifier. The differences include using KerasRegressor instead of KerasClassifier and adding keras.metrics.R2Score as a metric. Most of the Scikit-learn regressors use the coefficient of determination or R^2 as a metric function, which measures correlation between the true labels and predicted labels.

[12]:

import keras
import keras.models
from scikeras.wrappers import KerasRegressor


reg = KerasRegressor(
    model=get_reg,
    loss="mse",
    metrics=[keras.metrics.R2Score],
    hidden_layer_sizes=(100,),
    dropout=0.5,
)

[13]:

reg.fit(X_regr, y_regr);

 1/32 ━━━━━━━━━━━━━━━━━━━━ 17s 580ms/step - loss: 39734.1211 - r2_score: -0.0590: </pre>
textbf{ 1/32} textcolor{ansi-white}{━━━━━━━━━━━━━━━━━━━━} textbf{17s} 580ms/step - loss: 39734.1211 - r2_score: -0.0590: end{sphinxVerbatim}

[1m 1/32[0m [37m━━━━━━━━━━━━━━━━━━━━[0m [1m17s[0m 580ms/step - loss: 39734.1211 - r2_score: -0.0590

32/32 ━━━━━━━━━━━━━━━━━━━━ 1s 907us/step - loss: 42994.3867 - r2_score: -3.3056e-04: </pre>
textbf{32/32} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{1s} 907us/step - loss: 42994.3867 - r2_score: -3.3056e-04: end{sphinxVerbatim}

[1m32/32[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m1s[0m 907us/step - loss: 42994.3867 - r2_score: -3.3056e-04

3.4 Making predictions, regression¶

You may call predict or predict_proba on the fitted model. For regressions, both methods return the same value.

[14]:

y_pred = reg.predict(X_regr[:5])
y_pred

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 29ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 29ms/step

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 29ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 29ms/step

[14]:

array([ 0.38163126, -0.36355966,  1.3995478 ,  0.07941712, -0.1637274 ],
      dtype=float32)

4. Saving and loading a model¶

Save and load either the whole model by using pickle, or use Keras’ specialized save methods on the KerasClassifier.model_ or KerasRegressor.model_ attribute that is created after fitting. You will want to use Keras’ model saving utilities if any of the following apply:

You wish to save only the weights or only the training configuration of your model.
You wish to share your model with collaborators. Pickle is a relatively unsafe protocol and it is not recommended to share or load pickle objects publically.
You care about performance, especially if doing in-memory serialization.

For more information, see Keras’ saving documentation.

4.1 Saving the whole model¶

[15]:

import pickle


bytes_model = pickle.dumps(reg)
new_reg = pickle.loads(bytes_model)
new_reg.predict(X_regr[:5])  # model is still trained

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 34ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 34ms/step

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step: </pre>
textbf{1/1} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 34ms/step: end{sphinxVerbatim}

[1m1/1[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 34ms/step

[15]:

array([ 0.38163126, -0.36355966,  1.3995478 ,  0.07941712, -0.1637274 ],
      dtype=float32)

4.2 Saving using Keras’ saving methods¶

This efficiently and safely saves the model to disk, including trained weights. You should use this method if you plan on sharing your saved models.

[16]:

# Save to disk
pred_old = reg.predict(X_regr)
reg.model_.save("/tmp/my_model.keras")  # saves just the Keras model

 1/32 ━━━━━━━━━━━━━━━━━━━━ 1s 39ms/step: </pre>
textbf{ 1/32} textcolor{ansi-white}{━━━━━━━━━━━━━━━━━━━━} textbf{1s} 39ms/step: end{sphinxVerbatim}

[1m 1/32[0m [37m━━━━━━━━━━━━━━━━━━━━[0m [1m1s[0m 39ms/step

32/32 ━━━━━━━━━━━━━━━━━━━━ 0s 650us/step: </pre>
textbf{32/32} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 650us/step: end{sphinxVerbatim}

[1m32/32[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 650us/step

[17]:

# Load the model back into memory
new_reg_model = keras.saving.load_model("/tmp/my_model.keras")
# Now we need to instantiate a new SciKeras object
# since we only saved the Keras model
reg_new = KerasRegressor(new_reg_model)
# use initialize to avoid re-fitting
reg_new.initialize(X_regr, y_regr)
pred_new = reg_new.predict(X_regr)
np.testing.assert_allclose(pred_old, pred_new)

 1/32 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step: </pre>
textbf{ 1/32} textcolor{ansi-white}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 27ms/step: end{sphinxVerbatim}

[1m 1/32[0m [37m━━━━━━━━━━━━━━━━━━━━[0m [1m0s[0m 27ms/step

32/32 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step: </pre>
textbf{32/32} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 1ms/step: end{sphinxVerbatim}

[1m32/32[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 1ms/step

5. Usage with an sklearn Pipeline¶

It is possible to put the KerasClassifier inside an sklearn Pipeline, as you would with any sklearn classifier.

[18]:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


pipe = Pipeline([
    ('scale', StandardScaler()),
    ('clf', clf),
])


y_proba = pipe.fit(X, y).predict(X)

 1/32 ━━━━━━━━━━━━━━━━━━━━ 17s 570ms/step - loss: 0.7106: </pre>
textbf{ 1/32} textcolor{ansi-white}{━━━━━━━━━━━━━━━━━━━━} textbf{17s} 570ms/step - loss: 0.7106: end{sphinxVerbatim}

[1m 1/32[0m [37m━━━━━━━━━━━━━━━━━━━━[0m [1m17s[0m 570ms/step - loss: 0.7106

32/32 ━━━━━━━━━━━━━━━━━━━━ 1s 816us/step - loss: 0.7350: </pre>
textbf{32/32} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{1s} 816us/step - loss: 0.7350: end{sphinxVerbatim}

[1m32/32[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m1s[0m 816us/step - loss: 0.7350

 1/32 ━━━━━━━━━━━━━━━━━━━━ 0s 27ms/step: </pre>
textbf{ 1/32} textcolor{ansi-white}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 27ms/step: end{sphinxVerbatim}

[1m 1/32[0m [37m━━━━━━━━━━━━━━━━━━━━[0m [1m0s[0m 27ms/step

32/32 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step: </pre>
textbf{32/32} textcolor{ansi-green}{━━━━━━━━━━━━━━━━━━━━} textbf{0s} 1ms/step: end{sphinxVerbatim}

[1m32/32[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m0s[0m 1ms/step

To save the whole pipeline, including the Keras model, use pickle.

6. Callbacks¶

Adding a new callback to the model is straightforward. Below we define a threashold callback to avoid training past a certain accuracy. This a rudimentary for of early stopping.

[19]:

class MaxValLoss(keras.callbacks.Callback):

    def __init__(self, monitor: str, threashold: float):
        self.monitor = monitor
        self.threashold = threashold

    def on_epoch_end(self, epoch, logs=None):
        if logs[self.monitor] > self.threashold:
            print("Threashold reached; stopping training")
            self.model.stop_training = True

Define a test dataset:

[20]:

from sklearn.datasets import make_moons

X, y = make_moons(n_samples=100, noise=0.2, random_state=0)

And try fitting it with and without the callback:

[21]:

kwargs = dict(
    model=get_clf,
    loss="binary_crossentropy",
    dropout=0.5,
    hidden_layer_sizes=(100,),
    metrics=["binary_accuracy"],
    fit__validation_split=0.2,
    epochs=20,
    verbose=False,
    random_state=0
)

# First test without the callback
clf = KerasClassifier(**kwargs)
clf.fit(X, y)
print(f"Trained {len(clf.history_['loss'])} epochs")
print(f"Final accuracy: {clf.history_['val_binary_accuracy'][-1]}")  # get last value of last fit/partial_fit call

Trained 20 epochs
Final accuracy: 0.8999999761581421

And with:

[22]:

# Test with the callback

cb = MaxValLoss(monitor="val_binary_accuracy", threashold=0.75)

clf = KerasClassifier(
    **kwargs,
    callbacks=[cb]
)
clf.fit(X, y)
print(f"Trained {len(clf.history_['loss'])} epochs")
print(f"Final accuracy: {clf.history_['val_binary_accuracy'][-1]}")  # get last value of last fit/partial_fit call

Threashold reached; stopping training
Trained 4 epochs
Final accuracy: 0.8999999761581421

For information on how to write custom callbacks, have a look at the Advanced Usage notebook.

7. Usage with sklearn GridSearchCV¶

7.1 Special prefixes¶

SciKeras allows to direct access to all parameters passed to the wrapper constructors, including deeply nested routed parameters. This allows tunning of paramters like hidden_layer_sizes as well as optimizer__learning_rate.

This is exactly the same logic that allows to access estimator parameters in sklearn Pipelines and FeatureUnions.

This feature is useful in several ways. For one, it allows to set those parameters in the model definition. Furthermore, it allows you to set parameters in an sklearn GridSearchCV as shown below.

To differentiate paramters like callbacks which are accepted by both keras.Model.fit and keras.Model.predict you can add a fit__ or predict__ routing suffix respectively. Similar, the model__ prefix may be used to specify that a paramter is destined only for get_clf/get_reg (or whatever callable you pass as your model argument).

For more information on parameter routing with special prefixes, see the Advanced Usage Docs

7.2 Performing a grid search¶

Below we show how to perform a grid search over the learning rate (optimizer__learning_rate), the model’s number of hidden layers (model__hidden_layer_sizes), the model’s dropout rate (model__dropout).

[23]:

from sklearn.model_selection import GridSearchCV


clf = KerasClassifier(
    model=get_clf,
    loss="binary_crossentropy",
    optimizer="adam",
    optimizer__learning_rate=0.1,
    model__hidden_layer_sizes=(100,),
    model__dropout=0.5,
    verbose=False,
)

Note: We set the verbosity level to zero (verbose=False) to prevent too much print output from being shown.

[24]:

params = {
    'optimizer__learning_rate': [0.05, 0.1],
    'model__hidden_layer_sizes': [(100, ), (50, 50, )],
    'model__dropout': [0, 0.5],
}

gs = GridSearchCV(clf, params, scoring='accuracy', n_jobs=-1, verbose=True)

gs.fit(X, y)

print(gs.best_score_, gs.best_params_)

Fitting 5 folds for each of 8 candidates, totalling 40 fits

WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7faf09385080> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 5 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7faf09233e20> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7f940a505080> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 5 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f940a5afe20> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7f5bbcc6bf60> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 5 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f5bbcb43240> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7faf48ffbf60> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 5 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7faf490d3240> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7faf09163ec0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:6 out of the last 6 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7faf090407c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7f940a4e3ec0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:6 out of the last 6 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f940a3c47c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7f5bbccaf7e0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:6 out of the last 6 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7f5bbc990720> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 13 calls to <function TensorFlowTrainer.make_train_function.<locals>.one_step_on_iterator at 0x7faf4903f7e0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

WARNING:tensorflow:6 out of the last 6 calls to <function TensorFlowTrainer.make_predict_function.<locals>.one_step_on_data_distributed at 0x7faf48d20720> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

0.86 {'model__dropout': 0, 'model__hidden_layer_sizes': (100,), 'optimizer__learning_rate': 0.1}

Of course, we could further nest the KerasClassifier within an sklearn.pipeline.Pipeline, in which case we just prefix the parameter by the name of the net (e.g. clf__model__hidden_layer_sizes).