Skip to content

ValueError: ngrams when loading a model with a TextVectorization layer #19180

@mykolaskrynnyk

Description

@mykolaskrynnyk

Describe a bug

Loading a model that contains a TextVectorization layer with ngram set to a tuple results in a ValueError.

Code to Reproduce

import numpy as np
import tensorflow as tf
from tensorflow import keras

texts = np.array(['foo bar', 'bar baz', 'baz bada boom'])

model = keras.Sequential([
    keras.layers.Input(dtype=tf.string, shape=(1,)),
    keras.layers.TextVectorization(ngrams=(1, 2)),
])

model.layers[0].adapt(texts)
model(texts)
<tf.Tensor: shape=(3, 5), dtype=int64, numpy=
array([[ 5,  3,  4,  0,  0],
       [ 3,  2,  8,  0,  0],
       [ 2, 10,  6,  7,  9]])>
model.save('model.keras')
model = tf.keras.models.load_model('model.keras')  # raises `ValueError`
ValueError: `ngrams` must be None, an integer, or a tuple of integers. Received: ngrams=[1, 2]

Expected Results

The model is loaded. No error is raised.

Actual Results

ValueError is raised.

Cause and Possible Solutions

The error is raised in __init__ method of TextVectorization class in text_vectorisation.py. Perhaps, checking if the ngram parameter is a list and, if so, coercing it to a tuple would be a viable solution in this case.

Versions

Python 3.11.4

tensorflow == 2.14.1
tensorflow-metal == 1.1.0

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions