5 Best Ways to Build Ragged Tensor from List of Words Using TensorFlow and Python

by

💡 Problem Formulation: When working with natural language data, developers often encounter lists of words where each list can have a varying number of elements. The challenge is to transform this data into a format suitable for machine learning models. For example, given a list of sentences [“TensorFlow shines”, “Python is fun”, “Ragged tensors are useful”], the goal is to convert this irregularly-shaped data into a ragged tensor, where each inner list of words corresponds to a different sentence, accommodating sentences of variable lengths.

Method 1: Direct RaggedTensor Construction

Using TensorFlow’s RaggedTensor class, developers can directly construct a ragged tensor from a nested list or sequence of words. The RaggedTensor.from_nested_row_splits() or RaggedTensor.from_row_lengths() functions handle variable-length sequences, making them ideal for irregularly-shaped data such as sentences with different word counts.

Here’s an example:

import tensorflow as tf

list_of_words = [['TensorFlow', 'shines'], ['Python', 'is', 'fun'], ['Ragged', 'tensors', 'are', 'useful']]
ragged_tensor = tf.ragged.constant(list_of_words)

print(ragged_tensor)

Output:

<tf.RaggedTensor [[b'TensorFlow', b'shines'], [b'Python', b'is', b'fun'], [b'Ragged', b'tensors', b'are', b'useful']]>

This code snippet converts the list_of_words into a ragged tensor using the tf.ragged.constant() function which is a convenient and straightforward method for ragged tensor creation in TensorFlow.

Method 2: Using tf.strings.split

For a list of sentences, another approach is to use tf.strings.split() which splits strings in Tensor or RaggedTensor by a delimiter and returns a ragged tensor. This is useful for raw string input where sentences are separated by spaces or another delimiter and need to be split into words.

Here’s an example:

import tensorflow as tf

sentences = ['TensorFlow shines', 'Python is fun', 'Ragged tensors are useful']
tensor_of_sentences = tf.constant(sentences)
ragged_tensor = tf.strings.split(tensor_of_sentences, sep=' ')

print(ragged_tensor)

Output:

<tf.RaggedTensor [[b'TensorFlow', b'shines'], [b'Python', b'is', b'fun'], [b'Ragged', b'tensors', b'are', b'useful']]>

The tf.strings.split() function effectively splits each sentence into words, and the resulting words form the elements of the inner lists in the ragged tensor, preserving the varying sentence lengths.

Method 3: Using padding and tf.RaggedTensor.from_tensor

If the list of words is initially in the form of a padded tensor, tf.RaggedTensor.from_tensor() along with tf.boolean_mask() can convert it to a ragged tensor. This method involves masking out the padding values to revert a padded representation back to ragged format.

Here’s an example:

import tensorflow as tf

# Padded tensor of words (0 is used as padding value)
padded_tensor = tf.constant([['TensorFlow', 'shines', '', ''],
                             ['Python', 'is', 'fun', ''],
                             ['Ragged', 'tensors', 'are', 'useful']])
mask = tf.strings.not_equal(padded_tensor, '')
ragged_tensor = tf.RaggedTensor.from_tensor(tensor=padded_tensor, padding='')

print(ragged_tensor)

Output:

<tf.RaggedTensor [[b'TensorFlow', b'shines'], [b'Python', b'is', b'fun'], [b'Ragged', b'tensors', b'are', b'useful']]>

This code demonstrates conversion from a padded tensor to a ragged tensor by first creating a boolean mask that identifies non-padding elements and then applies this mask to obtain the ragged tensor.

Method 4: Building Ragged Tensor from String Tensors Using tf.data

The tf.data.Dataset API allows building complex input pipelines from simple, reusable pieces. By converting a list of string tensors to a dataset and then using batch and map transformations along with tf.RaggedTensor.from_tensor(), one can achieve a ragged tensor representation.

Here’s an example:

import tensorflow as tf

sentences = tf.data.Dataset.from_tensor_slices(['TensorFlow shines', 'Python is fun', 'Ragged tensors are useful'])
ragged_tensor_ds = sentences.map(tf.strings.split).batch(1)

for rt in ragged_tensor_ds.take(3):
    print(rt)

Output:

<tf.RaggedTensor [[[b'TensorFlow', b'shines']]]>
<tf.RaggedTensor [[[b'Python', b'is', b'fun']]]>
<tf.RaggedTensor [[[b'Ragged', b'tensors', b'are', b'useful']]]>

The dataset consisting of sentences is split into words using a map transformation and batched accordingly. The take(3) methods allow for inspection of the first three ragged tensors.

Bonus One-Liner Method 5: List Comprehension with tf.constant

This one-liner leverages Python’s list comprehension feature, combined with tf.constant(), to generate a ragged tensor. The list comprehension handles the word splitting, and tf.constant() handles the tensor construction.

Here’s an example:

import tensorflow as tf

sentences = ['TensorFlow shines', 'Python is fun', 'Ragged tensors are useful']
ragged_tensor = tf.ragged.constant([sentence.split(' ') for sentence in sentences])

print(ragged_tensor)

Output:

<tf.RaggedTensor [[b'TensorFlow', b'shines'], [b'Python', b'is', b'fun'], [b'Ragged', b'tensors', b'are', b'useful']]>

This concise snippet demonstrates the use of a list comprehension for splitting each sentence into words, and tf.ragged.constant() function then converts this list into a ragged tensor, which TensorFlow can easily work with.

Summary/Discussion

  • Method 1: Direct RaggedTensor Construction. Straightforward and efficient for nested lists. Less flexible for raw string processing.
  • Method 2: Using tf.strings.split. Ideal for splitting strings into tokens. Requires additional steps if starting with lists.
  • Method 3: From Padding to Ragged Tensor. Useful to revert padded tensors. Overhead of creating and removing padding.
  • Method 4: Using tf.data. Best for large datasets and complex pipelines. Complexity increases with pipeline customization.
  • Method 5: One-Liner with List Comprehension. Quick and elegant for simple use cases. May not handle more complex situations efficiently.