Skip to content

writer

This section contains the documentation of movinets_helper\writer.py.

Helper to write mp4 videos to TFRecordDataset.

convert_mp4_to_tfrecord(dataset_df, target_path, n_videos_in_record=15, n_frames_per_video=10, resolution=224)

Saves videos in mp4 format to tfrecords.

It creates a new dataset of homogeneus videos, all with shape: (n_frames, resolution, resolution, channels=3), along with its label and shape features to be loaded back.

Parameters:

Name Type Description Default
dataset_df pd.DataFrame

Dataframe with 2 columns: classes, representing the labels, and files, with the path to the videos.

 required
target_path Path

Path where the files will be generated. It must be already generated.

 required
n_videos_in_record int

Number of videos per file. See the notes in https://www.tensorflow.org/tutorials/load_data/tfrecord Defaults to 10.

 15
n_frames_per_video int

Number of frames to select per video. Defaults to 10.

 10
resolution int

Try to keep to as low as possible to save space and computing time during writing. This can be updated later. Defaults to 224.

 224
Example

Generate a dataset of tf records

convert_mp4_to_tfrecord(dataset_df[["classes", "files"]], target_path)

Split into train/test

convert_mp4_to_tfrecord(train_dataset_df[["classes", "files"]], target_path_train) convert_mp4_to_tfrecord(test_dataset_df[["classes", "files"]], target_path_test)

Note

This data format isn't storage friendly, even after compression.

Source code in movinets_helper/writer.py 
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
def convert_mp4_to_tfrecord(
    dataset_df: pd.DataFrame,
    target_path: Path,
    n_videos_in_record: int = 15,
    n_frames_per_video: int = 10,
    resolution: int = 224,
):
    r"""Saves videos in mp4 format to tfrecords.

    It creates a new dataset of homogeneus videos, all with shape:
    (n_frames, resolution, resolution, channels=3), along
    with its label and shape features to be loaded back.

    Args:
        dataset_df (pd.DataFrame):
            Dataframe with 2 columns: classes, representing the labels, and files,
            with the path to the videos.
        target_path (Path):
            Path where the files will be generated.
            It must be already generated.
        n_videos_in_record (int, optional):
            Number of videos per file.
            See the notes in https://www.tensorflow.org/tutorials/load_data/tfrecord
            Defaults to 10.
        n_frames_per_video (int, optional):
            Number of frames to select per video.
            Defaults to 10.
        resolution (int, optional):
            Try to keep to as low as possible to save space and computing time during
            writing. This can be updated later.
            Defaults to 224.

    Example:

        Generate a dataset of tf records

        >>> convert_mp4_to_tfrecord(dataset_df[["classes", "files"]], target_path)

        Split into train/test

        >>> convert_mp4_to_tfrecord(train_dataset_df[["classes", "files"]], target_path_train)
        >>> convert_mp4_to_tfrecord(test_dataset_df[["classes", "files"]], target_path_test)

    Note:
        This data format isn't storage friendly, even after compression.
    """
    if not Path(target_path).is_dir():
        print(f"{target_path} doesn't exist, create it first.")
        return

    assert dataset_df.columns.tolist() == [
        "classes",
        "files",
    ], "dataset_df doesn't have the expected columns"
    files = dataset_df.to_records(index=False)  # List of tuples (label, filename)
    filenames_split = list(get_chunks(files, n_videos_in_record))

    total_batches = len(filenames_split)

    for i, batch in enumerate(tqdm.tqdm(filenames_split)):
        tfrecord_filename = str(
            Path(target_path) / f"video_batch_{i}_{total_batches}.tfrecords"
        )
        with tf.io.TFRecordWriter(
            tfrecord_filename, options=tf.io.TFRecordOptions(compression_type="GZIP")
        ) as file_writer:
            for label, file in batch:
                try:
                    video = load_video_tf(file)
                except tf.errors.InvalidArgumentError:
                    print(f"Video without length, passing")
                    continue
                video = tf.image.resize(video, (resolution, resolution))
                frame_count = get_frame_count(Path(file))
                video = _select_frames(video, frame_count, n_frames_per_video)
                try:
                    file_writer.write(serialize_example(label, video))
                except Exception as exc:
                    print(f"Couldn't serialize file: {file}")
                    print(f"err: {exc}")

serialize_example(label, video)

Creates a tf.train.Example message ready to be written to a file.

The output can be sent to a TFRecordWriter to be written.

Parameters:

Name Type Description Default
label tf.Tensor

int encoded label.

 required
video tf.Tensor

video as a tensor, maybe loaded using load_video_tf.

 required

Returns:

Name Type Description
str str

representation of the features to be stored as .tfrecords.
Note

Depending on the size (i.e., videos), jupyter won't be able to show them, a javascript error will raise.

Source code in movinets_helper/writer.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
def serialize_example(label: tf.Tensor, video: tf.Tensor) -> str:
    """Creates a tf.train.Example message ready to be written to a file.

    The output can be sent to a TFRecordWriter to be written.

    Args:
        label (tf.Tensor): int encoded label.
        video (tf.Tensor): video as a tensor, maybe loaded using load_video_tf.

    Returns:
        str: representation of the features to be stored as .tfrecords.

    Note:
        Depending on the size (i.e., videos), jupyter won't be able to show them,
        a javascript error will raise.
    """
    # dict mapping the feature name to the tf.train.Example-compatible
    # data type.
    features = {
        "label": _int64_feature(label),
        "n_frames": _int64_feature(video.shape[0]),
        "width": _int64_feature(video.shape[1]),
        "height": _int64_feature(video.shape[2]),
        "n_channels": _int64_feature(video.shape[3]),
        "video": _bytes_feature(tf.io.serialize_tensor(video.numpy())),
    }
    # Create a Features message using tf.train.Example.

    example_proto = tf.train.Example(features=tf.train.Features(feature=features))
    return example_proto.SerializeToString()