Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

Commit 7ae6d28

Browse files
syzymoncopybara-github
authored andcommitted
Merge of PR #1895
PiperOrigin-RevId: 392071163
1 parent 874389b commit 7ae6d28

File tree

1 file changed

+6
-4
lines changed

1 file changed

+6
-4
lines changed

tensor2tensor/data_generators/enwik8.py

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -131,10 +131,12 @@ def generate_encoded_samples(self, data_dir, tmp_dir, dataset_split):
131131

132132
@registry.register_problem
133133
class Enwik8L2k(Enwik8L65k):
134-
"""Enwiki8, with examples up to 2048 characters long. Reads the input
135-
byte-wise and chunks it into fragments of maximum length of 2048. Does not
136-
shift byte indices (we do not assume cls or pad are used),
137-
unlike the base class!"""
134+
"""Enwiki8, with examples up to 2048 characters long.
135+
136+
Reads the input byte-wise and chunks it into fragments of maximum
137+
length of 2048. Does not shift byte indices (we do not assume cls or
138+
pad are used), unlike the base class!
139+
"""
138140

139141
READ_MODE = "rb"
140142

0 commit comments

Comments
 (0)