Skip to content

Commit b74a955

Browse files
authored
fix rum_clm.py seeking text column name twice (#16624)
1 parent 3663fca commit b74a955

File tree

1 file changed

+0
-4
lines changed

1 file changed

+0
-4
lines changed

examples/tensorflow/language-modeling/run_clm.py

-4
Original file line numberDiff line numberDiff line change
@@ -347,10 +347,6 @@ def main():
347347
column_names = raw_datasets["train"].column_names
348348
text_column_name = "text" if "text" in column_names else column_names[0]
349349

350-
# First we tokenize all the texts.
351-
column_names = raw_datasets["train"].column_names
352-
text_column_name = "text" if "text" in column_names else column_names[0]
353-
354350
def tokenize_function(examples):
355351
return tokenizer(examples[text_column_name])
356352

0 commit comments

Comments
 (0)