fix max seq len

[ebsmothers]

Test plan:

tune run lora_finetune_single_device --config llama3/8B_lora_single_device
...
1|21|Loss: 2.320160388946533:   0%|▏ 

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/807

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 8a71824 with merge base b74fd3a ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kartikayk]

Thanks for the fix. I finally found the reference:

We trained the models on sequences of 8,192 tokens, using a mask to ensure self-attention does not cross document boundaries.

[kartikayk]

One note, we do double are RoPE cache. But I dont think this will meaningfully impact training.

[rohan-varma]

@kartikayk Is there a way to prevent this that's used in pretraining or other finetuning libs that we don't have? Seems our implementation makes RoPE directly proportional to max_seq_len, right? https:/pytorch/torchtune/blob/main/torchtune/modules/position_embeddings.py