Alpaca prompt template #515
Conversation
RdoubleA
requested review from
gokulavasan,
msaroufim,
joecummings,
ebsmothers,
SLR722 and
kartikayk
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/515
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit a8be21b with merge base e164402 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for torchtune-preview ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
torchtune/data/templates.py
Outdated
| @@ -0,0 +1,87 @@ | |||
| # Copyright (c) Meta Platforms, Inc. and affiliates. | |||
There was a problem hiding this comment.
This should have a leading underscore and an init.py file that exposes AlpacaPromptTemplate
There was a problem hiding this comment.
oops, forgot to add that
|
|
||
|
|
||
| class TestAlpacaInstructTemplate: | ||
| def test_format(self): |
There was a problem hiding this comment.
I actually configured the original test to include real data from the alpaca test. Can you do the same? It helps to verify that the test passes on real data.
There was a problem hiding this comment.
In fact why not just refactor data from that test directly?
|
|
||
| Args: | ||
| sample (Mapping): a single data sample with instruction | ||
| column_map (Optional[Dict[str, str]]): a mapping from the expected |
There was a problem hiding this comment.
column_map seems like a nice generalization, but I don't quite understand the use case. Can you expand on this?
There was a problem hiding this comment.
see all the prompting strategies inheriting from InstructionPromptTokenizingStrategy here in Axolotl: https:/OpenAccess-AI-Collective/axolotl/blob/2ea70ebbd8f1d8d46e692afd05773dcf06626601/src/axolotl/prompt_tokenizers.py#L148
column map serves the purpose of making sure the right columns are used for instruction and input. For samsum and grammar datasets, we will need to use this because the datasets on the hub will not be using "instruction" or "input" as their column names since they are specific types of instruct tasks
|
Shall we move dataset folder under data folder? |
Saving this for a later PR, as that will require considerable refactoring |
Context
We need to have a standardized set of prompt templates for our flagship datasets and to enable users to configure their own custom dataset, as discussed in the RFC (#493).
First, we create the
PromptTemplateinterface which all templates will be based on.AlpacaTemplateis added to demonstrate the interface and theAlpacaDatasetis refactored to use this.Test plan
pytest tests/torchtune/datasets/test_alpaca_dataset.pypytest tests/torchtune/data/test_templates.py