Guidance on describing a text loss?

[kgourgou]

Hi all,

I've been using textgrad in experiments and it is fun to see how it updates prompts, etc., but I have a practical problem. While I'm doing my best to describe what I want in the text loss, I see drift in the objective.

Specifically, I'm trying to optimize a large system prompt over three examples. The objective is for the final answer by the llm to finish with a particular formatting, e.g.,

code: <some-code-here>

However, I see that the LLM provides feedback on other aspects of the answer as well, causing a bit of drift and growing the system_prompt.

If you have any general suggestions on how to describe a text loss, I think that would be useful to the community. Thanks!

[mertyg]

We should absolutely add more guidance about how to do this in our tutorials! Thank you for this question @kgourgou .
Here are a couple things I would try:

• Use TGD(..., constraints=["your constraint"]): We can try to impose any 'constraint' through the constraints argument. In this case, we can try to have constraints like Your answer should end in the following format: 'Code: $IMPLEMENTATION' where IMPLEMENTATION is your implementation; or Do not grow the system prompt too much etc. In general, this is one major source to inject user guidance to the optimization process.
• Use other loss functions: You can use some structured loss function to give feedback only to the parts of interest in the inputs. For instance, if you use textgrad.loss.MultiFieldEvaluation, you can partition the inputs to the loss function into variables, and set requires_grad=True for only variables you want to optimize. I know this can be a bit confusing, but here's how we use it for a BigBench Hard experiment
• Use role descriptions proactively: We often find role descriptions are pretty helpful in guiding optimization, and you can also inject information there. For instance, if you set the role to be concise system prompt to a code generating language model (compared to e.g., system prompt), you'll see that you'll get much more specific/targeted prompts.

In general, there are a bunch of places to modulate the behavior of optimizers, which is a blessing as it is expressive, but curse because grows the search space. I'd recommend starting with these -- and we'll make sure to add more tutorials on this!

Thanks again @kgourgou -- great to have you in the community.

[kgourgou]

Very interesting! Thanks for sharing all the tips.

[mertyg]

Moved this to discussions in case it's more accessible!