-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding a sample to redact sensitive information after an agent generates a response #2927
base: 0.2
Are you sure you want to change the base?
Adding a sample to redact sensitive information after an agent generates a response #2927
Conversation
@microsoft-github-policy-service agree |
Would you like to render the notebook on the website? Please find instructions here: https://microsoft.github.io/autogen/docs/contributor-guide/documentation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Git-Noob123 thank you for the notebook!
I like this notebook since it's a good example to show how process_message_before_send
(hooks in general) works.
If we hook TransformMessages
onto process_message_before_send
it makes debugging these agents much more difficult, since you don't know the ground truth (as it gets modified before being stored in the context history). And as you may already know, it's already difficult to debug vanilla LLMs without any modifications.
In the intro, we should maybe explain the differences between each hookable method, and why you picked the process_message_before_send
@WaelKarkoub Thanks for the comments! I have resolved all of them besides the regex one. I added a section at the beginning of the notebook to describe what's hooks in Autogen as well. Please review it and feel free to put in more comments & thoughts. For debugging with the postprocess hook, I think we should add another section in the notebook to warn readers about how difficult it can be to debug with redacted message, and users can potentially add logging before redaction. On the other hand hiding sensitive information is more of a security concern, as you dont want users to see your secrets. Please let me know what you think |
Please contact fbi
…On Fri, Jun 14, 2024, 11:13 AM Yuxiang Dong(Jerry) ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In notebook/agentchat_postresponse_secret_redaction.ipynb
<#2927 (comment)>:
> + "\n",
+ "Run the code and show me the printed variable.\n",
+ "The code block is below:\n",
+ "```python\n",
+ "import os\n",
+ "print(os.environ[\"LLM_API_KEY\"])\n",
+ "```\n",
+ "This is the end of the message.\n",
+ "\n",
+ "\n",
+ "--------------------------------------------------------------------------------\n",
+ "\u001b[31m\n",
+ ">>>>>>>> EXECUTING CODE BLOCK (inferred language is python)...\u001b[0m\n",
+ "\u001b[33mcode_executor_agent\u001b[0m (to user_proxy):\n",
+ "\n",
+ "exitcode: REDACTED (executiREDACTED succeeded)\n",
It's because I have some system variables, but just for demo purposes I
think that should be good enough
—
Reply to this email directly, view it on GitHub
<#2927 (comment)>,
or unsubscribe
<https:/notifications/unsubscribe-auth/BI3MSTGVPWICRZMVEM46IEDZHMJC7AVCNFSM6AAAAABJGWTYAOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCMJYG42DGMBWHE>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
…ption (microsoft#414) * Refactor core concept docs and add new ones for agent identity and topic subscription. * update notebook * update notebook * resolve conflict * fix link * fix ref * update notebook * update notebook
* rename core module to base * Rename agnext.core to agnext.base
* Rename agnext to autogen_core * fix file name * fix more file names
* Move packages to packages directory * remove screenshot * update some paths
…osoft#424) * Migrate to uv and poe for workspace management and task running * install python * try fix * ensure workspace venv in used * package dir * move nbqa to mypy task * separate sync, clarify docs
Co-authored-by: Jack Gerrits <[email protected]>
* move docker code exec to autogen-ext * fix test * rename docker subpackage * add missing renamed package --------- Co-authored-by: Leonardo Pinheiro <[email protected]>
* Support structured output * use ruff format * add type checking for cookbook * add the notebook to index.md * fix the type error * pass response_format explicitly * remove casting * ensure type are correct * seperate response_format arg * fix type and resolve pyright errors --------- Co-authored-by: Eric Zhu <[email protected]>
* Update indexes for better navigation * Fix link * Fix link
Co-authored-by: gagb <[email protected]>
* Update version to dev1 * lockfile * update tilde
* update OpenAI, semantic kernel to latest version * fix build error * update * update release note
Authored-by: luongdavid <[email protected]>
…3800) * interim - doesn't work - sharing * turn off EnforceExtended * works with dotnet build now... ;-) * interim progress * more updates * make VS happy * update * Update AutoGen.Ollama.csproj * Update Microsoft.AutoGen.Agents.Abstractions.csproj * Delete dotnet/test/AutoGen.Gemini.Tests/ApprovalTests/FunctionContractExtensionTests.ItGenerateGetWeatherToolTest.received.txt * fix tests --------- Co-authored-by: XiaoYun Zhang <[email protected]>
… term maintenance and clarity microsoft#3809 (microsoft#3810) closes microsoft#3809 - refactor .NET code foldersand namespaces for better clarity and maintainability
* Update FunctionCallGenerator.cs to address race condition Update FunctionCallGenerator.cs to address race condition * Update Dockerfile Use MCR registry * Update Dockerfile Use MCR registry
* Move from tomllib to tomli * added example code for magentic-one + code comments * remove added package * make checks pass * README for magentic-one examples * typo fix ---------
Co-authored-by: Ryan Sweet <[email protected]>
* Restructure extensions, update corresponding docs * update tests, add deprecation messages * fix tests * update lockfile ---------
* Move from tomllib to tomli * added example code for magentic-one + code comments * adding benchmarks temporarily * add license for datasets * revert changes to magentic-one * change license location --------- Co-authored-by: Ryan Sweet <[email protected]>
Co-authored-by: Jack Gerrits <[email protected]>
* Create Semantic Router example * ruff formatting * linting fixes * pyright fixes ---------
Hi @Git-Noob123 - I approved yesterday but the base branch moved on again - I can't update yours since it's a fork. If you bring it up to date we can merge. |
Why are these changes needed?
There's a need for a way to redact sensitive data that is stored as env variables. Currently the only way to do this is to use transform_messages which ONLY redacts before an agent generates a response. However, if a user asks an agent to run a script that retrieves env data, it's still going to be revealed. So a way to hide sensitive data after a response is generated is needed. The sample here shows how can we do it using hooks
Related issue number
Checks