Add CometLogger for metrics logging via comet.ml #1221

dzheng256 · 2024-07-25T04:23:44Z

Context

What is the purpose of this PR? Is it to

add a new feature
fix a bug
update tests and/or documentation
other (please add here)

My company uses torchtune for LLM finetuning but we use comet.ml for experiment tracking, not tensorboard or weights and biases. Adding a CometLogger class to make it easier for other Comet users to use torchtune with Comet.

Changelog

Introduce CometLogger class
Update docs to include CometLogger
Add TestCometLogger test

Test plan

Please make sure to do each of the following if applicable to your PR. (If you're not sure about any one of these just ask and we will happily help.)

run pre-commit hooks and linters (make sure you've first installed via pre-commit install)
add unit tests for any new functionality
update docstrings for any new or updated methods or classes
run unit tests via pytest tests
run recipe tests via pytest tests -m integration_test
manually run any new or modified recipes with sufficient proof of correctness
- include relevant commands and any other artifacts in this summary (pastes of loss curves, eval results, etc.)

pytorch-bot · 2024-07-25T04:23:47Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1221

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 27c90e3 with merge base 0057fe7 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-07-25T04:23:50Z

Hi @dzheng256!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

joecummings · 2024-07-29T22:12:48Z

@dzheng256 Would you mind signing the CLA agreement?

dzheng256 · 2024-07-29T22:34:19Z

Hey @joecummings I actually already had an email exchange with the meta CLA folks about this and I think I'm on the agreement already but it might be my corporate email ([email protected]) rather than the account I'm making this PR from ([email protected]). I've asked them to add my personal email to the corporate CLA as well but haven't heard back.

dzheng256 · 2024-07-30T20:55:17Z

It seems like the tests ran and passed, @joecummings does that mean the CLA issue is resolved?

RdoubleA · 2024-07-30T21:00:30Z

@dzheng256 It seems the CLA check is still failing. Have you confirmed that your personal email was added? Are you still able to sign a second time with your personal email?

facebook-github-bot · 2024-07-30T22:06:41Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

facebook-github-bot · 2024-07-30T22:06:46Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

dzheng256 · 2024-07-30T22:17:22Z

@joecummings sorry about that, I signed again with my personal email and it seems like CLA issues are resolved now.

SalmanMohammadi · 2024-08-01T16:27:46Z

Thank you so much for adding this @dzheng256. It's really cool to hear that you're using torchtune in your company.

Could you possibly attach an example comet logging output to the PR to confirm everything looks GOOD?

torchtune/utils/metric_logging.py

SalmanMohammadi · 2024-08-01T16:30:41Z

Could you also please shout out integration in the README.md here, please?

torchtune/README.md

Line 35 in 288ff44

RdoubleA · 2024-08-03T00:29:39Z

Thanks for adding this. There was also an earlier PR to add CometLogger that including a quick tutorial on how to use it. Would you be able to add that in this PR as well? #910

dzheng256 · 2024-08-03T04:45:16Z

Oh sorry, I hadn't realized there was an open PR implementing pretty much the same thing as this one. Should we stick with #910 then, since it came first? It also seems to be written by the a Comet dev so it might utilize the Comet API better. cc @Lothiraldan

RdoubleA · 2024-08-05T18:04:22Z

@dzheng256 I've pinged the other PR to see if he's still willing to work on it. If not, maybe you can incorporate the differences here?

dzheng256 · 2024-08-05T18:33:18Z

Yes sounds good, I will pull in the tutorial from the other PR if they no longer have the bandwidth to work on it.

Lothiraldan

Looking good so far, I will give it a try in the next days to generate an example public project.

@dzheng256 Feel free to pull the tutorial from my PR https:/pytorch/torchtune/pull/910/files#diff-5c3c14a3a87e5106e63f046499c113cd5b5c0771ac548bcb5b097fc40342bd44, I don't think I have the permission to push to your fork. Once I have a new example project, I will share the links to update the tutorial

There is one open question, whether to use our new way of initializing Comet Experiments. This is a new way but it's about to be more globally released, we have it stable for a while now and we already updated several of our integrations to use it, last one was Transformers: huggingface/transformers#31366

pyproject.toml

Lothiraldan · 2024-08-06T15:28:14Z

torchtune/utils/metric_logging.py

+ self.experiment = Experiment(
+ project_name=project_name,
+ workspace=workspace,
+ log_code=log_code,
+ **kwargs,
+ )


We are moving towards a new way of initializing Comet Experiment with a new top-level function: comet_ml.start. This has several benefits:

It support online and offline logging

It support logging a new experiment or appending to an existing Experiment

You can set name and tags directly at initialization time

I'm working right now on finalizing the documentation but I can share the reference doc for that new function already: https://www.comet.com/docs/v2/api-and-sdk/python-sdk/reference/Experiment-Creation/

If there is a chance that users might want to append data to an existing experiment or create log data to an offline Experiment (to upload later), I would recommend using that instead. Please let me know, happy to provide the code corresponding code change.

Thanks @Lothiraldan, I swapped it over to the new comet_ml.start() and verified the latest code works for me. Could you use it to make an example project which I can then link in the deep dive tutorial?

@Lothiraldan I also added you as a collaborator to my fork, feel free to directly push changes if you would like!

I've sent an update and I'm working on running a new example once I get my hands on a EC2 GPU instance

torchtune/utils/metric_logging.py

Co-authored-by: Boris Feld <[email protected]>

SalmanMohammadi · 2024-08-08T16:26:40Z

torchtune/utils/metric_logging.py

+ api_key (Optional[str]): Comet API key. It's recommended to configure the API Key from the environment.
+ workspace (Optional[str]): Comet workspace name. If not provided, uses the default workspace.
+ project (Optional[str]): Comet project name. Defaults to Uncategorized.
+ experiment_name (Optional[str]): The name for comet experiment to be used for logging.


Suggested change

experiment_name (Optional[str]): The name for comet experiment to be used for logging.

I think it's still unhappy about this @Lothiraldan

Yes, I'm working on understanding why is it complaining. It only is complaining with the latest version of pydocstring.

SalmanMohammadi · 2024-08-08T16:28:43Z

torchtune/utils/metric_logging.py

+ online (Optional[bool]): If True, the data will be logged to Comet server, otherwise it will be stored locally
+ in offline experiment. Default is `True`.
+ mode (Optional[str]): Control how the Comet experiment is started. "get": Continue logging to an existing
+ experiment identified by the `experiment_key` value. "create": Always creates of a new experiment, useful
+ for HPO sweeps. "get_or_create" (default): Starts a fresh experiment if required, or persists logging to
+ an existing one.


Suggested change

online (Optional[bool]): If True, the data will be logged to Comet server, otherwise it will be stored locally

in offline experiment. Default is `True`.

mode (Optional[str]): Control how the Comet experiment is started. "get": Continue logging to an existing

experiment identified by the `experiment_key` value. "create": Always creates of a new experiment, useful

for HPO sweeps. "get_or_create" (default): Starts a fresh experiment if required, or persists logging to

an existing one.

mode (Optional[str]): Control how the Comet experiment is started. "get": Continue logging to an existing

experiment identified by the `experiment_key` value. "create": Always creates of a new experiment, useful

for HPO sweeps. "get_or_create" (default): Starts a fresh experiment if required, or persists logging to

an existing one.

online (Optional[bool]): If True, the data will be logged to Comet server, otherwise it will be stored locally

in offline experiment. Default is `True`.

nit: ordering

Thanks, I was wondering why the linting was failing initially.

Lothiraldan · 2024-08-08T16:38:28Z

@SalmanMohammadi I got back access to the EC2 instance and I've got the example running: https://www.comet.com/examples/comet-example-torchtune-mistral/fffb3036880e41b5af2df932db4d3578?compareXAxis=step&experiment-tab=panels&showOutliers=true&smoothing=0&xAxis=step. I took a screenshot while it's running, it will probably looks better after it finishes but we can merge with the current screenshot.

I made some small changes and added back logging the configuration as a file, this way it's pretty easy to grab it from Comet and reproduce a previous run. I made several small commits, sorry about that. Let me know if you want me to squash them if you are not gonna squash the PR.

SalmanMohammadi · 2024-08-08T16:53:32Z

@SalmanMohammadi I got back access to the EC2 instance and I've got the example running: https://www.comet.com/examples/comet-example-torchtune-mistral/fffb3036880e41b5af2df932db4d3578?compareXAxis=step&experiment-tab=panels&showOutliers=true&smoothing=0&xAxis=step. I took a screenshot while it's running, it will probably looks better after it finishes but we can merge with the current screenshot.

I made some small changes and added back logging the configuration as a file, this way it's pretty easy to grab it from Comet and reproduce a previous run. I made several small commits, sorry about that. Let me know if you want me to squash them if you are not gonna squash the PR.

Amazing!! Thank you.

Feel free to send me an updated screenshot on Discord and I'm happy to add.

SalmanMohammadi · 2024-08-08T16:54:31Z

torchtune/utils/metric_logging.py

+ experiment_name (Optional[str]): Name of the experiment. If not provided, Comet will auto-generate a name.
+ tags (Optional[List[str]]): Tags to associate with the experiment.
+ log_code (bool): Whether to log the source code. Defaults to True.
+ **kwargs: additional arguments to pass to Comet.start. See


might need to type this (annoyingly)

Suggested change

**kwargs: additional arguments to pass to Comet.start. See

**kwargs: additional arguments to pass to Comet.start. See

SalmanMohammadi · 2024-08-08T16:54:38Z

torchtune/utils/metric_logging.py

+ experiment_name: Optional[str] = None,
+ tags: Optional[List[str]] = None,
+ log_code: bool = True,
+ **kwargs,


RdoubleA

Looks great overall. I have a lot of nit comments around RST docs rendering stuff, hope you don't mind :)

RdoubleA · 2024-08-08T16:37:41Z

docs/source/deep_dives/comet_logging.rst

+ * How to use the :class:`~torchtune.utils.metric_logging.CometLogger`
+ * How to log configs, metrics, and model checkpoints to Comet
+
+Torchtune supports logging your training runs to `Comet <https://www.comet.com/site/?utm_source=torchtune&utm_medium=docs&utm_content=docs>`_.


nit: we lowercase torchtune everywhere, even at the start of a sentence :)

docs/source/deep_dives/comet_logging.rst

RdoubleA · 2024-08-08T16:39:39Z

docs/source/deep_dives/comet_logging.rst

+
+ # enable logging to the built-in CometLogger
+ metric_logger:
+ _component_: torchtune.utils.metric_logging.CometLogger


RdoubleA · 2024-08-08T16:40:37Z

docs/source/deep_dives/comet_logging.rst

+
+.. note::
+
+ Click on this sample `Comet project to see how it will looks like after fine-tuning <https://www.comet.com/examples/comet-example-torchtune-mistral/>`_.


Suggested change

Click on this sample `Comet project to see how it will looks like after fine-tuning <https://www.comet.com/examples/comet-example-torchtune-mistral/>`_.

Click on this sample `Comet project to see the logged metrics after fine-tuning <https://www.comet.com/examples/comet-example-torchtune-mistral/>`_.

torchtune/utils/metric_logging.py

RdoubleA · 2024-08-08T16:50:24Z

torchtune/utils/metric_logging.py

+ experiment_key (Optional[str]): The key for comet experiment to be used for logging. Must be an alphanumeric
+ string whose length is between 32 and 50 characters.
+ mode (Optional[str]): Control how the Comet experiment is started. "get": Continue logging to an existing
+ experiment identified by the `experiment_key` value. "create": Always creates of a new experiment, useful


Suggested change

experiment identified by the `experiment_key` value. "create": Always creates of a new experiment, useful

experiment identified by the ``experiment_key`` value. "create": Always creates of a new experiment, useful

RdoubleA · 2024-08-08T16:50:37Z

torchtune/utils/metric_logging.py

+ for HPO sweeps. "get_or_create" (default): Starts a fresh experiment if required, or persists logging to
+ an existing one.
+ online (Optional[bool]): If True, the data will be logged to Comet server, otherwise it will be stored locally
+ in offline experiment. Default is `True`.


Suggested change

in offline experiment. Default is `True`.

in an offline experiment. Default is ``True``.

RdoubleA · 2024-08-08T16:51:23Z

torchtune/utils/metric_logging.py

+ experiment_name (Optional[str]): Name of the experiment. If not provided, Comet will auto-generate a name.
+ tags (Optional[List[str]]): Tags to associate with the experiment.
+ log_code (bool): Whether to log the source code. Defaults to True.
+ **kwargs: additional arguments to pass to Comet.start. See


Suggested change

**kwargs: additional arguments to pass to Comet.start. See

**kwargs: additional arguments to pass to ``Comet.start``. See

RdoubleA · 2024-08-08T16:51:58Z

torchtune/utils/metric_logging.py

+ >>> logger.close()
+
+ Raises:
+ ImportError: If `comet_ml` package is not installed.


Suggested change

ImportError: If `comet_ml` package is not installed.

ImportError: If ``comet_ml`` package is not installed.

RdoubleA · 2024-08-08T16:53:58Z

torchtune/utils/metric_logging.py

+ You can install it with ``pip install comet_ml``.
+ You need to set up your Comet.ml API key before using this logger.
+ You can do this by setting the COMET_API_KEY environment variable
+ or by calling ``comet_ml.login()`` with your API key.


personally I find comet login the easiest to do and is more comfortable for folks who don't know much code. What are your thoughts on just always recommending to setup via comet login so folks do not get confused with the different options?

comet login works great in interactive situations but won't help in automated setups. I'm happy to highlight the comet login option and links to other options in the Comet docs if that works for you.

Same comment as in #1221 (comment), if we expect users to be setting up a config and doing tune run ... how are they supposed to use comet.login()? We would have to call that inside the logger, and incorporate some API key via a secrets manager solution which would be different for every user.

As far as I can tell torchtune seems to heavily favor the CLI tune run style usage, rather than importing it and calling tune run programmatically. If this is indeed the case, I think we should actually remove the mention to comet.login() and have people export the api key, what do you guys think?

Sorry I was under the impression you can run comet login from CLI, is this not the case?

Either way I agree with @dzheng256, since CLI is the primary entry point for most users via tune run I would lean towards suggesting whatever fits that workflow best whether that's setting environment variables or a CLI command. I suppose for automated setups environment variable is better?

@RdoubleA @dzheng256 Sorry about the confusion, there is bot a comet login command-line script and comet_ml.login that one can call in their Python script. I will clean up and double-check everything to point people to use the CLI comet login.

codecov-commenter · 2024-08-08T20:55:37Z

Codecov Report

Attention: Patch coverage is 22.41379% with 45 lines in your changes missing coverage. Please review.

Project coverage is 27.13%. Comparing base (7eb89e2) to head (089adad).
Report is 37 commits behind head on main.

Files	Patch %	Lines
torchtune/utils/metric_logging.py	25.71%	26 Missing ⚠️
tests/torchtune/utils/test_metric_logging.py	17.39%	19 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (7eb89e2) and HEAD (089adad). Click for more details.

HEAD has 3 uploads less than BASE

Flag BASE (7eb89e2) HEAD (089adad)

5 2

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1221       +/-   ##
===========================================
- Coverage   67.81%   27.13%   -40.68%     
===========================================
  Files         219      255       +36     
  Lines        9908    11799     +1891     
===========================================
- Hits         6719     3202     -3517     
- Misses       3189     8597     +5408

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

SalmanMohammadi · 2024-08-09T12:59:40Z

tests/torchtune/utils/test_metric_logging.py

+class TestCometLogger:
+ def test_log(self) -> None:
+ with patch("comet_ml.Experiment") as mock_experiment:
+ logger = CometLogger(project_name="test_project")


Suggested change

logger = CometLogger(project_name="test_project")

logger = CometLogger(project="test_project")

SalmanMohammadi · 2024-08-09T12:59:52Z

tests/torchtune/utils/test_metric_logging.py

+
+ def test_log_dict(self) -> None:
+ with patch("comet_ml.Experiment") as mock_experiment:
+ logger = CometLogger(project_name="test_project")


Suggested change

logger = CometLogger(project_name="test_project")

logger = CometLogger(project="test_project")

SalmanMohammadi · 2024-08-09T13:00:05Z

tests/torchtune/utils/test_metric_logging.py

+
+ def test_log_config(self) -> None:
+ with patch("comet_ml.Experiment") as mock_experiment:
+ logger = CometLogger(project_name="test_project")


Suggested change

logger = CometLogger(project_name="test_project")

logger = CometLogger(project="test_project")

Also update the last uppercase references to Torchtune

Lothiraldan · 2024-08-12T12:23:41Z

Hi @SalmanMohammadi, I've pushed fixes for the unit tests, updated the links and screenshots, tried to clarify the docstring about experiment_key argument and found the correct format for the mode argument to render as a nested bullet point.

I think that we are all good now but please let me know if I missed anything

SalmanMohammadi · 2024-08-12T13:03:20Z

README.md

@@ -30,6 +30,7 @@ torchtune focuses on integrating with popular tools and libraries from the ecosy
 - [PyTorch FSDP](https://pytorch.org/docs/stable/fsdp.html) for distributed training
 - [torchao](https:/pytorch-labs/ao) for lower precision dtypes and [post-training quantization](recipes/quantize.py) techniques
 - [Weights & Biases](https://wandb.ai/site) for [logging](https://pytorch.org/torchtune/main/deep_dives/wandb_logging.html) metrics and checkpoints, and tracking training progress
+- [CometML](https://www.comet.com/site/) as another option for [logging](https://pytorch.org/torchtune/stable/api_ref_utilities.html#metric-logging)


Suggested change

- [CometML](https://www.comet.com/site/) as another option for [logging](https://pytorch.org/torchtune/stable/api_ref_utilities.html#metric-logging)

- [CometML](https://www.comet.com/site/) as another option for [logging](https://pytorch.org/torchtune/main/deep_dives/comet_logging.html)

one last minor nit, I promise

SalmanMohammadi · 2024-08-12T13:04:39Z

Thank you so much for your patience in addressing all our comments, and for your contributions @dzheng256, @Lothiraldan. This looks fantastic to me.

One last tiny nit and feel free to merge once CI is green.

Lothiraldan · 2024-08-12T13:33:18Z

@SalmanMohammadi Good catch, thanks you and @RdoubleA for all of your time and reviews. I hope that my millions of small commits were not spamming your inbox too much 😅

dzheng256 · 2024-08-12T22:06:46Z

Thanks everyone! Looking forward to deleting my internal CometLogger and switching to torchtune.utils.metric_logging.CometLogger

dzheng256 added 5 commits July 25, 2024 03:49

Add comet_ml dep

bd90921

Add CometLogger

fb86fef

Add TestCometLogger

d241cd7

Update docs

93d75f6

Formatting

a5d253e

dzheng256 changed the title ~~Add comet logging~~ Add CometLogger for metrics logging via comet.ml Jul 25, 2024

dzheng256 added 3 commits July 25, 2024 04:34

Prefer comet_ml.login to comet_ml.init

14eadcd

Using typing.List for older python

edb00cf

empty commit

a3cff9e

dzheng256 marked this pull request as ready for review July 29, 2024 22:03

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 30, 2024

SalmanMohammadi reviewed Aug 1, 2024

View reviewed changes

torchtune/utils/metric_logging.py Outdated Show resolved Hide resolved

RdoubleA mentioned this pull request Aug 5, 2024

New integration - CometLogger #910

Closed

3 tasks

Lothiraldan reviewed Aug 6, 2024

View reviewed changes

Update pyproject.toml

6b39e87

Co-authored-by: Boris Feld <[email protected]>

SalmanMohammadi reviewed Aug 8, 2024

View reviewed changes

Lothiraldan added 2 commits August 8, 2024 18:29

Fix lints and doc issues

1a4e1f1

Add missing screenshot and add Comet Logging to the Nav

4abe775

SalmanMohammadi reviewed Aug 8, 2024

View reviewed changes

RdoubleA reviewed Aug 8, 2024

View reviewed changes

Lothiraldan added 2 commits August 8, 2024 21:12

Fix docstring format

3195e78

Add Comet description to the docstring

089adad

Lothiraldan added 3 commits August 9, 2024 09:50

Point to comet login for setting up Comet authorization

e930526

Fix docstring lint issue

68b1f7b

Re-order valid values for mode parameter

025ad3a

SalmanMohammadi reviewed Aug 9, 2024

View reviewed changes

Fix unit tests

0b61dbc

SalmanMohammadi reviewed Aug 9, 2024

View reviewed changes

Lothiraldan added 3 commits August 12, 2024 14:11

Update Comet links and screenshot

48a70f8

Try to clarify experiment_key docstring.

bfde03b

Also update the last uppercase references to Torchtune

Update mode argument docstring with proper format

4c94eb1

SalmanMohammadi reviewed Aug 12, 2024

View reviewed changes

SalmanMohammadi approved these changes Aug 12, 2024

View reviewed changes

Update Comet name and doc link in the Readme

27c90e3

SalmanMohammadi merged commit 6be89c0 into pytorch:main Aug 12, 2024
20 checks passed

dzheng256 deleted the add-comet-logging branch August 12, 2024 22:05

	**kwargs: additional arguments to pass to Comet.start. See
	**kwargs: additional arguments to pass to Comet.start. See


		.. note::

		Click on this sample `Comet project to see how it will looks like after fine-tuning <https://www.comet.com/examples/comet-example-torchtune-mistral/>`_.

	experiment identified by the `experiment_key` value. "create": Always creates of a new experiment, useful
	experiment identified by the ``experiment_key`` value. "create": Always creates of a new experiment, useful

	in offline experiment. Default is `True`.
	in an offline experiment. Default is ``True``.

	ImportError: If `comet_ml` package is not installed.
	ImportError: If ``comet_ml`` package is not installed.

	logger = CometLogger(project_name="test_project")
	logger = CometLogger(project="test_project")

	- [CometML](https://www.comet.com/site/) as another option for [logging](https://pytorch.org/torchtune/stable/api_ref_utilities.html#metric-logging)
	- [CometML](https://www.comet.com/site/) as another option for [logging](https://pytorch.org/torchtune/main/deep_dives/comet_logging.html)

Add CometLogger for metrics logging via comet.ml #1221

Add CometLogger for metrics logging via comet.ml #1221

Conversation

dzheng256 commented Jul 25, 2024

Context

Changelog

Test plan

pytorch-bot bot commented Jul 25, 2024 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1221

✅ No Failures

facebook-github-bot commented Jul 25, 2024

Action Required

Process

joecummings commented Jul 29, 2024

dzheng256 commented Jul 29, 2024

dzheng256 commented Jul 30, 2024

RdoubleA commented Jul 30, 2024

facebook-github-bot commented Jul 30, 2024

facebook-github-bot commented Jul 30, 2024

dzheng256 commented Jul 30, 2024 • edited Loading

SalmanMohammadi commented Aug 1, 2024

SalmanMohammadi commented Aug 1, 2024

RdoubleA commented Aug 3, 2024

dzheng256 commented Aug 3, 2024 • edited Loading

RdoubleA commented Aug 5, 2024

dzheng256 commented Aug 5, 2024

Lothiraldan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dzheng256 Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lothiraldan commented Aug 8, 2024

SalmanMohammadi commented Aug 8, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RdoubleA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RdoubleA Aug 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 8, 2024

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lothiraldan commented Aug 12, 2024

Choose a reason for hiding this comment

SalmanMohammadi commented Aug 12, 2024

Lothiraldan commented Aug 12, 2024

dzheng256 commented Aug 12, 2024

pytorch-bot bot commented Jul 25, 2024 •

edited

Loading

dzheng256 commented Jul 30, 2024 •

edited

Loading

dzheng256 commented Aug 3, 2024 •

edited

Loading

dzheng256 Aug 8, 2024 •

edited

Loading

RdoubleA Aug 9, 2024 •

edited

Loading