【Hackathon 4 No52】add fp16 for dist

[enkilee]

PR types

Others

PR changes

Others

Describe

add fp16 for dist
add docs: PaddlePaddle/docs#5730

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[enkilee]

Traceback (most recent call last):
2023-03-22 23:09:51 10 File "test_dist_op.py", line 119, in test_check_grad
2023-03-22 23:09:51 11 self.check_grad(
2023-03-22 23:09:51 12 File "/mnt/paddle/build/python/paddle/fluid/tests/unittests/eager_op_test.py", line 2211, in check_grad
2023-03-22 23:09:51 13 self.check_grad_with_place(
2023-03-22 23:09:51 14 File "/mnt/paddle/build/python/paddle/fluid/tests/unittests/eager_op_test.py", line 2387, in check_grad_with_place
2023-03-22 23:09:51 15 self._assert_is_close(
2023-03-22 23:09:51 16 File "/mnt/paddle/build/python/paddle/fluid/tests/unittests/eager_op_test.py", line 2184, in _assert_is_close
2023-03-22 23:09:51 17 self.assertLessEqual(max_diff, max_relative_error, err_msg())
2023-03-22 23:09:51 18 AssertionError: 0.04123 not less than or equal to 0.005 : Operator dist error, Gradient Check On Place(gpu:0) variable X (shape: (4, 1, 4, 8), dtype: float16) max gradient diff 4.122925e-02 over limit 5.000000e-03, the first error element is 36, expected 2.960205e-03, but got 2.838135e-03.

请问，像这种grad check超了的，该如何分析？

[zhangting2020]

Traceback (most recent call last): 2023-03-22 23:09:51 10 File "test_dist_op.py", line 119, in test_check_grad 2023-03-22 23:09:51 11 self.check_grad( 2023-03-22 23:09:51 12 File "/mnt/paddle/build/python/paddle/fluid/tests/unittests/eager_op_test.py", line 2211, in check_grad 2023-03-22 23:09:51 13 self.check_grad_with_place( 2023-03-22 23:09:51 14 File "/mnt/paddle/build/python/paddle/fluid/tests/unittests/eager_op_test.py", line 2387, in check_grad_with_place 2023-03-22 23:09:51 15 self._assert_is_close( 2023-03-22 23:09:51 16 File "/mnt/paddle/build/python/paddle/fluid/tests/unittests/eager_op_test.py", line 2184, in _assert_is_close 2023-03-22 23:09:51 17 self.assertLessEqual(max_diff, max_relative_error, err_msg()) 2023-03-22 23:09:51 18 AssertionError: 0.04123 not less than or equal to 0.005 : Operator dist error, Gradient Check On Place(gpu:0) variable X (shape: (4, 1, 4, 8), dtype: float16) max gradient diff 4.122925e-02 over limit 5.000000e-03, the first error element is 36, expected 2.960205e-03, but got 2.838135e-03.

请问，像这种grad check超了的，该如何分析？

从前、反向的实现上去检查，看哪个计算过程对精度损失比较严重

[paddle-ci-bot]

Sorry to inform you that 8eae1b5's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[luotao1]

close due to the following PR is merged:

• 【Hackathon No.52】为 Paddle dist 算子实现 float16 数据类型支持 #50915