Auto scaling factor tuning on FP8 weight gradients reduction for Megatron-LM

[Kan Wu (wkcn)]

Related PR in MS-AMP repo: Azure/MS-AMP#140

Add 3 arguments in Megatron-LM patch.

+    # Auto scaling factor tuning for FP8 collective communication
+    group.add_argument('--wgrad-auto-scaling', action='store_true', default=False,
+                       help='whether to enable auto scaling factor tuning on weight gradients reduction')
+    group.add_argument('--wgrad-auto-scaling-freq', type=int, default=10,
+                       help='the frequency of checking whether overflow exists in the result of weight gradients reduction')
+    group.add_argument('--wgrad-auto-scaling-ratio', type=float, default=1e-3,
+                       help='the threshold of overflow ratio for auto scaling factor tuning on weight gradients reduction')
+    group.add_argument('--wgrad-auto-scaling-window', type=int, default=100,
+                       help='the window size for auto scaling factor tuning on weight gradients reduction')