Request for Test Results Comparison: Potential Issues in My Implementation

[01xjw]

Dear LPhgh,

Thank you for sharing your project and detailed instructions! I recently followed the steps outlined in your documentation to run the primary test, but I encountered some unexpected results that I’d like to clarify.

During my testing, the preheating time was significantly longer than anticipated, and the final performance metrics were not satisfactory . To better understand whether the issue lies in my implementation or environment configuration, I would be extremely grateful if you could kindly share:

A sample of your test results (output files).
Any specific configurations ( hardware) used during your testing.
By comparing my results with yours, I hope to identify potential gaps in my setup or misunderstandings in the process. If there are additional details I should provide, please let me know—I’m happy to assist in troubleshooting.

I truly appreciate your time and effort in addressing this request. Thank you for your patience and support!

Best regards!

[LPhgh]

hi @01xjw,

We think it reasonable to show sub-optimal or even bad performance because this demo is not oriented for performance optimization.

Just FYI, we use Hopper GPUs with 200Gbps RDMA to run benchmark, shown in https://github.com/LPhgh/sglang/blob/afd/afd/poc_evaluation.md

We haven't tested it in other environments yet. Maybe you can try to run with zmq first, which should not take much time to setup (but with StepMesh this takes a bit longer). And it's suggested to check the network bandwidth, which plays a significant role in AFD performance.

We are working on a more graceful design and plan to post pr in a month. Then we can see if the problem solved :)

[01xjw]

Thank you very much for your support! I have successfully run the related tests on H800 using the startup commands you provided, including ZMQ and StepMesh. I look forward to your future improvements and designs, and once again, thank you for your valuable response and advice.