How to make cuda plot fast

[liyujcx]

Now my device create one plot used about 5.5 min, how to make it faster? Someone use 80 seconds....it's too fast, how to improve this.

OS:ubantu 22.04
CPU:2670v2 *2
RAM:280GB
NVME SSD: intel p3600 800G *2
NVIDIA GeForce RTX 3070 with 8GB vram

Bladebit Chia Plotter
Version : 3.0.0-alpha3-dev
Git Commit : unknown
Compiled With: gcc 11.3.0

[Global Plotting Config]
Will create plots indefinitely.
Thread count : 40
Warm start enabled : false
NUMA disabled : false
CPU affinity disabled : false
Farmer public key : xxx
Pool public key : xxx
Benchmark mode : disabled

[Bladebit CUDA Plotter]
Selected cuda device 0 : NVIDIA GeForce RTX 3070
CUDA Compute Capability : 8.6
SM count : 46
Max blocks per SM : 16
Max threads per SM : 1536
Async Engine Count : 2
L2 cache size : 4.00 MB
L2 persist cache max size : 3.00 MB
Stack Size : 1.00 KB
Memory:
Total : 7.78 GB
Free : 7.63 GB

Allocating buffers (this may take a few seconds)...
Kernel RAM required : 90240524288 bytes ( 86060.07 MiB or 84.04 GiB )
Intermediate RAM required : 2999001088 bytes ( 2860.07 MiB or 2.79 GiB )
Host RAM required : 168443248640 bytes ( 160640.00 MiB or 156.88 GiB )
Total Host RAM required : 258683772928 bytes ( 246700.07 MiB or 240.92 GiB )
GPU RAM required : 6140243968 bytes ( 5855.79 MiB or 5.72 GiB )
Allocating buffers

Generating plot 1 / 4294967295: 02533bf1293e9bed404b86a05ea5e831a2ce729d207c18582cf48dae198457e5
Plot temporary file: /mnt/chia/tmp1/plot-k32-2023-03-28-22-45-02533bf1293e9bed404b86a05ea5e831a2ce729d207c18582cf48dae198457e5.plot.tmp
Start to check out dir :/mnt/chia/tmp1/ space, if space not enough, plot will pause!
Finish to check out dir /mnt/chia/tmp1/ , pause time 0.00 seconds, total 1465 GB , 1263 GB free

Generating F1
Finished F1 in 5.42 seconds.
Table 2 completed in 44.40 seconds with 4294963568 entries.
Table 3 completed in 25.14 seconds with 4294933103 entries.
Table 4 completed in 24.52 seconds with 4294916009 entries.
Table 5 completed in 24.50 seconds with 4294721601 entries.
Table 6 completed in 19.21 seconds with 4294414058 entries.
Table 7 completed in 13.43 seconds with 4293870521 entries.
Finalizing Table 7
Finalized Table 7 in 7.36 seconds.
Completed Phase 1 in 164.01 seconds
Marked Table 6 in 4.29 seconds.
Marked Table 5 in 4.28 seconds.
Marked Table 4 in 4.28 seconds.
Marked Table 3 in 4.28 seconds.
Marked Table 2 in 4.28 seconds.
Completed Phase 2 in 21.41 seconds
Compressing Table 1 and 2...
Step 1 completed step in 9.06 seconds.
Step 2 completed step in 8.78 seconds.
Completed table 1 in 17.84 seconds with 3429376545 / 4294963568 entries ( 79.85% ).
Compressing tables 2 and 3...
Step 1 completed step in 8.86 seconds.
Step 2 completed step in 11.66 seconds.
Step 3 completed step in 8.68 seconds.
Completed table 2 in 29.20 seconds with 3439872925 / 4294933103 entries ( 80.09% ).
Compressing tables 3 and 4...
Step 1 completed step in 8.85 seconds.
Step 2 completed step in 11.75 seconds.
Step 3 completed step in 8.66 seconds.
Completed table 3 in 29.26 seconds with 3465957977 / 4294916009 entries ( 80.70% ).
Compressing tables 4 and 5...
Step 1 completed step in 8.85 seconds.
Step 2 completed step in 11.91 seconds.
Step 3 completed step in 8.73 seconds.
Completed table 4 in 29.49 seconds with 3532686831 / 4294721601 entries ( 82.26% ).
Compressing tables 5 and 6...
Step 1 completed step in 8.84 seconds.
Step 2 completed step in 12.33 seconds.
Step 3 completed step in 9.05 seconds.
Completed table 5 in 30.22 seconds with 3713072453 / 4294414058 entries ( 86.46% ).
Compressing tables 6 and 7...
Step 1 completed step in 8.77 seconds.
Step 2 completed step in 13.72 seconds.
Step 3 completed step in 9.45 seconds.
Completed table 6 in 31.94 seconds with 4293870521 / 4293870521 entries ( 100.00% ).
Serializing P7 entries
Completed serializing P7 entries in 5.76 seconds.
Completed Phase 3 in 173.72 seconds
Completed Plot 1 in 359.15 seconds ( 5.99 minutes )

/mnt/chia/tmp1/plot-k32-2023-03-28-22-45-02533bf1293e9bed404b86a05ea5e831a2ce729d207c18582cf48dae198457e5.plot.tmp -> /mnt/chia/tmp1/plot-k32-2023-03-28-22-45-02533bf1293e9bed404b86a05ea5e831a2ce729d207c18582cf48dae198457e5.plot
Completed writing plot in 46.42 seconds
Final plot table pointers:
Table 1: 1289885828 ( 0x000000004ce21884 )
Table 2: 16129313690 ( 0x00000003c161cb9a )
Table 3: 30112200140 ( 0x0000000702d3b5cc )
Table 4: 44201122115 ( 0x0000000a4a979943 )
Table 5: 58561289240 ( 0x0000000da2865c18 )
Table 6: 73654714040 ( 0x000000112629beb8 )
Table 7: 91109050565 ( 0x000000153685c8c5 )
C 1 : 4096 ( 0x0000000000001000 )
C 2 : 1721652 ( 0x00000000001a4534 )
C 3 : 1721828 ( 0x00000000001a45e4 )

Final plot table sizes:
Table 1: 14151.98 MiB
Table 2: 13335.12 MiB
Table 3: 13436.24 MiB
Table 4: 13694.92 MiB
Table 5: 14394.21 MiB
Table 6: 16645.75 MiB
Table 7: 16891.69 MiB
C 1 : 1.64 MiB
C 2 : 0.00 MiB
C 3 : 1228.49 MiB

[A1pha2er0]

This is a little bit slow,I am still using ram plot.

[scoville07]

Same here, my system wa able to create madmax gigahorse plots in 2,5 minutes. With bladebit it takes around 6 minutes. This slows down replotting significantly, any idea if it is possible to tune that?