Skip to content

Fix HOLD SCPI float formatting precision - #6

Open
grasswang wants to merge 1 commit into
epics-modules:masterfrom
grasswang:precision
Open

Fix HOLD SCPI float formatting precision#6
grasswang wants to merge 1 commit into
epics-modules:masterfrom
grasswang:precision

Conversation

@grasswang

Copy link
Copy Markdown

Fixed float formatting precision for the HOLD SCPI command.

The code was using %e, which truncated the value and reduced delay/holdoff precision in readback/write formatting. This changes it to %.11e so the full intended precision is preserved.

@anjohnson

Copy link
Copy Markdown
Member

The commit log doesn't mention that it actually updates all of the SCPI commands that use %e to %.11e, not just HOLD.

@grasswang

Copy link
Copy Markdown
Author

Thank you for pointing it out.
This modified TRIG_HOLDOFF, BURST_DELAY, and all 8 Delays.
I tested all of the modified parameters with our device. With the modification, it can now take 0.123456789123. (But the device would get 0.0123456789125 because the device's resolution is 5 ps.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants