Write the rules out over several files, so RULES=c can be built 32-bit - #6
Open
masonasons wants to merge 3 commits into
Open
Write the rules out over several files, so RULES=c can be built 32-bit#6masonasons wants to merge 3 commits into
masonasons wants to merge 3 commits into
Conversation
Takes a path to a library and a line of text, prints the sample count and a checksum of the samples. Built for both word sizes, it lets a sixty-four bit OpenEVV build and IBM's thirty-two bit library be asked the same question -- which is how the text-mode divergence in the punctuation was found. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
eciSetParam(eciSampleRate) rebuilt the output as a device whatever it had been. For an instance that had registered a buffer with eciSetOutputBuffer that was fatal and silent: ev_setOutputToDevice hands the engine a null buffer on its way past WHERE_SAMPLES, so the caller's was forgotten, and the instance went on reporting the new rate while never answering another sample. Setting the rate back did not recover it -- that lost the buffer a second time -- and neither did setting it to the value it already had. ev_sendChangedEnvironment has always chosen what to rebuild by looking at OI_WHERE. This makes ev_setParam make the same choice. Before: six rate changes, six silent. After: all six speak, at the right length for the rate they were set to. The audio is otherwise unchanged -- identical SHA-256 over three voices and seven texts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All of them in one file is thirteen megabytes, and a thirty-two bit compiler runs out of address space on it -- at -O1 as much as at -O2 -- so RULES=c could not be built thirty-two bit at all. That matters because interpreted the engine takes about five times as long to produce the first audio of an utterance as IBM's own library does, and a thirty-two bit host had no way out of it. tools/delta-decompile.py now takes --parts=N, and the Makefile passes RULE_PARTS. The rules are shared out by how big they are rather than how many there are: they are nothing like the same size, and dividing by count put a quarter of the whole thirteen megabytes into one file, which is the thing this is here to avoid. By size the largest piece is 795 KB. Each piece carries the global offsets, because each has rules that reach through them. The rules become linkable rather than static, since the table naming them is elsewhere -- in the file the Makefile already knew about, which now also carries the offsets and a declaration for every rule. Thirty-two bit RULES=c now builds: 41 seconds with -j8, where before it would not compile at all. Sixty-four bit goes from 4m20s to 50s, the pieces compiling at the same time as each other. The audio is unchanged either way: identical SHA-256 over three voices and seven texts against the single-file build. make distclean removes the pieces, so a later run with fewer of them cannot leave one behind to be compiled and define its rules a second time. They are ignored rather than kept, like the single file was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
masonasons
force-pushed
the
feat/split-rules
branch
from
August 22, 2026 05:06
fd8866a to
084dd6d
Compare
Author
|
Force-pushed: the first push had the generated pieces committed by mistake -- 388,581 lines of them. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RULES=ccannot be built for a thirty-two bit host at all:All 3377 rules go into one file of 12.9 MB and a thirty-two bit compiler runs
out of address space on it, at
-O1as much as at-O2. That matters becauseinterpreted, the engine takes about five times as long to produce the first
audio of an utterance as IBM's own library does (#3), and a thirty-two bit
host had no way out of it.
tools/delta-decompile.pynow takes--parts=N, and the Makefile passesRULE_PARTS, 16 by default.Shared out by size, not by count. The rules are nothing like the same
size: dividing them evenly by count put 3.7 MB of the 13 into one file, which
is the thing this is here to avoid. By size the largest piece is 795 KB.
What changed in the generated code. Each piece carries the
DG_*globaloffsets, because each has rules that reach through them. The rules are
linkable rather than
static, since the table that names them is in anotherfile -- and that file is the one the Makefile already knew about, now
carrying the offsets, a declaration for every rule, and the table.
make distcleanremoves the pieces. Without it a later run with fewer partswould leave one behind to be compiled, defining its rules a second time.
Results. Thirty-two bit
RULES=cbuilds, in 41 s with-j8, where beforeit did not compile. Sixty-four bit drops from 4m20s to 50s, because the
pieces compile at the same time as each other.
The audio is unchanged. Identical SHA-256 over three voices and seven texts
against the single-file build, sixty-four bit; and
RULES=cwas alreadyverified identical to
RULES=bytecodeover the same cases.tools/engine_probe.ccomes with it: it takes a path to aneci.dlland aline of text and prints the sample count and a checksum, built for either word
size, so a build here and IBM's own library can be asked the same question.
That is what the comparisons above and in #3 and #4 were made with.