DEV_NOTES.md

Notes for development

Unorganized notes which might be useful during development.

Releasing

Release process for tagged versions

These are the "regular" versions, e.g., 0.4.0 or 0.4.0-RC1.

1. Commit every change.
2. Make sure there are no untracked files in git.
3. Push any new commits.
4. Wait for CI to become green.
5. Tag the release (e.g., git tag -s "v1.2.3"), but don't push the new tag.
6. Start sbt, and:
   • clean
   • ;staticAnalysis;++3.3.7;staticAnalysis (precompile seems to help publish faster)
   • exit
7. In a new sbt shell:
   • release (this step requires Sonatype credentials)
8. If everything looks right, push the new tag (git push --tags).
9. Create a "release" on github for the new tag.

Release process for "hash" versions

These are "preview" versions, e.g., 0.4-39d987a or 0.4.3-2-39d987a.

1. Commit every change.
2. Make sure there are no untracked files in git.
3. Push any new commits.
4. In sbt, call releaseHash (requires Sonatype credentials).

Development

Lincheck

A warning like this appears when running tests in stressLinchk:

OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

According to https://stackoverflow.com/a/57957031, this is harmless.

Exceptions

Some types of exceptions must never be thrown, because they are (probably) undefined behavior on Scala.js:

• ArrayIndexOutOfBoundsException and StringIndexOutOfBoundsException: use IndexOutOfBoundsException instead. Indexing into an array (or string, though that's not typical) must check the index; see jsCheckIdx, which is a NOP in Scala, but checks the index (and throws if necessary) in Scala.js.
• NegativeArraySizeException: when allocating an array, care must be taken, so the length is non-negative; if it is, throw an AssertionError instead.

The following types of exceptions are allowed, even though they are similarly UB in Scala.js. (Nevertheless, they should not be thrown directly, as throw new ....) The users are recommended to use the corresponding Scala.js linker settings, or otherwise avoid the situations which can lead to these exceptions.

• NullPointerException: users are directed to never pass null to library methods, or use the Compliant Scala.js linker setting (null as a payload is allowed). See the [README.md].

Further reading: https://www.scala-js.org/doc/semantics.html#undefined-behaviors

Historical decisions

Older versions used to have (sometimes significantly) different API and internals. Some of the decisions to change them are documented here.

API

• The choam-mcas module used to be public, and the MCAS algorithm was configurable. Now it is private, as (primarily due to the changes necessary to support opacity) it is not anymore a simple and clean MCAS library, but somewhat intertwined with higher-level (Rxn) concerns. The algorithm is not selectable now, as EMCAS is clearly the fastest (of the ones we tried). In fact, CASN was removed (SpinLockMcas remains, because it is so simple, that it's not a burden to maintain it, and might be useful for testing).
• Rxn used to have two type parameters (i.e., Rxn[-A, +B]), and it formed an arrow (more specifically an ArrowChoice), besides forming a monad in its second type parameter. This was "inherited" from the Reagents paper. It was interesting, and sometimes useful, but rarely used in practice. In theory, using a static structure of Rxns combined with the arrow combinators could have a better performance than using monadic composition (which typically needs to allocate always new Rxn instances). In practice though, benchmarking showed that these performance wins are very small in semi-realistic situations (typically 2-3%), and it is somewhat non-intuitive whether it's even worth using arrow combinators instead of the monadic ones (in some very similar situations the arrow ones are slower). Moreover, for performance optimization the "unsafe API" (dev.tauri.choam.unsafe) proved much more effective (10-15% performance improvement in the same situations). For this reason (and to simplify the API and implementation), the input type parameter of Rxn was removed. (Note, that it still forms a monad.)

Internals

• EMCAS used to employ an IBR (interval-based reclamation) scheme to determine if a descriptor is still in use by a helper. This was replaced by using the JVM GC for this purpose (by using weakrefs). This solution proved to be faster (although it was necessary to "reuse" weakrefs to avoid having too much of them, because that slows down the GC; see getReusableMarker/getReusableWeakRef).
• The Rxn interpreter used to match on Int tags (JVM tableswitch). This was inspired by an old optimization in the Scala compiler for matching on sealed subclasses (see https://github.com/scala/scala/commit/b98eb1d74141a4159539d373e6216e799d6b6dcd). The Cats Effect runloop is doing something very similar, and ZIO also used to do something like this in version 1. This was removed from Rxn, and now it's a simple match on a sealed type. This way it's easier to maintain (e.g., we get non-exhaustive match warnings), and some benchmarking showed that it might even be slightly faster this way.
• An even older version of Rxn was based on calling continuations recursively (like in https://github.com/aturon/ChemistrySet). That was not stack-safe, so it was changed to a stack-safe interpreter.
• Refs used to have a 256-bit ID (4 longs generated by ThreadLocalRandom) out of fear of collisions (which would be catastrophic). Now they are generated deterministically (but pseudorandomly), so 64 bits (a long) should be enough (see ṘefIdGen).

System properties

• dev.tauri.choam.stats
  • enable/disable performance stats collection (available through JMX)
  • specify a Boolean (default: false)
• dev.tauri.choam.internal.mcas.impl
  • override the MCAS impl
  • specify the name of an Mcas class, e.g., "Emcas" (default: "")