Capturing things outside the env

[keynmol]

Just listened to your talk :)

I asked about the captured bound identifiers which are not part of the env because I cannot make this snippet (scala-cli) fail at compile time:

//> using lib "com.phaller::spores3::0.1.0"

import com.phaller.spores.Spore
import com.phaller.spores.{Spore, SporeData, PackedSporeData}
import com.phaller.spores.upickle.given

val str = "anonymous function"
val BLA = "hello"
val s = Spore(str) { // `str` is the environment of the spore
  env => (x: Int) => x + env.length + BLA.length
}

val z = 25
object MySpore
    extends Spore.Builder[Int, Int, Int](env => (x: Int) => env + x + 1 + z)

import upickle.default._

@main def hello =
  val x = 12
  val data =
    SporeData(MySpore, Some(x)) // `x` is the environment, as before
  val pickledData = write(data)
  val unpickledData = read[PackedSporeData](pickledData)
  val unpickledSpore = unpickledData.toSpore[Int, Int]
  assert(unpickledSpore(3) == 16)

[lolgab]

I think this is hard to impossible to do in a macro at the moment: scala/scala3#19349

[jspenger]

Hi, I encountered the same issue when I first used spores, and it should perhaps be explained better.

The variables str, BLA, and z are all fields in a "top-level" object. These are not captured in the sense that their value (or a reference thereof) is copied into the closure (at least, to my understanding). Instead, it suffices for the closure to keep a static reference to the fields. For this reason it does not cause any issues with pickling/unpickling.

We tried to explain this more in our talk at Scala Days 2025 as it otherwise is not explained elsewhere. One slide shows three examples of what is captured/not captured https://speakerdeck.com/jspenger/simple-and-safe-pickling-of-closures-in-scala-3?slide=4.

I took the liberty to update your code example to work with spores3-0.2.0 on Scastie: https://scastie.scala-lang.org/uIX8hNKsSGCZaAUWNujrUQ.

An example of capturing that is checked and causes a compile-time error looks like this (in the style of spores3-0.2.0) https://scastie.scala-lang.org/tFtJYk5wSlWY61bMkbsnew.

object Main {
  def main(args: Array[String]): Unit = {
    val z = 12
    val spore = Spore[Int => Int] {
      x => x + z
      // Error: Invalid capture of variable `z`. Use the first parameter of a spore's body to refer to the spore's environment.
    }
  }
}

This is fixed by writing it like so (https://scastie.scala-lang.org/Pf8mN0WvRQOeQYcTJAhlAA):

object Main {
  def main(args: Array[String]): Unit = {
    val z = 12
    val spore = Spore.applyWithEnv[Int, Int => Int](z) {
      env => x => x + env
    }
  }
}

[jspenger]

I should note for completeness that a SporeBuilder must itself be defined on a top-level. For this reason, it cannot capture any non-top-level variables.

There is a similar check that causes a compile-time error if a SporeBuilder is not top-level (https://scastie.scala-lang.org/XTA6sHJ9TuO5ZA2wj0RiqA):

object Main {
  def main(args: Array[String]): Unit = {
    val z = 12
    object MySpore extends SporeBuilder[Int => Int]({
      x => x + z
    })
    val spore = MySpore.build()
    // Error: The provided SporeBuilder `Main$._$MySpore$` is not a top-level object; its owner `main` is not a top-level object nor a package.
  }
}