All pull requests

Pull requests

• fix(tk-train): ParityBpeTrainer: train into Vocab and Merges

  #2474·SBrandeis opened Sep 30, 2026Collaborator·

  ·

• chore(tk-train): fmt, remove dead code, stale docs, and unused tk-convert dependency

  #2473·SBrandeis opened Sep 30, 2026Collaborator·

  ·
  1

• fix(tk-encode): remove unicode-normalization-alignments dependency

  #2472·SBrandeis opened Sep 30, 2026Collaborator·

  ·
  1

• chore(tk-encode): drop itertools, paste and unstable_wasm

  #2471·SBrandeis opened Sep 30, 2026Collaborator·

  ·
  1

• chore(dx): tk-train in workspace, make the umbrella crate more convenient

  #2470·SBrandeis opened Sep 30, 2026Collaborator·

  ·
  1

• feat(encode): honour truncation stride as Encoding::overflowing, derive masks from a layout

  #2468·ysjprojects opened Sep 29, 2026·

  ·

• fix(BucketVocabStore): Explicit duplicate handling

  #2467·SBrandeis opened Sep 28, 2026Collaborator·

  ·
  1

• fix(byte-level): Support sparse byte atoms

  #2466·SBrandeis opened Sep 28, 2026Collaborator·

  ·
  2

• chore(bitcannon): drop unused dependency, lint

  #2465·SBrandeis opened Sep 28, 2026Collaborator·

  ·
  2

• fix(python): xxx_array getters copy the underlying data

  #2464·SBrandeis opened Sep 28, 2026Collaborator·

  ·
  1

  3

• chore(deps): bump the actions group with 4 updates

  dependenciesgithub_actions
  #2463·dependabot[bot] opened Sep 27, 2026Bot·

  ·
  1

• Support ByteLevel add_prefix_space during canonicalization

  #2462·ducanhnguyen223 opened Sep 27, 2026·

  ·
  1

  1

• Load byte-fallback BPE vocabs that lack some <0xNN> codes

  #2461·daaain opened Sep 26, 2026·

  ·

• fix: make tk-train compile and add a CI workflow

  #2457·SBrandeis opened Sep 25, 2026Collaborator·

  ·
  1

• chore(cleanup): Remove dead code, rename, simplify

  #2455·SBrandeis opened Sep 24, 2026Collaborator·

  ·
  1

• feat(python): role_to_token

  #2454·SBrandeis opened Sep 24, 2026Collaborator·

  ·
  1

• feat(python): Support encoding pairs of text

  #2453·SBrandeis opened Sep 24, 2026Collaborator·

  ·
  1

• chore(python): Remove unecessary encoding copy at the rust-python boundary

  #2452·SBrandeis opened Sep 24, 2026Collaborator·

  ·
  3

• Load sparse byte-level BPE vocabularies without fallback policy

  #2451·dlowzzxx opened Sep 23, 2026·

  ·

• feat: bindings for WebAssembly

  #2450·SBrandeis opened Sep 23, 2026Collaborator·

  ·
  7

• Pass workflow expressions through the environment

  #2449·hf-security-analysis[bot] opened Sep 23, 2026Bot·

  ·
  1

• chore(release): bump dev version + trigger the release workflow on release/* branches

  #2446·SBrandeis opened Sep 21, 2026Collaborator·

  ·
  3

• fix(tk-encode): longest-prefix matching in PrecompiledNormalizer

  #2430·SIDDARTHAREDDY8 opened Sep 20, 2026·

  ·
  1

  1

• docs: fix 'deserilized' typo in the utils doc comment

  #2429·simpleqt opened Sep 18, 2026·

  ·
  1

• docs(python): drop the dead free-threading-audit.md link

  #2428·simpleqt opened Sep 18, 2026·

  ·
  1