Skip to content

Fix list-to-array conversion for integer arrays - #1225

Merged
jgabry merged 6 commits into
masterfrom
fix-817-int-container-data
Jul 25, 2026
Merged

Fix list-to-array conversion for integer arrays#1225
jgabry merged 6 commits into
masterfrom
fix-817-int-container-data

Conversation

@jgabry

@jgabry jgabry commented Jul 24, 2026

Copy link
Copy Markdown
Member

Submission Checklist

  • Run unit tests
  • Declare copyright holder and agree to license (see below)

Summary

This PR and the description below was co-written by me and Claude (most of the code was written by me directly and checked by claude, most of the text was written by Claude directly and checked by me)

Fixes #817.

process_data() makes sure variables declared as int are of integer type by calling mode<- on them, which errors on lists and data frames. The list-to-array conversion that would make those inputs atomic happens later, in write_stan_json(), so it never had a chance to run. The result was that the documented list convenience worked only for real variables: a list of matrices for array[K,I,J] real was fine, while the same list for array[K,I,J] int failed. Data frames failed for int variables for the same reason.

This moves the container conversions (table, logical, data.frame, list) out of write_stan_json() into a convert_to_array() helper shared by both functions, and calls it in process_data() before the length-1 and int-coercion checks. Two smaller things fall out of the reordering: the length-1 check now measures the converted array rather than counting list elements, and a list containing NA reports "has NA values" instead of failing inside a coercion.

While in there we also tightened three cases where the same value was accepted in one position and rejected or silently coerced in another:

  • Data frame columns. data.matrix() silently converted character columns to factor codes and date/time columns to their numeric representation, so a value the data list rejects as a variable was accepted as a column. A character column reached Stan as alphabetically ordered integer codes. The variable-level type check is now applied per column.
  • Logical list elements. list_to_array() rejected them even though logical variables and logical data frame columns are both accepted. They are now allowed, and the logical-to-inr the container conversions so a list of logicals isnot written as JSON true/false.
  • Factors. A factor supplied for a variable not declared as int silently became level indices, which are not meaningful as continuous data. That is now an error. Factors for int variables are unchanged, since level indices are exactly what a grouping variable wants (Fix factor data handling #999).

Documentation: factor handling was not documented anywhere, so write_stan_json() now describes it, including that the indices depend on level order rather than on the values (factor(c(10, 9, 8)) is written as [3, 2, 1]). It also corrects the claim that a list cannot be used for arrays with more than one dimension. A list contributes exactly one leading dimension, so array[K,L] vector[J] v can be supplied as a list of K LxJ matrices; the real limitation is that nested lists are not supported.

One limitation is inherent rather than deferred. write_stan_json() called directly cannot do the factor check, because it has no variable declarations to compare against, so it still converts factors silently. Only the fitting methods of a model compiled from a Stan file have the information needed. This is noted in the write_stan_json() documentation.

Copyright and Licensing

Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
Jonah Gabry

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

jgabry added 4 commits July 24, 2026 12:46
process_data() coerced variables declared as int with mode<-, which
errors on lists and data frames. The list-to-array conversion happens
later, in write_stan_json(), so the documented list convenience worked
only for real variables. Extract the container conversions into
convert_to_array(), shared by both functions, and apply them in
process_data() before the length-1 and int-coercion checks.

Document factor conversion, which was undocumented, and correct the
claim that lists cannot be used for arrays with more than one
dimension. A list contributes exactly one leading dimension, so
array[K,L] vector[J] can be supplied as a list of K LxJ matrices;
the real limitation is that nested lists are not supported.

closes #817
data.matrix() silently coerced character columns to factor codes and
date/time columns to numeric, so a value the data list would reject as
a variable was accepted as a column. Apply the same type check per
column.
list_to_array() rejected logical elements even though logical
variables and logical data frame columns are both accepted. Allow
them, and apply the logical-to-integer conversion after the container
conversions so a list of logicals is not written as JSON true/false.

Factors are written as their level indices, which are only meaningful
for variables declared as int. Error when one is supplied for any
other type.
@jgabry jgabry changed the title Fix 817 int container data Fix list-to-array conversion for integer arrays Jul 24, 2026
array() drops the factor class, so the length-1 reshaping left a
character array behind and factors failed for array[1] int. The
factor check also ran after data.matrix(), so a factor column
supplied for a real variable was silently converted to level codes.

Move factor handling ahead of both conversions: convert to integer for int variables, and error for any other type, including data frames with factor columns. Describe level ordering in terms of levels() rather than "alphabetical", which is wrong for numeric
input, and correct the list element error message now that logical
elements are accepted.
@codecov-commenter

codecov-commenter commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.90%. Comparing base (d83f62d) to head (4156dc9).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1225      +/-   ##
==========================================
+ Coverage   91.79%   91.90%   +0.11%     
==========================================
  Files          15       15              
  Lines        6200     6260      +60     
==========================================
+ Hits         5691     5753      +62     
+ Misses        509      507       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jgabry
jgabry merged commit cd4bb05 into master Jul 25, 2026
14 of 15 checks passed
@jgabry
jgabry deleted the fix-817-int-container-data branch July 25, 2026 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot supply list of matrices to populate 3 dimensional array int

2 participants