show textbooks

[abeglova]

What are the relevant tickets?

closes https://github.com/mitodl/hq/issues/12337

Description (What does it do?)

OCW textbooks are ready to be made visible to all users in production. We want to create them regardless of CREATE_OCW_LEARNING_MATERIALS and show them in the main search without requiring show_ocw_files=true.

How can this be tested?

Set CREATE_HIDDEN_OCW_LEARNING_MATERIALS=True

Run
docker-compose run web ./manage.py backpopulate_ocw_data --course-name 6-011-introduction-to-communication-control-and-signal-processing-spring-2010

go to http://open.odl.local:8062/search?resource_type_group=learning_material. You should see "Open Textbook" as an option

go to http://open.odl.local:8062/search?resource_type_group=learning_material&show_ocw_files=true You should also see "Practice and Assignment"

Set CREATE_HIDDEN_OCW_LEARNING_MATERIALS=False

Run
docker-compose run web ./manage.py update_ocw_learning_material_resources

go to http://open.odl.local:8062/search?resource_type_group=learning_material. You should still see "Open Textbook" as an option

go to http://open.odl.local:8062/search?resource_type_group=learning_material&show_ocw_files=true You should no longer see "Practice and Assignment"

run
docker-compose run web ./manage.py backpopulate_ocw_data --course-name 6-011-introduction-to-communication-control-and-signal-processing-spring-2010

search should not change

Set CREATE_HIDDEN_OCW_LEARNING_MATERIALS=True again

Run
docker-compose run web ./manage.py update_ocw_learning_material_resources

Go to
go to http://open.odl.local:8062/search?resource_type_group=learning_material&show_ocw_files=true You should see "Practice and Assignment" again

[github-actions]

OpenAPI Changes

No changes detected

View full changelog

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).

[Copilot]

Pull request overview

This PR updates the OCW “learning materials” ingestion and search behavior so that OCW Open Textbooks are created and surfaced by default, while other OCW learning-material file types remain gated behind a new “hidden materials” feature flag.

Changes:

• Always run OCW learning-material promotion, but only promote Open Textbooks by default; promote additional OCW categories only when CREATE_HIDDEN_OCW_LEARNING_MATERIALS=True.
• Update OCW ETL fixtures/tests to include a textbook content file and validate the resulting learning-material resources.
• Adjust search filtering so that OCW Open Textbooks are included even when show_ocw_files is false.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test_json/courses/16-01-unified-engineering-i-ii-iii-iv-fall-2005-spring-2006/resources/textbook/data.json	Adds OCW fixture data for a textbook resource.
main/settings.py	Replaces prior flag with CREATE_HIDDEN_OCW_LEARNING_MATERIALS for hidden OCW learning materials.
learning_resources/tasks.py	Removes the “flag disabled” runtime guard so the task can run unconditionally.
learning_resources/tasks_test.py	Updates task test to match unconditionally-running behavior.
learning_resources/etl/pipelines.py	Always calls load_learning_materials during OCW ETL.
learning_resources/etl/pipelines_test.py	Updates ETL expectations for new textbook file and hidden-material flag behavior.
learning_resources/etl/ocw_test.py	Updates OCW content-file transform expectations to include textbook fixture.
learning_resources/etl/loaders.py	Changes learning-material promotion logic to allow textbooks by default and gate additional categories behind the new flag.
learning_resources/etl/loaders_test.py	Updates promotion tests for new flag/behavior.
learning_resources/constants.py	Introduces the “visible” OCW categories list used when hidden materials are disabled.
learning_resources_search/api.py	Includes Open Textbooks in results when show_ocw_files is false.
learning_resources_search/api_test.py	Updates search filter-clause test expectations accordingly.

[abeglova]

fixed

[mbertrand]

Since this task is now the mechanism for rolling hidden materials out (and back) in production, it could use a guard for a published course that has no published run — in that case course_run ends up as None and the error on the next line stops the loop for all remaining courses (it happens before the try/except). Something like if not course_run: continue, or moving these lines inside the try, would keep one bad course from halting the whole run.

[abeglova]

done

[mbertrand]

Now that this runs unconditionally, there's a case that used to be gated off: when load_content_files takes its "preserving existing content files" path and returns an empty list, this call will still run and clear all of the course's learning-material relationships at the end (resources.set([])), even though the promoted resources stay published. Wrapping this in if content_file_ids: would keep the preserve behavior intact.

[abeglova]

done

[mbertrand]

👍