Pre-trained HF Aggregation: Config files for all conditions

[jlunder00]

Task 5: Configuration Files

Depends on: #6 (train_unified.py changes)

Create ~24 config files

All in Tree_Matching_Networks/LinguisticTrees/configs/experiment_configs/.

Use existing prop_heavy configs as templates — copy and modify:

• Contrastive template: ttn_embedding_prop_heavy_contrastive_config.yaml
• Primary template: ttn_embedding_prop_heavy_snli_primary_config.yaml
• Finetune template: ttn_embedding_prop_heavy_snli_finetune_config.yaml

Config Differences by Condition

Condition A (text mode, Phase 3 only)

• Files: pretrained_text_{embedding,matching}_snli_{primary,finetune}_config.yaml
• text_mode: true
• model.name: pretrained_text
• Add model.pretrained.model_name: "sentence-transformers/all-MiniLM-L6-v2"
• No model.graph.pretrained_transformer section needed
• Keep model.graph section for graph_rep_dim: 2048
• train.pretrained_lr_scale: 1.0 (full LR, everything trains)

Condition B (tree mode, no prop, Phase 3 only)

• Files: pretrained_noprop_{embedding,matching}_snli_{primary,finetune}_config.yaml
• text_mode: false
• model.name: pretrained_noprop
• Add model.graph.pretrained_transformer block (see below)
• Keep full model.graph section (node_feature_dim=804 used by aggregator)
• train.pretrained_lr_scale: 0.1

Condition D (full pipeline, both train)

• Files: pretrained_tree_{embedding,matching}_{contrastive,snli_primary,snli_finetune}_config.yaml
• text_mode: false
• model.name: pretrained_tree
• model.strict_checkpoint_load: false
• Full model.graph section (identical to prop_heavy)
• Add model.graph.pretrained_transformer block
• train.pretrained_lr_scale: 0.1

Condition E (frozen transformer)

• Files: pretrained_tree_frozen_xfmr_{embedding,matching}_{contrastive,snli_primary,snli_finetune}_config.yaml
• Same as D but add:

  model:
    freeze:
      freeze_transformer: true

• train.pretrained_lr_scale: 0.0 (zero, transformer is frozen)

Condition F (frozen GNN, Phase 3 only)

• Files: pretrained_tree_frozen_gnn_{embedding,matching}_snli_{primary,finetune}_config.yaml
• Same as D but add:

  model:
    freeze:
      freeze_propagation: true
    strict_checkpoint_load: false  # Loading existing GNN checkpoint

• train.pretrained_lr_scale: 1.0 (transformer trains fully)

Common pretrained_transformer Block

Add to model.graph for conditions B/D/E/F:

    pretrained_transformer:
      model_name: "sentence-transformers/all-MiniLM-L6-v2"
      max_nodes: 64
      use_cls_token: false
      cls_token_type: "virtual"
      freeze_transformer: false  # Overridden by model.freeze for E
      positional_features:
      - depth
      - num_siblings
      - num_children
      - num_grandparent_children
      - subtree_size
      - parent_num_children
      - distance_to_leaf
      - nodes_at_level
      positional_max_values:
        depth: 15
        num_siblings: 10
        num_children: 10
        num_grandparent_children: 10
        subtree_size: 40
        parent_num_children: 10
        distance_to_leaf: 10
        nodes_at_level: 20

Wandb Tags

Use tags like: pretrained, condition_a/condition_b/etc, embedding/matching, stage name

Tip

Write a script to generate configs from templates to avoid copy-paste errors. The differences between configs are small and systematic.