Skip to content

Conversation

@meilame-tayebjee
Copy link
Member

This pull request updates the handling of label attention configuration and tokenizer saving in the torchTextClassifiers codebase. The main changes simplify and clarify how label attention is configured and instantiated, and ensure compatibility with the tokenizer saving method.

Label Attention Configuration Updates:

  • The ModelConfig class now uses n_heads_label_attention (an integer) instead of the previous label_attention_config object, and makes num_classes a required field. This simplifies configuration and ensures that the number of classes is always specified.
  • The model initialization now enables label attention if n_heads_label_attention is set, rather than checking for a full config object.
  • When constructing the TextEmbedderConfig, a new LabelAttentionConfig is created on the fly using n_heads_label_attention and num_classes from the model config, instead of passing through a possibly pre-existing config object.
  • The model loading logic no longer attempts to reconstruct label_attention_config from a dictionary, since label attention is now configured via n_heads_label_attention.

Tokenizer Saving:

  • The tokenizer is now saved using the save_pretrained method instead of save, ensuring compatibility with Hugging Face-style tokenizers.

not expose LabelAttentionConfig and build it directly from the wrapper
only provide num_heads
@meilame-tayebjee meilame-tayebjee force-pushed the fix_label_att_parametrization branch from 100815a to ca2ba2a Compare February 9, 2026 15:43
@meilame-tayebjee meilame-tayebjee merged commit 4577614 into main Feb 9, 2026
1 of 4 checks passed
@meilame-tayebjee meilame-tayebjee deleted the fix_label_att_parametrization branch February 9, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant