Skip to content

Conversation

JoJoBarthold2
Copy link
Collaborator

Adds a little to the documentation and also puts the dropout to 0.7 in the Embedder Model so that Dropout is corrected in the .eval mode of the mode.
It is currently untested if this works

Merge readthe docs changes into my training documentation branch
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR aims to update the training documentation and correct the dropout behavior in the Embedder Model by setting the dropout rate to 0.7 during instantiation.

  • Updates the dropout parameter when creating the EmbeddingModel instance
  • Improves documentation regarding the dropout change in eval mode
Files not reviewed (3)
  • README.rst: Language not supported
  • docs/source/index.rst: Language not supported
  • docs/source/training.rst: Language not supported
Comments suppressed due to low confidence (1)

paule/paule.py:165

  • Consider adding test cases to verify that the new dropout parameter behaves as expected in eval mode.
self.embedder = EmbeddingModel(num_lstm_layers=2, hidden_size=720, dropout=0.7).double()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant