Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ParametricAttention.v2 #913

Merged
merged 7 commits into from
Dec 14, 2023
Merged

Add ParametricAttention.v2 #913

merged 7 commits into from
Dec 14, 2023

Commits on Dec 12, 2023

  1. Add ParametricAttention.v2 

    This layer is an extension of the existing `ParametricAttention` layer,
adding support for transformations (such as a non-linear layer) of the
key representation. This brings the model closer to the paper that
suggested it (Yang et al, 2016) and gave slightly better results in
experiments.
    @danieldk
    danieldk committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    a2e178f View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Use noop for when key_transform is None

    @danieldk
    danieldk committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    f394c84 View commit details
    Browse the repository at this point in the history

  2. Remove stray import

    @danieldk
    danieldk committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    137a457 View commit details
    Browse the repository at this point in the history

  3. Add constant for key transform ref

    @danieldk
    danieldk committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    e92d581 View commit details
    Browse the repository at this point in the history

  4. Check that we correctly set the key transform

    @danieldk
    danieldk committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    06be6dd View commit details
    Browse the repository at this point in the history

  5. isooooooort

    @danieldk
    danieldk committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    2369b34 View commit details
    Browse the repository at this point in the history

  6. Update citation to ACL link 

    Co-authored-by: Adriane Boyd <[email protected]>
    @svlandeg @adrianeboyd
    svlandeg and adrianeboyd authored Dec 13, 2023
    Configuration menu
    Copy the full SHA
    80f47b7 View commit details
    Browse the repository at this point in the history