Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Indonesian model #2752

Merged
merged 15 commits into from
Sep 14, 2018
Merged

Update Indonesian model #2752

merged 15 commits into from
Sep 14, 2018

Commits on Sep 10, 2018

  1. adding e-KTP in tokenizer exceptions list

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    a62b8db View commit details
    Browse the repository at this point in the history

  2. add exception token

    @williamgun007 @aongko
    williamgun007 authored and aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    87c8515 View commit details
    Browse the repository at this point in the history

  3. removing lines with containing space as it won't matter since we use … 

    ….split() method in the end, added new tokens in exception
    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    520cdf8 View commit details
    Browse the repository at this point in the history

  4. add tokenizer exceptions list

    @irchanbani @aongko
    irchanbani authored and aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    17a0308 View commit details
    Browse the repository at this point in the history

  5. combining base_norms with norm_exceptions

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    775ff27 View commit details
    Browse the repository at this point in the history

  6. adding norm_exception

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    06bc1bc View commit details
    Browse the repository at this point in the history

  7. fix double key in lemmatizer

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    19cab8b View commit details
    Browse the repository at this point in the history

  8. remove unused import on punctuation.py

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    f60e013 View commit details
    Browse the repository at this point in the history

  9. reformat stop_words to reduce number of lines, improve readibility

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    536ba22 View commit details
    Browse the repository at this point in the history

  10. updating tokenizer exception

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    a37e68b View commit details
    Browse the repository at this point in the history

  11. implement is_currency for lang/id

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    57c43fd View commit details
    Browse the repository at this point in the history

  12. adding orth_first_upper in tokenizer_exceptions

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    d4acfe2 View commit details
    Browse the repository at this point in the history

  13. update the norm_exception list

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    5dd82ad View commit details
    Browse the repository at this point in the history

  14. remove bunch of abbreviations

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    8aa806d View commit details
    Browse the repository at this point in the history

  15. adding contributors file

    @aongko
    aongko committed Sep 10, 2018
    Configuration menu
    Copy the full SHA
    ca8e2f0 View commit details
    Browse the repository at this point in the history