Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/lucene search engine #2892

Open
wants to merge 95 commits into
base: main
Choose a base branch
from
Open

Feature/lucene search engine #2892

wants to merge 95 commits into from

Commits on May 31, 2024

  1. Elasticsearch code.

    phix committed May 31, 2024
    Configuration menu
    Copy the full SHA
    f34e016 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Adding basic responses for elasticsearch.

    phix committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    70c4ce0 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Saving highlights.

    phix committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    51a5fa8 View commit details
    Browse the repository at this point in the history

  2. Saving code cleanup.

    phix committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    52606f4 View commit details
    Browse the repository at this point in the history

  3. Adding EsSearch.

    phix committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c489de4 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Saving partial progress.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    f05bfb1 View commit details
    Browse the repository at this point in the history

  2. Refinements.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    b5667f2 View commit details
    Browse the repository at this point in the history

  3. Small bug fixes.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    e78bafb View commit details
    Browse the repository at this point in the history

  4. Fixing alignment

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    9f9abfe View commit details
    Browse the repository at this point in the history

  5. Migrating es jobs naming to be specific.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    545f7cb View commit details
    Browse the repository at this point in the history

  6. Adding boilerplate for dataset es search

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    ec4c0c6 View commit details
    Browse the repository at this point in the history

  7. Adding datasets.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    897f1be View commit details
    Browse the repository at this point in the history

  8. Adding polish for more data.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    38c8db6 View commit details
    Browse the repository at this point in the history

  9. Empty state and other small enhancements.

    phix committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    6713e1d View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Adding arrow key functionality.

    phix committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    bb5ce2d View commit details
    Browse the repository at this point in the history

  2. Removing console log

    phix committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    cffd9cc View commit details
    Browse the repository at this point in the history

  3. Spotless

    phix committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    6f62898 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Refinements to queries.

    phix committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    4a5f1aa View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Adding debounce.

    phix committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    40bc497 View commit details
    Browse the repository at this point in the history

  2. Fixing alignment issues.

    phix committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    19c76c8 View commit details
    Browse the repository at this point in the history

  3. Saving updates for password setting via env config for elasticsearch.

    phix committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c7f5860 View commit details
    Browse the repository at this point in the history

  4. Setting up startup scripts and adding corresponding waits.

    phix committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    fd37df9 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Adding logs and more fields for jobs.

    phix committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    0a4cdcc View commit details
    Browse the repository at this point in the history

  2. Resolving jackson serialization issue.

    phix committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    f9023fb View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Small updates for search display.

    phix committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    96ab8f8 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Adding onClick handlers.

    phix committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    e828b2b View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. Fixing null cases, adding more search options for datasets.

    @phixMe
    phixMe committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    5133c42 View commit details
    Browse the repository at this point in the history

  2. Handling enter key.

    @phixMe
    phixMe committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    9241307 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Fixing minor encoding and layout issues for spark related open lineag… 

    …e events.
    @phixMe
    phixMe committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    259fea7 View commit details
    Browse the repository at this point in the history

  2. Additional fixes for text overflow on names and namespaces.

    @phixMe
    phixMe committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    5440917 View commit details
    Browse the repository at this point in the history

  3. Fixing indexing problem.

    @phixMe
    phixMe committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    df43ab8 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. Transitioning to opensearch.

    @phixMe
    phixMe committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    b6f4691 View commit details
    Browse the repository at this point in the history

  2. Removing elasticsearch references.

    @phixMe
    phixMe committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    82e00e2 View commit details
    Browse the repository at this point in the history

  3. Isolation of search code, calling services.

    @phixMe
    phixMe committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    37e80ed View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. Adding config to support multiple instances.

    @phixMe
    phixMe committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    843baaf View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. Spotless

    @phixMe
    phixMe committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    4faba34 View commit details
    Browse the repository at this point in the history

  2. Adding helm files.

    @phixMe
    phixMe committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    df0bc84 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Adding in stronger password for search.

    @phixMe
    phixMe committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    bf459ef View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Handling debouncing.

    @phixMe
    phixMe committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    5d40b2e View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Adding "ADVANCED_SEARCH" configurable variable for web.

    @phixMe
    phixMe committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    085a3ef View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Fixing some tests.

    @phixMe
    phixMe committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    635c27b View commit details
    Browse the repository at this point in the history

  2. Moving indexing down a row.

    @phixMe
    phixMe committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    df446e7 View commit details
    Browse the repository at this point in the history

  3. Spotless

    @phixMe
    phixMe committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    a8cee81 View commit details
    Browse the repository at this point in the history

  4. Putting back removed code.

    @phixMe
    phixMe committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    986cd22 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. Merge branch 'refs/heads/main' into feature/es-client 

    # Conflicts:
#	.env.example
#	api/src/main/java/marquez/MarquezApp.java
#	build.gradle
#	web/src/routes/events/Events.tsx
#	web/src/routes/jobs/Jobs.tsx
    @phixMe
    phixMe committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    fc1bc5d View commit details
    Browse the repository at this point in the history

  2. Merge spotless resolution.

    @phixMe
    phixMe committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    3db2bda View commit details
    Browse the repository at this point in the history

  3. Skipping over search for db migration tests.

    @phixMe
    phixMe committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    22c0be4 View commit details
    Browse the repository at this point in the history

  4. Adding search back to migration

    @phixMe
    phixMe committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    cd3fa50 View commit details
    Browse the repository at this point in the history

  5. Trying out ci config setting.

    @phixMe
    phixMe committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    06dadb3 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. Removing search from base config as a whole.

    @phixMe
    phixMe committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    04d9546 View commit details
    Browse the repository at this point in the history

  2. Pushing out header updates.

    @phixMe
    phixMe committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    038cfc9 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. Review comment on search service init.

    @phixMe
    phixMe committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    2767eda View commit details
    Browse the repository at this point in the history

  2. Fixing up dependencies in docker to apply migrations.

    @phixMe
    phixMe committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    ec4ab41 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Code review updates and naming changes.

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    593d7b8 View commit details
    Browse the repository at this point in the history

  2. newline

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    d13bb06 View commit details
    Browse the repository at this point in the history

  3. Updating for beta vs. non beta endpoints in search resource.

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    d099b77 View commit details
    Browse the repository at this point in the history

  4. Moving search resource to its own place.

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    e8bfb3b View commit details
    Browse the repository at this point in the history

  5. Removing prints.

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    7608b58 View commit details
    Browse the repository at this point in the history

  6. Removing all helm changes for this work stream.

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    2c9cd6c View commit details
    Browse the repository at this point in the history

  7. Adding back lock file contents.

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    3cb90b2 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into feature/es-client

    @phixMe
    phixMe authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    247b79d View commit details
    Browse the repository at this point in the history

  9. Adding header

    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    4158d54 View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'origin/feature/es-client' into feature/… 

    …es-client
    @phixMe
    phixMe committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    e672bf1 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Adding middleware proxy.

    @phixMe
    phixMe committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f05bd15 View commit details
    Browse the repository at this point in the history

  2. Code review updates.

    @phixMe
    phixMe committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    61672cd View commit details
    Browse the repository at this point in the history

  3. Moving from outer gradle to api gradle.

    @phixMe
    phixMe committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    fdeaaa1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'refs/heads/main' into feature/es-client 

    # Conflicts:
#	.env.example
#	docker-compose.web.yml
    @phixMe
    phixMe committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f04e9b1 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2024

  1. Removing extra containers.

    @phixMe
    phixMe committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    dfd60e6 View commit details
    Browse the repository at this point in the history

  2. Removing extra containers.

    @phixMe @wslulciuc
    phixMe authored and wslulciuc committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    d4649ff View commit details
    Browse the repository at this point in the history

  3. Resolve merge conflicts 

    Signed-off-by: wslulciuc <[email protected]>
    @wslulciuc
    wslulciuc committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    ac5102b View commit details
    Browse the repository at this point in the history

  4. Set timeout for seed container to 60s 

    Signed-off-by: wslulciuc <[email protected]>
    @wslulciuc
    wslulciuc committed Aug 10, 2024
    Configuration menu
    Copy the full SHA
    b83254c View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. Fixing --no-search and frontend config.

    @phixMe
    phixMe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    73726a7 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/feature/es-client' into feature/… 

    …es-client
    @phixMe
    phixMe committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    907d96e View commit details
    Browse the repository at this point in the history

  3. Add check before indexing ol event 

    Signed-off-by: wslulciuc <[email protected]>
    @wslulciuc
    wslulciuc committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    15c6697 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. feat: add search subproject 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    d47a6c4 View commit details
    Browse the repository at this point in the history

  2. feat: init search engine 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    2c2040c View commit details
    Browse the repository at this point in the history

  3. feat: add healthcheck 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    f6dd50b View commit details
    Browse the repository at this point in the history

  4. feat: replicate Opensearch API behavior 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    b4f6930 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. feat: properly index and search datasets and jobs 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    ecf172a View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. chore: solve conflicts 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    e906d37 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. feat: working search response 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e9609a4 View commit details
    Browse the repository at this point in the history

  2. feat: changing the analyzer to an NGramAnalyzer 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    dc77ead View commit details
    Browse the repository at this point in the history

  3. WIP: load event at startup 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    c506eee View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. WIP: dedup event at startup 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    61631bb View commit details
    Browse the repository at this point in the history

  2. WIP: remove log 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    4fd9535 View commit details
    Browse the repository at this point in the history

  3. WIP: batch load event from lineage_events 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    dda476e View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. WIP: reduce memory footprint by removing duplicates 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    c7921f5 View commit details
    Browse the repository at this point in the history

  2. WIP: remove debug log 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    f37d6b5 View commit details
    Browse the repository at this point in the history

  3. WIP: thread safe index writing 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    00455d6 View commit details
    Browse the repository at this point in the history

  4. WIP: load init events in background 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    b8589bc View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2024

  1. WIP: remove the ERROR StatusLogger due to unnecessary opensearch deps 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    17c0d85 View commit details
    Browse the repository at this point in the history

  2. WIP: spotlessApply 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    bf3808a View commit details
    Browse the repository at this point in the history

  3. WIP: tidy up 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    7422cbd View commit details
    Browse the repository at this point in the history

  4. WIP: add a banner 

    Signed-off-by: Yannick Libert <[email protected]>
    @yanlibert
    yanlibert committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    52a088d View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Merge branch 'MarquezProject:main' into feature/lucene-search-engine

    @yanlibert
    yanlibert authored Sep 9, 2024
    Configuration menu
    Copy the full SHA
    5689a67 View commit details
    Browse the repository at this point in the history