Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/417 File upload integration #464

Merged
merged 33 commits into from
Sep 4, 2024

Conversation

ErykKul
Copy link
Contributor

@ErykKul ErykKul commented Aug 22, 2024

What this PR does / why we need it:

File upload works now, integration test is added

Which issue(s) this PR closes:

Special notes for your reviewer:

Suggestions on how to test this:

Upload some file to a dataset.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

No interface changes.

Is there a release notes update needed for this change?:

Something like "added files upload feature"

Additional documentation:

@coveralls
Copy link

coveralls commented Aug 22, 2024

Coverage Status

coverage: 97.42% (-0.05%) from 97.466%
when pulling f1ff52d on feature/417-upload-integration
into c71e287 on develop.

@g-saracca g-saracca added Size: 10 A percentage of a sprint. 7 hours. integration Tasks involving the connection and interaction of UI features with the Dataverse API SPA: File Upload Page GREI Re-arch GREI re-architecture-related FY25 Sprint 4 FY25 Sprint 4 labels Aug 23, 2024
@g-saracca g-saracca changed the title Feature/417 upload integration Feature/417 File upload integration Aug 23, 2024
@g-saracca g-saracca self-assigned this Aug 23, 2024
Copy link
Contributor

@g-saracca g-saracca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ErykKul great work! just some comments.
Also one thing I noticed in the UI, the checkboxes to select an uploaded file from the UploadedFiles list are not being checked/unchecked on the first click, somehow they only are checked/unchecked on the second click.

src/files/infrastructure/FileJSDataverseRepository.ts Outdated Show resolved Hide resolved
src/sections/upload-dataset-files/UploadDatasetFiles.tsx Outdated Show resolved Hide resolved
src/files/domain/useCases/addUploadedFiles.ts Outdated Show resolved Hide resolved
@GPortas GPortas self-assigned this Sep 3, 2024
@ErykKul
Copy link
Contributor Author

ErykKul commented Sep 3, 2024

I have added the gap. I think that the header warning is something that the js-client sets. I can look at it, as well as improving the progress updates (I think that one would also require changes at the js-client).

@GPortas
Copy link
Contributor

GPortas commented Sep 4, 2024

@ErykKul Thanks for addressing the changes. I have retested the feature for .java files and it works great.

I think this is ready to be merged, and open a separate issue in js-dataverse to review the header warning and progress updates. @g-saracca

GPortas
GPortas previously approved these changes Sep 4, 2024
@ErykKul
Copy link
Contributor Author

ErykKul commented Sep 4, 2024

I tried fixing the header and progress bar issue: IQSS/dataverse-client-javascript#187
I will test it locally now. I will let you know if it worked. It is fine for me if it will be a separate issue.

@ErykKul
Copy link
Contributor Author

ErykKul commented Sep 4, 2024

I think it fixed it, I will add it to this PR, we can revert it if needed.

@ErykKul
Copy link
Contributor Author

ErykKul commented Sep 4, 2024

image

The progress bar moves nicely and there are no errors in the log. However, multipart upload does not work as it should, it looks like it "resets" with every part (starts from 0 and never reaches 100%). I will try fixing it, it should be easy to do (I think that my assumption on the index and part size were incorrect, or I messed up the formula).

g-saracca
g-saracca previously approved these changes Sep 4, 2024
@ErykKul
Copy link
Contributor Author

ErykKul commented Sep 4, 2024

I think that it was a correct behavior: resetting was due to the upload failing, from the log it looks like uploading first part always failed with some retries (maybe the file is too large for the test-bucket)? Anyway, it is the best I could do for now. I updated the dependency again, after a small fix on the multipart progress updates. I will stop committing now, such that the review and QA can continue. Thanks @g-saracca and @GPortas

@g-saracca
Copy link
Contributor

@ErykKul you made changes to the file upload use case on the js-dataverse repo?

@g-saracca
Copy link
Contributor

If that is the case we should first I think , review and approve the JS-dataverse PR 👍🏼

@ErykKul
Copy link
Contributor Author

ErykKul commented Sep 4, 2024

Yes, it was a small change in the js-dataverse. It is not urgent, I think it is most important to get it right, so I think it is a good idea to take a look at my change at the js-dataverse first: PR 187

Copy link
Contributor

@GPortas GPortas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works great now. I just merged the js-dataverse PR and tested it on this branch. All issues are solved.

testjsdv.mov

Copy link
Contributor

@g-saracca g-saracca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job @ErykKul ! approving again and merging!

@g-saracca g-saracca merged commit 08511d7 into develop Sep 4, 2024
11 of 14 checks passed
@g-saracca g-saracca deleted the feature/417-upload-integration branch September 4, 2024 13:07
@GPortas GPortas added SPA.Q3.5 Enhanced File upload Original size: 10 labels Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FY25 Sprint 4 FY25 Sprint 4 FY25 Sprint 5 FY25 sprint 5 GREI Re-arch GREI re-architecture-related integration Tasks involving the connection and interaction of UI features with the Dataverse API Original size: 10 Size: 10 A percentage of a sprint. 7 hours. SPA: File Upload Page SPA.Q3.5 Enhanced File upload
Projects
Status: Done 🧹
Development

Successfully merging this pull request may close these issues.

Integrate file upload page with js-dataverse use cases
6 participants