Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure indexer output is deterministic and repeatable #22383

Closed
wants to merge 1 commit into from

Conversation

vpavic
Copy link
Contributor

@vpavic vpavic commented Feb 7, 2019

The output file that spring-context-indexer produces is currently written using Properties#store. This prevents the build of a project that uses indexer from being repeatable, due to current timestamp being included in comment inside output file, and also entries possibly not being ordered deterministically.

This PR ensures that indexer output is written in deterministic and repeatable manner by filtering out comments and ordering entries.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Feb 7, 2019
@vpavic
Copy link
Contributor Author

vpavic commented Feb 20, 2019

Any feedback on this PR? The current behavior is preventing us from adopting spring-context-indexer as our builds must be repeatable.

@snicoll
Copy link
Member

snicoll commented Feb 20, 2019

I had a chat with @wilkinsona as we have a similar requirement for Spring Boot. I am not a huge fan of the manual handling and assuming that values are not multi-lines. I hadn't enough cycles to try to improve it though but perhaps you do?

@snicoll
Copy link
Member

snicoll commented Feb 20, 2019

Out of curiosity, have you actually noticed a significant improvement using the indexer? If so, can you share a bit more details?

@vpavic
Copy link
Contributor Author

vpavic commented Feb 20, 2019

On application with over 1000 beans I've observed an improvement in startup time of roughly half a second. In relative terms that is around 3%, so quite measurable.

I'm not a fan of manual handling of writing Properties, but unfortunately the API leaves a lot to be desired. Will multi-line values ever be the case in indexer's output? Alternative to the current approach could be something involving a custom OutputStream that filters out undesirable values.

@vpavic
Copy link
Contributor Author

vpavic commented May 28, 2019

With #23018 now resolved, I assumed this is unblocked and can be addressed in 5.2?

The only question is how to leverage the new SortedProperties, having in mind spring-context-indexer doesn't have dependencies to any other modules - is it OK to add dependency on spring-core?

@snicoll
Copy link
Member

snicoll commented May 28, 2019

is it OK to add dependency on spring-core

I don't think it is so we'll have to decide what we want to do here and if we accept copy/pasting this.

@sbrannen sbrannen added in: core Issues in core modules (aop, beans, core, context, expression) type: enhancement A general enhancement labels May 28, 2019
@sbrannen
Copy link
Member

sbrannen commented May 28, 2019

I don't foresee any real issue with having a copy of SortedProperties used internally within the indexer. We'd just have to make sure they stay in sync in case a bug is ever fixed in one of the implementations.

The only dependency that would need to be worked around is the current use of StringUtils.tokenizeToStringArray(...), but that also should not be a problem.

@vpavic
Copy link
Contributor Author

vpavic commented May 28, 2019

Assuming copy of SortedProperties within spring-context-indexer, what's the preferred way of handling StringUtils#tokenizeToStringArray dependency? Trimmed-down copy of StringUtils or something else?

@sbrannen
Copy link
Member

Assuming copy of SortedProperties within spring-context-indexer, what's the preferred way of handling StringUtils#tokenizeToStringArray dependency? Trimmed-down copy of StringUtils or something else?

Maybe something as simple as String.split(...). We could even make the same change within SortedProperties to ensure consistency.

The reason I used StringUtils#tokenizeToStringArray was simply that that's how it's often done within core Spring, but I don't see that as a must.

@vpavic
Copy link
Contributor Author

vpavic commented May 28, 2019

I've update the PR to take the approach with a copy of SortedProperties. I've also added a commit to update the original SortedProperties to use String#split instead of StringUtils as you suggested @sbrannen.

@sbrannen
Copy link
Member

I've update the PR to take the approach with a copy of SortedProperties.

Looks fine to me.

I've also added a commit to update the original SortedProperties to use String#split instead of StringUtils as you suggested @sbrannen.

I actually pushed the same change to master just now in 53597f9. So that's already taken care of.

@sbrannen sbrannen added this to the 5.2 M3 milestone May 28, 2019
@sbrannen sbrannen removed the status: waiting-for-triage An issue we've not yet triaged or decided on label May 28, 2019
@sbrannen sbrannen self-assigned this Jun 11, 2019
@sbrannen sbrannen closed this in b51e553 Jun 11, 2019
sbrannen added a commit that referenced this pull request Jun 11, 2019
@vpavic vpavic deleted the indexer-repeatability branch June 11, 2019 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: core Issues in core modules (aop, beans, core, context, expression) type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants