-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Java] Allocate the FieldReader for Arrow Vectors only on demand. Also introduce getTransferPair which takes Field as parameter. #15187
Comments
take |
Other than my comment about trying to lift the code for a thread-safe supplier from Guava, this looks good to me. FWIW, we did something similar with org.apache.arrow.util.Preconditions in the memory component. |
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 4, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 4, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 14, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 14, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 14, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 16, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 16, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 16, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 17, 2023
…ader type when asked for impl
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 17, 2023
rtadepalli
added a commit
to rtadepalli/arrow
that referenced
this issue
Jun 17, 2023
lidavidm
pushed a commit
that referenced
this issue
Jun 18, 2023
…TransferPair()` function that takes in a `Field` type (#34424) ### Rationale for this change This PR closes #15187. `FieldReader` is being allocated directly in the constructor today, and this PR changes it such that the initialization becomes lazy. Additionally, a new function `getTransferPair(Field, Allocator)` is introduced so that a new `Field` method is not constructed each time `getTransferPair` is called on the Vector. ### What changes are included in this PR? 1. Introduce a new `getTransferPair` method. 2. Make initializing `FieldReader` lazy. ### Are these changes tested? Yes, some tests have been added to verify these changes. ### Are there any user-facing changes? I am not 100% sure if there are any user facing changes. There should not be any breaking changes. * Closes: #15187 Authored-by: Ramasai <[email protected]> Signed-off-by: David Li <[email protected]>
lriggs
pushed a commit
to dremio/arrow
that referenced
this issue
Sep 7, 2023
…w `getTransferPair()` function that takes in a `Field` type (apache#34424) This PR closes apache#15187. `FieldReader` is being allocated directly in the constructor today, and this PR changes it such that the initialization becomes lazy. Additionally, a new function `getTransferPair(Field, Allocator)` is introduced so that a new `Field` method is not constructed each time `getTransferPair` is called on the Vector. 1. Introduce a new `getTransferPair` method. 2. Make initializing `FieldReader` lazy. Yes, some tests have been added to verify these changes. I am not 100% sure if there are any user facing changes. There should not be any breaking changes. * Closes: apache#15187 Authored-by: Ramasai <[email protected]> Signed-off-by: David Li <[email protected]>
lriggs
added a commit
to dremio/arrow
that referenced
this issue
Sep 8, 2023
* apacheGH-15187: [Java] Made `reader` initialization lazy and added new `getTransferPair()` function that takes in a `Field` type (apache#34424) This PR closes apache#15187. `FieldReader` is being allocated directly in the constructor today, and this PR changes it such that the initialization becomes lazy. Additionally, a new function `getTransferPair(Field, Allocator)` is introduced so that a new `Field` method is not constructed each time `getTransferPair` is called on the Vector. 1. Introduce a new `getTransferPair` method. 2. Make initializing `FieldReader` lazy. Yes, some tests have been added to verify these changes. I am not 100% sure if there are any user facing changes. There should not be any breaking changes. * Closes: apache#15187 Authored-by: Ramasai <[email protected]> Signed-off-by: David Li <[email protected]> * Fix import * Fix imports --------- Signed-off-by: David Li <[email protected]> Co-authored-by: rtadepalli <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
[Java] Allocate the FieldReader for Arrow Vectors only on demand. Also introduce getTransferPair which takes Field as parameter.
FieldReader objects are created in the constructor of arrow vectors. FieldReader objects might not be needed in all cases. So create them only if required. https:/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/BitVector.java#L83
Introduce a new getTransferPair method which takes 'Field' as a parameter. In some cases its useful to pass a 'Field' object directly and not create a new 'Field' everytime. https:/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/BitVector.java#L553
Component(s)
Java
The text was updated successfully, but these errors were encountered: