Skip to content

Conversation

@schlosna
Copy link
Contributor

What's Changed

Presize JsonStringArrayLists when constructing them as part of ValueVector#getObject conversions.

FixedSizeListVector#getObject already performs this optimization; however, ListVector, ListViewVector, LargeListVector, and LargeListViewVector do not yet presize the result JsonStringArrayList requiring dynamic reallocations as elements are converted & added. This can become a scalability bottleneck when using these types.

Closes #816.

@lidavidm lidavidm changed the title Presize JsonStringArrayList vector results GH-816: Presize JsonStringArrayList vector results Aug 11, 2025
@lidavidm
Copy link
Member

Looks like the formatter isn't happy:

 Error:  Failed to execute goal com.diffplug.spotless:spotless-maven-plugin:2.44.4:check (spotless-check) on project arrow-vector: The following files had format violations:
Error:      src/main/java/org/apache/arrow/vector/complex/LargeListVector.java
Error:          @@ -865,7 +865,8 @@
Error:           ····final·long·start·=·offsetBuffer.getLong((long)·index·*·OFFSET_WIDTH);
Error:           ····final·long·end·=·offsetBuffer.getLong(((long)·index·+·1L)·*·OFFSET_WIDTH);
Error:           ····final·ValueVector·vv·=·getDataVector();
Error:          -····final·List<Object>·vals·=·new·JsonStringArrayList<>(LargeMemoryUtil.checkedCastToInt(end·-·start));
Error:          +····final·List<Object>·vals·=
Error:          +········new·JsonStringArrayList<>(LargeMemoryUtil.checkedCastToInt(end·-·start));
Error:           ····for·(long·i·=·start;·i·<·end;·i++)·{
Error:           ······vals.add(vv.getObject(checkedCastToInt(i)));
Error:           ····}
Error:  Run 'mvn spotless:apply' to fix these violations.

@lidavidm
Copy link
Member

There's more formatting errors than just that. Can you mvn spotless:apply?

@schlosna
Copy link
Contributor Author

There's more formatting errors than just that. Can you mvn spotless:apply?

Thanks for the quick review. Apologies about the format issue, I had tried to fix up while mobile. I've run mvn spotless:apply & pushed.

@lidavidm lidavidm merged commit 52f7a86 into apache:main Aug 13, 2025
30 of 36 checks passed
@schlosna schlosna deleted the ds/presize-JsonStringArrayList branch August 15, 2025 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance: Presize JsonStringArrayList vector results

2 participants