graph: fix inefficient query for IsPublicNode #10356
Conversation
Summary of ChangesHello @Abdulkbk, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a significant performance bottleneck in the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the IsPublicV1Node SQL query to improve performance, particularly for the SQLite backend. The change replaces an OR condition in a JOIN with a UNION ALL construct, which is a sound optimization strategy to help the query planner use indexes more effectively. The change is logical and well-explained. I have one minor suggestion regarding SQL formatting to ensure consistency.
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
2 times, most recently
from
9c698ba to
ca19c8e
Compare
|
can you create some stress tests to see whether union is actually better ? |
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
from
ca19c8e to
08ce7a8
Compare
I added a benchmark as the first commit, the following is what I found after running: go test -tags=test_db_sqlite -bench=BenchmarkIsPublicNode > sqlite_or.txt
# then
go test -tags=test_db_sqlite -bench=BenchmarkIsPublicNode > sqlite_union.txt
# finally
benchstat sqlite_or.txt sqlite_union.txt
The fact that this benchmark is run with approximately 8k nodes means that, if the number of nodes is increased to around ~80k, the difference will become more noticeable. |
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
from
08ce7a8 to
a0fa327
Compare
There was a problem hiding this comment.
Hello @Abdulkbk, I ran the benchmark as is and got the following numbers.
go test -tags=test_db_sqlite -bench=BenchmarkIsPublicNode
*** SELECT WITH UNION ***
goos: linux
goarch: amd64
pkg: github.com/lightningnetwork/lnd/graph/db
cpu: AMD Ryzen 7 7840HS w/ Radeon 780M Graphics
BenchmarkIsPublicNode-16 122 9719683 ns/op
--- BENCH: BenchmarkIsPublicNode-16
test_sqlite.go:49: Creating new SQLite DB for testing
PASS
ok github.com/lightningnetwork/lnd/graph/db 119.003s
Then changed the go.mod line 216:
from replace github.com/lightningnetwork/lnd/sqldb => ./sqldb
to //replace github.com/lightningnetwork/lnd/sqldb => ./sqldb
And ran it again :
*** SELECT USING OR ***
goos: linux
goarch: amd64
pkg: github.com/lightningnetwork/lnd/graph/db
cpu: AMD Ryzen 7 7840HS w/ Radeon 780M Graphics
BenchmarkIsPublicNode-16 93 11786240 ns/op
--- BENCH: BenchmarkIsPublicNode-16
test_sqlite.go:49: Creating new SQLite DB for testing
PASS
ok github.com/lightningnetwork/lnd/graph/db 120.685s
The BenchmarkIsPublicNode using SELECT WITH UNION took 9.7ms and using SELECT WITH OR 11.7ms.
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
2 times, most recently
from
398364b to
004d49c
Compare
|
Thanks @MPins, for taking a look. I've addressed the feedback. |
GustavoStingelin
left a comment
GustavoStingelin
left a comment
There was a problem hiding this comment.
Just ran an EXPLAIN ANALYZE btw the approaches for PostgreSQL
graph_explain_postgres_test.go:108: old_or_join EXPLAIN ANALYZE output:
Result (cost=15.65..15.66 rows=1 width=1) (actual time=0.036..0.036 rows=1 loops=1)
Buffers: shared hit=8
InitPlan 1 (returns $1)
-> Nested Loop (cost=8.95..62.51 rows=8 width=0) (actual time=0.035..0.035 rows=1 loops=1)
Buffers: shared hit=8
-> Index Scan using graph_nodes_unique on graph_nodes n (cost=0.28..8.30 rows=1 width=8) (actual time=0.016..0.016 rows=1 loops=1)
Index Cond: (pub_key = '\x02470e769bf7c452d32e804976916efbe2b88fb3f66b554c6f615af7ecd30cc3dd'::bytea)
Buffers: shared hit=3
-> Bitmap Heap Scan on graph_channels c (cost=8.67..54.09 rows=12 width=16) (actual time=0.018..0.019 rows=1 loops=1)
Recheck Cond: ((n.id = node_id_1) OR (n.id = node_id_2))
Filter: ((bitcoin_1_signature IS NOT NULL) AND (version = 1))
Heap Blocks: exact=1
Buffers: shared hit=5
-> BitmapOr (cost=8.67..8.67 rows=12 width=0) (actual time=0.006..0.006 rows=0 loops=1)
Buffers: shared hit=4
-> Bitmap Index Scan on graph_channels_node_id_1_idx (cost=0.00..4.33 rows=6 width=0) (actual time=0.003..0.003 rows=0 loops=1)
Index Cond: (n.id = node_id_1)
Buffers: shared hit=2
-> Bitmap Index Scan on graph_channels_node_id_2_idx (cost=0.00..4.33 rows=6 width=0) (actual time=0.003..0.003 rows=8 loops=1)
Index Cond: (n.id = node_id_2)
Buffers: shared hit=2
Planning Time: 0.329 ms
Execution Time: 0.056 ms
graph_explain_postgres_test.go:108: new_union_all EXPLAIN ANALYZE output:
Result (cost=4.71..4.72 rows=1 width=1) (actual time=0.028..0.029 rows=1 loops=1)
Buffers: shared hit=11
InitPlan 1 (returns $2)
-> Append (cost=0.57..33.66 rows=8 width=4) (actual time=0.028..0.028 rows=1 loops=1)
Buffers: shared hit=11
-> Nested Loop (cost=0.57..16.77 rows=4 width=4) (actual time=0.020..0.020 rows=0 loops=1)
Buffers: shared hit=5
-> Index Scan using graph_nodes_unique on graph_nodes n (cost=0.28..8.30 rows=1 width=8) (actual time=0.002..0.002 rows=1 loops=1)
Index Cond: (pub_key = '\x02470e769bf7c452d32e804976916efbe2b88fb3f66b554c6f615af7ecd30cc3dd'::bytea)
Buffers: shared hit=3
-> Index Scan using graph_channels_node_id_1_idx on graph_channels c (cost=0.29..8.41 rows=6 width=8) (actual time=0.017..0.017 rows=0 loops=1)
Index Cond: (node_id_1 = n.id)
Filter: ((bitcoin_1_signature IS NOT NULL) AND (version = 1))
Buffers: shared hit=2
-> Nested Loop (cost=0.57..16.77 rows=4 width=4) (actual time=0.008..0.008 rows=1 loops=1)
Buffers: shared hit=6
-> Index Scan using graph_nodes_unique on graph_nodes n_1 (cost=0.28..8.30 rows=1 width=8) (actual time=0.001..0.001 rows=1 loops=1)
Index Cond: (pub_key = '\x02470e769bf7c452d32e804976916efbe2b88fb3f66b554c6f615af7ecd30cc3dd'::bytea)
Buffers: shared hit=3
-> Index Scan using graph_channels_node_id_2_idx on graph_channels c_1 (cost=0.29..8.41 rows=6 width=8) (actual time=0.006..0.006 rows=1 loops=1)
Index Cond: (node_id_2 = n_1.id)
Filter: ((bitcoin_1_signature IS NOT NULL) AND (version = 1))
Buffers: shared hit=3
Planning Time: 0.213 ms
Execution Time: 0.050 ms
|
Thanks for taking a look @GustavoStingelin. From the output, we can see that indeed the |
|
yeah I think having a union call makes sense, can land after the cache PR if we still see major bottlenecks, however I think the cache will already solve our problems. The speed gain with a union is probably around ~20-30% given that node ids will be equally queried meaning that in the UnionCall we will likely also have to scan the table 2 times for 50% the node announcements we receive. I think planning plans are cached in PG so the planning time is not really super critical in a query. |
|
Given @yyforyongyu comment and another analysis I run on a bigger channel graph, it seems these queries compare O(n) vs O(log n) so might might prioritize this one first before the cache. here the OR query could not effectively use a bitmap and had to scan through the whole channel table. |
|
can you fix the linter and add 20.1 release notes here @Abdulkbk |
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
2 times, most recently
from
a0c35a1 to
98ec010
Compare
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
from
98ec010 to
a1edfec
Compare
In this commit we add a benchmark to test the performance of IsPublicNode query.
In this commit we updated the IsPublicV1Node query to use UNION instead of OR, since sqlite struggles to efficiently use multiple indexes in a single query involving OR conditions across different columns. We use UNION ALL since the query doesn't care about duplicates.
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
from
a1edfec to
2141516
Compare
|
@Abdulkbk could you analyze why the itest on windows failed ? It is not related to the PR but we should make sure we at least document flakes. |
Abdulkbk
force-pushed
the
ispublicnode-perf
branch
from
2141516 to
68f558c
Compare
|
@Abdulkbk, remember to re-request review from reviewers when ready |
ziggie1984
added
the
backport-v0.20.x-branch
|
Successfully created backport PR for |
…20.x-branch [v0.20.x-branch] Backport #10356: graph: fix inefficient query for IsPublicNode
7 participants
fixes #10337 (partially)
Change Description
In this PR, we rewrite the query for
IsPublicV1Node, which returns a boolean based on if a node has a public or private channel. This particularly fixes an issuesqlitebackend as it struggles to efficiently use multiple indexes across seperate columns in one query.Step to test