feat: virtualized scrolling + paginated message loading for long sessions (#8535)

[CasualDeveloper]

Following up on feedback from #6138 - @rekram1-node mentioned virtualized scrolling would be the better long-term solution.

Problem

Loading thousands of messages at once is slow and memory-heavy.

Proposed approach

1. Paginated fetch - load messages in chunks as user scrolls up
2. Virtualized rendering - only render visible messages, placeholder the rest

Questions for maintainers

• Is this something you'd want contributed, or is it already on the roadmap?
• Any pointers on how virtualization fits with opentui/SolidJS?

Happy to take a stab at this with guidance.

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• Cannot scroll to beginning of long conversations (message limit hardcoded to 100) (#6138) #6137: Cannot scroll to beginning of long conversations (message limit hardcoded to 100) - directly related to message loading limitations in long sessions
• feature request: pagination for messages & sessions #4918: Feature request: pagination for messages & sessions - requesting pagination functionality similar to what you're proposing
• High CPU usage (100%+) during LLM streaming in long sessions due to O(n) text buffer rendering #6172: High CPU usage (100%+) during LLM streaming in long sessions - addresses the performance problem that virtualized scrolling is meant to solve

Feel free to ignore if none of these address your specific case.

[CasualDeveloper]

Background

I originally just wanted to scroll to the beginning of long conversations - that's what led to #6137/#6138 (message_limit config). But as @rekram1-node pointed out, that's a stopgap. The real fix is pagination + virtualization.

My longest session has 710 messages. With message_limit: 500, I can see more history, but it's slower to load and uses more memory. Related: #6172 (CPU issues), #4918 (original pagination request).

Proposed Approach

After digging into the codebase with Claude (my experience is mostly in Java/backend stuff, so I needed some assistance):

Phase 1: Paginated Loading (MVP)

Task	File	Change
1.1	session/index.ts:262	Add before cursor param to Session.messages()
1.2	server/server.ts:1161	Add before query param to endpoint
1.3	sdk/*	Regenerate types
1.4	tui/context/sync.tsx	Add pagination state tracking (hasMore, oldestId, loading)
1.5	tui/context/sync.tsx	Add loadMore(sessionID) method
1.6	tui/routes/session/index.tsx	Trigger loadMore when scrolling near top

Phase 2: Virtualization (later)

• Only render visible messages + buffer, spacers for the rest
• Would need custom implementation - Claude says opentui's viewportCulling skips painting but still mounts all Solid components + Yoga layout nodes
• Anchor-based scroll preservation on prepend
• Update findNextVisibleMessage() to work with windowed children

Recommend Phase 1 first - simpler, solves 80% of the problem.

The Tricky Part

The API currently returns an array. Changing to { messages, hasMore } would break SDK users (@opencode-ai/sdk is published on npm).

Options:

Option	Approach	Tradeoff
1	Add optional paginated param that opts into new response shape	Cleanest typing, but two response shapes to maintain
2	Return pagination info via Link header (body stays array)	Industry standard (GitHub uses this, RFC 8288) - fully backwards compatible
3	Breaking change (bump SDK major)	Simplest code, but affects external SDK users

Claude found that GitHub uses Link headers for pagination (RFC 8288). Option 2 seems like the safest path?

Questions

1. Does this approach make sense?
2. Which option do you prefer for the API?
3. Any concerns before I start on a PR?

[ry2009]

did something along th esimilar lines: #6656

[CasualDeveloper]

did something along similar lines: #6656

Nice work! Your implementation seems to align closely with what was proposed above - my Claude says you followed the right approach.

An observation I noticed after reviewing the diff:

• Config interaction with tui.message_limit
  My PR feat(tui): add session_list_limit for session picker (#6137) #6138 adds a message_limit config option. This PR hard-codes limit: 100 in both the initial load (sync.tsx:356) and loadMore (sync.tsx:387), and removes the 100-message trim. Should the page size use the config value instead? Or if pagination makes that config obsolete, we should probably close feat(tui): add session_list_limit for session picker (#6137) #6138 and remove/deprecate it.

Claude also offered some minor suggestions (non-blocking, could be follow-ups):

1. Silent loadMore failures (sync.tsx:377) - The catch block just flips loading to false. Users get no feedback if pagination fails, and can keep triggering broken requests. A toast or short backoff would help.
2. Brittle Link header parsing (sync.tsx:361) - includes('rel="next"') works with current server output, but won't handle rel=next (no quotes) or multiple Link values. Could parse more defensively or use a small helper.
3. before cursor validation (server.ts:1162) - Invalid cursors currently fail inside Session.messages. Validating with Identifier.schema("message") in the query schema would return consistent 400s at the boundary.
4. N+1 queries on deep pagination (session/index.ts:274) - Each page does Storage.list + per-message MessageV2.get, so page 10 of a 1000-message session still scans all IDs. Probably fine for MVP, but could cache IDs or extend Storage for cursor-based listing later.
5. What this doesn't solve yet (as you noted in your PR's comments):
   CPU/memory will still grow as users paginate since all loaded messages render. Virtualization (Phase 2) would address High CPU usage (100%+) during LLM streaming in long sessions due to O(n) text buffer rendering #6172. But that's a reasonable follow-up - this PR solves the "can't scroll to beginning" problem which is the more urgent issue.

@ry2009 Happy to help test or iterate on this. If it's easier, I can submit these fixes as commits to your branch - just let me know.

[CasualDeveloper]

Decided to take a whack at this. My PR #8535 also aims to implement cursor-based pagination for message loading. Phase 2 (virtualization) can be a follow-up if the memory bounding in my PR proves insufficient.

What's Implemented

Backend:

• Bi-directional cursor support (before and after params)
• RFC 5005 compliant Link headers (rel="prev" for older, rel="next" for newer)
• Binary search on ULIDs for efficient cursor lookup
• Graceful handling of deleted messages via Promise.allSettled

TUI:

• loadOlder/loadNewer with scroll position restoration
• Memory bounding: 500-message sliding window with 50-message eviction
• Live SSE updates handled correctly when viewing history (hasNewer=true)
• Error state feedback for failed pagination requests

Tests:

• 25 tests across 4 new test files covering pagination, caching, Link header parsing, and API validation

Related

Supersedes @ry2009's PR #6656. (Thanks for taking time to look into it as well!)