Skip to content

Comments

🐛 Resolve N+1 query problem to prevent SIGABRT/SIGSEGV errors (#2653)#2689

Merged
KATO-Hiro merged 8 commits intostagingfrom
#2653
Oct 13, 2025
Merged

🐛 Resolve N+1 query problem to prevent SIGABRT/SIGSEGV errors (#2653)#2689
KATO-Hiro merged 8 commits intostagingfrom
#2653

Conversation

@KATO-Hiro
Copy link
Collaborator

@KATO-Hiro KATO-Hiro commented Oct 13, 2025

plan.md の phase 2以降は、動作確認中です。

Summary by CodeRabbit

  • Bug Fixes

    • Workbook pages load faster and more reliably; task results and statuses display consistently for large workbooks.
  • Refactor

    • Data fetching now batches related task and answer records and merges them in-memory, reducing queries, memory pressure, and improving scalability.
  • Documentation

    • Added a detailed rollout plan with phased implementation, testing, monitoring, rollback, and success criteria.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 13, 2025

Walkthrough

Adds a planning document and implements bulk-fetch helpers for Tasks and TaskAnswers, refactors task result assembly to eliminate N+1 queries by loading data with IN queries and merging in-memory via a new mergeTaskAndAnswer helper, adds timing logs and guards for empty inputs.

Changes

Cohort / File(s) Summary
Planning doc
docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md
Adds a rollout and implementation plan addressing N+1 queries on /workbooks/[slug], detailing root causes, phased implementation, concrete code-change proposals, testing, monitoring, rollback, success criteria, risks, and progress tracking.
Answers helper
src/lib/services/answers.ts
Adds getAnswersWithSelectedTaskIds(selectedTaskIds: string[], userId: string) returning selected fields (task_id, user_id, status_id, updated_at) for answers matching task_id IN (...) and user_id; returns [] for empty input.
Tasks helper
src/lib/services/tasks.ts
Adds getTasksWithSelectedTaskIds(selectedTaskIds: string[]) which deduplicates IDs and returns selected Task fields (contest_id, task_table_index, task_id, title, grade) via a Prisma findMany with an in filter; updates type imports.
Task results refactor
src/lib/services/task_results.ts
Rewrites getTaskResultsByTaskId to early-return on empty taskIds, bulk-load tasks and answers via the new helpers, build lookup maps, merge entries in-memory using a new internal mergeTaskAndAnswer helper, log load duration/counts, and update getTaskResult to delegate to the same merge helper; removes the previous per-task N+1 pattern.
Types update
src/lib/types/answer.ts
Adds updated_at: Date to the TaskAnswer interface.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Page as Workbook Page
  participant TR as getTaskResultsByTaskId
  participant TS as getTasksWithSelectedTaskIds
  participant AN as getAnswersWithSelectedTaskIds
  participant M as mergeTaskAndAnswer

  Page->>TR: request(taskIds[], userId)
  alt taskIds empty
    TR-->>Page: []
  else
    TR->>TS: fetch Tasks by IN(taskIds)
    TR->>AN: fetch Answers by IN(taskIds) & userId
    TS-->>TR: Tasks[]
    AN-->>TR: Answers[]
    TR->>TR: build maps (taskById, answerByTaskId, statusById)
    loop per taskId
      TR->>M: mergeTaskAndAnswer(Task?, Answer?, statusById)
      M-->>TR: TaskResult
    end
    TR-->>Page: TaskResult[]
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

I thump my paw: no N plus one!
I fetch in bulk, then merge with fun.
Maps and logs, a tidy trail,
Answers joined without the hail.
A carrot cheers — the queries run. 🥕🐇

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Title Check ⚠️ Warning The pull request title accurately summarizes the main change of resolving the N+1 query problem to prevent process aborts, but it includes extraneous noise in the form of an emoji and the issue number reference which reduces clarity and conciseness. Please remove the leading emoji and the parenthetical issue reference so the title reads simply “Resolve N+1 query problem to prevent SIGABRT/SIGSEGV errors,” which concisely conveys the primary change without added noise.
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch #2653

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 681b2a0 and a48c0ac.

📒 Files selected for processing (2)
  • src/lib/services/task_results.ts (4 hunks)
  • src/lib/types/answer.ts (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: preview
🔇 Additional comments (5)
src/lib/types/answer.ts (1)

5-5: LGTM! Essential field addition for tracking answer timestamps.

The updated_at field is correctly typed and necessary for the N+1 query fix implemented in task_results.ts. This enables tracking when users last submitted their answers.

src/lib/services/task_results.ts (4)

8-8: LGTM! Import changes support the bulk loading refactor.

The addition of getTasksWithSelectedTaskIds and the type change from WorkBookTaskBase to WorkBookTasksBase align correctly with the new bulk-fetch approach.

Also applies to: 22-22


191-232: Excellent refactoring! Successfully addresses the N+1 query problem.

This implementation effectively eliminates N+1 queries by:

  1. Bulk fetching tasks and answers with just 2 queries (lines 203-204) instead of N+1 queries
  2. In-memory merging using Map-based O(1) lookups (lines 207-224)
  3. Type-safe filtering with proper guards (line 196)
  4. Early returns for empty inputs to avoid unnecessary queries (lines 198-200)
  5. Performance monitoring with duration logging (lines 226-229)

The defensive programming with null checks and missing task warnings (lines 215-218) ensures robustness.


234-270: LGTM! Past critical issue has been resolved.

The mergeTaskAndAnswer helper correctly implements the merge logic and addresses the issue flagged in previous reviews:

Fixed: Lines 264-266 now correctly use answer.updated_at instead of the incorrect status.updated_at reference from earlier versions.

The function properly:

  • Creates a default TaskResult as a baseline (line 248)
  • Returns early when no answer exists (lines 250-252)
  • Guards against missing status in the map (line 256)
  • Correctly assigns the answer timestamp to track actual submission time (lines 264-266)

This matches the correct pattern shown at line 170 in getTaskResultsOnlyResultExists.


291-301: Good refactoring for consistency.

The migration to use mergeTaskAndAnswer (line 301) improves code maintainability by eliminating duplication. The deprecation note (line 291) clearly signals the migration path toward bulk operations.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (5)
docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md (1)

13-17: Add languages to fenced code blocks (markdownlint MD040).

Please annotate code fences with languages to satisfy linting and improve readability. Examples:

  • Lines 13-17: use text
    Node.js process exited with signal: 6 (SIGABRT) ...
    
  • Lines 56-60: use text
    200個のPrismaクエリエンジン × 40-60MB/プロセス = 8GB〜12GB必要
    

Similarly apply typescript, sql, bash where appropriate throughout the doc.

Also applies to: 56-60

src/lib/services/answers.ts (1)

23-44: Type the return, guard empty input, and consider selecting updated_at.

  • Add an explicit return type to avoid type drift downstream.
  • Early-return on empty selectedTaskIds to skip a no-op DB call.
  • If future merges will use timestamps, include updated_at now.

Apply:

-export async function getAnswersWithSelectedTaskIds(selectedTaskIds: string[], userId: string) {
-  return await prisma.taskAnswer.findMany({
+export async function getAnswersWithSelectedTaskIds(
+  selectedTaskIds: string[],
+  userId: string,
+): Promise<Array<Pick<TaskAnswer, 'task_id' | 'user_id' | 'status_id' | 'updated_at'>>> {
+  if (!selectedTaskIds?.length) return [];
+  return await prisma.taskAnswer.findMany({
     where: {
       task_id: { in: selectedTaskIds },
       user_id: userId,
     },
     select: {
       task_id: true,
       user_id: true,
       status_id: true,
+      updated_at: true,
     },
   });
 }

If you prefer not to expose updated_at, keep the select as-is but still add the explicit return type and early return.

Ensure a composite index exists on (task_id, user_id) to fully benefit from the combined filter.

src/lib/services/tasks.ts (1)

14-34: Tighten JSDoc, guard/normalize inputs, and verify return type compatibility.

  • Fix JSDoc param name and stray backtick.
  • Avoid duplicate IDs and skip empty queries.
  • Confirm Tasks matches the selected projection.

Apply:

 /**
  * Fetches tasks with the specified task IDs.
- * @param taskIds - An array of task IDs to filter the tasks.
+ * @param selectedTaskIds - An array of task IDs to filter the tasks.
  *
  * @returns A promise that resolves to an array of Task objects.
- * @note conditions: { task_id: { in: taskIds } }` for efficient filtering
+ * @note conditions: { task_id: { in: taskIds } } for efficient filtering
  */
 export async function getTasksWithSelectedTaskIds(selectedTaskIds: string[]): Promise<Tasks> {
-  return await db.task.findMany({
+  if (!selectedTaskIds?.length) return [];
+  const ids = Array.from(new Set(selectedTaskIds));
+  return await db.task.findMany({
     where: {
-      task_id: { in: selectedTaskIds }, // SQL: WHERE task_id IN ('id1', 'id2', ...)
+      task_id: { in: ids }, // SQL: WHERE task_id IN ('id1', 'id2', ...)
     },
     select: {
       contest_id: true,
       task_table_index: true,
       task_id: true,
       title: true,
       grade: true,
     },
   });
 }

Also, please verify that type Tasks equals the selected shape (or adjust to Pick<Task, 'contest_id' | 'task_table_index' | 'task_id' | 'title' | 'grade'>[]).

src/lib/services/task_results.ts (2)

191-229: Great N+1 elimination; align typing and minor robustness.

  • Change answersMap construction to avoid forcing a broader type than the selected shape.
  • Optionally dedupe taskIds before querying to cut noise.
  • Consider chunking if ids can approach DB/driver param limits; keep current simple path for typical sizes.

Apply:

-  const answersMap = new Map(answers.map((answer: TaskAnswer) => [answer.task_id, answer]));
+  const answersMap = new Map(answers.map((answer) => [answer.task_id, answer]));

Optional:

-  const taskIds = workBookTasks
+  const taskIds = Array.from(
+    new Set(
+      workBookTasks
       .map((workBookTask) => workBookTask.taskId)
-    .filter((id): id is string => id !== null && id !== undefined);
+        .filter((id): id is string => id !== null && id !== undefined),
+    ),
+  );

And consider gating the console.log under a debug flag or using your logger to avoid prod noise.


243-266: Consider propagating updated_at when an answer exists.

If answers include updated_at (see suggested change in answers.ts), set taskResult.updated_at accordingly to keep parity with other paths (e.g., getTaskResultsOnlyResultExists).

Example:

   if (status) {
     taskResult.status_id = status.id;
     taskResult.status_name = status.status_name;
     taskResult.submission_status_image_path = status.image_path;
     taskResult.submission_status_label_name = status.label_name;
     taskResult.is_ac = status.is_ac;
     taskResult.user_id = userId;
+    if ((answer as any).updated_at) {
+      taskResult.updated_at = (answer as any).updated_at;
+    }
   }

Prefer a proper AnswerSummary type over any-casts if you adopt this.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 7e294cf and d4b23d9.

📒 Files selected for processing (4)
  • docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md (1 hunks)
  • src/lib/services/answers.ts (1 hunks)
  • src/lib/services/task_results.ts (4 hunks)
  • src/lib/services/tasks.ts (2 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.18.1)
docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md

13-13: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


56-56: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: preview
🔇 Additional comments (1)
src/lib/services/task_results.ts (1)

287-298: LGTM on refactor to reuse mergeTaskAndAnswer.

Keeps behavior while centralizing merge logic.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between d4b23d9 and 62cb320.

📒 Files selected for processing (4)
  • docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md (1 hunks)
  • src/lib/services/answers.ts (1 hunks)
  • src/lib/services/task_results.ts (4 hunks)
  • src/lib/services/tasks.ts (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/lib/services/answers.ts
  • src/lib/services/tasks.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: preview
🔇 Additional comments (5)
docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md (1)

1-740: Excellent planning document!

This is a very thorough and well-structured planning document that clearly explains:

  • The root cause analysis (N+1 queries causing memory exhaustion)
  • The solution approach (bulk queries with IN clause)
  • Implementation phases with clear checklists
  • Success criteria and risk mitigation
  • Detailed code examples and technical explanations

The documentation will be valuable for future reference and for other team members to understand the fix.

src/lib/services/task_results.ts (4)

191-200: Good defensive programming with early return.

The early return for empty input prevents unnecessary database queries and potential errors downstream.


226-229: Excellent addition of performance logging.

The duration logging will help monitor the performance improvement and detect any regressions in production.


287-298: Good refactoring to use shared helper.

The refactoring eliminates code duplication and ensures consistent merge logic across the codebase.


202-209: Cannot locate bulk query functions or confirm selected fields.
Please verify that getTasksWithSelectedTaskIds and getAnswersWithSelectedTaskIds exist in tasks.ts and answers.ts, respectively, and that they each select the required fields (contest_id, task_table_index, task_id, title, grade for tasks; task_id, user_id, status_id, updated_at for answers`).

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 62cb320 and 681b2a0.

📒 Files selected for processing (3)
  • docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md (1 hunks)
  • src/lib/services/answers.ts (1 hunks)
  • src/lib/services/task_results.ts (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/services/answers.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: preview
🔇 Additional comments (3)
src/lib/services/task_results.ts (2)

187-232: LGTM! Excellent N+1 resolution with bulk queries.

The refactor successfully addresses the N+1 query problem:

  • Type-safe filtering (lines 194-196): Proper type guard ensures only valid task IDs are processed
  • Early return (lines 198-200): Prevents unnecessary queries when no tasks exist
  • Bulk fetches (lines 203-204): Reduces 200 queries to 2 queries (99% reduction)
  • O(1) lookup (lines 207-209): Map-based lookups for efficient in-memory merging
  • Error handling (lines 215-218): Gracefully handles missing tasks with warning and continues
  • Performance logging (lines 226-229): Enables monitoring of execution time in production

The implementation follows best practices for batch operations and maintains type safety throughout.


291-302: LGTM! Good refactor for consistency.

The function now correctly delegates merge logic to the shared mergeTaskAndAnswer helper, ensuring consistent behavior with getTaskResultsByTaskId. The deprecation note is appropriate for future bulk-only operations.

Once the updated_at bug in mergeTaskAndAnswer (line 264-266) is fixed, this function will automatically benefit from the correction.

docs/dev-notes/2025-10-13/fix-n-plus-1-query-for-prisma/plan.md (1)

222-224: Inconsistency: Plan is correct, but implementation differs.

The example code here correctly shows using answer.updated_at:

if (answer.updated_at) {
  taskResult.updated_at = answer.updated_at;
}

However, the actual implementation in src/lib/services/task_results.ts (lines 264-266) incorrectly uses status.updated_at. Please ensure the implementation matches this correct pattern shown in your plan.

Copy link
Collaborator Author

@KATO-Hiro KATO-Hiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KATO-Hiro KATO-Hiro merged commit 9e0f556 into staging Oct 13, 2025
3 checks passed
@KATO-Hiro KATO-Hiro deleted the #2653 branch October 13, 2025 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant