Skip to content

Commit d9cd3c5

Browse files
author
Bob Strahan
committed
Update S3 partition key format for improved date range filtering
1 parent f336d04 commit d9cd3c5

File tree

2 files changed

+8
-4
lines changed

2 files changed

+8
-4
lines changed

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,10 @@ SPDX-License-Identifier: MIT-0
6161
- Defend against non-numeric confidence_threshold values in the configuration - avoid float conversion or numeric comparison exceptions in Assessement step
6262
- Prevent creation of empty configuration fields in UI
6363
- Firefox browser issues with signed URLs (PR #14)
64+
- Improved S3 Partition Key Format for Better Date Range Filtering:
65+
- Updated reporting data partition keys to use YYYY-MM format for month and YYYY-MM-DD format for day
66+
- Enables easier date range filtering in analytics queries across different months and years
67+
- Partition structure now: `year=2024/month=2024-03/day=2024-03-15/` instead of `year=2024/month=03/day=15/`
6468

6569
## [0.3.3]
6670

docs/reporting-database.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ The `document_evaluations` table contains document-level evaluation metrics:
3737
| false_discovery_rate | double | False discovery rate (0-1) |
3838
| execution_time | double | Time taken to evaluate (seconds) |
3939

40-
This table is partitioned by year, month, day, and document ID.
40+
This table is partitioned by year, month (YYYY-MM format), day (YYYY-MM-DD format), and document ID.
4141

4242
### Section Evaluations
4343

@@ -56,7 +56,7 @@ The `section_evaluations` table contains section-level evaluation metrics:
5656
| false_discovery_rate | double | Section false discovery rate (0-1) |
5757
| evaluation_date | timestamp | When the evaluation was performed |
5858

59-
This table is partitioned by year, month, day, and document ID.
59+
This table is partitioned by year, month (YYYY-MM format), day (YYYY-MM-DD format), and document ID.
6060

6161
### Attribute Evaluations
6262

@@ -78,7 +78,7 @@ The `attribute_evaluations` table contains attribute-level evaluation metrics:
7878
| confidence_threshold | string | Confidence threshold used |
7979
| evaluation_date | timestamp | When the evaluation was performed |
8080

81-
This table is partitioned by year, month, day, and document ID.
81+
This table is partitioned by year, month (YYYY-MM format), day (YYYY-MM-DD format), and document ID.
8282

8383
## Metering Table
8484

@@ -94,7 +94,7 @@ The `metering` table captures detailed usage metrics for each document processin
9494
| number_of_pages | int | Number of pages in the document |
9595
| timestamp | timestamp | When the operation was performed |
9696

97-
This table is partitioned by year, month, day, and document ID.
97+
This table is partitioned by year, month (YYYY-MM format), day (YYYY-MM-DD format), and document ID.
9898

9999
The metering table is particularly valuable for:
100100
- Cost analysis and allocation

0 commit comments

Comments
 (0)