in dremio, removing the micro/nano seconds because dremio support mil…

[GuyEshdat]

…i seconds only when filtering

Summary by CodeRabbit

• Bug Fixes
  • Improved timestamp casting to handle inputs with fractional seconds by trimming them to millisecond precision before conversion. This reduces parsing errors and ensures consistent timestamp values across sources.
  • Enhances compatibility with datasets that include extended fractional seconds, preventing failed loads or inconsistent results during transformations and queries.
  • Provides more predictable behavior when working with time-based fields, improving data reliability in downstream reports and analytics.

[linear]

ELE-5134 dremio miliscenods precision

[github-actions]

👋 @GuyEshdat
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

[coderabbitai]

Walkthrough

Updated timestamp casting logic in dremio__edr_cast_as_timestamp to first normalize input by trimming fractional seconds to three digits via REGEXP_REPLACE before performing the cast.

Changes

Cohort / File(s)	Summary
Timestamp casting normalization macros/utils/data_types/cast_column.sql	Modified dremio__edr_cast_as_timestamp to apply REGEXP_REPLACE that trims fractional seconds to millisecond precision prior to casting to TIMESTAMP.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

A nibble of time, a tick to prune,
I trim the sparks past three too soon—
Millis neat, then cast I hop,
Into the TIMESTAMP carrot shop.
Thump-thump! The query’s right on cue—
Precise as dew, and swift to chew. 🥕⏱️

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title clearly describes the core change of stripping micro- and nanosecond precision in Dremio because it only supports milliseconds, which matches the update in the cast_column.sql macro to trim fractional seconds before casting. Despite minor stylistic issues, it directly summarizes the main functional adjustment without introducing unrelated details.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ele-5134-dremio-miliseconds-precision

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 1f753cb and 958487d.

📒 Files selected for processing (1)

• macros/utils/data_types/cast_column.sql (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (14)

• GitHub Check: test (latest_official, redshift) / test
• GitHub Check: test (fusion, bigquery) / test
• GitHub Check: test (latest_official, athena) / test
• GitHub Check: test (latest_official, databricks_catalog) / test
• GitHub Check: test (latest_official, trino) / test
• GitHub Check: test (latest_official, dremio) / test
• GitHub Check: test (latest_official, snowflake) / test
• GitHub Check: test (fusion, databricks_catalog) / test
• GitHub Check: test (fusion, redshift) / test
• GitHub Check: test (fusion, snowflake) / test
• GitHub Check: test (latest_official, clickhouse) / test
• GitHub Check: test (latest_official, postgres) / test
• GitHub Check: test (latest_official, bigquery) / test
• GitHub Check: test (latest_pre, postgres) / test

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

Fix the Dremio backreference syntax.

REGEXP_REPLACE in Dremio expects \1-style backreferences, not $1. With the current replacement string the fractional part becomes literal $1, and the subsequent CAST will fail. Please switch the replacement to '\1' (escaping as needed in Jinja) so the truncated milliseconds are preserved correctly.

🤖 Prompt for AI Agents

In macros/utils/data_types/cast_column.sql around line 32, the REGEXP_REPLACE
replacement uses a POSIX-style $1 backreference which Dremio does not recognize;
change the replacement to use a Dremio backreference '\1' and ensure it is
properly escaped for Jinja (e.g. use '\\1' or a quoted literal that yields '\1'
in the rendered SQL) so the fractional milliseconds are truncated correctly
before casting.