Get timeseries table generalize #685
Merged
+675
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduces a new endpoint `/job/hourly_rate_table` that generates an Excel file with hourly rate data for one or more scenarios. Adds `hourly_rate_helpers.py` with utility functions for datetime generation and safe dictionary access, and implements the `hourly_rate_table` view to assemble and return the Excel file based on scenario data.
Enhanced the Excel export in hourly_rate_table by adding specific formats for datetime, integers, and energy rates. Rate headers now use the rate name from urdb_metadata for clarity. Data columns for load and peak load use integer formatting, and datetime strings are written as Excel datetimes for better compatibility.
Renamed hourly_rate_helpers.py to timeseries_table_helpers.py and updated all references accordingly. Added (but not currently used) custom_timeseries_table_config.py to define flexible timeseries table configurations. Refactored the hourly_rate_table view and URL to get_timeseries_table, enhanced Excel formatting with colored headers, and improved code organization for future extensibility.
…ures easily Moved and expanded timeseries table configuration to support column formatting, worksheet naming, and Excel output customization. Renamed and enhanced helper functions for safer data extraction. Updated the view to dynamically use the new configuration structure, allowing selection of table layouts and improved formatting for Excel exports.
Documented new /job/get_timeseries_table endpoint, custom_timeseries_energy_demand table option, and increased monthly_totals_kwh max value. Prepares changelog for v3.17.1 minor updates.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please check if the PR fulfills these requirements
What kind of change does this PR introduce?
This pull request introduces a new, extensible system for configuring and generating custom timeseries tables for Excel export, along with utility helpers to safely extract and format timeseries data. It also adds a new API endpoint for retrieving these tables and updates the database schema to allow for larger monthly energy totals. The changes are grouped below by theme.
Custom Timeseries Table System:
custom_timeseries_table_config.py, which defines a flexible configuration structure for timeseries Excel tables, including column definitions, formatting options, and example configurations for energy/demand, emissions, and load profiles. This enables easier customization and extension of exported timeseries data.custom_timeseries_table_helpers.py, providing utility functions (safe_get_list,safe_get_value,safe_get,generate_datetime_column, etc.) for robust extraction and formatting of nested timeseries data, including special handling for datetime columns and monthly peaks.API and Routing:
/job/get_timeseries_table/tourls.pyto serve the generated timeseries tables.Database and Validation:
monthly_totals_kwhfield in theElectricLoadInputsmodel to allow values up to 1,000,000,000 (1e9) kWh, both in the Django model and via a new migration, to support larger energy datasets. [1] [2]Miscellaneous:
datetimeinviews.pyto support new datetime handling in the timeseries helpers.builds off the get_timeseries_table and add functionality by making it more generalized using custom_timeseries_table_config to allow the user to specify which table they want to use.What is the current behavior?
The user can specify the timeseries table endpoint and specify which timeseries configuration they want to be used. Each timeseries table configuration has its own formatting options so they can be easily changed.
Base columns set to TRUE don't change across scenarios (or run_uuids). Base columns set to FALSE change for each new rate scenario.
If the user inputs a year, that sets the first timestep in the table. If year is a leap year, the last day is removed.
What is the new behavior (if this is a feature change)?
N/A
Does this PR introduce a breaking change?
Hopefully, no changes will need to be made. This is a new endpoint and doesn't change the previous logic and code used to create other tables.
Other information:
Date Timestep column format has been set to be a date time in excel so that plotting is much simpler.