Feature/solution storage change+plotting acessors

[FBumann]

Description

Brief description of the changes in this PR.

Type of Change

• Bug fix
• New feature
• Documentation update
• Code refactoring

Related Issues

Closes #(issue number)

Testing

• I have tested my changes
• Existing tests still pass

Checklist

• My code follows the project style
• I have updated documentation if needed
• I have added tests for new functionality (if applicable)

Summary by CodeRabbit

Release Notes

• New Features

  • Added Statistics API for accessing aggregated flow rates, hours, sizes, and effects data
  • Introduced comprehensive plotting interface with balance, heatmap, flows, duration curves, and Sankey diagrams
  • Enhanced topology visualization with network info accessor
  • Added custom constraints support for advanced optimization workflows
  • New model verification capabilities

• Deprecations

  • Network visualization methods now accessible via topology API

• Documentation

  • Expanded results and plotting guides with end-to-end examples
  • Added custom constraints and model verification documentation
  • Updated migration guide with new API patterns

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[FBumann]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

pyproject.toml (1)

107-114: Critical: Duplicate dependency with conflicting version constraints.

The mkdocs-plotly-plugin package appears twice in the docs optional dependencies list:

• Line 107: mkdocs-plotly-plugin==0.1.3 (exact pin)
• Line 114: mkdocs-plotly-plugin>=0.1.3 (relaxed constraint)

This creates ambiguity and the second entry will likely override the first, making the exact pin ineffective.

Apply this diff to remove the duplicate:

     "mkdocs-literate-nav==0.6.2",
-    "mkdocs-plotly-plugin==0.1.3",
     "markdown-include==0.8.1",
     "pymdown-extensions==10.16.1",
     "pygments==2.19.2",

🧹 Nitpick comments (6)

flixopt/plotting.py (1)

1403-1414: Verify dimension name consistency after squeezing.

The fallback logic squeezes the data first, then uses data.name when expanding 1D to 2D. However, data.name refers to the original data before squeezing. If squeezing modifies the data structure significantly, using the original name might be inconsistent.

Consider using squeezed_data.name for consistency:

         squeezed_data = data.squeeze()
         if squeezed_data.ndim == 1:
             # If only 1D after squeezing, expand to 2D
-            squeezed_data = squeezed_data.expand_dims({'variable': [data.name or 'value']})
+            squeezed_data = squeezed_data.expand_dims({'variable': [squeezed_data.name or 'value']})
         fallback_args = {

tests/test_effect.py (1)

226-345: Effect share tests align with the new statistics API; consider clarifying the inline “issue” comment

The rewritten test_shares correctly:

• Solves via flow_system.optimize(highs_solver),
• Uses flow_system.statistics.effect_share_factors to check the temporal/periodic share factors implied by the share_from_temporal / share_from_periodic setup, and
• Verifies temporal_effects, periodic_effects, and total_effects against the corresponding entries in flow_system.solution.

The hard‑coded factors (including the composed paths from costs → Effect1 → Effect2/Effect3) are consistent with the share graph. The “# This is where the issue lies” comment on the ('costs', 'Effect3') factor is now ambiguous; consider either removing it or replacing it with a brief explanation of the expected composition to avoid confusion for future maintainers.

flixopt/results.py (1)

544-577: Deprecation of flow_rates, flow_hours, and sizes is clear; avoid double warnings in flow_hours

The new .. deprecated:: blocks plus warnings.warn(..., DeprecationWarning, stacklevel=2) for flow_rates(), flow_hours(), and sizes() communicate the migration path to results.plot.all_flow_rates / all_flow_hours / all_sizes and to element‑level accessors well.

One minor ergonomics point: flow_hours() currently calls self.flow_rates() to build its cache, so a call to flow_hours() will emit two deprecation warnings (one from flow_hours, one from flow_rates). If you want to keep the deprecation noise down, consider reusing the cached _flow_rates directly, e.g.:

-        if self._flow_hours is None:
-            self._flow_hours = (self.flow_rates() * self.hours_per_timestep).rename('flow_hours')
+        if self._flow_hours is None:
+            if self._flow_rates is None:
+                # Populate underlying cache once, without re‑emitting warnings
+                self._flow_rates = self._assign_flow_coords(
+                    xr.concat(
+                        [flow.flow_rate.rename(flow.label) for flow in self.flows.values()],
+                        dim=pd.Index(self.flows.keys(), name='flow'),
+                    )
+                ).rename('flow_rates')
+            self._flow_hours = (self._flow_rates * self.hours_per_timestep).rename('flow_hours')

Same idea could be applied if you introduce an internal helper for building the flow cache.

Also applies to: 598-657, 664-703

flixopt/statistics_accessor.py (3)

86-89: Consider handling multi-dimensional datasets in to_csv.

xr.Dataset.to_dataframe() can produce a MultiIndex DataFrame for multi-dimensional data, which may not export cleanly to CSV without additional handling. Consider adding reset_index() or documenting this behavior.

     def to_csv(self, path: str | Path, **kwargs: Any) -> PlotResult:
         """Export the underlying data to CSV. Returns self for chaining."""
-        self.data.to_dataframe().to_csv(path, **kwargs)
+        self.data.to_dataframe().reset_index().to_csv(path, index=False, **kwargs)
         return self

---

137-146: Minor: to_dataframe() already resets index; melt may duplicate coordinate columns.

The reset_index() call on line 144 flattens all coordinates. However, list(ds.coords.keys()) on line 145 may include non-dimension coordinates (e.g., auxiliary coords like component or component_type), which could cause issues if they're not present as columns after reset_index().

Consider using dimension names rather than all coordinate names:

     df = ds.to_dataframe().reset_index()
-    coord_cols = list(ds.coords.keys())
+    coord_cols = [c for c in ds.coords.keys() if c in df.columns]
     return df.melt(id_vars=coord_cols, var_name=var_name, value_name=value_name)

---

539-549: Nested loop inefficiency with include_flows.

The flow iteration logic (lines 540-549) is inside the conversion factor loop, meaning flows are processed repeatedly for each target effect. Consider hoisting the flow discovery outside the loop for better efficiency.

This is a minor performance concern for typical use cases but could improve clarity:

+        # Pre-compute flows if needed
+        flows_to_check = []
+        if include_flows:
+            if element not in self._fs.components:
+                raise ValueError(f'Only use Components when retrieving Effects including flows. Got {element}')
+            comp = self._fs.components[element]
+            flows_to_check = [f.label_full.split('|')[0] for f in comp.inputs + comp.outputs]
+
         for target_effect, conversion_factor in relevant_conversion_factors.items():
             label = f'{element}->{target_effect}({mode})'
             if label in self._fs.solution:
                 share_exists = True
                 da = self._fs.solution[label]
                 total = da * conversion_factor + total

-            if include_flows:
-                if element not in self._fs.components:
-                    raise ValueError(f'Only use Components when retrieving Effects including flows. Got {element}')
-                comp = self._fs.components[element]
-                flows = [f.label_full.split('|')[0] for f in comp.inputs + comp.outputs]
-                for flow in flows:
+            for flow in flows_to_check:
                     label = f'{flow}->{target_effect}({mode})'
                     ...

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e0312d3 and 173ed3a.

📒 Files selected for processing (17)

• docs/home/quick-start.md (2 hunks)
• docs/user-guide/core-concepts.md (2 hunks)
• docs/user-guide/migration-guide-v6.md (1 hunks)
• docs/user-guide/optimization/index.md (2 hunks)
• docs/user-guide/results-plotting.md (1 hunks)
• docs/user-guide/results/index.md (1 hunks)
• examples/01_Simple/simple_example.py (1 hunks)
• flixopt/flow_system.py (6 hunks)
• flixopt/plotting.py (1 hunks)
• flixopt/results.py (9 hunks)
• flixopt/statistics_accessor.py (1 hunks)
• flixopt/topology_accessor.py (1 hunks)
• mkdocs.yml (1 hunks)
• pyproject.toml (1 hunks)
• tests/test_effect.py (3 hunks)
• tests/test_flow_system_resample.py (2 hunks)
• tests/test_topology_accessor.py (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (5)

tests/test_topology_accessor.py (2)

flixopt/flow_system.py (2)

• topology (1033-1059)
• network_infos (1111-1123)

flixopt/topology_accessor.py (1)

• infos (59-99)

examples/01_Simple/simple_example.py (1)

flixopt/statistics_accessor.py (2)

• balance (648-725)
• duration_curve (1054-1130)

tests/test_flow_system_resample.py (2)

flixopt/flow_system.py (2)

• build_model (842-873)
• resample (1730-1768)

flixopt/structure.py (2)

• variables (1583-1589)
• constraints (1574-1580)

flixopt/topology_accessor.py (3)

flixopt/flow_system.py (5)

• FlowSystem (45-1772)
• connected_and_transformed (1771-1772)
• flows (1288-1294)
• plot_network (1061-1081)
• _connect_network (1198-1222)

flixopt/elements.py (1)

• Bus (183-296)

flixopt/network_app.py (2)

• flow_graph (117-180)
• shownetwork (393-790)

flixopt/statistics_accessor.py (4)

flixopt/config.py (2)

• CONFIG (181-814)
• Plotting (547-576)

flixopt/flow_system.py (7)

• FlowSystem (45-1772)
• coords (1297-1303)
• sel (1476-1504)
• solution (944-946)
• solution (949-952)
• statistics (1006-1030)
• to_dataset (541-577)

flixopt/effects.py (1)

• calculate_effect_share_factors (606-630)

flixopt/plotting.py (2)

• reshape_data_for_heatmap (654-822)
• heatmap_with_plotly (1195-1416)

🪛 LanguageTool

docs/user-guide/results-plotting.md

[style] ~3-~3: Consider a different adjective to strengthen your wording.
Context: ...ss to common plots while still allowing deep customization. ## The Plot Accessor A...

(DEEP_PROFOUND)

🔇 Additional comments (30)

mkdocs.yml (1)

29-29: LGTM! Navigation entry added for new plotting documentation.

The new "Plotting Results" entry is appropriately placed in the User Guide navigation structure after "Analyzing Results".

docs/user-guide/migration-guide-v6.md (1)

299-321: LGTM! Clear documentation of the new statistics accessor API.

The new "Statistics Accessor" section provides comprehensive examples of the new aggregated data access patterns, including flow data, effect breakdowns, and groupby capabilities. The examples clearly show the clean label format (without |flow_rate suffix) which aligns with the new API design.

docs/home/quick-start.md (2)

91-116: LGTM! Quick-start guide updated to reflect new visualization workflow.

The documentation now appropriately guides users through:

1. Optional pre-optimization topology visualization using flow_system.topology.plot()
2. Direct FlowSystem optimization
3. Multi-level result access via flow_system.solution, flow_system.statistics, and component-specific results
4. New plotting capabilities via flow_system.statistics.plot.* methods

The progression from raw solution data → aggregated statistics → visualization is clear and well-structured.

---

145-149: Well-structured workflow steps.

The updated workflow steps now include:

• Pre-optimization structure verification (step 5)
• Explicit statistics-based analysis (step 7)
• Dedicated visualization step (step 8)

This provides a more comprehensive guide for users following best practices.

docs/user-guide/optimization/index.md (2)

5-18: LGTM! Helpful pre-optimization verification guide.

The new "Verifying Your Model" section provides practical guidance on using the topology accessor to inspect system structure before optimization. The examples show both visual (plot) and programmatic (infos) approaches, which is valuable for different use cases.

---

96-196: Excellent custom constraints documentation.

The new "Custom Constraints" section provides:

• Clear workflow: build_model() → access variables → add constraints → solve()
• Practical examples (minimum runtime, linked flows, seasonal constraints)
• Comprehensive variable naming convention reference
• Model inspection guidance

This empowers advanced users to extend flixOpt while maintaining the framework's convenience.

docs/user-guide/core-concepts.md (2)

130-153: LGTM! Core concepts updated to reflect new FlowSystem-centric API.

The workflow section now correctly guides users to:

1. Use flow_system.optimize() directly instead of creating an Optimization object
2. Access results via flow_system.solution for raw data and flow_system.statistics for aggregated data
3. Use component-specific access via flow_system.components['Name'].solution

This simplifies the API and makes the workflow more intuitive.

---

190-205: Clear guidance on extending with linopy.

The advanced section now correctly shows the flow_system.build_model() → flow_system.model access pattern for custom constraints. This is consistent with the detailed examples in the optimization guide.

tests/test_flow_system_resample.py (2)

189-213: Modeling integration after resample is wired to the new build_model API correctly

Using fs_r.build_model() and asserting fs_r.model is non-None with a non‑empty variables collection is the right smoke‑test for the new workflow; the fixture setup and parameterization over with_dim still look consistent.

---

215-238: Structure comparison focuses on variable/constraint types, which matches FlowSystemModel semantics

Comparing len(fs.model.variables) / .constraints and the sets of labels.data_vars.keys() between original and resampled systems is an appropriate way to verify that resampling preserves the model’s variable/constraint types while allowing dimension sizes (e.g. time) to differ.

tests/test_topology_accessor.py (3)

11-63: Topology infos() tests give good coverage of node/edge structure

The fixture normalization plus checks on tuple shape, dict types, node/edge key presence, and per‑entry fields (label, class, infos, start, end) exercise the accessor well for a simple system and align with the implementation in TopologyAccessor.infos().

---

69-126: Plotting tests correctly handle optional pyvis and validate HTML output

The try/except ImportError pattern in test_plot_returns_network_or_none and pytest.importorskip('pyvis') in the HTML test match the contract of topology.plot(), and the content check for basic HTML markers is a pragmatic sanity check.

---

100-126: Deprecation coverage for legacy network APIs looks solid

Asserting DeprecationWarning for network_infos() and plot_network(), and then checking that network_infos() returns exactly the same (nodes, edges) as topology.infos() ensures the deprecation shims in FlowSystem stay behavior‑compatible.

docs/user-guide/results-plotting.md (1)

19-25: Please verify the PlotResult reference target

The docs link to [PlotResult][flixopt.plot_accessors.PlotResult]; just confirm that this matches the actual module/class location after the refactor (vs. e.g. a statistics accessor module) so the generated API docs hyperlink works.

flixopt/results.py (1)

1178-1185: variable_names refactor is consistent across element results and helpers

Switching _ElementResults to store variable_names and using it in:

• self.solution = self._results.solution[self.variable_names],
• variables access (self._results.model.variables[self.variable_names]),
• ComponentResults.is_storage (self._charge_state in self.variable_names),
• EffectResults.get_shares_from (prefix filter on self.variable_names), and
• Results.setup_colors.get_all_variable_names,

keeps variable selection logic centralized and avoids reliance on a private _variable_names attribute. This is coherent and shouldn’t change external behavior as long as callers construct _ElementResults with the same variables list as before.

Also applies to: 1660-1661, 1921-1922, 394-400

docs/user-guide/results/index.md (1)

199-215: Docs are in line with the new results/statistics/topology APIs; double‑check extra name consistency

This page accurately documents:

• flow_system.solution as the primary xarray Dataset,
• flow_system.statistics (including flow_rates, flow_hours, sizes, charge_states, and effect aggregates),
• flow_system.statistics.plot.* for visualization, and
• flow_system.topology.* for network visualization and infos().

One thing to verify is that the recommended extra flixopt[network_viz] in the “Optional Dependencies” note matches the actual extras name(s) used in runtime ImportError messages (some existing code still mentions flixopt[viz]). Aligning those names will avoid confusion for users installing optional deps.

Also applies to: 134-181, 3-22

flixopt/topology_accessor.py (3)

59-99: Topology infos() implementation matches test and doc expectations

infos():

• Lazily connects/transforms the FlowSystem if needed,
• Produces node entries keyed by label_full with label, class (Bus vs Component), and textual infos, and
• Produces edge entries keyed by flow.label_full with start/end derived from is_input_in_component.

This aligns with how tests assert membership and with the documented example usage for topology inspection.

---

101-181: plot() and start_app() cleanly delegate visualization and handle optional deps

The plot() method reuses infos() and delegates to plotting.plot_network, correctly honoring path, controls, and show (defaulting via CONFIG.Plotting.default_show). start_app():

• Warns that the feature is experimental,
• Fails fast with a clear ImportError when Dash/Cytoscape deps are missing, and
• Ensures the network is connected (via _connect_network()) before building the NetworkX graph.

This matches the new flow_system.topology.* docs and the legacy behavior wrapped by the deprecated FlowSystem methods.

---

192-220: stop_app() shutdown logic is robust against missing deps and missing app instance

stop_app() mirrors the dependency check from start_app(), logs a warning when no app is running, and guards the shutdown in a try/except, always clearing self._fs._network_app in the finally block. This is a reasonable lifecycle wrapper around the underlying Dash server instance.

flixopt/flow_system.py (3)

210-218: solution backing field and statistics cache invalidation are implemented correctly

Introducing _solution with a solution property and a setter that clears _statistics ensures:

• Code that uses flow_system.solution remains clean and typed as xr.Dataset | None, and
• Any change of solution (e.g. after solve() or from_dataset()) forces the next access to flow_system.statistics to rebuild its accessor, preventing stale cached statistics for the common “access via property” pattern.

This is a sensible way to manage statistics lifetime without touching the underlying FlowSystemModel.

Also applies to: 943-953

---

1006-1030: statistics accessor wiring matches the intended API

The statistics property:

• Lazily creates StatisticsAccessor(self) on first use,
• Caches it in _statistics, and
• Relies on the solution setter to invalidate the cache on new solves.

The docstring and examples (flow_system.statistics.plot.balance(...), .heatmap(...), .flow_rates) align with the new user‑facing API described in the docs. Just ensure StatisticsAccessor itself guards against self._fs.solution is None where appropriate.

---

1032-1123: Topology accessor property and deprecation wrappers are clean and backward‑compatible

• topology consistently returns a TopologyAccessor(self), exposing .plot(), .start_app(), .stop_app(), and .infos() under the new namespace.
• plot_network, start_network_app, stop_network_app, and network_infos all:
  • Emit a DeprecationWarning with the shared DEPRECATION_REMOVAL_VERSION, and
  • Delegate directly to the corresponding topology methods.

This keeps existing code working while clearly steering users toward the new accessor‑based API.

flixopt/statistics_accessor.py (8)

1-39: LGTM!

The module structure, imports, and docstring are well-organized. Good use of TYPE_CHECKING to avoid circular imports with FlowSystem.

---

41-46: LGTM!

Clear type aliases with helpful docstrings explaining their purpose.

---

1132-1271: LGTM!

The effects method is well-designed with proper validation, flexible grouping options (by='component', by='contributor', by='time'), and correct handling of dimension aggregation for each case.

---

875-989: LGTM!

The Sankey diagram implementation correctly handles flow direction semantics (bus→component vs component→bus), applies proper weight normalization, and produces meaningful visualization data.

---

648-726: LGTM!

The balance method correctly handles both Bus and Component nodes, properly negates input flows for visualization semantics, and has good error handling for edge cases.

---

831-854: LGTM!

The flow filtering logic correctly handles flow direction semantics using is_input_in_component to determine source/target relationships. The filtering by start/end/component is intuitive.

---

555-569: Consider edge case when no coordinates are present.

Line 568-569 returns a scalar xr.DataArray(np.nan) when no coordinates are available. This is valid but worth noting that it produces a 0-dimensional array, which may require special handling downstream when concatenating with dimensioned arrays (handled on line 608-609).

---

904-906: No action needed; scenario weight normalization is correct.

The code properly normalizes scenario_weights before applying them. FlowSystem.scenario_weights stores raw, unnormalized values by design—normalization is applied contextually depending on usage. In the statistics accessor, normalizing ensures the weighted result preserves the dataset's overall magnitude while distributing it across scenarios proportionally. This pattern aligns with how scenario weights are used throughout the codebase.

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Find statistics_accessor.py file
find . -name "statistics_accessor.py" -type f

Repository: flixOpt/flixopt

Length of output: 91

---

🏁 Script executed:

#!/bin/bash
# Read the duration_curve method from statistics_accessor.py
sed -n '1053,1129p' ./flixopt/statistics_accessor.py

Repository: flixOpt/flixopt

Length of output: 2614

---

🏁 Script executed:

#!/bin/bash
# Check the exact code at line 130 in simple_example.py
sed -n '125,135p' ./examples/01_Simple/simple_example.py

Repository: flixOpt/flixopt

Length of output: 260

---

🏁 Script executed:

#!/bin/bash
# Search for how flow_rates is constructed or documented
rg "flow_rates" ./flixopt/statistics_accessor.py -B 2 -A 2 | head -40

Repository: flixOpt/flixopt

Length of output: 1391

---

Fix duration_curve label to use clean flow name without suffix.

The duration_curve method accesses the flow_rates dataset, which stores flow variables with clean labels (the |flow_rate suffix is removed during processing). Line 130 uses 'Boiler(Q_th)|flow_rate', which is the raw solution variable name and will fail when accessed in the flow_rates dictionary.

-    optimization.results.plot.duration_curve('Boiler(Q_th)|flow_rate')
+    optimization.results.plot.duration_curve('Boiler(Q_th)')

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	optimization.results.plot.balance('Fernwärme')
	optimization.results.plot.duration_curve('Boiler(Q_th)|flow_rate')
	optimization.results.plot.balance('Fernwärme')
	optimization.results.plot.duration_curve('Boiler(Q_th)')

🤖 Prompt for AI Agents

In examples/01_Simple/simple_example.py around lines 129 to 130, the call to
duration_curve uses the raw solution variable name with the '|flow_rate' suffix
which doesn't match the cleaned keys in the flow_rates dataset; change the label
passed to duration_curve to the cleaned flow name (remove the '|flow_rate'
suffix), e.g. use 'Boiler(Q_th)' instead of 'Boiler(Q_th)|flow_rate' so the
method can find the flow_rates entry.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Asymmetric NaN handling in validation check.

The validation compares computed.values directly against found.fillna(0).values, but computed doesn't receive the same fillna(0) treatment. This asymmetry could cause false warnings when computed contains NaN values that would otherwise match.

-                if not np.allclose(computed.values, found.fillna(0).values):
+                if not np.allclose(computed.fillna(0).values, found.fillna(0).values, equal_nan=True):

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	computed = ds[effect].sum('contributor')
	found = self._fs.solution[label]
	if not np.allclose(computed.values, found.fillna(0).values):
	logger.critical(
	f'Results for {effect}({mode}) in effects_dataset doesnt match {label}\n{computed=}\n, {found=}'
	)
	computed = ds[effect].sum('contributor')
	found = self._fs.solution[label]
	if not np.allclose(computed.fillna(0).values, found.fillna(0).values, equal_nan=True):
	logger.critical(
	f'Results for {effect}({mode}) in effects_dataset doesnt match {label}\n{computed=}\n, {found=}'
	)

🤖 Prompt for AI Agents

In flixopt/statistics_accessor.py around lines 625 to 630, the validation
compares computed.values to found.fillna(0).values asymmetrically causing false
mismatches when computed contains NaNs; fix by ensuring symmetric NaN handling —
either apply the same fillna(0) to computed (e.g., computed = computed.fillna(0)
before comparing) or use np.allclose(computed.values, found.fillna(0).values,
equal_nan=True); update the logger conditional to use the chosen symmetric
comparison so NaNs are treated consistently.