Skip to content

Commit 1a0712f

Browse files
jfriedri-niCopilot
andauthored
Add system metadata example (#70)
* Create new example project for hardware and software metadata * Update mypy settings for nisyscfg * Add the new example to the checks * Exclude non-package subfolders from tool checks * Update docs with Python script descriptions --------- Signed-off-by: Joe Friedrichsen <114173023+jfriedri-ni@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 7d0895a commit 1a0712f

File tree

9 files changed

+1359
-13
lines changed

9 files changed

+1359
-13
lines changed

.github/workflows/check_examples.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
matrix:
1212
os: [windows-latest, ubuntu-latest, macos-latest]
1313
python-version: [3.14]
14-
example-name: ['overview']
14+
example-name: ['overview', 'system']
1515
runs-on: ${{ matrix.os }}
1616
steps:
1717
- name: Check out repo
@@ -28,4 +28,4 @@ jobs:
2828
- name: Analyze Python Project
2929
uses: ni/python-actions/analyze-project@9768589f3e50672173dad75a6fc181e4a85d33fa # v0.7.0
3030
with:
31-
project-directory: ${{ github.workspace }}/examples/${{ matrix.example-name }}
31+
project-directory: ${{ github.workspace }}/examples/${{ matrix.example-name }}

docs/examples/index.rst

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,14 @@
22
Examples
33
########
44

5-
This section contains Jupyter notebook examples demonstrating how to use the NI Data Store.
5+
This section contains Jupyter notebook and Python script examples demonstrating how to use the NI Data Store.
66

77
All example notebooks are located in the ``examples/notebooks/`` directory of the repository.
88

9-
OData Query Examples
10-
====================
9+
Each example script has its own project folder under ``examples/``.
10+
11+
OData Query Example Notebooks
12+
=============================
1113

1214
These notebooks demonstrate how to query data and metadata using OData syntax:
1315

@@ -20,8 +22,8 @@ These notebooks demonstrate how to query data and metadata using OData syntax:
2022
* `query_metadata.ipynb <https://github.com/ni/datastore-python/blob/main/examples/notebooks/query/query_metadata.ipynb>`_ - Demonstrates metadata queries (operators, hardware, UUTs, etc.)
2123
* `query_measurements.ipynb <https://github.com/ni/datastore-python/blob/main/examples/notebooks/query/query_measurements.ipynb>`_ - Demonstrates measurement and condition queries
2224

23-
Other Examples
24-
==============
25+
Other Example Notebooks
26+
=======================
2527

2628
Additional examples for various use cases:
2729

@@ -30,11 +32,21 @@ Additional examples for various use cases:
3032
* `custom_metadata.ipynb <https://github.com/ni/datastore-python/blob/main/examples/notebooks/custom-metadata/custom_metadata.ipynb>`_ - Custom metadata examples
3133
* `publish_waveforms.ipynb <https://github.com/ni/datastore-python/blob/main/examples/notebooks/voltage-regulator/publish_waveforms.ipynb>`_ - Publishing waveform data
3234

33-
Getting Started
34-
===============
35+
Getting Started with Notebooks
36+
==============================
3537

3638
1. **Run the setup notebook first:** Start with ``publish_sample_data.ipynb`` to create sample data
3739
2. **Explore queries:** Try the OData query examples to learn filtering and data retrieval
3840
3. **Adapt for your use case:** Use the other examples as templates for your specific needs
3941

40-
All notebooks include detailed explanations and can be run in any Jupyter environment with the ``ni.datastore`` package installed.
42+
All notebooks include detailed explanations and can be run in any Jupyter environment with the ``ni.datastore`` package installed.
43+
44+
Python Example Scripts
45+
======================
46+
47+
These Python scripts demonstrate how to publish and query both data and metadata.
48+
49+
* `overview.py <https://github.com/ni/datastore-python/blob/main/examples/overview>`_ - Publishing and querying measurement data
50+
* `system.py <https://github.com/ni/datastore-python/blob/main/examples/system>`_ - Publishing and querying system metadata
51+
52+
Each has a ``README.md`` file that describes its runtime requirements and usage.

examples/system/README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# System Example
2+
3+
`system` contains sample Python code for detecting system hardware
4+
and software resources and publishing their metadata to the
5+
[NI Measurement Data Store](https://github.com/ni/datastore-service).
6+
7+
## Required Software
8+
9+
`system` requires Python 3.10 or greater and the
10+
[NI System Configuration API](https://www.ni.com/en/support/downloads/drivers/download.system-configuration.html).
11+
Alternatively, install at least one NI driver like NI-DAQmx or NI-SCOPE.
12+
13+
## Usage
14+
15+
```python
16+
poetry install
17+
poetry run system
18+
```

examples/system/poetry.lock

Lines changed: 1091 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

examples/system/poetry.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[virtualenvs]
2+
in-project = true

examples/system/pyproject.toml

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
[project]
2+
name = "system_example"
3+
version = "0.1.0.dev0"
4+
license = "MIT"
5+
description = "Example demonstrating how to detect, publish, and query system hardware and software resources"
6+
authors = [{name = "NI", email = "opensource@ni.com"}]
7+
maintainers = [
8+
{name = "Johann Scholtz", email = "johann.scholtz@emerson.com"},
9+
{name = "Joel Dixon", email = "joel.dixon@emerson.com"}
10+
]
11+
readme = "README.md"
12+
keywords = ["system", "example"]
13+
classifiers = [
14+
"Development Status :: 3 - Alpha",
15+
"Intended Audience :: Developers",
16+
"Intended Audience :: Manufacturing",
17+
"Intended Audience :: Science/Research",
18+
"License :: OSI Approved :: MIT License",
19+
"Operating System :: Microsoft :: Windows",
20+
"Operating System :: POSIX",
21+
"Programming Language :: Python :: 3",
22+
"Programming Language :: Python :: 3.10",
23+
"Programming Language :: Python :: 3.11",
24+
"Programming Language :: Python :: 3.12",
25+
"Programming Language :: Python :: 3.13",
26+
"Programming Language :: Python :: 3.14",
27+
"Programming Language :: Python :: Implementation :: CPython",
28+
]
29+
dynamic = ["dependencies"]
30+
requires-python = ">=3.10,<4.0"
31+
32+
[project.urls]
33+
repository = "https://github.com/ni/datastore-python"
34+
35+
[project.scripts]
36+
system = "src.system:main"
37+
38+
[tool.poetry]
39+
requires-poetry = ">=2.1,<3.0"
40+
41+
[[tool.poetry.packages]]
42+
include = "src"
43+
44+
[tool.poetry.dependencies]
45+
nisyscfg = { version = ">=0.2.1", allow-prereleases = true }
46+
47+
[tool.poetry.group.dev.dependencies]
48+
ni-datastore = {path = "../..", develop = true}
49+
50+
[tool.poetry.group.lint.dependencies]
51+
ni-python-styleguide = ">=0.4.1"
52+
mypy = ">=1.0"
53+
54+
[tool.black]
55+
line-length = 100
56+
57+
[tool.mypy]
58+
mypy_path = "src"
59+
files = "."
60+
namespace_packages = true
61+
strict = true
62+
explicit_package_bases = true
63+
warn_unused_ignores = true
64+
65+
[[tool.mypy.overrides]]
66+
module = [
67+
"nisyscfg",
68+
"nisyscfg.component_info",
69+
"nisyscfg.hardware_resource",
70+
]
71+
ignore_missing_imports = true # no library stubs or py.typed marker
72+
73+
[build-system]
74+
requires = ["poetry-core>=2.1.0,<3.0.0"]
75+
build-backend = "poetry.core.masonry.api"

examples/system/src/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Detect, publish, and query system hardware and software resources."""

examples/system/src/system.py

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
"""How to detect, publish, and query system hardware and software resources."""
2+
3+
import os
4+
import platform
5+
from dataclasses import dataclass
6+
7+
import nisyscfg
8+
import nisyscfg.component_info
9+
import nisyscfg.hardware_resource
10+
from ni.datastore.data import (
11+
DataStoreClient,
12+
TestResult,
13+
)
14+
from ni.datastore.metadata import (
15+
HardwareItem,
16+
MetadataStoreClient,
17+
Operator,
18+
SoftwareItem,
19+
TestStation,
20+
)
21+
22+
23+
@dataclass(frozen=True)
24+
class SystemMetadata:
25+
"""Represents the available resources on a system."""
26+
27+
operator: Operator
28+
test_station: TestStation
29+
hardware_items: list[HardwareItem]
30+
software_items: list[SoftwareItem]
31+
32+
33+
def main() -> None:
34+
"""Detect, publish, and query hardware and software resources from the local system."""
35+
print("Scanning system for metadata...")
36+
system_metadata = detect_system_resources()
37+
38+
print("Publishing detected system metadata...")
39+
test_result_id = publish_empty_test_result(system_metadata)
40+
41+
print("Querying system metadata...")
42+
test_result = query_test_result(test_result_id)
43+
44+
print()
45+
print(f"TestResult ID: {test_result.id}")
46+
print(f"- Operator: {test_result.operator_id}")
47+
print(f"- Test Station: {test_result.test_station_id}")
48+
print(f"- Installed Software: {len(test_result.software_item_ids)} packages")
49+
print(f"- Available Hardware: {len(test_result.hardware_item_ids)} devices")
50+
51+
52+
def detect_system_resources(system_target: str = "localhost") -> SystemMetadata:
53+
"""Scan the specified system_target and return SystemMetadata describing the system."""
54+
with nisyscfg.Session(target=system_target) as session:
55+
ni_device_filter = session.create_filter()
56+
ni_device_filter.is_ni_product = True
57+
ni_device_filter.is_device = True
58+
ni_device_filter.is_present = True
59+
60+
operator = create_operator()
61+
test_station = create_test_station(session)
62+
hardware = [
63+
create_hardware_item(entry) for entry in session.find_hardware(ni_device_filter)
64+
]
65+
software = [
66+
create_software_item(entry) for entry in session.get_installed_software_components()
67+
]
68+
system_metadata = SystemMetadata(operator, test_station, hardware, software)
69+
return system_metadata
70+
71+
72+
def create_hardware_item(
73+
hardware_entry: nisyscfg.hardware_resource.HardwareResource,
74+
) -> HardwareItem:
75+
"""Create a new HardwareItem instance from the specified nisyscfg entry."""
76+
manufacturer = hardware_entry.vendor_name
77+
model = hardware_entry.product_name
78+
serial_number = hardware_entry.serial_number
79+
new_instance = HardwareItem(manufacturer=manufacturer, model=model, serial_number=serial_number)
80+
return new_instance
81+
82+
83+
def create_software_item(software_entry: nisyscfg.component_info.ComponentInfo) -> SoftwareItem:
84+
"""Create a new SoftwareItem instance from the specified nisyscfg entry."""
85+
new_instance = SoftwareItem(product=software_entry.id, version=software_entry.version)
86+
return new_instance
87+
88+
89+
def create_test_station(session: nisyscfg.Session) -> TestStation:
90+
"""Create a new TestStation instance from the specified nisyscfg session."""
91+
new_instance = TestStation(name=session.hostname)
92+
return new_instance
93+
94+
95+
def create_operator(name: str = "") -> Operator:
96+
"""Create a new Operator instance using the specified name. Otherwise, use the active user."""
97+
if not name:
98+
host_os = platform.system()
99+
if host_os == "Windows":
100+
username_variable = "USERNAME"
101+
elif host_os == "Linux":
102+
username_variable = "USER"
103+
else:
104+
raise NotImplementedError(f"{host_os} support not implemented")
105+
name = os.environ.get(username_variable, "Unknown operator")
106+
107+
new_instance = Operator(name=name)
108+
return new_instance
109+
110+
111+
def publish_empty_test_result(system_metadata: SystemMetadata) -> str:
112+
"""Publish a TestResult with the specified system_metadata and return its ID."""
113+
with MetadataStoreClient() as metadata_store_client:
114+
operator_id = metadata_store_client.create_operator(system_metadata.operator)
115+
test_station_id = metadata_store_client.create_test_station(system_metadata.test_station)
116+
hardware_item_ids = [
117+
metadata_store_client.create_hardware_item(entry)
118+
for entry in system_metadata.hardware_items
119+
]
120+
software_item_ids = [
121+
metadata_store_client.create_software_item(entry)
122+
for entry in system_metadata.software_items
123+
]
124+
125+
empty_result = TestResult(
126+
name="system metadata result",
127+
operator_id=operator_id,
128+
test_station_id=test_station_id,
129+
software_item_ids=software_item_ids,
130+
hardware_item_ids=hardware_item_ids,
131+
)
132+
133+
with DataStoreClient() as datastore_client:
134+
test_result_id = datastore_client.create_test_result(empty_result)
135+
136+
return test_result_id
137+
138+
139+
def query_test_result(test_result_id: str) -> TestResult:
140+
"""Query the NI DataStore Service for the specified TestResult and return it."""
141+
with DataStoreClient() as datastore_client:
142+
test_result = datastore_client.get_test_result(test_result_id)
143+
return test_result
144+
145+
146+
if __name__ == "__main__":
147+
main()

pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ ipykernel = ">=6.0"
5050
plotly = ">=5.0"
5151
nbformat = ">=4.2.0"
5252
ipython = ">=7.0"
53-
jupyter = ">=1.0"
53+
jupyter = ">=1.0"
5454

5555
[tool.poetry.group.lint.dependencies]
5656
bandit = { version = ">=1.7", extras = ["toml"] }
@@ -92,7 +92,7 @@ skips = [
9292
extend_exclude = "docs,examples"
9393

9494
[tool.black]
95-
extend-exclude = 'docs/|_pb2(_grpc)?\.(py|pyi)$'
95+
extend-exclude = 'docs/|examples/|_pb2(_grpc)?\.(py|pyi)$'
9696
line-length = 100
9797

9898
[tool.mypy]
@@ -101,7 +101,7 @@ files = "."
101101
namespace_packages = true
102102
strict = true
103103
explicit_package_bases = true
104-
exclude = ["docs"]
104+
exclude = ["docs", "examples"]
105105

106106
[tool.pyright]
107107
include = ["src/", "tests/"]

0 commit comments

Comments
 (0)