Skip to content

Commit 289796e

Browse files
haochengxiaCopilot
haochengxia
and
Copilot
authored
[DOC] Add description for Pybind in main README.md (#262)
* Add description for Pybind in main README.md * Update README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * CI: add test pypi release * Bump to v0.3.2 * Add permission for workflows --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent deb3134 commit 289796e

File tree

15 files changed

+520
-346
lines changed

15 files changed

+520
-346
lines changed

‎.github/workflows/npm-release.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ on:
1010
- 'libCacheSim-node/**'
1111

1212
permissions:
13-
contents: read # Default permission for reading repository contents
13+
contents: read
14+
actions: read
1415

1516
env:
1617
BUILD_TYPE: Release

‎.github/workflows/pypi-release.yml‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
name: Python Package
2+
3+
on:
4+
release:
5+
types: [published]
6+
workflow_dispatch: # Allow manual triggering
7+
8+
permissions:
9+
contents: read
10+
actions: read
11+
id-token: write
12+
13+
jobs:
14+
build-wheels:
15+
name: Build wheels on ${{ matrix.os }}
16+
runs-on: ${{ matrix.os }}
17+
strategy:
18+
matrix:
19+
os: [ubuntu-latest]
20+
21+
steps:
22+
- uses: actions/checkout@v4
23+
with:
24+
submodules: recursive
25+
26+
- name: Set up Python
27+
uses: actions/setup-python@v4
28+
with:
29+
python-version: '3.11'
30+
31+
- name: Set up Docker Buildx (Linux only)
32+
if: runner.os == 'Linux'
33+
uses: docker/setup-buildx-action@v3
34+
35+
- name: Install dependencies
36+
run: |
37+
python -m pip install --upgrade pip
38+
python -m pip install cibuildwheel
39+
40+
- name: Sync Python version
41+
run: python scripts/sync_python_version.py
42+
43+
- name: Build main libCacheSim library
44+
run: |
45+
sudo apt-get update && sudo apt-get install -y cmake ninja-build libglib2.0-dev || true
46+
brew install cmake ninja glib || true
47+
rm -rf ./build
48+
cmake -G Ninja -B build
49+
ninja -C build
50+
51+
- name: Verify Docker (Linux only)
52+
if: runner.os == 'Linux'
53+
run: |
54+
docker --version
55+
docker info
56+
echo "Docker is ready for cibuildwheel"
57+
58+
- name: Build wheels
59+
env:
60+
CIBW_SKIP: "*-musllinux* *-manylinux_i686"
61+
CIBW_ARCHS_LINUX: "x86_64"
62+
CIBW_BEFORE_ALL_LINUX: |
63+
yum install -y yum-utils && yum-config-manager --set-enabled crb && yum install -y ninja-build cmake libzstd-devel glib2-devel
64+
CIBW_TEST_COMMAND: "python -c 'import libcachesim; print(\"Import successful\")'"
65+
run: python -m cibuildwheel libCacheSim-python --output-dir wheelhouse
66+
67+
- name: Upload wheels as artifacts
68+
uses: actions/upload-artifact@v4
69+
with:
70+
name: wheels-${{ matrix.os }}
71+
path: wheelhouse/*.whl
72+
73+
build-sdist:
74+
name: Build source distribution
75+
runs-on: ubuntu-latest
76+
steps:
77+
- uses: actions/checkout@v4
78+
with:
79+
submodules: recursive
80+
81+
- name: Set up Python
82+
uses: actions/setup-python@v4
83+
with:
84+
python-version: '3.11'
85+
86+
- name: Install build dependencies
87+
run: |
88+
python -m pip install --upgrade pip
89+
python -m pip install build
90+
91+
- name: Sync Python version
92+
run: python scripts/sync_python_version.py
93+
94+
- name: Build source distribution
95+
run: python -m build --sdist libCacheSim-python --outdir dist/
96+
97+
- name: Upload sdist as artifact
98+
uses: actions/upload-artifact@v4
99+
with:
100+
name: sdist
101+
path: dist/*.tar.gz
102+
103+
publish-to-pypi:
104+
name: Publish to PyPI
105+
needs: [build-wheels, build-sdist]
106+
runs-on: ubuntu-latest
107+
if: github.event_name == 'release' && github.event.action == 'published'
108+
environment:
109+
name: pypi
110+
url: https://pypi.org/p/libcachesim
111+
permissions:
112+
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
113+
114+
steps:
115+
- name: Download all artifacts
116+
uses: actions/download-artifact@v4
117+
with:
118+
path: dist/
119+
120+
- name: Flatten artifacts directory
121+
run: |
122+
mkdir -p final-dist
123+
find dist/ -name "*.whl" -exec cp {} final-dist/ \;
124+
find dist/ -name "*.tar.gz" -exec cp {} final-dist/ \;
125+
ls -la final-dist/
126+
127+
- name: Publish to PyPI
128+
uses: pypa/gh-action-pypi-publish@release/v1
129+
with:
130+
packages-dir: final-dist/
131+
skip-existing: true
132+
133+
publish-to-test-pypi:
134+
name: Publish to TestPyPI
135+
needs: [build-wheels, build-sdist]
136+
runs-on: ubuntu-latest
137+
if: github.event_name == 'workflow_dispatch'
138+
environment:
139+
name: testpypi
140+
url: https://test.pypi.org/p/libcachesim
141+
permissions:
142+
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
143+
144+
steps:
145+
- name: Download all artifacts
146+
uses: actions/download-artifact@v4
147+
with:
148+
path: dist/
149+
150+
- name: Flatten artifacts directory
151+
run: |
152+
mkdir -p final-dist
153+
find dist/ -name "*.whl" -exec cp {} final-dist/ \;
154+
find dist/ -name "*.tar.gz" -exec cp {} final-dist/ \;
155+
ls -la final-dist/
156+
157+
- name: Publish to TestPyPI
158+
uses: pypa/gh-action-pypi-publish@release/v1
159+
with:
160+
repository-url: https://test.pypi.org/legacy/
161+
packages-dir: final-dist/
162+
skip-existing: true

‎.github/workflows/python.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ name: Python
22

33
on: [push, pull_request]
44

5+
permissions:
6+
contents: read
7+
58
jobs:
69
build:
710
runs-on: ubuntu-latest

‎README.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
11
# libCacheSim - building and running cache simulations
22

33
[![build](https://github.com/1a1a11a/libCacheSim/actions/workflows/build.yml/badge.svg)](https://github.com/1a1a11a/libCacheSim/actions/workflows/build.yml)
4+
[![Python-package GitHub Actions Build Status](https://github.com/1a1a11a/libCacheSim/actions/workflows/pypi-release.yml/badge.svg)](https://github.com/1a1a11a/libCacheSim/actions/workflows/pypi-release.yml)
5+
[![Python Versions](https://img.shields.io/pypi/pyversions/libcachesim.svg?logo=python&logoColor=white)](https://pypi.org/project/libcachesim)
6+
[![PyPI Version](https://img.shields.io/pypi/v/libcachesim.svg?logo=pypi&logoColor=white)](https://pypi.org/project/libcachesim)
47
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/1a1a11a/libCacheSim/badge)](https://scorecard.dev/viewer/?uri=github.com/1a1a11a/libCacheSim)
58

9+
610
<!-- TOC start (generated with https://github.com/derlin/bitdowntoc) -->
711
- [libCacheSim - building and running cache simulations](#libcachesim---building-and-running-cache-simulations)
812
- [News](#news)
@@ -29,6 +33,9 @@
2933
- [Miss ratio curves profiling](#miss-ratio-curves-profiling)
3034
- [Using libCacheSim as a library](#using-libcachesim-as-a-library)
3135
- [Extending libCacheSim (new algorithms and trace types)](#extending-libcachesim-new-algorithms-and-trace-types)
36+
- [Python package](#python-package)
37+
- [Simulation with python](#simulation-with-python)
38+
- [Extending new algorithm](#extending-new-algorithm)
3239
- [Open source cache traces](#open-source-cache-traces)
3340
- [Contributions](#contributions)
3441
- [Reference](#reference)
@@ -309,6 +316,66 @@ If you need to add a new trace type or a new algorithm, please see [here](/doc/a
309316

310317
We encourage the users to check [deepWiki](https://deepwiki.com/1a1a11a/libCacheSim) for a more detailed documentation.
311318

319+
---
320+
<!-- TOC --><a name="python-package"></a>
321+
## Python package
322+
323+
If you are not extremely sensitive to the performance, our python binding can offer you an easier way to access the core feature of libCacheSim.
324+
325+
```shell
326+
pip install libcachesim
327+
```
328+
329+
> [!NOTE]
330+
> Built python package only support linux now. We will support more platform soon. You can also install it from source, see [instructions](./libCacheSim-python/README.md#installation-from-sources).
331+
332+
### Simulation with python
333+
334+
```python
335+
import libcachesim as lcs
336+
337+
reader = lcs.open_trace("./data/cloudPhysicsIO.oracleGeneral.bin")
338+
cache = lcs.FIFO(cache_size=1024*1024)
339+
miss_ratio = cache.process_trace(reader)
340+
print(f"Miss ratio: {miss_ratio:.4f}")
341+
```
342+
343+
### Extending new algorithm
344+
345+
With python package, you can extend new algorithm to test your own eviction design **without any C/C++ compilation**.
346+
347+
```python
348+
import libcachesim as lcs
349+
from collections import deque
350+
351+
cache = lcs.PythonHookCachePolicy(cache_size=1024, cache_name="CustomFIFO")
352+
353+
def init_hook(cache_size):
354+
return deque() # Use deque for FIFO order
355+
356+
def hit_hook(fifo_queue, obj_id, obj_size):
357+
pass # FIFO doesn't reorder on hit
358+
359+
def miss_hook(fifo_queue, obj_id, obj_size):
360+
fifo_queue.append(obj_id) # Add to end of queue
361+
362+
def eviction_hook(fifo_queue, obj_id, obj_size):
363+
return fifo_queue[0] # Return first item (oldest)
364+
365+
def remove_hook(fifo_queue, obj_id):
366+
if fifo_queue and fifo_queue[0] == obj_id:
367+
fifo_queue.popleft()
368+
369+
# Set the hooks and test
370+
cache.set_hooks(init_hook, hit_hook, miss_hook, eviction_hook, remove_hook)
371+
372+
reader = lcs.open_trace("./data/cloudPhysicsIO.oracleGeneral.bin")
373+
miss_ratio = cache.process_trace(reader)
374+
print(f"Miss ratio: {miss_ratio:.4f}")
375+
```
376+
377+
378+
See more information in [README.md](./libCacheSim-python/README.md) of the Python binding.
312379

313380
---
314381
<!-- TOC --><a name="open-source-cache-traces"></a>

0 commit comments

Comments
 (0)