Skip to content

Commit 191ce8f

Browse files
Fabio SalernoFabio Salerno
authored andcommitted
add evaluation/ directory
1 parent b1a2918 commit 191ce8f

26 files changed

+336256
-0
lines changed

evaluation/README.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Evaluation Directory Documentation
2+
3+
This directory contains code, data, and results for data extraction experiments. The evaluation is organized into three main areas: forgetting attacks, memorization attacks, and data inspection.
4+
5+
## Directory Structure
6+
7+
```
8+
evaluation/
9+
├── forgetting/
10+
│ ├── experiments/
11+
│ └── results/
12+
├── memorization/
13+
│ ├── experiments/
14+
│ └── results/
15+
└── data-inspection/
16+
└── data/
17+
```
18+
19+
## Forgetting Analysis
20+
Pre-training code attack experiments on base and fine-tuned models (RQ1-2).
21+
22+
- **experiments/**: Contains data extraction experiment code and extracted code datasets
23+
- **results/**: Contains generated tables and plots of experimental results
24+
25+
## Memorization Analysis
26+
Fine-tuning code attack experiments on fine-tuned models (RQ3).
27+
28+
- **experiments/**: Contains data extraction experiment code and extracted code datasets
29+
- **results/**: Contains generated tables and plots of experimental results
30+
31+
## Data Inspection
32+
Code data inspection for memorized code samples (RQ4).
33+
34+
### Files
35+
- **data/**: Contains tagged dataset files
36+
- **manual_tag.ipynb**: Jupyter notebook for:
37+
- Dataset preparation for manual labeling
38+
- Inter-rater agreement statistics
39+
- **rq4.ipynb**: Jupyter notebook containing tables and analysis for Research Question 4
40+
41+
The evaluation directory provides a comprehensive framework for analyzing model behavior through data extraction attacks and detailed code inspection.

evaluation/data-inspection/data/fine-tune_tags.csv

Lines changed: 79474 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)