Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 43 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,56 @@
# CodeSecTools
<!--start-include-->
# CodeSecTools <!-- omit in toc -->

<div align="center">
<img src="docs/assets/logo.svg" alt="Logo" style="width: 200px; height: auto;" />
</div>

A framework for code security that provides abstractions for static analysis tools and datasets to support their integration, testing, and evaluation.

<!--end-include-->

## Table Of Contents <!-- omit in toc -->
- [Overview](#overview)
- [Features](#features)
- [SAST Tool Integration Status](#sast-tool-integration-status)
- [Installation](#installation)
- [Usage](#usage)
- [Command-line interface](#command-line-interface)
- [Python API](#python-api)

<!--start-include-->
## Overview

**CodeSecTools** is a collection of scripts and wrappers that abstract external resources (such as SAST tools, datasets, and codebases), providing standardized interfaces to help them interact easily.

For more details on the design and integration of SAST tools and datasets in CodeSecTools, please refer to the [documentation](https://oppida.github.io/CodeSecTools).

<div align="center">
<img src="docs/assets/overview.svg" alt="CodeSecTools Overview" style="width: 75%; height: auto;" />
</div>

## Features

- **Standardized SAST Tool Integration**: Provides a common abstraction layer for integrating various SAST tools. Once a tool is integrated, it automatically benefits from the framework’s core functionalities.
- **Unified Dataset Integration**: Uses a similar abstraction for handling datasets, allowing for consistent benchmarking of SAST tools across different sets of codebases, whether they are collections of individual files or entire Git repositories.
- **Project Analysis and Benchmarking**: Users can analyze their own projects or benchmark SAST tools against curated datasets to evaluate their effectiveness, including metrics like true positives, false positives, and false negatives.
- **Concurrent Analysis for Cross-Verification**: CodeSecTools can run multiple SAST tools simultaneously on the same project. This allows for the aggregation and cross-verification of results, increasing confidence in the identified vulnerabilities by highlighting findings reported by multiple tools.
- **Automated Reporting and Visualization**: The framework can generate detailed reports in HTML format and create graphs to visualize analysis results, helping to identify trends such as the most common CWEs or the files with the highest number of defects.

> [!WARNING]
> This project provides wrappers and scripts to integrate with various third-party static analysis security testing (SAST) tools and datasets. It is important to note that this project **does not include** these third-party tools or datasets, unless otherwise specified. When a tool or dataset is included, its associated license file is also provided.
> This project provides wrappers and scripts to integrate with various third-party static analysis security testing (SAST) tools and datasets. By default, this project **does not include third-party tools or datasets**. In the few instances where they are included, their associated license files are provided.
>
> Users of this project are solely responsible for reviewing, understanding, and complying with the licenses and terms of use associated with any third-party tools or datasets they choose to use through this framework. The respective licenses and terms can be found on the official websites or in the documentation of each tool or dataset.

## SAST Tool Integration Status

|SAST Tool|Languages|Maintained|Continuous Testing|Last Test Date|
|:---:|:---:|:---:|:---:|:---:|
|Coverity|Java|⚠️<br>(Deprioritized)|❌<br>(Proprietary)|October 2025|
|Semgrep Community Edition|C/C++, Java|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
|Snyk Code|C/C++, Java|✅|❌<br>(Rate limited)|November 2025|
|Bearer|Java|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
|SpotBugs|Java|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
|Cppcheck|C/C++|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|

## Installation

Expand All @@ -45,24 +77,14 @@ cd CodeSecTools
pip install .
```

## Features

- **Standardized SAST Tool Integration**: Provides a common abstraction layer for integrating various SAST tools. Once a tool is integrated, it automatically benefits from the framework’s core functionalities.
- **Unified Dataset Integration**: Uses a similar abstraction for handling datasets, allowing for consistent benchmarking of SAST tools across different sets of codebases, whether they are collections of individual files or entire Git repositories.
- **Project Analysis and Benchmarking**: Users can analyze their own projects or benchmark SAST tools against curated datasets to evaluate their effectiveness, including metrics like true positives, false positives, and false negatives.
- **Concurrent Analysis for Cross-Verification**: CodeSecTools can run multiple SAST tools simultaneously on the same project. This allows for the aggregation and cross-verification of results, increasing confidence in the identified vulnerabilities by highlighting findings reported by multiple tools.
- **Automated Reporting and Visualization**: The framework can generate detailed reports in HTML format and create graphs to visualize analysis results, helping to identify trends such as the most common CWEs or the files with the highest number of defects.
- Update the project:

- Pull the latest changes:
```bash
git pull
```

### SAST Tool Integration Status

|SAST Tool|Languages|Maintained|Continuous Testing|Last Test Date|
|:---:|:---:|:---:|:---:|:---:|
|Coverity|Java|❌<br>(Proprietary)|❌<br>(Proprietary)|October 2025|
|Semgrep Community Edition|C/C++, Java|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
|Snyk Code|C/C++, Java|✅|❌<br>(Rate limited)|November 2025|
|Bearer|Java|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
|SpotBugs|Java|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
|Cppcheck|C/C++|✅|✅|[Latest PR](https://github.com/OPPIDA/CodeSecTools/actions/workflows/ci.yaml)|
- Reinstall (in case dependencies changed)

## Usage

Expand Down Expand Up @@ -123,13 +145,4 @@ for plot_function in graphics.plot_functions:
fig = plot_function()
fig.show()
```

## Documentation

The documentation is available [online](https://oppida.github.io/CodeSecTools/).

Or, you can serve it locally:
```bash
pip install .[docs]
mkdocs serve
```
<!--end-include-->
1 change: 1 addition & 0 deletions codesectools/sasts/all/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Initializes the All SAST module."""
4 changes: 4 additions & 0 deletions docs/api/allsast.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
::: codesectools.sasts.all
options:
show_submodules: true
show_if_no_docstring: true
1 change: 1 addition & 0 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@
Available APIs:

- [`sasts`](./sast.md)
- [`allsast`](./allsast.md)
- [`datasets`](./dataset.md)
- [`shared`](./shared.md)
120 changes: 105 additions & 15 deletions docs/assets/overview.excalidraw
Original file line number Diff line number Diff line change
Expand Up @@ -181,11 +181,41 @@
"autoResize": true,
"lineHeight": 1.25
},
{
"id": "Yp-Ops1lm44S4mFwzv_v2",
"type": "rectangle",
"x": 719.2928134448903,
"y": 269.6718345779226,
"width": 362.3999938964843,
"height": 141.5999298095703,
"angle": 0,
"strokeColor": "#1e1e1e",
"backgroundColor": "#e9ecef",
"fillStyle": "solid",
"strokeWidth": 4,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"frameId": null,
"index": "b22G",
"roundness": {
"type": 3
},
"seed": 485793748,
"version": 1598,
"versionNonce": 1114887916,
"isDeleted": false,
"boundElements": [],
"updated": 1764004560433,
"link": null,
"locked": false
},
{
"id": "13dkZHvSDLQKbfPK5JpGL",
"type": "rectangle",
"x": 709.7904386974515,
"y": 281.89503914969293,
"x": 708.9904509044827,
"y": 281.09506661551325,
"width": 362.3999938964843,
"height": 141.5999298095703,
"angle": 0,
Expand All @@ -203,24 +233,24 @@
"type": 3
},
"seed": 1483036838,
"version": 1499,
"versionNonce": 1672335482,
"version": 1500,
"versionNonce": 1068640724,
"isDeleted": false,
"boundElements": [
{
"type": "text",
"id": "fsuA80u19oDfd_WXy5fbY"
}
],
"updated": 1760631995840,
"updated": 1764004513531,
"link": null,
"locked": false
},
{
"id": "fsuA80u19oDfd_WXy5fbY",
"type": "text",
"x": 815.8244734264554,
"y": 286.89503914969293,
"x": 815.0244856334866,
"y": 286.09506661551325,
"width": 150.33192443847656,
"height": 35,
"angle": 0,
Expand All @@ -236,11 +266,11 @@
"index": "b24",
"roundness": null,
"seed": 1552723450,
"version": 1387,
"versionNonce": 894148965,
"version": 1388,
"versionNonce": 1022742356,
"isDeleted": false,
"boundElements": [],
"updated": 1760632001741,
"updated": 1764004513531,
"link": null,
"locked": false,
"text": "SAST Tool",
Expand All @@ -253,6 +283,36 @@
"autoResize": true,
"lineHeight": 1.25
},
{
"id": "ljpVgLZj0RDw1SkVJgIQY",
"type": "rectangle",
"x": 1107.2928287036793,
"y": 226.27181779325466,
"width": 172.79998779296875,
"height": 151.99999999999997,
"angle": 0,
"strokeColor": "#1e1e1e",
"backgroundColor": "#e9ecef",
"fillStyle": "solid",
"strokeWidth": 4,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"frameId": null,
"index": "b24G",
"roundness": {
"type": 3
},
"seed": 892714580,
"version": 820,
"versionNonce": 1133845228,
"isDeleted": false,
"boundElements": [],
"updated": 1764004596046,
"link": null,
"locked": false
},
{
"id": "ehh88Kle4w-5mpVadmmlV",
"type": "rectangle",
Expand Down Expand Up @@ -325,6 +385,36 @@
"autoResize": true,
"lineHeight": 1.25
},
{
"id": "Y1koFTBntLWjV_oBg9Et0",
"type": "rectangle",
"x": 1105.6928531177418,
"y": 400.07185136259056,
"width": 172.79998779296875,
"height": 261.59994506835926,
"angle": 0,
"strokeColor": "#1e1e1e",
"backgroundColor": "#e9ecef",
"fillStyle": "solid",
"strokeWidth": 4,
"strokeStyle": "solid",
"roughness": 1,
"opacity": 100,
"groupIds": [],
"frameId": null,
"index": "b26G",
"roundness": {
"type": 3
},
"seed": 1150040660,
"version": 1164,
"versionNonce": 891305940,
"isDeleted": false,
"boundElements": [],
"updated": 1764004639763,
"link": null,
"locked": false
},
{
"id": "bm_HYs349MNhCJXSJAr-s",
"type": "rectangle",
Expand Down Expand Up @@ -864,20 +954,20 @@
"index": "b2K",
"roundness": null,
"seed": 123664058,
"version": 1623,
"versionNonce": 199624230,
"version": 1625,
"versionNonce": 396867028,
"isDeleted": false,
"boundElements": [],
"updated": 1760631995840,
"updated": 1764004650574,
"link": null,
"locked": false,
"text": "Abstract\nSAST result",
"text": "Abstracted\nSAST result",
"fontSize": 16,
"fontFamily": 5,
"textAlign": "center",
"verticalAlign": "middle",
"containerId": "8qAhylsGA8gx_F27etVCZ",
"originalText": "Abstract\nSAST result",
"originalText": "Abstracted\nSAST result",
"autoResize": true,
"lineHeight": 1.25
},
Expand Down
2 changes: 1 addition & 1 deletion docs/assets/overview.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ nav:
- API Reference:
- api/index.md
- SAST Tools: api/sast.md
- All SAST: api/allsast.md
- Datasets: api/dataset.md
- Shared: api/shared.md

Expand Down Expand Up @@ -75,6 +76,8 @@ plugins:
nav_file: SUMMARY.md
- include-markdown:
rewrite_relative_urls: true
start: <!--start-include-->
end: <!--end-include-->
- mkdocstrings:
handlers:
python:
Expand Down