feat: add negative filter support with `!` prefix #258

digizeph · 2025-12-23T03:24:21Z

Add support for negative (exclusion) filters across the library and CLI. Filters can now be negated by prefixing the filter type with ! to match elements that do NOT satisfy the filter condition.

Library API:

Add Filter::Negated(Box<Filter>) variant to wrap any filter for negation
Filter::new() parses ! prefix to create negated filters
Example: .add_filter("!origin_asn", "13335") excludes AS 13335

CLI:

Add --filter / -f option for generic filter expressions
Supports key=value (positive) and key!=value (negative) syntax
Can be used multiple times to combine filters
Example: bgpkit-parser --filter "origin_asn!=13335" file.mrt

All existing filter types support negation:

origin_asn, prefix, peer_ip, peer_ips, peer_asn
type, as_path, community, ip_version

Includes comprehensive tests for negative filters on both unit level and with actual MRT file data.

Add support for negative (exclusion) filters across the library and CLI. Filters can now be negated by prefixing the filter type with `!` to match elements that do NOT satisfy the filter condition. Library API: - Add `Filter::Negated(Box<Filter>)` variant to wrap any filter for negation - `Filter::new()` parses `!` prefix to create negated filters - Example: `.add_filter("!origin_asn", "13335")` excludes AS 13335 CLI: - Add `--filter` / `-f` option for generic filter expressions - Supports `key=value` (positive) and `key!=value` (negative) syntax - Can be used multiple times to combine filters - Example: `bgpkit-parser --filter "origin_asn!=13335" file.mrt` All existing filter types support negation: - origin_asn, prefix, peer_ip, peer_ips, peer_asn - type, as_path, community, ip_version Includes comprehensive tests for negative filters on both unit level and with actual MRT file data.

codecov · 2025-12-23T03:29:02Z

Codecov Report

❌ Patch coverage is 96.75926% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.39%. Comparing base (6ca8ce5) to head (c176aa6).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/parser/filter.rs	96.75%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #258      +/-   ##
==========================================
+ Coverage   91.31%   91.39%   +0.07%     
==========================================
  Files          84       84              
  Lines       14862    15078     +216     
==========================================
+ Hits        13571    13780     +209     
- Misses       1291     1298       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

This PR adds comprehensive support for negative (exclusion) filters across the bgpkit-parser library and CLI tool. Filters can now be negated by prefixing the filter type with ! to match elements that do NOT satisfy a given condition.

Key changes:

Introduces Filter::Negated(Box<Filter>) variant that wraps any filter for negation with recursive evaluation
Adds CLI --filter / -f option supporting both key=value (positive) and key!=value (negative) syntax
Includes comprehensive test coverage for negative filters with both unit tests and integration tests using real MRT data

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/parser/filter.rs	Core implementation: adds `Filter::Negated` variant, refactors `Filter::new()` to parse `!` prefix and create negated filters via new `new_base()` helper, implements negation matching logic, includes extensive unit and integration tests
src/bin/main.rs	CLI support: adds `--filter` option with `parse_filter_expression()` function to parse `key=value` and `key!=value` syntax, integrates with existing filter processing
src/lib.rs	Documentation: adds negative filter usage examples and documentation for the library API
README.md	Documentation: adds negative filter examples and CLI usage documentation
CHANGELOG.md	Documents the new negative filter feature with examples for both library and CLI usage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-23T03:29:49Z

src/parser/filter.rs

    pub fn new(filter_type: &str, filter_value: &str) -> Result<Filter, ParserError> {
+        // Check for negation prefix
+        let (negated, actual_filter_type) = if let Some(stripped) = filter_type.strip_prefix('!') {
+            (true, stripped)
+        } else {
+            (false, filter_type)
+        };
+
+        let base_filter = Self::new_base(actual_filter_type, filter_value)?;
+
+        if negated {
+            Ok(Filter::Negated(Box::new(base_filter)))
+        } else {
+            Ok(base_filter)
+        }


Double negation is possible but may not be intentional. If a user accidentally provides "!!origin_asn" as a filter type, the code will strip the first !, call new_base("!origin_asn", ...), which will fail with "unknown filter type: !origin_asn". This happens because new_base doesn't handle negation prefixes, only the outer new method does.

While this currently results in an error (which is reasonable), the error message could be confusing. Consider adding explicit validation to reject filter types starting with multiple ! characters, or document this behavior if it's intentional.

Copilot · 2025-12-23T03:29:50Z

src/parser/filter.rs

+    #[test]
+    fn test_negated_filters_on_mrt_file() {
+        let url = "https://spaces.bgpkit.org/parser/update-example.gz";
+        let parser = BgpkitParser::new(url).unwrap();
+        let elems = parser.into_elem_iter().collect::<Vec<BgpElem>>();
+
+        // Count all elems from peer 185.1.8.65
+        let filters = vec![Filter::PeerIp(IpAddr::from_str("185.1.8.65").unwrap())];
+        let count_with_peer = elems.iter().filter(|e| e.match_filters(&filters)).count();
+        assert_eq!(count_with_peer, 3393);
+
+        // Count all elems NOT from peer 185.1.8.65
+        let filters = vec![Filter::new("!peer_ip", "185.1.8.65").unwrap()];
+        let count_without_peer = elems.iter().filter(|e| e.match_filters(&filters)).count();
+        assert_eq!(count_without_peer, elems.len() - 3393);
+
+        // Verify total adds up
+        assert_eq!(count_with_peer + count_without_peer, elems.len());
+
+        // Test negated type filter
+        let filters = vec![Filter::Type(ElemType::WITHDRAW)];
+        let count_withdrawals = elems.iter().filter(|e| e.match_filters(&filters)).count();
+        assert_eq!(count_withdrawals, 379);
+
+        let filters = vec![Filter::new("!type", "w").unwrap()];
+        let count_not_withdrawals = elems.iter().filter(|e| e.match_filters(&filters)).count();
+        assert_eq!(count_not_withdrawals, elems.len() - 379);
+
+        // Test negated prefix filter
+        let filters = vec![Filter::Prefix(
+            IpNet::from_str("190.115.192.0/22").unwrap(),
+            PrefixMatchType::Exact,
+        )];
+        let count_with_prefix = elems.iter().filter(|e| e.match_filters(&filters)).count();
+
+        let filters = vec![Filter::new("!prefix", "190.115.192.0/22").unwrap()];
+        let count_without_prefix = elems.iter().filter(|e| e.match_filters(&filters)).count();
+        assert_eq!(count_with_prefix + count_without_prefix, elems.len());
+    }


The test coverage for negated filters doesn't include testing the prefix variant filters (prefix_super, prefix_sub, prefix_super_sub). While the basic !prefix filter is tested, it only tests the Exact match type. Since all prefix filter types should support negation according to the PR description, these variants should be tested as well to ensure they work correctly when negated.

Copilot · 2025-12-23T03:29:50Z

src/parser/filter.rs

+### Negative Filters
+
+All filters support negation by prefixing the filter type with `!`. For example:
+- `!origin_asn` -- matches elements where origin AS is NOT the specified value
+- `!prefix` -- matches elements where prefix is NOT the specified value
+- `!peer_ip` -- matches elements where peer IP is NOT the specified value
+


Negating timestamp filters (!ts_start and !ts_end) is technically supported but may lead to unintuitive behavior. For example:

ts_start=1000 matches elements with timestamp >= 1000

!ts_start=1000 matches elements with timestamp < 1000

This could be confusing because negating a "start" filter doesn't create an "end before" filter in the traditional sense—it creates a "before start" filter. Similarly, !ts_end would match anything after the end time.

Consider documenting this behavior explicitly in the documentation sections that mention timestamp filters can be negated, or consider whether timestamp filters should be excluded from negation support since their semantics are already range-based.

Copilot · 2025-12-23T03:29:50Z

src/parser/filter.rs

+```
+
 Note, by default, the prefix filtering is for the exact prefix. You can include super-prefixes or
 sub-prefixes when fitlering by using `"prefix_super"`, `"prefix_sub"`, or  `"prefix_super_sub"` as


There's a typo in the word "filtering" - it's spelled as "fitlering".

Suggested change

sub-prefixes when fitlering by using `"prefix_super"`, `"prefix_sub"`, or `"prefix_super_sub"` as

sub-prefixes when filtering by using `"prefix_super"`, `"prefix_sub"`, or `"prefix_super_sub"` as

Update docs and tests to reflect negation rules: timestamp filters do not support negation; double negation is rejected

digizeph requested a review from Copilot December 23, 2025 03:25

Copilot started reviewing on behalf of digizeph December 23, 2025 03:25 View session

Copilot AI reviewed Dec 23, 2025

View reviewed changes

Reject timestamp negation and double negation

c176aa6

Update docs and tests to reflect negation rules: timestamp filters do not support negation; double negation is rejected

digizeph merged commit a9d25a0 into main Dec 23, 2025
5 checks passed

digizeph deleted the feature/negative-filter branch December 23, 2025 03:47

digizeph mentioned this pull request Dec 23, 2025

[feature] negative filter bgpkit/monocle#74

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: add negative filter support with `!` prefix #258

feat: add negative filter support with `!` prefix #258

Uh oh!

digizeph commented Dec 23, 2025

Uh oh!

codecov bot commented Dec 23, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	sub-prefixes when fitlering by using `"prefix_super"`, `"prefix_sub"`, or `"prefix_super_sub"` as
	sub-prefixes when filtering by using `"prefix_super"`, `"prefix_sub"`, or `"prefix_super_sub"` as

Uh oh!

feat: add negative filter support with ! prefix #258

feat: add negative filter support with ! prefix #258

Uh oh!

Conversation

digizeph commented Dec 23, 2025

Uh oh!

codecov bot commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: add negative filter support with `!` prefix #258

feat: add negative filter support with `!` prefix #258

codecov bot commented Dec 23, 2025 •

edited

Loading