feat(gitlab): add projectQuery support for dynamic project discovery

[sudhanshu112233shukla]

Fixes:#679
This PR addresses issue #679 by adding support for a new projectQuery configuration options in the GitLab integration. This allows users to dynamically select projects to mirror using raw GitLab API query strings, offering greater flexibility than statically defining groups or projects.
Approach
Schema Update:
Modified :
schemas/v3/gitlab.json to include the projectQuery property (an array of strings).
Updated the description and added examples to guide users on how to construct API query paths (e.g., groups/my-group/projects?include_subgroups=true).

Backend Implementation:
Updated
packages/backend/src/gitlab.ts
to process the projectQuery array.
Implemented

• parseQuery helper to convert query strings into a format compatible with the gitbeaker library (handling booleans and numbers correctly).
• Used project.Projects.all({...}) to fetch projects matching the parsed queries.
• Deduplication: Implemented logic to ensure that if a project is returned by multiple queries (or overlaps with other config options), it is only added to the list once based on its Project ID.

Deduplication: Implemented logic to ensure that if a project is returned by multiple queries (or overlaps with other config options), it is only added to the list once based on its Project ID.

Testing
I added comprehensive tests in packages/backend/src/gitlab.test.ts to fully validate the new functionality. This includes unit tests to ensure the parseQuery helper correctly converts query strings into the expected object parameters. I also added mocked integration tests by stubbing the gitbeaker GitLab client and the api.Projects.all method, verifying that getGitLabReposFromConfig invokes the GitLab API with parameters accurately derived from the configuration strings. The tests confirm that results returned from multiple queries are properly merged and deduplicated, while also ensuring that existing exclusion logic (such as exclude.topics) continues to work as intended. All tests were successfully verified by running yarn workspace @sourcebot/backend test src/gitlab.test.ts.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sudhanshu112233shukla]

@msukkari @brendan-kellam do check it !!