Comprehensive Integration Test Suite

[RyanL2004]

Please read #139

Overview

This PR introduces a robust integration test suite for GitIngest, covering critical functionalities and edge cases. These tests ensure the reliability, scalability, and correctness of the application.

What Does This PR Do?

1. Adds Comprehensive Tests

• Tests for analyzing remote repositories (e.g., input validation, processing, and output generation).
• Tests for handling edge cases, including invalid URLs, special characters, and unsupported inputs.
• Tests for analyzing large repositories and handling nested structures.

2. Simulates Concurrent Requests

• Validates the server’s ability to handle multiple simultaneous requests without failures.
• Tests rate limiting and stability under load.

3. Mocks Template Rendering

• Bypasses template file dependencies, ensuring faster and isolated testing of core logic.

4. Improves Code Coverage

• Ensures that all major code paths, including edge cases and error scenarios, are tested.

5. Ensures Graceful Error Handling

• Validates proper user feedback for invalid inputs and large files.

How Does It Work?

• Test Fixtures: Mocked templates and a test client simulate real-world scenarios while ensuring fast and isolated tests.
• Concurrent Requests: Thread pooling simulates high traffic, validating server stability.
• Edge Case Tests: Specific inputs test the system’s robustness against unusual and unsupported cases.

What Does It Test?

1. End-to-End Flows

• Repository analysis from form submission to result rendering.

2. Error Handling

• Invalid inputs, large files, and unsupported patterns.

3. Performance

• Large repositories and high concurrency.

4. Scalability

• Stability under simultaneous requests.

Why Is This Critical?

1. Reduces the risk of bugs and regressions.
2. Enhances developer confidence in making changes.
3. Improves user experience by validating common and edge-case scenarios.
4. Ensures operational stability under load.

Conclusion

This PR significantly enhances GitIngest’s testing framework, ensuring robustness, reliability, and scalability. It provides a foundation for future development with confidence in the system’s core functionalities.

[RyanL2004]

I wanted to provide some context regarding the previous failures on this PR:

1. Initial Failure:
   The CI tests initially failed because the application was attempting to mount a static directory using this line in main.py:

   app.mount("/static", StaticFiles(directory="static"), name="static")```

However, the static directory was missing in the CI environment, resulting in the following error:

RuntimeError: Directory 'static' does not exist

2. Fix:
   I corrected the directory path to src/static since that’s the actual location of thestatic folder in the project. Here’s the updated line in main.py:

app.mount("/static", StaticFiles(directory="src/static"), name="static")

3. Resolution:
   After making this change, I pushed the fix to this branch, which resolved the CI errors. The tests now pass successfully, and the PR is ready for review.

[cyclotruc]

Hey @RyanL2004 thank you very much for this work

I'll take a deeper look at the tests themselves very soon but there's an issue that needs to be tackled first:

The changes you made in main.py are breaking the web server

That makes me think that this is the first thing we should try to test:

• is the server runnable and is it possible to trigger an ingest?

Feel free to reach out on discord if you need any help in structuring those new tests

[RyanL2004]

Hi @cyclotruc,
Thank you for your feedback and for taking the time to review the work!

After some troubleshooting,
I’ve resolved the Internal Server Error (500) issue that was occurring only in the local environment while the server ran smoothly on Docker.

Root Cause:
The issue was related to how the server handled file paths, particularly for:

• Resolving static files (src/static).
• Locating templates (src/templates).
• In the local environment, the server wasn’t consistently resolving paths correctly unless the application was run explicitly from the root directory. This discrepancy didn’t appear in Docker due to its controlled environment and consistent working directory.

1. Issue:
   The web server failed to run due to an incorrect path for the static directory in the following line:

   app.mount("/static", StaticFiles(directory="src/static"), name="static")

   The directory static was assumed to be at the root level, but it was actually located in /static.

Fix: I’ve updated main.py to dynamically resolve the correct path for the static directory relative to main.py and added an if to handle potential use cases where if static is not found or unrecognized & the server is ran from a different root, it would prevent everything from crashing.

static_dir = Path(__file__).parent / "static"
if static_dir.exists():
    app.mount("/static", StaticFiles(directory=static_dir), name="static")
else:
    print(f"Warning: Static directory '{static_dir}' does not exist. Skipping static file mount.")

I may still make some modifications, based on your next feedback

[cyclotruc]

@RyanL2004 Hi sorry for the delay, finally got back to your PR thank you very much :)