TestGen refactor (#93) #96
Conversation
alexgarzao
requested review from
XavierFilipe,
Copilot,
eminetto,
fabriciogpp,
felipeagger,
leocavalcante and
mfbmina
There was a problem hiding this comment.
Pull Request Overview
This pull request adds comparative performance benchmarking tests between ValidGen and the go-playground/validator library. The changes introduce a new test generation framework for benchmarking and refactor the existing test generation code for better maintainability.
Key Changes
- Adds comparative performance benchmark test generation infrastructure
- Introduces
excludeIfflag system to conditionally skip test generation for specific scenarios - Populates
validatorTagfields to enable comparison with go-playground/validator - Refactors test generation code to use centralized error handling and template execution
- Adds new benchmark test templates for pointer and non-pointer types
Reviewed Changes
Copilot reviewed 10 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| testgen/validations.go | Adds excludeIf flag system and populates validatorTag for all validation types to enable comparative benchmarking |
| testgen/generate_cmp_perf_tests.go | New file implementing comparative performance test generation logic |
| testgen/execute_template.go | New utility file centralizing template execution logic |
| testgen/generate_validation_types_tests.go | Refactored to use centralized template execution and error handling |
| testgen/generate_validation_code_tests.go | Refactored to use centralized template execution and error handling |
| testgen/generate_function_code_tests.go | Refactored to use centralized template execution and error handling |
| testgen/generate_tests.go | Updated main function with proper error handling and new benchmark test generation call |
| testgen/cmp_perf_pointer_tests.tpl | New template for pointer-based benchmark tests |
| testgen/cmp_perf_no_pointer_tests.tpl | New template for non-pointer benchmark tests |
| tests/cmpbenchtests/generated_tests/validator__.go | Generated validator functions (auto-generated file) |
| tests/endtoend/generated_numeric_int_tests.go | Minor whitespace fix |
| Makefile | Updated to include benchmark test generation in build pipeline |
| Field: &validInput, | ||
| } | ||
|
|
||
| for b.Loop() { |
There was a problem hiding this comment.
The method b.Loop() does not exist in the testing.B API. This should be i := 0; i < b.N; i++ to properly iterate in a benchmark test.
| Field: {{.ValidInput}}, | ||
| } | ||
|
|
||
| for b.Loop() { |
There was a problem hiding this comment.
The method b.Loop() does not exist in the testing.B API. This should be i := 0; i < b.N; i++ to properly iterate in a benchmark test.
| Field: &validInput, | ||
| } | ||
|
|
||
| for b.Loop() { |
There was a problem hiding this comment.
The method b.Loop() does not exist in the testing.B API. This should be i := 0; i < b.N; i++ to properly iterate in a benchmark test.
| Field: {{.ValidInput}}, | ||
| } | ||
|
|
||
| for b.Loop() { |
There was a problem hiding this comment.
The method b.Loop() does not exist in the testing.B API. This should be i := 0; i < b.N; i++ to properly iterate in a benchmark test.
This PR ntroduces significant improvements to the test generation system, focusing on code organization and maintainability. The main changes include refactoring the test generation logic to use a shared template execution helper, improving error handling, and adding new comparative performance benchmark tests for validation. The Makefile is updated to support these changes and to properly clean up generated test files.
Test generation refactoring and error handling:
generateValidationTypesEndToEndTests,generateValidationCodeUnitTests,generateFunctionCodeUnitTests) now return errors and propagate them tomain, which exits on failure, improving robustness and error visibility. ([testgen/generate_tests.goR5-R29](https://github.com/opencodeco/validgen/pull/96/files#diff-16af27a0e4fc1a51af4d65efde17f3f02bd2bec3b338968de08915d0d677f129R5-R29))ExecTemplateintestgen/execute_template.go, reducing duplication and centralizing template logic. ([[1]](https://github.com/opencodeco/validgen/pull/96/files#diff-029984c4d009db611889f43c97b3cf8ef992c0152a7b86b882b15a55825efd4aR1-R37),[[2]](https://github.com/opencodeco/validgen/pull/96/files#diff-5078c470f44fdb9cf8a2be0e92277ead39c1128b0de9351e1cf08762160fadf0L96-R98),[[3]](https://github.com/opencodeco/validgen/pull/96/files#diff-69c699826520c6e9d36080750cedfafd968644d48bc35ad96b0e06314076ceb5L94-R95),[[4]](https://github.com/opencodeco/validgen/pull/96/files#diff-f32b12148daad96f1b536542e4a41eb32fdcd173a8d46aceacc13296415dd43eL118-R127))excludeIf&noPointer) for consistency and clarity, replacing previous string-based logic. ([[1]](https://github.com/opencodeco/validgen/pull/96/files#diff-5078c470f44fdb9cf8a2be0e92277ead39c1128b0de9351e1cf08762160fadf0L55-R61),[[2]](https://github.com/opencodeco/validgen/pull/96/files#diff-69c699826520c6e9d36080750cedfafd968644d48bc35ad96b0e06314076ceb5L50-L58),[[3]](https://github.com/opencodeco/validgen/pull/96/files#diff-f32b12148daad96f1b536542e4a41eb32fdcd173a8d46aceacc13296415dd43eL68-L76))Comparative performance benchmark test generation:
testgen/generate_cmp_perf_tests.go,testgen/cmp_perf_no_pointer_tests.tpl, andtestgen/cmp_perf_pointer_tests.tplto generate comparative performance benchmarks betweenvalidgenandgo-playground/validator, supporting both pointer and non-pointer types. ([[1]](https://github.com/opencodeco/validgen/pull/96/files#diff-1a44c75455be18ee4c4ddd609b3ee6e09b3a2bc84be26cebd878fb4572ae5c29R1-R100),[[2]](https://github.com/opencodeco/validgen/pull/96/files#diff-14df08054696f979de87abe56e2a775ece5d0faffcf69465116430544279d245R1-R48),[[3]](https://github.com/opencodeco/validgen/pull/96/files#diff-cef0fa02ef5f0edad57244d8b87d833bd2d3385eb825e51eb07785f55366efb7R1-R52))[[1]](https://github.com/opencodeco/validgen/pull/96/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L34-R34),[[2]](https://github.com/opencodeco/validgen/pull/96/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L44-R44))General codebase improvements:
[[1]](https://github.com/opencodeco/validgen/pull/96/files#diff-5078c470f44fdb9cf8a2be0e92277ead39c1128b0de9351e1cf08762160fadf0L36-R44),[[2]](https://github.com/opencodeco/validgen/pull/96/files#diff-69c699826520c6e9d36080750cedfafd968644d48bc35ad96b0e06314076ceb5L31-R39),[[3]](https://github.com/opencodeco/validgen/pull/96/files#diff-f32b12148daad96f1b536542e4a41eb32fdcd173a8d46aceacc13296415dd43eL37-R45))fmt.Errorfand returns errors instead of callinglog.Fatalf, making the code more testable and modular. ([[1]](https://github.com/opencodeco/validgen/pull/96/files#diff-5078c470f44fdb9cf8a2be0e92277ead39c1128b0de9351e1cf08762160fadf0L96-R98),[[2]](https://github.com/opencodeco/validgen/pull/96/files#diff-69c699826520c6e9d36080750cedfafd968644d48bc35ad96b0e06314076ceb5L76-R77),[[3]](https://github.com/opencodeco/validgen/pull/96/files#diff-f32b12148daad96f1b536542e4a41eb32fdcd173a8d46aceacc13296415dd43eL92-R89),[[4]](https://github.com/opencodeco/validgen/pull/96/files#diff-f32b12148daad96f1b536542e4a41eb32fdcd173a8d46aceacc13296415dd43eL118-R127))These changes collectively make the test generation process more reliable, easier to maintain, and extend with new test types.