OdbDesign Test Suite Documentation
Overview
This document describes the automated test suite for the OdbDesign project. The test suite provides comprehensive coverage including unit tests, integration tests, performance tests, and error condition testing.
Test Framework
The project uses GoogleTest (gtest) and GoogleMock (gmock) frameworks for automated testing. Tests are automatically discovered and executed by CMake’s CTest runner.
Test Structure
Test Categories
- Unit Tests - Test individual components in isolation
- Integration Tests - Test component interactions and workflows
- Performance Tests - Measure performance and detect regressions
- Error Condition Tests - Test error handling and edge cases
Test Files
| File | Purpose | Test Types |
|---|---|---|
TestTests.cpp |
Basic test infrastructure verification | Unit |
ArchiveTests.cpp |
Archive functionality | Unit |
ArchiveExtractorTests.cpp |
Archive extraction | Unit |
FileArchiveTests.cpp |
File archive operations | Unit |
FileArchiveLoadTests.cpp |
File archive loading | Unit |
FileReaderTests.cpp |
File reading functionality | Unit |
CrossPlatformTests.cpp |
Cross-platform compatibility | Unit |
DesignCacheLoadTests.cpp |
Design cache operations | Unit |
ProtobufSerializationTests.cpp |
Protobuf serialization | Unit |
PerformanceTests.cpp |
Performance benchmarks and stress tests | Performance |
EnhancedTests.cpp |
Error conditions and edge cases | Unit/Error |
IntegrationTests.cpp |
End-to-end workflows and component integration | Integration |
Test Fixtures
| Fixture | Purpose |
|---|---|
TestDataFixture |
Base fixture providing test data access |
FileArchiveLoadFixture |
Fixture for file archive testing |
TestUtils |
Utility functions for test automation |
Running Tests
Local Development
Using the Test Runner Script
The easiest way to run tests locally is using the provided test runner script:
# Basic test run
./scripts/run-tests.sh
# Clean build with verbose output
./scripts/run-tests.sh --clean --verbose
# Release build with coverage
./scripts/run-tests.sh --release --coverage
# Performance tests only
./scripts/run-tests.sh --performance
# Show all options
./scripts/run-tests.sh --help
Manual CMake/CTest
# Configure and build
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug
cmake --build .
# Run all tests
ctest --output-on-failure
# Run tests with verbose output
ctest --verbose
# Run specific test pattern
ctest -R "Performance"
# Run tests in parallel
ctest -j 4
Prerequisites
- Required Tools:
- CMake 3.21+
- Ninja build system (recommended)
- C++17 compatible compiler
- Dependencies:
- vcpkg package manager (for dependencies)
- GoogleTest (automatically fetched by CMake)
- Environment Variables:
ODB_TEST_DATA_DIR- Path to test data directory (optional)VCPKG_ROOT- Path to vcpkg installation
Test Data
Some tests require test data files. The test suite can work in two modes:
- With Test Data: Set
ODB_TEST_DATA_DIRenvironment variable pointing to test data - Without Test Data: Tests requiring test data will be skipped gracefully
Setting up Test Data
# Option 1: Use environment variable
export ODB_TEST_DATA_DIR=/path/to/test/data
# Option 2: Let the test runner create basic test data
./scripts/run-tests.sh --test-data-dir ./test_data
Continuous Integration
Tests run automatically on every push and pull request via GitHub Actions. The CI pipeline:
- Builds on Windows, Linux, and macOS
- Runs the full test suite
- Generates test reports
- Publishes test results
CI Test Execution
- name: CMake Test
run: ctest --test-dir ./out/build/${{matrix.preset}}/OdbDesignTests
--output-log testlog.txt
--output-junit testlog.xml
--output-on-failure
- name: Report Test Results
uses: dorny/test-reporter@v1.9.1
with:
name: test-results
path: testlog.xml
reporter: java-junit
Test Development
Adding New Tests
- Create Test File:
#include <gtest/gtest.h> #include "Fixtures/TestDataFixture.h" namespace Odb::Test { TEST(MyComponentTest, BasicFunctionality) { EXPECT_TRUE(true); } } - Add to CMakeLists.txt:
add_executable(OdbDesignTests # ... existing files ... "MyNewTests.cpp" ) - Use Test Utilities:
#include "Fixtures/TestUtils.h" auto tempFile = TestUtils::createManagedTempFile("content"); auto [result, time] = TestUtils::measureExecutionTime([]() { // Your test code here });
Test Naming Conventions
- Test Suites: Use descriptive names ending with “Test” or “Tests”
- Test Cases: Use descriptive names in PascalCase
- Fixtures: Use descriptive names ending with “Fixture”
Examples:
TEST(ComponentNameTest, SpecificFunctionality)
TEST_F(MyFixture, SpecificScenario)
class PerformanceTests : public testing::Test
Test Utilities
The TestUtils class provides utilities for:
- Temporary Resources: Automatic cleanup of temp files/directories
- Random Data Generation: Generate test data with various patterns
- Performance Measurement: Time execution and benchmark functions
- File Operations: Create, compare, and manage test files
- Retry Logic: Implement retry patterns for flaky operations
Example usage:
// Create temporary file with automatic cleanup
auto tempFile = TestUtils::createManagedTempFile("test content");
// Measure execution time
auto [result, duration] = TestUtils::measureExecutionTime([&]() {
return myFunction();
});
// Generate random test data
auto randomData = TestUtils::generateRandomString(1024);
// Retry flaky operations
bool success = TestUtils::retryWithBackoff([&]() {
return attemptOperation();
});
Performance Testing
Benchmark Tests
Performance tests measure execution time and detect regressions:
TEST_F(PerformanceTestBase, MyBenchmark)
{
auto avgTime = benchmarkFunction([&]() {
return myFunction();
}, 100); // 100 iterations
EXPECT_LT(avgTime.count(), 1000) << "Function should complete in <1ms";
}
Stress Tests
Stress tests verify behavior under high load:
TEST_F(PerformanceTestBase, ConcurrentStressTest)
{
const int numThreads = std::thread::hardware_concurrency();
// Launch concurrent operations and verify results
}
Error Condition Testing
Edge Cases
Test boundary conditions and edge cases:
TEST_F(ErrorConditionTests, BoundaryConditions)
{
// Test with empty input
// Test with maximum size input
// Test with invalid input
}
Exception Testing
Use test utilities for exception testing:
TestAssertions::assertThrows<std::runtime_error>([&]() {
functionThatShouldThrow();
}, "expected error message");
Integration Testing
Workflow Testing
Test complete end-to-end workflows:
TEST_F(IntegrationTestSuite, CompleteWorkflow)
{
// Step 1: Setup
// Step 2: Execute workflow
// Step 3: Verify results
// Step 4: Cleanup
}
Component Interaction
Test how components work together:
TEST_F(IntegrationTestSuite, ComponentInteraction)
{
// Test data flow between components
// Test error propagation
// Test resource sharing
}
Troubleshooting
Common Issues
- Missing Dependencies:
# Install vcpkg dependencies vcpkg install - Test Data Not Found:
# Set test data directory export ODB_TEST_DATA_DIR=/path/to/test/data - Build Failures:
# Clean and rebuild ./scripts/run-tests.sh --clean
Debug Mode
Enable verbose logging in tests:
// In test fixtures, set ENABLE_TEST_LOGGING = true
static inline constexpr bool ENABLE_TEST_LOGGING = true;
Performance Issues
If tests are running slowly:
- Check system resources
- Reduce test iterations for development
- Use
--performanceflag to run only performance tests - Run tests in parallel:
ctest -j N
Best Practices
Test Writing
- Keep tests focused - One concept per test
- Use descriptive names - Test names should explain what is being tested
- Make tests independent - Tests should not depend on each other
- Use fixtures for common setup - Avoid code duplication
- Test both success and failure paths - Include error condition testing
Performance
- Use benchmark utilities - Consistent performance measurement
- Set reasonable expectations - Performance limits should be achievable
- Test on different hardware - Consider CI environment limitations
- Monitor for regressions - Track performance over time
Maintenance
- Update tests with code changes - Keep tests in sync with implementation
- Remove obsolete tests - Clean up tests for removed functionality
- Refactor common patterns - Extract reusable test utilities
- Document complex test scenarios - Help future developers understand tests
Contributing
When contributing new tests:
- Follow existing patterns and conventions
- Add appropriate documentation
- Ensure tests pass on all platforms
- Update this documentation if needed
- Consider both positive and negative test cases
For questions or issues with the test suite, please refer to the project’s issue tracker or documentation.