Update testing documentation
This commit is contained in:
adilallo
@@ -1,308 +0,0 @@
|
|||||||
# Performance Monitoring System
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Community Rule platform includes a comprehensive performance monitoring system designed to detect performance regressions, maintain performance budgets, and ensure optimal user experience across all components and user interactions.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Core Components
|
|
||||||
|
|
||||||
1. **Performance Monitor Module** (`tests/performance/performance-monitor.js`)
|
|
||||||
- Base `PerformanceMonitor` class for metric collection and analysis
|
|
||||||
- `WebPerformanceMonitor` for browser-based performance monitoring
|
|
||||||
- `PlaywrightPerformanceMonitor` for E2E performance testing
|
|
||||||
|
|
||||||
2. **Performance Tests** (`tests/e2e/performance.spec.ts`)
|
|
||||||
- Comprehensive E2E performance tests using Playwright
|
|
||||||
- Core Web Vitals monitoring
|
|
||||||
- Component render performance testing
|
|
||||||
- Interaction performance testing
|
|
||||||
|
|
||||||
3. **Lighthouse CI Integration** (`lighthouserc.json`)
|
|
||||||
- Automated performance audits
|
|
||||||
- Performance budget enforcement
|
|
||||||
- Core Web Vitals validation
|
|
||||||
|
|
||||||
4. **Performance Budgets** (`performance-budgets.json`)
|
|
||||||
- Resource size limits
|
|
||||||
- Timing budgets
|
|
||||||
- Resource count limits
|
|
||||||
|
|
||||||
5. **Monitoring Script** (`scripts/performance-monitor.js`)
|
|
||||||
- Standalone performance monitoring
|
|
||||||
- Regression detection
|
|
||||||
- Report generation
|
|
||||||
|
|
||||||
## Performance Budgets
|
|
||||||
|
|
||||||
### Timing Budgets
|
|
||||||
|
|
||||||
| Metric | Budget | Baseline | Description |
|
|
||||||
| ------------------------ | ------ | -------- | ------------------------- |
|
|
||||||
| Page Load Time | 3000ms | 2000ms | Total page load time |
|
|
||||||
| First Contentful Paint | 2000ms | 1500ms | First content appears |
|
|
||||||
| Largest Contentful Paint | 2500ms | 2000ms | Largest content element |
|
|
||||||
| First Input Delay | 100ms | 50ms | First user interaction |
|
|
||||||
| TTFB | 600ms | 400ms | Time to First Byte |
|
|
||||||
| Component Render | 500ms | 300ms | Component rendering time |
|
|
||||||
| Interaction Time | 100ms | 50ms | User interaction response |
|
|
||||||
| Scroll Performance | 50ms | 30ms | Scroll operation time |
|
|
||||||
|
|
||||||
### Resource Budgets
|
|
||||||
|
|
||||||
| Resource Type | Size Limit | Count Limit | Description |
|
|
||||||
| ------------- | ---------- | ----------- | ---------------------- |
|
|
||||||
| Scripts | 300KB | 10 | JavaScript files |
|
|
||||||
| Stylesheets | 50KB | 5 | CSS files |
|
|
||||||
| Images | 100KB | 20 | Image files |
|
|
||||||
| Fonts | 50KB | 5 | Font files |
|
|
||||||
| Total | 500KB | 50 | All resources combined |
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### Running Performance Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run all performance tests
|
|
||||||
npm run e2e:performance
|
|
||||||
|
|
||||||
# Run specific performance test
|
|
||||||
npx playwright test tests/e2e/performance.spec.ts --grep="homepage load performance"
|
|
||||||
|
|
||||||
# Run with specific browser
|
|
||||||
npx playwright test tests/e2e/performance.spec.ts --project=chromium
|
|
||||||
```
|
|
||||||
|
|
||||||
### Running Lighthouse CI
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run Lighthouse CI with default settings
|
|
||||||
npm run lhci
|
|
||||||
|
|
||||||
# Run with mobile preset
|
|
||||||
npm run lhci:mobile
|
|
||||||
|
|
||||||
# Run with desktop preset
|
|
||||||
npm run lhci:desktop
|
|
||||||
|
|
||||||
# Run with performance budgets
|
|
||||||
npm run performance:budget
|
|
||||||
```
|
|
||||||
|
|
||||||
### Running Performance Monitoring
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run comprehensive performance monitoring
|
|
||||||
npm run performance:monitor
|
|
||||||
|
|
||||||
# Run monitoring script directly
|
|
||||||
node scripts/performance-monitor.js
|
|
||||||
```
|
|
||||||
|
|
||||||
## Performance Metrics
|
|
||||||
|
|
||||||
### Core Web Vitals
|
|
||||||
|
|
||||||
1. **Largest Contentful Paint (LCP)**
|
|
||||||
- Measures loading performance
|
|
||||||
- Target: < 2.5 seconds
|
|
||||||
- Baseline: < 2.0 seconds
|
|
||||||
|
|
||||||
2. **First Input Delay (FID)**
|
|
||||||
- Measures interactivity
|
|
||||||
- Target: < 100ms
|
|
||||||
- Baseline: < 50ms
|
|
||||||
|
|
||||||
3. **Cumulative Layout Shift (CLS)**
|
|
||||||
- Measures visual stability
|
|
||||||
- Target: < 0.1
|
|
||||||
- Baseline: < 0.05
|
|
||||||
|
|
||||||
### Navigation Timing
|
|
||||||
|
|
||||||
- **DNS Lookup**: Domain name resolution time
|
|
||||||
- **TCP Connection**: Connection establishment time
|
|
||||||
- **TTFB**: Time to First Byte
|
|
||||||
- **Download**: Resource download time
|
|
||||||
- **DOM Content Loaded**: DOM parsing completion
|
|
||||||
- **Load**: Full page load completion
|
|
||||||
|
|
||||||
### Component Performance
|
|
||||||
|
|
||||||
- **Render Time**: Component rendering duration
|
|
||||||
- **Interaction Time**: User interaction response time
|
|
||||||
- **Scroll Performance**: Smooth scrolling performance
|
|
||||||
- **Memory Usage**: JavaScript heap memory consumption
|
|
||||||
|
|
||||||
## Regression Detection
|
|
||||||
|
|
||||||
### Automatic Detection
|
|
||||||
|
|
||||||
The performance monitoring system automatically detects regressions by:
|
|
||||||
|
|
||||||
1. **Comparing against baselines**: Current metrics vs. established baselines
|
|
||||||
2. **Threshold monitoring**: Real-time threshold violation detection
|
|
||||||
3. **Trend analysis**: Performance degradation over time
|
|
||||||
4. **Statistical analysis**: Variance and consistency monitoring
|
|
||||||
|
|
||||||
### Regression Thresholds
|
|
||||||
|
|
||||||
- **20% degradation**: Triggers regression warning
|
|
||||||
- **50% degradation**: Triggers regression error
|
|
||||||
- **Consistent degradation**: Pattern-based regression detection
|
|
||||||
|
|
||||||
### Alert System
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// Example regression detection output
|
|
||||||
🚨 Performance regression detected: scroll_performance = 111ms (baseline: 30ms)
|
|
||||||
⚠️ Performance threshold exceeded: interaction_time = 1368ms (threshold: 100ms)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Performance Reports
|
|
||||||
|
|
||||||
### Generated Reports
|
|
||||||
|
|
||||||
1. **Console Output**: Real-time performance metrics and warnings
|
|
||||||
2. **JSON Reports**: Structured performance data (`performance-report.json`)
|
|
||||||
3. **Lighthouse Reports**: Detailed performance audits
|
|
||||||
4. **Playwright Reports**: E2E test results with performance data
|
|
||||||
|
|
||||||
### Report Structure
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"timestamp": "2024-01-01T12:00:00.000Z",
|
|
||||||
"summary": {
|
|
||||||
"totalMetrics": 15,
|
|
||||||
"regressions": 2,
|
|
||||||
"warnings": 3
|
|
||||||
},
|
|
||||||
"regressions": [
|
|
||||||
{
|
|
||||||
"metric": "scroll_performance",
|
|
||||||
"current": 111,
|
|
||||||
"baseline": 30,
|
|
||||||
"regression": "270.0%"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"warnings": [
|
|
||||||
"Performance threshold exceeded: interaction_time = 1368ms (threshold: 100ms)"
|
|
||||||
],
|
|
||||||
"metrics": {
|
|
||||||
"page_load_time": {
|
|
||||||
"latest": 1704,
|
|
||||||
"average": 1704,
|
|
||||||
"min": 1704,
|
|
||||||
"max": 1704,
|
|
||||||
"count": 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## CI/CD Integration
|
|
||||||
|
|
||||||
### GitHub Actions Integration
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# Example CI workflow
|
|
||||||
- name: Performance Tests
|
|
||||||
run: |
|
|
||||||
npm run e2e:performance
|
|
||||||
npm run lhci
|
|
||||||
npm run performance:budget
|
|
||||||
```
|
|
||||||
|
|
||||||
### Performance Gates
|
|
||||||
|
|
||||||
- **Performance Score**: Must be > 90
|
|
||||||
- **Core Web Vitals**: All metrics within budgets
|
|
||||||
- **Regression Detection**: No significant regressions
|
|
||||||
- **Resource Budgets**: All resources within limits
|
|
||||||
|
|
||||||
## Best Practices
|
|
||||||
|
|
||||||
### Development Workflow
|
|
||||||
|
|
||||||
1. **Pre-commit Checks**: Run performance tests before commits
|
|
||||||
2. **Baseline Updates**: Update baselines after performance improvements
|
|
||||||
3. **Budget Reviews**: Regular budget review and adjustment
|
|
||||||
4. **Regression Investigation**: Immediate investigation of detected regressions
|
|
||||||
|
|
||||||
### Performance Optimization
|
|
||||||
|
|
||||||
1. **Code Splitting**: Implement dynamic imports for better loading
|
|
||||||
2. **Image Optimization**: Use modern formats and proper sizing
|
|
||||||
3. **Caching**: Implement effective caching strategies
|
|
||||||
4. **Bundle Analysis**: Regular bundle size monitoring
|
|
||||||
|
|
||||||
### Monitoring Strategy
|
|
||||||
|
|
||||||
1. **Continuous Monitoring**: Automated performance testing in CI/CD
|
|
||||||
2. **Real User Monitoring**: Collect performance data from real users
|
|
||||||
3. **Alert Thresholds**: Set appropriate alert thresholds
|
|
||||||
4. **Performance Budgets**: Enforce strict performance budgets
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Common Issues
|
|
||||||
|
|
||||||
1. **Test Timeouts**
|
|
||||||
- Increase timeout values for slow operations
|
|
||||||
- Add proper wait conditions
|
|
||||||
- Check for network issues
|
|
||||||
|
|
||||||
2. **False Positives**
|
|
||||||
- Adjust baseline values
|
|
||||||
- Review test environment
|
|
||||||
- Check for external dependencies
|
|
||||||
|
|
||||||
3. **Performance Fluctuations**
|
|
||||||
- Run multiple test iterations
|
|
||||||
- Use statistical analysis
|
|
||||||
- Consider environmental factors
|
|
||||||
|
|
||||||
### Debugging Performance Issues
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Enable detailed logging
|
|
||||||
DEBUG=playwright:* npm run e2e:performance
|
|
||||||
|
|
||||||
# Run with specific browser and debugging
|
|
||||||
npx playwright test tests/e2e/performance.spec.ts --project=chromium --debug
|
|
||||||
|
|
||||||
# Generate detailed reports
|
|
||||||
npm run performance:monitor -- --verbose
|
|
||||||
```
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
### Planned Features
|
|
||||||
|
|
||||||
1. **Real User Monitoring (RUM)**
|
|
||||||
- Collect performance data from real users
|
|
||||||
- User-centric performance metrics
|
|
||||||
- Geographic performance analysis
|
|
||||||
|
|
||||||
2. **Advanced Analytics**
|
|
||||||
- Machine learning-based regression detection
|
|
||||||
- Predictive performance modeling
|
|
||||||
- Automated performance optimization suggestions
|
|
||||||
|
|
||||||
3. **Performance Dashboard**
|
|
||||||
- Web-based performance monitoring dashboard
|
|
||||||
- Real-time performance metrics visualization
|
|
||||||
- Historical performance trends
|
|
||||||
|
|
||||||
4. **Integration with APM Tools**
|
|
||||||
- New Relic integration
|
|
||||||
- DataDog integration
|
|
||||||
- Custom APM tool integration
|
|
||||||
|
|
||||||
## Conclusion
|
|
||||||
|
|
||||||
The performance monitoring system provides comprehensive coverage of application performance, enabling early detection of regressions and maintaining high performance standards. Regular monitoring and proactive optimization ensure optimal user experience across all platforms and devices.
|
|
||||||
|
|
||||||
For questions or issues with the performance monitoring system, please refer to the testing documentation or create an issue in the project repository.
|
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
# Testing Documentation
|
||||||
|
|
||||||
|
This directory contains comprehensive testing documentation for the CommunityRule platform.
|
||||||
|
|
||||||
|
## 📚 Documentation Structure
|
||||||
|
|
||||||
|
### 1. [testing-framework.md](./testing-framework.md) - **Main Guide**
|
||||||
|
|
||||||
|
**Complete testing framework documentation** covering all aspects of testing:
|
||||||
|
|
||||||
|
- Testing architecture and philosophy
|
||||||
|
- Unit, integration, and E2E testing
|
||||||
|
- Visual regression testing
|
||||||
|
- Accessibility testing
|
||||||
|
- Performance testing
|
||||||
|
- CI/CD pipeline
|
||||||
|
- Development workflow
|
||||||
|
- Best practices and troubleshooting
|
||||||
|
|
||||||
|
**Use this for**: Learning the testing framework, understanding architecture, comprehensive reference
|
||||||
|
|
||||||
|
### 2. [testing-quick-reference.md](./testing-quick-reference.md) - **Quick Reference**
|
||||||
|
|
||||||
|
**Essential commands and quick troubleshooting** for daily development:
|
||||||
|
|
||||||
|
- Essential test commands
|
||||||
|
- Current test status and metrics
|
||||||
|
- Common test patterns
|
||||||
|
- Troubleshooting solutions
|
||||||
|
- Performance budgets
|
||||||
|
- CI/CD pipeline overview
|
||||||
|
|
||||||
|
**Use this for**: Daily development, quick commands, troubleshooting, reference
|
||||||
|
|
||||||
|
### 3. [visual-regression-guide.md](./visual-regression-guide.md) - **Visual Testing Guide**
|
||||||
|
|
||||||
|
**Focused guide for visual regression testing**:
|
||||||
|
|
||||||
|
- Visual regression workflow
|
||||||
|
- Snapshot management
|
||||||
|
- Configuration and best practices
|
||||||
|
- Troubleshooting visual test issues
|
||||||
|
- CI/CD integration for visual tests
|
||||||
|
|
||||||
|
**Use this for**: Visual regression testing, snapshot management, visual test troubleshooting
|
||||||
|
|
||||||
|
## 🎯 How to Use These Documents
|
||||||
|
|
||||||
|
### For New Team Members
|
||||||
|
|
||||||
|
1. Start with **testing-framework.md** to understand the complete testing strategy
|
||||||
|
2. Use **testing-quick-reference.md** for daily development
|
||||||
|
3. Reference **visual-regression-guide.md** when working with visual tests
|
||||||
|
|
||||||
|
### For Daily Development
|
||||||
|
|
||||||
|
1. Use **testing-quick-reference.md** for commands and troubleshooting
|
||||||
|
2. Reference **testing-framework.md** for detailed explanations
|
||||||
|
3. Use **visual-regression-guide.md** for visual test workflows
|
||||||
|
|
||||||
|
### For Troubleshooting
|
||||||
|
|
||||||
|
1. Check **testing-quick-reference.md** for common solutions
|
||||||
|
2. Use **testing-framework.md** for detailed troubleshooting
|
||||||
|
3. Reference **visual-regression-guide.md** for visual test issues
|
||||||
|
|
||||||
|
## 📊 Current Testing Status
|
||||||
|
|
||||||
|
- **Unit Tests**: 94.88% coverage (exceeds 85% target)
|
||||||
|
- **Integration Tests**: 5 comprehensive test suites
|
||||||
|
- **E2E Tests**: 92 tests across 4 browsers
|
||||||
|
- **Visual Regression**: 23 tests per browser
|
||||||
|
- **Accessibility**: WCAG 2.1 AA compliance
|
||||||
|
- **Performance**: Lighthouse CI with budgets
|
||||||
|
|
||||||
|
## 🔄 Documentation Updates
|
||||||
|
|
||||||
|
This documentation is maintained by the CommunityRule development team and updated regularly to reflect:
|
||||||
|
|
||||||
|
- Current testing framework status
|
||||||
|
- Best practices and patterns
|
||||||
|
- Troubleshooting solutions
|
||||||
|
- CI/CD pipeline changes
|
||||||
|
- New testing features
|
||||||
|
|
||||||
|
## 📚 Additional Resources
|
||||||
|
|
||||||
|
- **Vitest Documentation**: https://vitest.dev/
|
||||||
|
- **Playwright Documentation**: https://playwright.dev/
|
||||||
|
- **React Testing Library**: https://testing-library.com/
|
||||||
|
- **Lighthouse CI**: https://github.com/GoogleChrome/lighthouse-ci
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated**: December 2024
|
||||||
|
**Maintained by**: CommunityRule Development Team
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
# Visual Regression Snapshot Workflow
|
|
||||||
|
|
||||||
Quick reference for managing visual regression snapshots.
|
|
||||||
|
|
||||||
## 🚀 **First-Time Setup**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Generate baseline snapshots (choose one)
|
|
||||||
npm run seed-snapshots # Docker (recommended for CI consistency)
|
|
||||||
npm run seed-snapshots:local # Local environment
|
|
||||||
|
|
||||||
# 2. Commit the snapshots
|
|
||||||
git add tests/e2e/visual-regression.spec.ts-snapshots/
|
|
||||||
git commit -m "Add baseline visual regression snapshots"
|
|
||||||
|
|
||||||
# 3. Verify setup
|
|
||||||
npm run visual:test
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🔄 **Daily Workflow**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run visual regression tests
|
|
||||||
npm run visual:test
|
|
||||||
|
|
||||||
# Run with UI for debugging
|
|
||||||
npm run visual:ui
|
|
||||||
|
|
||||||
# Update snapshots after UI changes
|
|
||||||
npm run visual:update
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📝 **When UI Changes**
|
|
||||||
|
|
||||||
1. **Make your UI changes** (design updates, component modifications, etc.)
|
|
||||||
|
|
||||||
2. **Update snapshots to reflect new design:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run visual:update
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Review changes:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git diff tests/e2e/visual-regression.spec.ts-snapshots/
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Commit updated snapshots:**
|
|
||||||
```bash
|
|
||||||
git add tests/e2e/visual-regression.spec.ts-snapshots/
|
|
||||||
git commit -m "Update snapshots for [describe changes]"
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🐛 **Troubleshooting**
|
|
||||||
|
|
||||||
### **"Snapshot doesn't exist" errors**
|
|
||||||
|
|
||||||
- **Cause**: Baseline snapshots haven't been generated
|
|
||||||
- **Fix**: Run `npm run seed-snapshots:local`
|
|
||||||
|
|
||||||
### **Platform differences (macOS vs Linux)**
|
|
||||||
|
|
||||||
- **Cause**: Different font rendering between platforms
|
|
||||||
- **Fix**: Use `npm run seed-snapshots` (Docker container)
|
|
||||||
|
|
||||||
### **Minor pixel differences**
|
|
||||||
|
|
||||||
- **Cause**: Font rendering, anti-aliasing differences
|
|
||||||
- **Fix**: Check tolerance settings in `playwright.config.ts`
|
|
||||||
|
|
||||||
### **Animation-related failures**
|
|
||||||
|
|
||||||
- **Cause**: Animations not fully disabled
|
|
||||||
- **Fix**: Ensure `animations: "disabled"` is set (already configured)
|
|
||||||
|
|
||||||
## 📁 **File Structure**
|
|
||||||
|
|
||||||
```
|
|
||||||
tests/e2e/
|
|
||||||
├── visual-regression.spec.ts # Test definitions
|
|
||||||
└── visual-regression.spec.ts-snapshots/ # Baseline images
|
|
||||||
├── homepage-full-chromium.png
|
|
||||||
├── homepage-viewport-chromium.png
|
|
||||||
├── hero-banner-chromium.png
|
|
||||||
└── ...
|
|
||||||
```
|
|
||||||
|
|
||||||
## ⚡ **Quick Commands Reference**
|
|
||||||
|
|
||||||
| Command | Purpose |
|
|
||||||
| ------------------------------ | ----------------------------------------------- |
|
|
||||||
| `npm run visual:test` | Run visual regression tests |
|
|
||||||
| `npm run visual:update` | Update snapshots after UI changes |
|
|
||||||
| `npm run visual:ui` | Run tests with UI for debugging |
|
|
||||||
| `npm run seed-snapshots` | Generate baselines with Docker (CI consistency) |
|
|
||||||
| `npm run seed-snapshots:local` | Generate baselines locally |
|
|
||||||
|
|
||||||
## 💡 **Best Practices**
|
|
||||||
|
|
||||||
1. **Always review changes** before committing updated snapshots
|
|
||||||
2. **Use descriptive commit messages** when updating snapshots
|
|
||||||
3. **Test locally first** before pushing to CI
|
|
||||||
4. **Use Docker for consistency** when generating baselines
|
|
||||||
5. **Update snapshots promptly** after UI changes to avoid drift
|
|
||||||
|
|
||||||
## 🔗 **Related Documentation**
|
|
||||||
|
|
||||||
- [Visual Regression Setup](./VISUAL_REGRESSION_SETUP.md) - Detailed setup guide
|
|
||||||
- [Testing Strategy](../TESTING_STRATEGY.md) - Overall testing approach
|
|
||||||
@@ -1,150 +0,0 @@
|
|||||||
# Visual Regression Testing Setup
|
|
||||||
|
|
||||||
This document explains how to set up and maintain visual regression tests for the CommunityRule platform.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Visual regression tests capture screenshots of key UI components and compare them against baseline images to detect unintended visual changes. The tests are configured to work consistently across different environments (local development, CI/CD).
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### Playwright Configuration
|
|
||||||
|
|
||||||
The visual regression tests are configured in `playwright.config.ts` with the following key settings:
|
|
||||||
|
|
||||||
- **OS-agnostic snapshots**: Uses `{testDir}/{testFileName}-snapshots/{arg}-{projectName}.png` template
|
|
||||||
- **Consistent rendering**: Fixed viewport (1280x800), device scale factor (1), and color scheme (light)
|
|
||||||
- **Tolerance settings**: Allows 1% pixel difference or 200 pixels maximum difference
|
|
||||||
- **Animation handling**: Disables animations during screenshot capture
|
|
||||||
|
|
||||||
### CI Environment
|
|
||||||
|
|
||||||
The CI workflow uses:
|
|
||||||
|
|
||||||
- **Ubuntu Linux** with Playwright Docker image for consistent rendering
|
|
||||||
- **One-time snapshot seeding** on the main branch
|
|
||||||
- **Artifact packaging** to reduce file count and improve upload performance
|
|
||||||
|
|
||||||
## Setting Up Snapshots
|
|
||||||
|
|
||||||
### Option 1: CI Seeding (Recommended for New Projects)
|
|
||||||
|
|
||||||
1. **Push to main branch**: The CI will automatically generate baseline snapshots
|
|
||||||
2. **Download artifacts**: Download the `visual-regression-results` artifact
|
|
||||||
3. **Extract snapshots**: Extract the `tests/e2e/visual-regression.spec.ts-snapshots/` folder
|
|
||||||
4. **Commit snapshots**: Add and commit the snapshot files to your repository
|
|
||||||
5. **Remove seeding step**: After snapshots are committed, remove the seeding step from CI
|
|
||||||
|
|
||||||
### Option 2: Local Seeding with Docker (Recommended for Existing Projects)
|
|
||||||
|
|
||||||
Use the provided script to generate snapshots in the same environment as CI:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Using the Docker container (ensures CI consistency)
|
|
||||||
npm run seed-snapshots
|
|
||||||
|
|
||||||
# Or using local environment (may have slight differences)
|
|
||||||
npm run seed-snapshots:local
|
|
||||||
```
|
|
||||||
|
|
||||||
The Docker approach ensures:
|
|
||||||
|
|
||||||
- Same fonts and rendering as CI
|
|
||||||
- Same file naming conventions
|
|
||||||
- Same performance characteristics
|
|
||||||
|
|
||||||
## Running Visual Regression Tests
|
|
||||||
|
|
||||||
### Local Development
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run all visual regression tests
|
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts
|
|
||||||
|
|
||||||
# Run with UI for debugging
|
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts --ui
|
|
||||||
|
|
||||||
# Update snapshots (use with caution)
|
|
||||||
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
### CI/CD
|
|
||||||
|
|
||||||
Visual regression tests run automatically in the CI pipeline:
|
|
||||||
|
|
||||||
- **Main branch**: Generates new snapshots if needed
|
|
||||||
- **Feature branches**: Compares against existing snapshots
|
|
||||||
- **Artifacts**: Uploads test results and snapshots for review
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Common Issues
|
|
||||||
|
|
||||||
1. **"Snapshot doesn't exist" errors**
|
|
||||||
- **Cause**: Baseline snapshots haven't been generated
|
|
||||||
- **Solution**: Run snapshot seeding (see above)
|
|
||||||
|
|
||||||
2. **Platform-specific failures**
|
|
||||||
- **Cause**: Snapshots generated on different OS (macOS vs Linux)
|
|
||||||
- **Solution**: Use Docker container for local snapshot generation
|
|
||||||
|
|
||||||
3. **Minor pixel differences**
|
|
||||||
- **Cause**: Font rendering differences, anti-aliasing, etc.
|
|
||||||
- **Solution**: Check tolerance settings in `playwright.config.ts`
|
|
||||||
|
|
||||||
4. **Animation-related failures**
|
|
||||||
- **Cause**: Animations not fully disabled
|
|
||||||
- **Solution**: Ensure `animations: "disabled"` is set in test configuration
|
|
||||||
|
|
||||||
### Updating Snapshots
|
|
||||||
|
|
||||||
When intentional UI changes are made:
|
|
||||||
|
|
||||||
1. **Local update**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Review changes**: Check the updated snapshots in `tests/e2e/visual-regression.spec.ts-snapshots/`
|
|
||||||
|
|
||||||
3. **Commit changes**: Add and commit the updated snapshot files
|
|
||||||
|
|
||||||
4. **Verify**: Run tests again to ensure they pass
|
|
||||||
|
|
||||||
### Performance Considerations
|
|
||||||
|
|
||||||
- **CI environment**: Tests run slower due to container overhead
|
|
||||||
- **Local development**: Faster execution with native browser
|
|
||||||
- **Artifact size**: Snapshots are compressed and packaged to reduce upload time
|
|
||||||
|
|
||||||
## Best Practices
|
|
||||||
|
|
||||||
1. **Consistent environment**: Always use the same environment for snapshot generation and testing
|
|
||||||
2. **Meaningful test names**: Use descriptive names for snapshot files
|
|
||||||
3. **Selective testing**: Test only critical UI components to maintain reasonable test duration
|
|
||||||
4. **Regular updates**: Update snapshots when making intentional UI changes
|
|
||||||
5. **Review failures**: Always review visual regression failures before updating snapshots
|
|
||||||
|
|
||||||
## File Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
tests/e2e/
|
|
||||||
├── visual-regression.spec.ts # Test definitions
|
|
||||||
└── visual-regression.spec.ts-snapshots/ # Baseline images
|
|
||||||
├── homepage-full-chromium.png
|
|
||||||
├── homepage-viewport-chromium.png
|
|
||||||
├── hero-banner-chromium.png
|
|
||||||
└── ...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Integration with Other Tests
|
|
||||||
|
|
||||||
Visual regression tests complement:
|
|
||||||
|
|
||||||
- **Unit tests**: Verify component logic
|
|
||||||
- **Integration tests**: Verify component interactions
|
|
||||||
- **E2E tests**: Verify user workflows
|
|
||||||
- **Accessibility tests**: Verify accessibility compliance
|
|
||||||
|
|
||||||
Together, these tests provide comprehensive coverage of the application's functionality and appearance.
|
|
||||||
@@ -3,22 +3,22 @@
|
|||||||
## 📋 Table of Contents
|
## 📋 Table of Contents
|
||||||
|
|
||||||
- [Overview](#overview)
|
- [Overview](#overview)
|
||||||
|
- [Testing Architecture](#testing-architecture)
|
||||||
- [Quick Start](#quick-start)
|
- [Quick Start](#quick-start)
|
||||||
- [Test Structure](#test-structure)
|
- [Test Types & Coverage](#test-types--coverage)
|
||||||
- [Unit & Integration Testing](#unit--integration-testing)
|
- [Unit & Integration Testing](#unit--integration-testing)
|
||||||
- [E2E Testing](#e2e-testing)
|
- [E2E Testing](#e2e-testing)
|
||||||
- [Visual Regression Testing](#visual-regression-testing)
|
- [Visual Regression Testing](#visual-regression-testing)
|
||||||
|
- [Accessibility Testing](#accessibility-testing)
|
||||||
- [Performance Testing](#performance-testing)
|
- [Performance Testing](#performance-testing)
|
||||||
- [Storybook Testing](#storybook-testing)
|
|
||||||
- [CI/CD Pipeline](#cicd-pipeline)
|
- [CI/CD Pipeline](#cicd-pipeline)
|
||||||
- [Development Workflow](#development-workflow)
|
- [Development Workflow](#development-workflow)
|
||||||
- [Best Practices](#best-practices)
|
- [Best Practices](#best-practices)
|
||||||
- [Troubleshooting](#troubleshooting)
|
- [Troubleshooting](#troubleshooting)
|
||||||
- [Monitoring & Metrics](#monitoring--metrics)
|
|
||||||
|
|
||||||
## 🎯 Overview
|
## 🎯 Overview
|
||||||
|
|
||||||
This project uses a comprehensive testing framework with multiple layers of testing to ensure code quality, functionality, and visual consistency across all browsers and devices.
|
The CommunityRule platform uses a comprehensive testing framework with multiple layers to ensure code quality, functionality, visual consistency, and accessibility across all browsers and devices.
|
||||||
|
|
||||||
### Testing Stack
|
### Testing Stack
|
||||||
|
|
||||||
@@ -26,16 +26,32 @@ This project uses a comprehensive testing framework with multiple layers of test
|
|||||||
- **E2E**: Playwright (Chromium, Firefox, WebKit, Mobile)
|
- **E2E**: Playwright (Chromium, Firefox, WebKit, Mobile)
|
||||||
- **Visual Regression**: Playwright Screenshots
|
- **Visual Regression**: Playwright Screenshots
|
||||||
- **Performance**: Lighthouse CI
|
- **Performance**: Lighthouse CI
|
||||||
- **Accessibility**: Axe-core + Storybook
|
- **Accessibility**: Axe-core + Playwright
|
||||||
- **CI/CD**: Gitea Actions
|
- **CI/CD**: Gitea Actions
|
||||||
|
|
||||||
### Test Coverage
|
### Current Status
|
||||||
|
|
||||||
- ✅ **124 Unit Tests** (8 components + 1 integration)
|
- ✅ **305 Unit Tests** (94.88% coverage - exceeds 85% target)
|
||||||
- ✅ **308 E2E Tests** (4 browsers × 77 tests)
|
- ✅ **92 E2E Tests** across 4 browsers
|
||||||
- ✅ **92 Visual Regression Screenshots**
|
- ✅ **23 Visual Regression Tests** per browser
|
||||||
- ✅ **Performance Budgets**
|
- ✅ **Performance Budgets** with Lighthouse CI
|
||||||
- ✅ **Accessibility Compliance**
|
- ✅ **WCAG 2.1 AA Compliance** with automated testing
|
||||||
|
|
||||||
|
## 🏗 Testing Architecture
|
||||||
|
|
||||||
|
### Test Pyramid
|
||||||
|
|
||||||
|
- **Unit Tests**: Fast, focused, high coverage (94.88%)
|
||||||
|
- **Integration Tests**: Component interactions, data flow
|
||||||
|
- **E2E Tests**: Critical user journeys, cross-browser compatibility
|
||||||
|
|
||||||
|
### Testing Philosophy
|
||||||
|
|
||||||
|
**JSDOM Limitations**: Unit tests in JSDOM can't truly test responsive behavior since CSS media queries aren't evaluated. Therefore:
|
||||||
|
|
||||||
|
- **Unit/Integration Tests**: Test component structure, accessibility, and configuration
|
||||||
|
- **E2E Tests**: Test real responsive behavior at actual viewport widths
|
||||||
|
- **Visual Tests**: Capture visual consistency across breakpoints
|
||||||
|
|
||||||
## 🚀 Quick Start
|
## 🚀 Quick Start
|
||||||
|
|
||||||
@@ -49,18 +65,18 @@ npm install
|
|||||||
npx playwright install
|
npx playwright install
|
||||||
```
|
```
|
||||||
|
|
||||||
### Running Tests
|
### Essential Commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# All unit tests with coverage
|
# Unit tests with coverage
|
||||||
npm test
|
npm test
|
||||||
|
|
||||||
# Unit tests in watch mode
|
|
||||||
npm run test:watch
|
|
||||||
|
|
||||||
# E2E tests
|
# E2E tests
|
||||||
npm run e2e
|
npm run e2e
|
||||||
|
|
||||||
|
# Visual regression tests
|
||||||
|
npm run visual:test
|
||||||
|
|
||||||
# Performance tests
|
# Performance tests
|
||||||
npm run lhci
|
npm run lhci
|
||||||
|
|
||||||
@@ -68,40 +84,43 @@ npm run lhci
|
|||||||
npm run test:sb
|
npm run test:sb
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📁 Test Structure
|
## 🧪 Test Types & Coverage
|
||||||
|
|
||||||
|
### Test Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
tests/
|
tests/
|
||||||
├── unit/ # Component unit tests
|
├── unit/ # Component unit tests
|
||||||
│ ├── Button.test.jsx # 113 lines
|
│ ├── Button.test.jsx # 12 tests
|
||||||
│ ├── HeroBanner.test.jsx # 143 lines
|
│ ├── Logo.test.jsx # 12 tests
|
||||||
│ ├── FeatureGrid.test.jsx # 146 lines
|
│ ├── RuleCard.test.jsx # 18 tests
|
||||||
│ ├── LogoWall.test.jsx # 170 lines
|
│ ├── SectionHeader.test.jsx # 17 tests
|
||||||
│ ├── NumberedCards.test.jsx # 196 lines
|
│ ├── NumberedCard.test.jsx # 18 tests
|
||||||
│ ├── RuleStack.test.jsx # 207 lines
|
│ └── accessibility.test.jsx # 18 tests
|
||||||
│ ├── QuoteBlock.test.jsx # 223 lines
|
|
||||||
│ └── AskOrganizer.test.jsx # 294 lines
|
|
||||||
├── integration/ # Component integration tests
|
├── integration/ # Component integration tests
|
||||||
│ └── ContentLockup.integration.test.jsx # 157 lines
|
│ ├── component-interactions.integration.test.jsx
|
||||||
|
│ ├── page-flow.integration.test.jsx
|
||||||
|
│ ├── user-journey.integration.test.jsx
|
||||||
|
│ ├── layout.integration.test.jsx
|
||||||
|
│ └── ContentLockup.integration.test.jsx
|
||||||
└── e2e/ # End-to-end tests
|
└── e2e/ # End-to-end tests
|
||||||
├── homepage.spec.ts # 18 tests per browser
|
├── homepage.spec.ts # Homepage functionality
|
||||||
├── user-journeys.spec.ts # 13 tests per browser
|
├── user-journeys.spec.ts # User workflows
|
||||||
├── edge-cases.spec.ts # 18 tests per browser
|
├── header.responsive.spec.js # Responsive header
|
||||||
└── visual-regression.spec.ts # 23 tests per browser
|
├── footer.responsive.spec.js # Responsive footer
|
||||||
|
├── visual-regression.spec.ts # Visual consistency
|
||||||
|
├── accessibility.spec.ts # Accessibility compliance
|
||||||
|
└── performance.spec.ts # Performance metrics
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🚀 Runner Management Scripts
|
### Coverage Requirements
|
||||||
|
|
||||||
```
|
- **Statements**: >85% (Current: 94.88%) ✅
|
||||||
community-rule/
|
- **Branches**: >80% (Current: 86.93%) ✅
|
||||||
├── start-runner.sh # Start Gitea Actions runner
|
- **Functions**: >80% (Current: 88.67%) ✅
|
||||||
├── stop-runner.sh # Stop Gitea Actions runner
|
- **Lines**: >85% (Current: 94.88%) ✅
|
||||||
├── status-runner.sh # Check runner status
|
|
||||||
├── config.yaml # Runner configuration
|
|
||||||
└── act_runner # Gitea Actions runner binary
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🧪 Unit & Integration Testing
|
## 🧩 Unit & Integration Testing
|
||||||
|
|
||||||
### Framework
|
### Framework
|
||||||
|
|
||||||
@@ -143,6 +162,42 @@ describe("Component", () => {
|
|||||||
render(<Component />);
|
render(<Component />);
|
||||||
expect(screen.getByRole("button")).toBeInTheDocument();
|
expect(screen.getByRole("button")).toBeInTheDocument();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("handles user interactions", async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
render(<Component />);
|
||||||
|
|
||||||
|
const button = screen.getByRole("button");
|
||||||
|
await user.click(button);
|
||||||
|
|
||||||
|
expect(button).toHaveClass("clicked");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Testing Library Queries (Priority Order)
|
||||||
|
|
||||||
|
1. **`getByRole`**: Most accessible, tests user experience
|
||||||
|
2. **`getByLabelText`**: For form inputs
|
||||||
|
3. **`getByText`**: For content
|
||||||
|
4. **`getByTestId`**: Last resort, avoid when possible
|
||||||
|
|
||||||
|
### Integration Testing
|
||||||
|
|
||||||
|
```jsx
|
||||||
|
test("components work together", () => {
|
||||||
|
render(
|
||||||
|
<div>
|
||||||
|
<Header />
|
||||||
|
<MainContent />
|
||||||
|
<Footer />
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
|
||||||
|
// Test that components complement each other
|
||||||
|
expect(screen.getByRole("banner")).toBeInTheDocument();
|
||||||
|
expect(screen.getByRole("main")).toBeInTheDocument();
|
||||||
|
expect(screen.getByRole("contentinfo")).toBeInTheDocument();
|
||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -172,44 +227,37 @@ export default defineConfig({
|
|||||||
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
|
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
|
||||||
{ name: "firefox", use: { ...devices["Desktop Firefox"] } },
|
{ name: "firefox", use: { ...devices["Desktop Firefox"] } },
|
||||||
{ name: "webkit", use: { ...devices["Desktop Safari"] } },
|
{ name: "webkit", use: { ...devices["Desktop Safari"] } },
|
||||||
{ name: "mobile", use: { ...devices["iPhone 12"] } },
|
{ name: "mobile", use: { ...devices["iPhone 13"] } },
|
||||||
],
|
],
|
||||||
|
use: {
|
||||||
|
timezoneId: "UTC",
|
||||||
|
locale: "en-US",
|
||||||
|
headless: true,
|
||||||
|
},
|
||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
### Test Categories
|
### Test Categories
|
||||||
|
|
||||||
#### 1. Homepage Tests (18 tests per browser)
|
#### 1. Functional Tests
|
||||||
|
|
||||||
- Page loading and sections
|
- Page loading and sections
|
||||||
- Component functionality
|
- Component functionality
|
||||||
- Navigation and interactions
|
- Navigation and interactions
|
||||||
- Responsive design
|
- User workflows
|
||||||
- Accessibility compliance
|
|
||||||
- Performance metrics
|
|
||||||
|
|
||||||
#### 2. User Journey Tests (13 tests per browser)
|
#### 2. Responsive Tests
|
||||||
|
|
||||||
- Complete user workflows
|
- Layout changes between breakpoints
|
||||||
- Feature exploration
|
- Component visibility at different viewports
|
||||||
- Contact flows
|
- Interactive behavior across screen sizes
|
||||||
- Learning paths
|
|
||||||
- Navigation patterns
|
|
||||||
|
|
||||||
#### 3. Edge Cases Tests (18 tests per browser)
|
#### 3. Accessibility Tests
|
||||||
|
|
||||||
- Network conditions
|
- WCAG 2.1 AA compliance
|
||||||
- Browser behavior
|
- Screen reader compatibility
|
||||||
- Error scenarios
|
- Keyboard navigation
|
||||||
- Accessibility edge cases
|
- Color contrast
|
||||||
- Performance under stress
|
|
||||||
|
|
||||||
#### 4. Visual Regression Tests (23 tests per browser)
|
|
||||||
|
|
||||||
- Full page screenshots
|
|
||||||
- Component screenshots
|
|
||||||
- Responsive screenshots
|
|
||||||
- Interactive states
|
|
||||||
|
|
||||||
### Writing E2E Tests
|
### Writing E2E Tests
|
||||||
|
|
||||||
@@ -226,6 +274,16 @@ test.describe("Feature", () => {
|
|||||||
await expect(page).toHaveTitle(/CommunityRule/);
|
await expect(page).toHaveTitle(/CommunityRule/);
|
||||||
await expect(page.locator("h1")).toBeVisible();
|
await expect(page.locator("h1")).toBeVisible();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("responsive behavior", async ({ page }) => {
|
||||||
|
// Test mobile viewport
|
||||||
|
await page.setViewportSize({ width: 375, height: 667 });
|
||||||
|
await expect(page.getByTestId("mobile-nav")).toBeVisible();
|
||||||
|
|
||||||
|
// Test desktop viewport
|
||||||
|
await page.setViewportSize({ width: 1280, height: 800 });
|
||||||
|
await expect(page.getByTestId("desktop-nav")).toBeVisible();
|
||||||
|
});
|
||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -241,7 +299,14 @@ npm run e2e:serve # Start dev server and run tests
|
|||||||
|
|
||||||
### Overview
|
### Overview
|
||||||
|
|
||||||
Visual regression testing ensures UI consistency across browsers and prevents unintended visual changes.
|
Visual regression testing ensures UI consistency across browsers and prevents unintended visual changes by comparing screenshots against baseline images.
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
- **Snapshot Template**: `{testDir}/{testFileName}-snapshots/{arg}-{projectName}.png`
|
||||||
|
- **Deterministic Rendering**: Fixed timezone (UTC), locale (en-US), viewport
|
||||||
|
- **Tolerance**: 2% pixel difference or 500 pixels maximum
|
||||||
|
- **Animation Handling**: Disabled during capture
|
||||||
|
|
||||||
### Screenshots Generated
|
### Screenshots Generated
|
||||||
|
|
||||||
@@ -250,33 +315,74 @@ Visual regression testing ensures UI consistency across browsers and prevents un
|
|||||||
- **Interactive states** (hover, focus, loading, error)
|
- **Interactive states** (hover, focus, loading, error)
|
||||||
- **Special modes** (dark mode, high contrast, reduced motion)
|
- **Special modes** (dark mode, high contrast, reduced motion)
|
||||||
|
|
||||||
### Baseline Screenshots
|
### Breakpoint Coverage
|
||||||
|
|
||||||
```
|
- **Mobile**: 375x667 (iPhone)
|
||||||
tests/e2e/visual-regression.spec.ts-snapshots/
|
- **Tablet**: 768x1024 (iPad)
|
||||||
├── homepage-full-chromium-darwin.png
|
- **Desktop**: 1280x800 (Standard)
|
||||||
├── homepage-mobile-chromium-darwin.png
|
- **Large Desktop**: 1920x1080 (Full HD)
|
||||||
├── hero-banner-chromium-darwin.png
|
|
||||||
├── logo-wall-chromium-darwin.png
|
|
||||||
└── ... (92 total screenshots)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Managing Visual Changes
|
### Managing Visual Changes
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Update baselines after intentional changes
|
# Update baselines after intentional changes
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts --update-snapshots
|
npm run visual:update
|
||||||
|
|
||||||
# Run visual regression tests
|
# Run visual regression tests
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts
|
npm run visual:test
|
||||||
|
|
||||||
|
# Run with UI for debugging
|
||||||
|
npm run visual:ui
|
||||||
```
|
```
|
||||||
|
|
||||||
### Cross-Browser Coverage
|
### Snapshot Management
|
||||||
|
|
||||||
- **Chromium** (Chrome/Edge)
|
```bash
|
||||||
- **Firefox**
|
# Update snapshots for all projects
|
||||||
- **WebKit** (Safari)
|
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts
|
||||||
- **Mobile** (Mobile Chrome)
|
|
||||||
|
# Update snapshots for specific project
|
||||||
|
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts --project=chromium
|
||||||
|
|
||||||
|
# View test results
|
||||||
|
npx playwright show-report
|
||||||
|
```
|
||||||
|
|
||||||
|
## ♿ Accessibility Testing
|
||||||
|
|
||||||
|
### Framework
|
||||||
|
|
||||||
|
- **Unit Level**: jest-axe with Vitest
|
||||||
|
- **E2E Level**: Playwright accessibility tests
|
||||||
|
- **Standards**: WCAG 2.1 AA compliance
|
||||||
|
|
||||||
|
### Automated Testing
|
||||||
|
|
||||||
|
```jsx
|
||||||
|
import { axe, toHaveNoViolations } from "jest-axe";
|
||||||
|
|
||||||
|
test("component has no accessibility violations", async () => {
|
||||||
|
const { container } = render(<Component />);
|
||||||
|
const results = await axe(container);
|
||||||
|
expect(results).toHaveNoViolations();
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Testing Checklist
|
||||||
|
|
||||||
|
- [ ] Screen reader compatibility
|
||||||
|
- [ ] Keyboard navigation
|
||||||
|
- [ ] Color contrast (WCAG AA)
|
||||||
|
- [ ] Focus management
|
||||||
|
- [ ] ARIA attributes
|
||||||
|
- [ ] Semantic HTML
|
||||||
|
|
||||||
|
### WCAG 2.1 AA Requirements
|
||||||
|
|
||||||
|
- **Perceivable**: Text alternatives, captions, adaptable content
|
||||||
|
- **Operable**: Keyboard accessible, timing adjustable, navigation
|
||||||
|
- **Understandable**: Readable, predictable, input assistance
|
||||||
|
- **Robust**: Compatible with assistive technologies
|
||||||
|
|
||||||
## ⚡ Performance Testing
|
## ⚡ Performance Testing
|
||||||
|
|
||||||
@@ -284,21 +390,27 @@ npx playwright test tests/e2e/visual-regression.spec.ts
|
|||||||
|
|
||||||
- **Lighthouse CI**: Automated performance testing
|
- **Lighthouse CI**: Automated performance testing
|
||||||
- **Performance Budgets**: Defined thresholds
|
- **Performance Budgets**: Defined thresholds
|
||||||
|
- **Core Web Vitals**: LCP, FID, CLS monitoring
|
||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
```json
|
```json
|
||||||
// lighthouserc.json
|
// .lighthouserc.json
|
||||||
{
|
{
|
||||||
"ci": {
|
"ci": {
|
||||||
"collect": {
|
"collect": {
|
||||||
"url": ["http://localhost:3000"],
|
"url": ["http://localhost:3010"],
|
||||||
"startServerCommand": "npm run preview"
|
"chromeFlags": [
|
||||||
|
"--no-sandbox",
|
||||||
|
"--disable-dev-shm-usage",
|
||||||
|
"--disable-gpu",
|
||||||
|
"--headless"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"assert": {
|
"assert": {
|
||||||
"assertions": {
|
"assertions": {
|
||||||
"categories:performance": ["warn", { "minScore": 0.9 }],
|
"categories:performance": ["warn", { "minScore": 0.8 }],
|
||||||
"categories:accessibility": ["error", { "minScore": 0.95 }]
|
"categories:accessibility": ["error", { "minScore": 0.8 }]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -307,57 +419,31 @@ npx playwright test tests/e2e/visual-regression.spec.ts
|
|||||||
|
|
||||||
### Performance Metrics
|
### Performance Metrics
|
||||||
|
|
||||||
- **Core Web Vitals**: LCP, FID, CLS
|
- **Core Web Vitals**: LCP < 2.5s, FID < 100ms, CLS < 0.1
|
||||||
- **Performance Score**: Overall performance rating
|
- **Performance Score**: >80
|
||||||
- **Accessibility Score**: WCAG compliance
|
- **Accessibility Score**: >80
|
||||||
- **Best Practices**: Web development standards
|
- **Best Practices**: >90
|
||||||
- **SEO Score**: Search engine optimization
|
|
||||||
|
### Performance Budgets
|
||||||
|
|
||||||
|
- **First Contentful Paint**: <3000ms
|
||||||
|
- **Largest Contentful Paint**: <5000ms
|
||||||
|
- **First Input Delay**: <100ms
|
||||||
|
- **TTFB**: <700ms
|
||||||
|
|
||||||
### Available Scripts
|
### Available Scripts
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm run lhci # Run Lighthouse CI
|
npm run lhci # Run Lighthouse CI
|
||||||
```
|
npm run lhci:mobile # Run with mobile preset
|
||||||
|
npm run lhci:desktop # Run with desktop preset
|
||||||
## 📚 Storybook Testing
|
|
||||||
|
|
||||||
### Framework
|
|
||||||
|
|
||||||
- **Storybook**: Component development environment
|
|
||||||
- **@storybook/test-runner**: Automated testing
|
|
||||||
- **@storybook/test**: Testing utilities
|
|
||||||
|
|
||||||
### Configuration
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
// .storybook/preview.js
|
|
||||||
export const parameters = {
|
|
||||||
a11y: { element: "#storybook-root", manual: false },
|
|
||||||
viewport: { defaultViewport: "responsive" },
|
|
||||||
chromatic: { viewports: [360, 768, 1024, 1440] },
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
### Testing Features
|
|
||||||
|
|
||||||
- **Accessibility Testing**: Automated WCAG compliance
|
|
||||||
- **Visual Testing**: Component screenshots
|
|
||||||
- **Interaction Testing**: User interactions
|
|
||||||
- **Responsive Testing**: Multiple viewports
|
|
||||||
|
|
||||||
### Available Scripts
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run storybook # Start Storybook dev server
|
|
||||||
npm run test:sb # Run Storybook tests
|
|
||||||
npm run build-storybook # Build Storybook
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🔄 CI/CD Pipeline
|
## 🔄 CI/CD Pipeline
|
||||||
|
|
||||||
### Gitea Actions Workflow
|
### Gitea Actions Workflow
|
||||||
|
|
||||||
Location: `.gitea/workflows/ci.yml`
|
Location: `.gitea/workflows/ci.yaml`
|
||||||
|
|
||||||
### Pipeline Jobs
|
### Pipeline Jobs
|
||||||
|
|
||||||
@@ -376,8 +462,7 @@ Location: `.gitea/workflows/ci.yml`
|
|||||||
#### 3. Visual Regression Tests
|
#### 3. Visual Regression Tests
|
||||||
|
|
||||||
- **Screenshot comparison**: Baseline vs current
|
- **Screenshot comparison**: Baseline vs current
|
||||||
- **Artifact upload**: Screenshot diffs
|
- **Cross-browser validation**: All 4 browser projects
|
||||||
- **Cross-browser validation**
|
|
||||||
|
|
||||||
#### 4. Performance Tests
|
#### 4. Performance Tests
|
||||||
|
|
||||||
@@ -395,7 +480,6 @@ Location: `.gitea/workflows/ci.yml`
|
|||||||
|
|
||||||
- **ESLint**: Code quality
|
- **ESLint**: Code quality
|
||||||
- **Prettier**: Code formatting
|
- **Prettier**: Code formatting
|
||||||
- **Type checking**: TypeScript validation
|
|
||||||
|
|
||||||
#### 7. Build Verification
|
#### 7. Build Verification
|
||||||
|
|
||||||
@@ -434,48 +518,6 @@ git add .
|
|||||||
git commit -m "feat: add new component with tests"
|
git commit -m "feat: add new component with tests"
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. Manual Runner Management
|
|
||||||
|
|
||||||
The Gitea Actions runner is managed manually to save resources and provide control over when CI runs.
|
|
||||||
|
|
||||||
#### Start Runner (Before Creating PR)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start the runner to execute CI jobs
|
|
||||||
./start-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Check Runner Status
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check if runner is running and see recent logs
|
|
||||||
./status-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Stop Runner (After PR Complete)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Stop the runner to free up resources
|
|
||||||
./stop-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Complete PR Workflow
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Start runner
|
|
||||||
./start-runner.sh
|
|
||||||
|
|
||||||
# 2. Create Pull Request
|
|
||||||
# Go to repository → New Pull Request
|
|
||||||
|
|
||||||
# 3. Monitor CI progress
|
|
||||||
./status-runner.sh
|
|
||||||
# Or check Gitea Actions page
|
|
||||||
|
|
||||||
# 4. Stop runner when done
|
|
||||||
./stop-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Pull Request Process
|
### 2. Pull Request Process
|
||||||
|
|
||||||
1. **Create PR** → CI pipeline starts automatically
|
1. **Create PR** → CI pipeline starts automatically
|
||||||
@@ -489,11 +531,14 @@ The Gitea Actions runner is managed manually to save resources and provide contr
|
|||||||
```bash
|
```bash
|
||||||
# Make visual changes
|
# Make visual changes
|
||||||
# Run visual regression tests
|
# Run visual regression tests
|
||||||
npm run e2e:serve
|
npm run visual:test
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts
|
|
||||||
|
|
||||||
# If changes are intentional, update baselines
|
# If changes are intentional, update baselines
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts --update-snapshots
|
npm run visual:update
|
||||||
|
|
||||||
|
# Review and commit updated snapshots
|
||||||
|
git add tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
git commit -m "Update visual regression snapshots for [describe changes]"
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4. Performance Monitoring
|
### 4. Performance Monitoring
|
||||||
@@ -503,7 +548,7 @@ npx playwright test tests/e2e/visual-regression.spec.ts --update-snapshots
|
|||||||
npm run lhci
|
npm run lhci
|
||||||
|
|
||||||
# Review performance budgets
|
# Review performance budgets
|
||||||
# Update lighthouserc.json if needed
|
# Update .lighthouserc.json if needed
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📋 Best Practices
|
## 📋 Best Practices
|
||||||
@@ -518,14 +563,14 @@ npm run lhci
|
|||||||
### 2. Component Testing
|
### 2. Component Testing
|
||||||
|
|
||||||
```jsx
|
```jsx
|
||||||
// Good: Test behavior, not implementation
|
// ✅ Good: Test behavior, not implementation
|
||||||
test("shows error message when form is invalid", () => {
|
test("shows error message when form is invalid", () => {
|
||||||
render(<Form />);
|
render(<Form />);
|
||||||
fireEvent.click(screen.getByRole("button"));
|
fireEvent.click(screen.getByRole("button"));
|
||||||
expect(screen.getByText("Please fill all fields")).toBeInTheDocument();
|
expect(screen.getByText("Please fill all fields")).toBeInTheDocument();
|
||||||
});
|
});
|
||||||
|
|
||||||
// Avoid: Testing implementation details
|
// ❌ Avoid: Testing implementation details
|
||||||
test("calls onSubmit with form data", () => {
|
test("calls onSubmit with form data", () => {
|
||||||
const mockSubmit = vi.fn();
|
const mockSubmit = vi.fn();
|
||||||
render(<Form onSubmit={mockSubmit} />);
|
render(<Form onSubmit={mockSubmit} />);
|
||||||
@@ -554,6 +599,21 @@ test("calls onSubmit with form data", () => {
|
|||||||
- Test on different network conditions
|
- Test on different network conditions
|
||||||
- Regular performance audits
|
- Regular performance audits
|
||||||
|
|
||||||
|
### 6. Responsive Testing
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// ✅ Good: Test real viewport sizes
|
||||||
|
await page.setViewportSize({ width: 640, height: 700 });
|
||||||
|
|
||||||
|
// ✅ Good: Test visibility at breakpoints
|
||||||
|
if (bp.name === "xs") {
|
||||||
|
await expect(page.getByTestId("auth-xs")).toBeVisible();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ❌ Avoid: Testing responsive behavior in JSDOM
|
||||||
|
// JSDOM doesn't evaluate CSS media queries
|
||||||
|
```
|
||||||
|
|
||||||
## 🔧 Troubleshooting
|
## 🔧 Troubleshooting
|
||||||
|
|
||||||
### Common Issues
|
### Common Issues
|
||||||
@@ -575,7 +635,6 @@ npm test
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Run locally first
|
# Run locally first
|
||||||
npm run e2e:serve
|
|
||||||
npm run e2e
|
npm run e2e
|
||||||
|
|
||||||
# Common issues:
|
# Common issues:
|
||||||
@@ -589,10 +648,10 @@ npm run e2e
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Check if changes are intentional
|
# Check if changes are intentional
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts
|
npm run visual:test
|
||||||
|
|
||||||
# Update baselines if needed
|
# Update baselines if needed
|
||||||
npx playwright test tests/e2e/visual-regression.spec.ts --update-snapshots
|
npm run visual:update
|
||||||
|
|
||||||
# Review screenshot diffs in CI artifacts
|
# Review screenshot diffs in CI artifacts
|
||||||
```
|
```
|
||||||
@@ -603,7 +662,7 @@ npx playwright test tests/e2e/visual-regression.spec.ts --update-snapshots
|
|||||||
# Run locally
|
# Run locally
|
||||||
npm run lhci
|
npm run lhci
|
||||||
|
|
||||||
# Check performance budgets in lighthouserc.json
|
# Check performance budgets in .lighthouserc.json
|
||||||
# Optimize slow components
|
# Optimize slow components
|
||||||
# Review bundle size
|
# Review bundle size
|
||||||
```
|
```
|
||||||
@@ -636,43 +695,6 @@ npx playwright test tests/e2e/homepage.spec.ts
|
|||||||
npx playwright test --headed
|
npx playwright test --headed
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📊 Monitoring & Metrics
|
|
||||||
|
|
||||||
### 1. Test Coverage
|
|
||||||
|
|
||||||
- **Target**: >85% line coverage
|
|
||||||
- **Monitoring**: Codecov integration
|
|
||||||
- **Trends**: Track coverage over time
|
|
||||||
- **Reports**: Available in CI artifacts
|
|
||||||
|
|
||||||
### 2. Performance Metrics
|
|
||||||
|
|
||||||
- **Core Web Vitals**: LCP < 2.5s, FID < 100ms, CLS < 0.1
|
|
||||||
- **Performance Score**: >90
|
|
||||||
- **Accessibility Score**: >95
|
|
||||||
- **Monitoring**: Lighthouse CI reports
|
|
||||||
|
|
||||||
### 3. Visual Regression
|
|
||||||
|
|
||||||
- **Baseline Screenshots**: 92 total
|
|
||||||
- **Cross-browser Coverage**: 4 browsers
|
|
||||||
- **Responsive Testing**: 4 viewports
|
|
||||||
- **Monitoring**: Screenshot diffs in CI
|
|
||||||
|
|
||||||
### 4. E2E Test Results
|
|
||||||
|
|
||||||
- **Total Tests**: 308 across 4 browsers
|
|
||||||
- **Success Rate**: Monitor test stability
|
|
||||||
- **Execution Time**: Track performance
|
|
||||||
- **Reports**: Available in CI artifacts
|
|
||||||
|
|
||||||
### 5. CI Pipeline Health
|
|
||||||
|
|
||||||
- **Job Success Rate**: Monitor pipeline stability
|
|
||||||
- **Execution Time**: Track build performance
|
|
||||||
- **Resource Usage**: Monitor CI costs
|
|
||||||
- **Failure Analysis**: Identify common issues
|
|
||||||
|
|
||||||
## 📚 Additional Resources
|
## 📚 Additional Resources
|
||||||
|
|
||||||
### Documentation
|
### Documentation
|
||||||
@@ -698,4 +720,5 @@ npx playwright test --headed
|
|||||||
---
|
---
|
||||||
|
|
||||||
**Last Updated**: December 2024
|
**Last Updated**: December 2024
|
||||||
**Framework Version**: Next.js 15 + React 19 + Tailwind 4 + Storybook 9
|
**Framework Version**: Next.js 15 + React 19 + Tailwind 4 + Storybook 9
|
||||||
|
**Maintained by**: CommunityRule Development Team
|
||||||
@@ -0,0 +1,327 @@
|
|||||||
|
# Testing Quick Reference
|
||||||
|
|
||||||
|
## 🚀 Essential Commands
|
||||||
|
|
||||||
|
### Daily Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run all tests with coverage
|
||||||
|
npm test
|
||||||
|
|
||||||
|
# Watch mode (during development)
|
||||||
|
npm run test:watch
|
||||||
|
|
||||||
|
# E2E tests
|
||||||
|
npm run e2e
|
||||||
|
|
||||||
|
# Visual regression tests
|
||||||
|
npm run visual:test
|
||||||
|
|
||||||
|
# Performance check
|
||||||
|
npm run lhci
|
||||||
|
|
||||||
|
# Storybook tests
|
||||||
|
npm run test:sb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test UI & Debugging
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Debug unit tests
|
||||||
|
npm run test:ui
|
||||||
|
|
||||||
|
# Debug E2E tests
|
||||||
|
npm run e2e:ui
|
||||||
|
|
||||||
|
# Debug with browser
|
||||||
|
npx playwright test --debug
|
||||||
|
|
||||||
|
# Run tests in headed mode
|
||||||
|
npx playwright test --headed
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📊 Current Test Status
|
||||||
|
|
||||||
|
- **Unit Tests**: 94.88% ✅ (Target: >85%)
|
||||||
|
- **Integration Tests**: 5 comprehensive test suites ✅
|
||||||
|
- **E2E Tests**: 92 tests across 4 browsers ✅
|
||||||
|
- **Visual Regression**: 23 tests per browser ✅
|
||||||
|
- **Accessibility Tests**: WCAG 2.1 AA compliance ✅
|
||||||
|
- **Performance Tests**: Lighthouse CI with budgets ✅
|
||||||
|
|
||||||
|
## 🔧 Common Test Commands
|
||||||
|
|
||||||
|
### Unit Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run specific test file
|
||||||
|
npm test -- --run tests/unit/Component.test.jsx
|
||||||
|
|
||||||
|
# Run tests matching pattern
|
||||||
|
npm test -- --run Component
|
||||||
|
|
||||||
|
# Run with coverage report
|
||||||
|
npm test -- --coverage
|
||||||
|
|
||||||
|
# Run in watch mode
|
||||||
|
npm run test:watch
|
||||||
|
```
|
||||||
|
|
||||||
|
### E2E Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run specific test file
|
||||||
|
npm run e2e -- tests/e2e/homepage.spec.ts
|
||||||
|
|
||||||
|
# Run specific project (browser)
|
||||||
|
npm run e2e -- --project=chromium
|
||||||
|
|
||||||
|
# Run with headed browser
|
||||||
|
npm run e2e -- --headed
|
||||||
|
|
||||||
|
# Run with debug mode
|
||||||
|
npm run e2e -- --debug
|
||||||
|
```
|
||||||
|
|
||||||
|
### Visual Regression
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Update snapshots for all projects
|
||||||
|
npm run visual:update
|
||||||
|
|
||||||
|
# Update snapshots for specific project
|
||||||
|
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts --project=chromium
|
||||||
|
|
||||||
|
# View test results
|
||||||
|
npx playwright show-report
|
||||||
|
```
|
||||||
|
|
||||||
|
### Performance Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run mobile performance test
|
||||||
|
npm run lhci:mobile
|
||||||
|
|
||||||
|
# Run desktop performance test
|
||||||
|
npm run lhci:desktop
|
||||||
|
|
||||||
|
# Run with custom budget
|
||||||
|
npm run performance:budget
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📱 Browser Support
|
||||||
|
|
||||||
|
| Browser | Project Name | Status |
|
||||||
|
| ----------- | ------------ | --------------- |
|
||||||
|
| **Chrome** | `chromium` | ✅ Full Support |
|
||||||
|
| **Firefox** | `firefox` | ✅ Full Support |
|
||||||
|
| **Safari** | `webkit` | ✅ Full Support |
|
||||||
|
| **Mobile** | `mobile` | ✅ Full Support |
|
||||||
|
|
||||||
|
## 🎯 Testing Best Practices
|
||||||
|
|
||||||
|
### 1. Test Structure (AAA Pattern)
|
||||||
|
|
||||||
|
```jsx
|
||||||
|
test("should do something", () => {
|
||||||
|
// Arrange: Set up test data
|
||||||
|
const data = { name: "Test" };
|
||||||
|
|
||||||
|
// Act: Perform the action
|
||||||
|
const result = processData(data);
|
||||||
|
|
||||||
|
// Assert: Verify the outcome
|
||||||
|
expect(result).toBe("Processed Test");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Query Priority
|
||||||
|
|
||||||
|
1. **`getByRole`** - Most accessible, tests user experience
|
||||||
|
2. **`getByLabelText`** - For form inputs
|
||||||
|
3. **`getByText`** - For content
|
||||||
|
4. **`getByTestId`** - Last resort, avoid when possible
|
||||||
|
|
||||||
|
### 3. Async Testing
|
||||||
|
|
||||||
|
```jsx
|
||||||
|
test("async operation", async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
|
||||||
|
render(<Component />);
|
||||||
|
const button = screen.getByRole("button");
|
||||||
|
|
||||||
|
await user.click(button);
|
||||||
|
await waitFor(() => {
|
||||||
|
expect(screen.getByText("Success")).toBeInTheDocument();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Responsive Testing
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// ✅ Good: Test real viewport sizes
|
||||||
|
await page.setViewportSize({ width: 640, height: 700 });
|
||||||
|
|
||||||
|
// ✅ Good: Test visibility at breakpoints
|
||||||
|
if (bp.name === "xs") {
|
||||||
|
await expect(page.getByTestId("auth-xs")).toBeVisible();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ❌ Avoid: Testing responsive behavior in JSDOM
|
||||||
|
// JSDOM doesn't evaluate CSS media queries
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔍 Common Issues & Solutions
|
||||||
|
|
||||||
|
### Visual Regression Failures
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Regenerate snapshots
|
||||||
|
npm run visual:update
|
||||||
|
|
||||||
|
# Check for environment differences
|
||||||
|
# Ensure deterministic rendering in Playwright config
|
||||||
|
```
|
||||||
|
|
||||||
|
### E2E Test Failures
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Use waitFor instead of waitForTimeout
|
||||||
|
await page.waitForSelector("button", { state: "visible" });
|
||||||
|
|
||||||
|
# Use role-based selectors
|
||||||
|
await page.getByRole("button", { name: "Submit" }).click();
|
||||||
|
|
||||||
|
# Check for selector changes
|
||||||
|
# Verify component structure hasn't changed
|
||||||
|
```
|
||||||
|
|
||||||
|
### Performance Test Failures
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check Chrome path on macOS
|
||||||
|
# Ensure arm64 Chrome for Apple Silicon
|
||||||
|
# Verify performance budgets in .lighthouserc.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Unit Test Failures
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check for missing imports
|
||||||
|
# Verify component exports
|
||||||
|
# Ensure test environment setup
|
||||||
|
# Check for component changes
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📈 Performance Budgets
|
||||||
|
|
||||||
|
### Lighthouse CI Targets
|
||||||
|
|
||||||
|
- **Performance Score**: >80
|
||||||
|
- **Accessibility Score**: >80
|
||||||
|
- **Best Practices**: >90
|
||||||
|
- **SEO Score**: >90
|
||||||
|
|
||||||
|
### Core Web Vitals
|
||||||
|
|
||||||
|
- **LCP**: <2.5s
|
||||||
|
- **FID**: <100ms
|
||||||
|
- **CLS**: <0.1
|
||||||
|
|
||||||
|
### Performance Budgets
|
||||||
|
|
||||||
|
- **First Contentful Paint**: <3000ms
|
||||||
|
- **Largest Contentful Paint**: <5000ms
|
||||||
|
- **First Input Delay**: <100ms
|
||||||
|
- **TTFB**: <700ms
|
||||||
|
|
||||||
|
## 🔄 CI/CD Pipeline Jobs
|
||||||
|
|
||||||
|
1. **Unit Tests** (Node 18, 20) - Coverage reporting
|
||||||
|
2. **E2E Tests** (Chromium, Firefox, WebKit) - Cross-browser testing
|
||||||
|
3. **Visual Regression Tests** - Screenshot comparison
|
||||||
|
4. **Performance Tests** - Lighthouse CI with budgets
|
||||||
|
5. **Storybook Tests** - Component testing & accessibility
|
||||||
|
6. **Lint & Format** - Code quality & formatting
|
||||||
|
7. **Build Verification** - Next.js & Storybook builds
|
||||||
|
|
||||||
|
## 📁 Test File Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
tests/
|
||||||
|
├── unit/ # Component tests
|
||||||
|
│ ├── Button.test.jsx # 12 tests
|
||||||
|
│ ├── Logo.test.jsx # 12 tests
|
||||||
|
│ ├── RuleCard.test.jsx # 18 tests
|
||||||
|
│ ├── SectionHeader.test.jsx # 17 tests
|
||||||
|
│ ├── NumberedCard.test.jsx # 18 tests
|
||||||
|
│ └── accessibility.test.jsx # 18 tests
|
||||||
|
├── integration/ # Integration tests
|
||||||
|
│ ├── component-interactions.integration.test.jsx
|
||||||
|
│ ├── page-flow.integration.test.jsx
|
||||||
|
│ ├── user-journey.integration.test.jsx
|
||||||
|
│ ├── layout.integration.test.jsx
|
||||||
|
│ └── ContentLockup.integration.test.jsx
|
||||||
|
└── e2e/ # E2E tests
|
||||||
|
├── homepage.spec.ts # Homepage functionality
|
||||||
|
├── user-journeys.spec.ts # User workflows
|
||||||
|
├── header.responsive.spec.js # Responsive header
|
||||||
|
├── footer.responsive.spec.js # Responsive footer
|
||||||
|
├── visual-regression.spec.ts # Visual consistency
|
||||||
|
├── accessibility.spec.ts # Accessibility compliance
|
||||||
|
└── performance.spec.ts # Performance metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎨 Visual Regression Screenshots
|
||||||
|
|
||||||
|
### Generated Screenshots
|
||||||
|
|
||||||
|
- Full page (mobile, tablet, desktop)
|
||||||
|
- Component sections (hero, logo wall, cards)
|
||||||
|
- Interactive states (hover, focus, loading)
|
||||||
|
- Special modes (dark, high contrast, reduced motion)
|
||||||
|
|
||||||
|
### Managing Changes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Intentional changes
|
||||||
|
npm run visual:update
|
||||||
|
|
||||||
|
# Review changes
|
||||||
|
git diff tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
|
||||||
|
# Commit updated snapshots
|
||||||
|
git add tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
git commit -m "Update visual regression snapshots for [describe changes]"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📈 Monitoring
|
||||||
|
|
||||||
|
### Test Metrics
|
||||||
|
|
||||||
|
- **Unit Tests**: 305 tests (94.88% coverage)
|
||||||
|
- **E2E Tests**: 92 tests (4 browsers)
|
||||||
|
- **Visual Screenshots**: 92 baselines per browser
|
||||||
|
- **Coverage**: >85% target (exceeded)
|
||||||
|
|
||||||
|
### CI Metrics
|
||||||
|
|
||||||
|
- **Pipeline Jobs**: 7 parallel jobs
|
||||||
|
- **Execution Time**: Monitor build performance
|
||||||
|
- **Success Rate**: Track pipeline stability
|
||||||
|
- **Artifacts**: Test results and screenshots
|
||||||
|
|
||||||
|
## 🔗 Useful Links
|
||||||
|
|
||||||
|
- **Full Testing Documentation**: [docs/testing-framework.md](./testing-framework.md)
|
||||||
|
- **Vitest Docs**: https://vitest.dev/
|
||||||
|
- **Playwright Docs**: https://playwright.dev/
|
||||||
|
- **React Testing Library**: https://testing-library.com/docs/react-testing-library/intro/
|
||||||
|
- **Lighthouse CI**: https://github.com/GoogleChrome/lighthouse-ci
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Quick Reference Version**: December 2024
|
||||||
|
**For detailed guidelines, see [testing-framework.md](./testing-framework.md)**
|
||||||
@@ -0,0 +1,390 @@
|
|||||||
|
# Visual Regression Testing Guide
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Visual regression testing ensures UI consistency across browsers and prevents unintended visual changes by comparing screenshots against baseline images. This guide covers the complete workflow for managing visual regression tests.
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
### First-Time Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Generate baseline snapshots for all projects
|
||||||
|
npm run visual:update
|
||||||
|
|
||||||
|
# 2. Verify snapshots were created
|
||||||
|
ls tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
|
||||||
|
# 3. Commit the snapshots
|
||||||
|
git add tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
git commit -m "Add baseline visual regression snapshots"
|
||||||
|
|
||||||
|
# 4. Verify setup works
|
||||||
|
npm run visual:test
|
||||||
|
```
|
||||||
|
|
||||||
|
### Daily Workflow
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run visual regression tests
|
||||||
|
npm run visual:test
|
||||||
|
|
||||||
|
# Run with UI for debugging
|
||||||
|
npm run visual:ui
|
||||||
|
|
||||||
|
# Update snapshots after UI changes
|
||||||
|
npm run visual:update
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📝 Managing Visual Changes
|
||||||
|
|
||||||
|
### When UI Changes Are Intentional
|
||||||
|
|
||||||
|
1. **Make your UI changes** (design updates, component modifications, etc.)
|
||||||
|
|
||||||
|
2. **Update snapshots to reflect new design:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run visual:update
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Review changes:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git diff tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Commit updated snapshots:**
|
||||||
|
```bash
|
||||||
|
git add tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
git commit -m "Update snapshots for [describe changes]"
|
||||||
|
```
|
||||||
|
|
||||||
|
### When UI Changes Are Unintentional
|
||||||
|
|
||||||
|
1. **Investigate the failure** - Check what changed and why
|
||||||
|
2. **Fix the regression** - Revert or fix the unintended change
|
||||||
|
3. **Re-run tests** - Ensure they pass without updating snapshots
|
||||||
|
4. **Commit the fix** - Don't update snapshots for bug fixes
|
||||||
|
|
||||||
|
## ⚙️ Configuration
|
||||||
|
|
||||||
|
### Playwright Configuration
|
||||||
|
|
||||||
|
The visual regression tests use these key settings in `playwright.config.ts`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export default defineConfig({
|
||||||
|
expect: {
|
||||||
|
toHaveScreenshot: {
|
||||||
|
animations: "disabled",
|
||||||
|
maxDiffPixelRatio: 0.02, // 2% tolerance
|
||||||
|
maxDiffPixels: 500, // 500 pixel tolerance
|
||||||
|
},
|
||||||
|
},
|
||||||
|
use: {
|
||||||
|
timezoneId: "UTC", // Consistent timezone
|
||||||
|
locale: "en-US", // Consistent locale
|
||||||
|
headless: true, // Headless for CI
|
||||||
|
},
|
||||||
|
snapshotPathTemplate:
|
||||||
|
"{testDir}/{testFileName}-snapshots/{arg}-{projectName}.png",
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deterministic Rendering
|
||||||
|
|
||||||
|
To ensure consistent screenshots across environments:
|
||||||
|
|
||||||
|
- **Fixed timezone**: UTC
|
||||||
|
- **Fixed locale**: en-US
|
||||||
|
- **Fixed viewport**: 1280x800 (configurable per test)
|
||||||
|
- **Disabled animations**: Prevents timing-related differences
|
||||||
|
- **Browser-specific snapshots**: Separate baselines per browser
|
||||||
|
|
||||||
|
## 📱 Breakpoint Coverage
|
||||||
|
|
||||||
|
### Standard Viewports
|
||||||
|
|
||||||
|
| Breakpoint | Width | Height | Description |
|
||||||
|
| ----------- | ------ | ------ | ---------------- |
|
||||||
|
| **Mobile** | 375px | 667px | iPhone portrait |
|
||||||
|
| **Tablet** | 768px | 1024px | iPad portrait |
|
||||||
|
| **Desktop** | 1280px | 800px | Standard desktop |
|
||||||
|
| **Large** | 1920px | 1080px | Full HD desktop |
|
||||||
|
|
||||||
|
### Custom Viewports
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
test("mobile layout", async ({ page }) => {
|
||||||
|
await page.setViewportSize({ width: 375, height: 667 });
|
||||||
|
await expect(page).toHaveScreenshot("mobile-layout.png");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("tablet layout", async ({ page }) => {
|
||||||
|
await page.setViewportSize({ width: 768, height: 1024 });
|
||||||
|
await expect(page).toHaveScreenshot("tablet-layout.png");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎨 Screenshot Types
|
||||||
|
|
||||||
|
### Full Page Screenshots
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
test("homepage full page", async ({ page }) => {
|
||||||
|
await expect(page).toHaveScreenshot("homepage-full.png", {
|
||||||
|
fullPage: true,
|
||||||
|
animations: "disabled",
|
||||||
|
scale: "css",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Component Screenshots
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
test("hero section", async ({ page }) => {
|
||||||
|
const hero = page.locator("[data-testid='hero-section']");
|
||||||
|
await expect(hero).toHaveScreenshot("hero-section.png");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Interactive States
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
test("button hover state", async ({ page }) => {
|
||||||
|
const button = page.getByRole("button", { name: "Submit" });
|
||||||
|
|
||||||
|
// Normal state
|
||||||
|
await expect(button).toHaveScreenshot("button-normal.png");
|
||||||
|
|
||||||
|
// Hover state
|
||||||
|
await button.hover();
|
||||||
|
await expect(button).toHaveScreenshot("button-hover.png");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Special Modes
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
test("dark mode", async ({ page }) => {
|
||||||
|
// Enable dark mode
|
||||||
|
await page.evaluate(() => {
|
||||||
|
document.documentElement.classList.add("dark");
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(page).toHaveScreenshot("dark-mode.png");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("high contrast", async ({ page }) => {
|
||||||
|
// Enable high contrast
|
||||||
|
await page.evaluate(() => {
|
||||||
|
document.body.style.filter = "contrast(200%)";
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(page).toHaveScreenshot("high-contrast.png");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔄 Snapshot Management
|
||||||
|
|
||||||
|
### Update Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Update all snapshots for all projects
|
||||||
|
npm run visual:update
|
||||||
|
|
||||||
|
# Update snapshots for specific project
|
||||||
|
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts --project=chromium
|
||||||
|
|
||||||
|
# Update snapshots for specific test
|
||||||
|
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts --grep="homepage"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Snapshot Naming Convention
|
||||||
|
|
||||||
|
Snapshots follow this pattern:
|
||||||
|
|
||||||
|
```
|
||||||
|
{testDir}/{testFileName}-snapshots/{arg}-{projectName}.png
|
||||||
|
```
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- `tests/e2e/visual-regression.spec.ts-snapshots/homepage-full-chromium.png`
|
||||||
|
- `tests/e2e/visual-regression.spec.ts-snapshots/hero-section-firefox.png`
|
||||||
|
- `tests/e2e/visual-regression.spec.ts-snapshots/button-hover-webkit.png`
|
||||||
|
- `tests/e2e/visual-regression.spec.ts-snapshots/mobile-layout-mobile.png`
|
||||||
|
|
||||||
|
### File Organization
|
||||||
|
|
||||||
|
```
|
||||||
|
tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
├── homepage-full-chromium.png
|
||||||
|
├── homepage-full-firefox.png
|
||||||
|
├── homepage-full-webkit.png
|
||||||
|
├── homepage-full-mobile.png
|
||||||
|
├── hero-section-chromium.png
|
||||||
|
├── hero-section-firefox.png
|
||||||
|
├── hero-section-webkit.png
|
||||||
|
├── hero-section-mobile.png
|
||||||
|
└── ... (92 total screenshots)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🐛 Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
#### 1. "Snapshot doesn't exist" errors
|
||||||
|
|
||||||
|
**Cause**: Baseline snapshots haven't been generated or are missing
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Regenerate all snapshots
|
||||||
|
npm run visual:update
|
||||||
|
|
||||||
|
# Or regenerate for specific project
|
||||||
|
PLAYWRIGHT_UPDATE_SNAPSHOTS=1 npx playwright test tests/e2e/visual-regression.spec.ts --project=chromium
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2. Platform differences (macOS vs Linux)
|
||||||
|
|
||||||
|
**Cause**: Different font rendering between platforms
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
|
||||||
|
- Use CI-generated snapshots for consistency
|
||||||
|
- Ensure deterministic rendering settings
|
||||||
|
- Check font availability across platforms
|
||||||
|
|
||||||
|
#### 3. Minor pixel differences
|
||||||
|
|
||||||
|
**Cause**: Font rendering, anti-aliasing, scaling differences
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
|
||||||
|
- Check tolerance settings in `playwright.config.ts`
|
||||||
|
- Use `scale: "css"` for consistent scaling
|
||||||
|
- Ensure deterministic CSS properties
|
||||||
|
|
||||||
|
#### 4. Animation-related failures
|
||||||
|
|
||||||
|
**Cause**: Animations not fully disabled
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
|
||||||
|
- Ensure `animations: "disabled"` is set in test configuration
|
||||||
|
- Wait for animations to complete before screenshots
|
||||||
|
- Use `waitForTimeout` if necessary
|
||||||
|
|
||||||
|
#### 5. Height differences (especially WebKit)
|
||||||
|
|
||||||
|
**Cause**: WebKit may render elements with slightly different heights
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
|
||||||
|
- Increase tolerance for height-sensitive tests
|
||||||
|
- Use `maxDiffPixels: 1000` for specific tests
|
||||||
|
- Consider using `ignoreSize: false` (default)
|
||||||
|
|
||||||
|
### Debug Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run with UI for visual debugging
|
||||||
|
npm run visual:ui
|
||||||
|
|
||||||
|
# Run specific test with debugging
|
||||||
|
npx playwright test tests/e2e/visual-regression.spec.ts --grep="homepage" --debug
|
||||||
|
|
||||||
|
# Run with headed browser
|
||||||
|
npx playwright test tests/e2e/visual-regression.spec.ts --headed
|
||||||
|
|
||||||
|
# View test results
|
||||||
|
npx playwright show-report
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment Consistency
|
||||||
|
|
||||||
|
To ensure consistent results:
|
||||||
|
|
||||||
|
1. **Use same Node.js version** across environments
|
||||||
|
2. **Use same Playwright version** across environments
|
||||||
|
3. **Use same browser versions** when possible
|
||||||
|
4. **Set consistent environment variables** (timezone, locale)
|
||||||
|
5. **Use deterministic CSS** (avoid random values, timestamps)
|
||||||
|
|
||||||
|
## 📊 CI/CD Integration
|
||||||
|
|
||||||
|
### CI Workflow
|
||||||
|
|
||||||
|
Visual regression tests run automatically in the CI pipeline:
|
||||||
|
|
||||||
|
- **Main branch**: Tests run against existing snapshots
|
||||||
|
- **Feature branches**: Tests run against existing snapshots
|
||||||
|
- **Artifacts**: Test results and screenshots uploaded for review
|
||||||
|
|
||||||
|
### CI Best Practices
|
||||||
|
|
||||||
|
1. **Don't regenerate snapshots in CI** for feature branches
|
||||||
|
2. **Use CI-generated snapshots** as the source of truth
|
||||||
|
3. **Review screenshot diffs** in CI artifacts
|
||||||
|
4. **Fail fast** on visual regressions
|
||||||
|
|
||||||
|
### Artifact Management
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Example CI artifact configuration
|
||||||
|
- name: Upload visual regression results
|
||||||
|
if: always()
|
||||||
|
uses: actions/upload-artifact@v3
|
||||||
|
with:
|
||||||
|
name: visual-regression-results
|
||||||
|
path: |
|
||||||
|
test-results/
|
||||||
|
tests/e2e/visual-regression.spec.ts-snapshots/
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 Best Practices
|
||||||
|
|
||||||
|
### 1. Snapshot Management
|
||||||
|
|
||||||
|
- **Update snapshots only for intentional changes**
|
||||||
|
- **Review all changes** before committing
|
||||||
|
- **Use descriptive names** for snapshot files
|
||||||
|
- **Keep snapshots in version control**
|
||||||
|
|
||||||
|
### 2. Test Design
|
||||||
|
|
||||||
|
- **Test critical UI components** first
|
||||||
|
- **Use consistent viewport sizes** across tests
|
||||||
|
- **Test responsive breakpoints** systematically
|
||||||
|
- **Include interactive states** when relevant
|
||||||
|
|
||||||
|
### 3. Performance
|
||||||
|
|
||||||
|
- **Limit snapshot count** to essential components
|
||||||
|
- **Use appropriate timeouts** for slow operations
|
||||||
|
- **Parallelize tests** when possible
|
||||||
|
- **Cache browser installations** in CI
|
||||||
|
|
||||||
|
### 4. Maintenance
|
||||||
|
|
||||||
|
- **Regular cleanup** of outdated snapshots
|
||||||
|
- **Update snapshots promptly** after UI changes
|
||||||
|
- **Monitor test execution time** and optimize
|
||||||
|
- **Review and update tolerance settings** as needed
|
||||||
|
|
||||||
|
## 📚 Additional Resources
|
||||||
|
|
||||||
|
- **Main Testing Documentation**: [testing-framework.md](./testing-framework.md)
|
||||||
|
- **Playwright Visual Testing**: https://playwright.dev/docs/screenshots
|
||||||
|
- **Visual Regression Best Practices**: https://storybook.js.org/docs/writing-tests/visual-testing
|
||||||
|
- **CI/CD Integration**: [testing-quick-reference.md](./testing-quick-reference.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated**: December 2024
|
||||||
|
**Maintained by**: CommunityRule Development Team
|
||||||
Reference in New Issue
Block a user