# CI Template Testing Guide

This document explains how to test and validate the CI workflow templates in this repository.

## Overview

The testing infrastructure includes:

- **Automated validation workflow** (`.gitea/workflows/validate-templates.yml`)
- **Manual validation script** (`scripts/validate-templates.sh`)
- **Test projects** (`test-projects/`)
- **Matrix testing** across multiple configurations

## Quick Start

### 1. Run Local Validation

```bash
# Make the script executable
chmod +x scripts/validate-templates.sh

# Validate all workflows
./scripts/validate-templates.sh

# List available templates
./scripts/validate-templates.sh list

# Show help
./scripts/validate-templates.sh help
```

### 2. Test with Sample Projects

```bash
# Navigate to a test project
cd test-projects/python-basic

# Install dependencies (if using UV)
uv sync --all-extras

# Run local tests
uv run pytest
uv run mypy .
uv run ruff check .
```

## Testing Infrastructure

### 1. Validation Workflow (`.gitea/workflows/validate-templates.yml`)

Automatically runs on every push and pull request:

- **Matrix testing** across Python versions (3.10, 3.11, 3.12)
- **Template validation** for all workflow files
- **Integration testing** with sample projects
- **Documentation checks**

**Triggered by:**

- Push to `main` branch
- Pull requests
- Manual dispatch
- Daily schedule (2 AM UTC)

**Test matrix:**

```yaml
strategy:
  matrix:
    python-version: ["3.10", "3.11", "3.12"]
    template: ["code-quality", "pytest"]
    include:
      - template: "formatting"
        python-version: "3.12"  # Only test once for web template
```

### 2. Validation Script (`scripts/validate-templates.sh`)

**Features:**

- YAML syntax validation with `yamllint`
- Workflow structure validation with `yq`
- Reusable workflow detection
- Input parameter validation
- Documentation checks
- Common issue detection

**Usage:**

```bash
# Basic validation
./scripts/validate-templates.sh

# List all templates
./scripts/validate-templates.sh list

# Help
./scripts/validate-templates.sh help
```

**Dependencies:**

- `yamllint` (optional): `pip install yamllint`
- `yq` (optional): `brew install yq` (macOS) or `apt install yq` (Ubuntu)

**Note:** The script works without dependencies but provides more detailed validation when they're available.

### 3. Test Projects

#### Python Basic (`test-projects/python-basic/`)

**Purpose:** Test Python-specific templates

**Structure:**

```text
python-basic/
├── .gitea/workflows/
│   └── test-templates.yml      # Tests both code-quality and pytest templates
├── src/test_package/
│   ├── __init__.py
│   ├── calculator.py           # Sample module with type hints
│   └── utils.py                # Utility functions
├── tests/
│   ├── test_calculator.py      # Pytest tests
│   └── test_utils.py           # More tests
├── pyproject.toml              # UV-compatible project config
└── README.md
```

**Features:**

- Type hints for mypy testing
- Pytest with coverage
- Ruff-compatible code style
- UV package management

#### Web Basic (`test-projects/web-basic/`)

**Purpose:** Test web formatting template

**Structure:**

```text
web-basic/
├── .gitea/workflows/
│   └── test-formatting.yml     # Tests formatting template
├── package.json                # Node.js project
├── index.js                    # JavaScript file
├── config.yml                  # YAML file for validation
├── .prettierrc.json           # Prettier configuration
└── README.md
```

**Features:**

- Prettier configuration
- YAML validation
- JSON validation
- Node.js setup

## Testing Workflow Templates

### 1. Local Testing with Act

Install and use Act to test workflows locally:

```bash
# Install Act
brew install act  # macOS
# or
curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash

# Test a specific workflow
act -W .gitea/workflows/python/code-quality.yml

# Test with inputs
act workflow_call -W .gitea/workflows/python/code-quality.yml \
  --input python-version=3.11 \
  --input working-directory=./test-projects/python-basic
```

### 2. Branch Testing

Test changes on feature branches:

```bash
# Create feature branch
git checkout -b feature/improve-python-template

# Make changes to templates
# ...

# Update test project to use feature branch
# In test-projects/python-basic/.gitea/workflows/test-templates.yml:
# uses: gitea.gt-proj.com/brian/CI-templates/.gitea/workflows/python/code-quality.yml@feature/improve-python-template

# Push and test
git push origin feature/improve-python-template
```

### 3. Matrix Testing

Test multiple configurations:

```yaml
# Example matrix test
strategy:
  matrix:
    python-version: ["3.10", "3.11", "3.12"]
    os: ["ubuntu-latest", "ubuntu-20.04"]
    working-directory: [".", "./src"]
```

## Validation Checklist

### Workflow File Validation

- [ ] Valid YAML syntax
- [ ] Contains required fields: `name`, `on`, `jobs`
- [ ] Uses `workflow_call` trigger for reusable workflows
- [ ] All inputs have descriptions and types
- [ ] No hardcoded values that should be inputs
- [ ] Proper defaults for optional inputs

### Template Testing

- [ ] All input combinations work correctly
- [ ] Error handling for invalid inputs
- [ ] Backwards compatibility maintained
- [ ] Documentation updated
- [ ] Test projects updated

### Integration Testing

- [ ] Works with real projects
- [ ] Performance is acceptable
- [ ] No conflicts between templates
- [ ] Secrets handling works correctly

## Common Issues and Solutions

### 1. YAML Syntax Errors

**Problem:** Invalid YAML syntax

```bash
Error: YAML syntax validation failed
```

**Solution:**

- Use `yamllint` to check syntax
- Validate indentation (use spaces, not tabs)
- Check for missing quotes around strings with special characters

### 2. Missing Required Fields

**Problem:** Workflow missing required fields

```bash
Error: Missing required field 'on' in workflow.yml
```

**Solution:**

- Ensure all workflows have `name`, `on`, and `jobs` fields
- For reusable workflows, use `workflow_call` trigger

### 3. Input Validation Issues

**Problem:** Missing input descriptions or types

```bash
Warning: Input 'python-version' missing description
```

**Solution:**

```yaml
inputs:
  python-version:
    description: 'Python version to use (e.g., "3.12")'
    required: false
    type: string
    default: "3.12"
```

### 4. Test Project Issues

**Problem:** Tests fail in sample projects

```bash
Error: Module not found
```

**Solutions:**

- Check `pyproject.toml` configuration
- Verify package structure
- Ensure dependencies are correctly specified
- Check import paths

## CI/CD Best Practices

### 1. Version Control

- Use semantic versioning for releases
- Tag stable versions: `v1.0.0`, `v1.1.0`, etc.
- Reference specific versions in production: `@v1.0.0`
- Use `@main` for development/testing

### 2. Backwards Compatibility

- Test with previous versions
- Provide migration guides for breaking changes
- Deprecate features gracefully
- Maintain changelog

### 3. Security

- Validate all inputs
- Use secrets for sensitive data
- Limit permissions to minimum required
- Regular security updates

### 4. Documentation

- Keep README files updated
- Document all input parameters
- Provide usage examples
- Include troubleshooting guides

## Troubleshooting

### Validation Script Issues

```bash
# Check dependencies
./scripts/validate-templates.sh

# Manual YAML validation
yamllint .gitea/workflows/python/code-quality.yml

# Manual structure check
yq eval '.on | has("workflow_call")' .gitea/workflows/python/code-quality.yml
```

### Test Project Issues

```bash
# Python projects
cd test-projects/python-basic
uv sync --all-extras
uv run pytest --verbose

# Web projects  
cd test-projects/web-basic
npm install
npm run format:check
```

### Workflow Issues

```bash
# Check workflow logs in Gitea
# Enable debug logging:
env:
  ACTIONS_STEP_DEBUG: true
  ACTIONS_RUNNER_DEBUG: true
```

## Contributing

When adding new templates or making changes:

1. **Create/update test projects** to validate the changes
2. **Run validation script** locally before committing
3. **Update documentation** as needed
4. **Test with feature branches** before merging
5. **Tag releases** for stable versions

## Resources

- [Gitea Actions Documentation](https://docs.gitea.io/en-us/actions/)
- [GitHub Actions Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions)
- [Act - Local Testing Tool](https://github.com/nektos/act)
- [yamllint Documentation](https://yamllint.readthedocs.io/)
- [yq Documentation](https://mikefarah.gitbook.io/yq/)