service-finder/docs/gitea_sync_blueprint.md

# Gitea Synchronization Blueprint
> Generated: 2026-03-29  
> Auditor: Agile Project Manager & System Auditor  
> Status: Analysis Complete - Ready for Implementation

## Executive Summary

This document outlines the synchronization strategy between the Service Finder codebase, Masterbook documentation (2.0.0 & 2.0.1), and the Gitea project management system. The audit reveals significant gaps between documentation, implementation, and project tracking that must be addressed to establish a "Documentation-Driven" and "Issue-Driven" workflow.

---

## 1. Gitea Manager Capabilities Assessment

### Current Functionality (`gitea_manager.py`)
The existing script provides **basic issue management** but lacks **full project management capabilities**:

**✅ Supported Features:**
- Issue CRUD operations (create, read, update, delete via state changes)
- Milestone management (create, list)
- Label system with automatic creation (Status, Scope, Type, Role categories)
- Time tracking with stopwatch integration
- Comment/note adding
- Pagination handling for API responses
- Hybrid network detection (internal/external Gitea)

**❌ Missing Project Management Features:**
1. **Project Board Management** - Cannot create/manage projects or boards
2. **Column Operations** - No ability to move cards between columns (To Do → In Progress → Done)
3. **Card Positioning** - Cannot set card order/priority within columns
4. **Project Membership** - Cannot assign users to projects
5. **Board Visualization** - No way to view the Kanban board structure
6. **Batch Operations** - Cannot move multiple issues at once
7. **Webhook Integration** - No automated sync with code changes

**⚠️ Known Bugs:**
- `list_milestones()` fails with `KeyError: 'completeness'` (Gitea API returns `closed_issues`/`open_issues`, not `completeness`)
- No error handling for network failures
- Limited validation of input parameters

---

## 2. Documentation vs. Reality Gap Analysis

### Masterbook 2.0.1 Documentation Status
The `/docs/v201/` directory contains comprehensive system documentation covering:
- 19 PostgreSQL schemas with 127+ tables
- 70% completion status (backend stable, frontend incomplete)
- Detailed robot ecosystem (10+ specialized workers)
- Container infrastructure (30+ Docker containers)
- API dependencies and rate limits

### Codebase Implementation Status
**✅ Implemented (Matches Documentation):**
- Database schema with 19 schemas confirmed via `sync_engine.py`
- Robot ecosystem operational (R0-GB to R4-Validator)
- Container infrastructure running
- Core authentication and identity management
- Vehicle data pipeline with DVLA/RDW integration

**⚠️ Partially Implemented:**
- **Historical Data (`occurrence_date` fields)**: Found in `marketplace.service` table but needs audit of other cost/expense tables
- **Analytics Service (TCO/km)**: Database tables exist but service implementation incomplete
- **Frontend-Backend Integration**: UI components exist but API wiring incomplete (mocked data)

**❌ Missing from Codebase (Documented but not implemented):**
1. **Gamification Admin Controls** - Endpoints for modifying game parameters
2. **TCO Financial Aggregation** - Backend routes for unified analytics
3. **Marketplace Booking Flow** - Service request and geofenced broadcast logic
4. **Epic 11 Public Frontend** - "Smart Garage" concept with profile selector
5. **Advanced Search with Filters** - Documented but not implemented
6. **Webhook & Notification System** - Mentioned in issues but not in code

### Critical Documentation Gaps
1. **No API endpoint inventory** - Missing comprehensive list of implemented vs. planned endpoints
2. **No test coverage documentation** - Unknown which features have automated tests
3. **No deployment runbook** - Missing step-by-step deployment procedures
4. **No data migration guides** - For schema changes affecting production data
5. **No performance benchmarks** - Documented performance characteristics but no actual measurements

---

## 3. Gitea Project Structure Analysis

### Current Project Board State
Based on `gitea_manager.py list` output:
- **28 open issues** across 4 milestones
- **Milestone distribution**:
  - Phase 4: Testing & Deployment (6 issues)
  - Phase 3: Advanced Features & Epic 11 (6 issues)
  - Phase 2: Dashboard & Analytics Wiring (5 issues)
  - Phase 1: Core Functionality Fixes (3 issues)
  - No milestone assigned (8 issues)

### Issue Quality Assessment
**Well-defined issues** (e.g., #152 "Implement Historical Data"):
- Clear objective and acceptance criteria
- Target files identified
- Execution steps outlined
- Dependencies and priority specified

**Poorly-defined issues** (e.g., #140 "Connect Service Moderation Map"):
- Vague requirements
- Missing acceptance criteria
- No technical implementation details

---

## 4. Proposed Gitea Structure Plan

### MasterBook Project Hierarchy
```
Master Book 2.0 (Project)
├── 🎯 Phase 1: Core Functionality Fixes (Milestone)
├── 📊 Phase 2: Dashboard & Analytics Wiring (Milestone)
├── ⚡ Phase 3: Advanced Features & Epic 11 (Milestone)
├── 🚀 Phase 4: Testing & Deployment (Milestone)
└── 🔧 Phase 5: Maintenance & Optimization (New Milestone)
```

### Label System Enhancement
**Current labels are sufficient** but need better utilization:
- `Status: To Do`, `In Progress`, `Done`, `Blocked`
- `Scope: Backend`, `Frontend`, `API`, `Core`, `Robot`, `Database`
- `Type: Script`, `Model`, `Database`, `Bug`, `Feature`, `Refactor`
- `Role: Admin`, `User`

**Recommended additions:**
- `Priority: P0` (Critical), `P1` (High), `P2` (Medium), `P3` (Low)
- `Complexity: Simple`, `Medium`, `Complex`, `Epic`
- `Risk: Low`, `Medium`, `High`

### Issue Creation/Update Plan
Based on documentation gaps, **27 new issues** should be created:

#### Category 1: Documentation Updates (5 issues)
1. **Create API Endpoint Inventory** - Document all implemented endpoints with status
2. **Generate Test Coverage Report** - Map tests to features and identify gaps
3. **Write Deployment Runbook** - Step-by-step production deployment guide
4. **Create Data Migration Guide** - Procedures for schema changes
5. **Document Performance Benchmarks** - Actual measurements vs. targets

#### Category 2: gitea_manager.py Enhancements (8 issues)
6. **Fix Milestone Listing Bug** - Resolve `KeyError: 'completeness'`
7. **Add Project Board Management** - Create/move cards between columns
8. **Implement Column Operations** - Support Kanban board workflows
9. **Add Card Positioning** - Set priority/order within columns
10. **Implement Batch Operations** - Move multiple issues simultaneously
11. **Add Webhook Integration** - Sync with code changes automatically
12. **Improve Error Handling** - Network failures and validation
13. **Add Board Visualization** - CLI view of Kanban structure

#### Category 3: Code-Documentation Sync (7 issues)
14. **Audit Historical Data Implementation** - Verify `occurrence_date` in all cost tables
15. **Implement Analytics Service** - Complete TCO/km calculations
16. **Wire Frontend to Real APIs** - Replace mocked data with live endpoints
17. **Implement Gamification Admin** - Control panel for game parameters
18. **Build Marketplace Booking Flow** - Service request and geofenced broadcast
19. **Develop Epic 11 Public Frontend** - Smart Garage with profile selector
20. **Create Advanced Search** - With filters and sorting

#### Category 4: Testing & Quality (7 issues)
21. **Write Integration Tests** - For critical user journeys
22. **Implement Performance Tests** - Validate <200ms API response time
23. **Create Security Test Suite** - Penetration testing and vulnerability scans
24. **Build Accessibility Tests** - WCAG 2.1 compliance
25. **Develop Load Testing** - 1000+ concurrent users simulation
26. **Create Monitoring Dashboard** - Real-time system health visualization
27. **Implement CI/CD Pipeline** - Automated testing and deployment

---

## 5. Manager Upgrade Requirements

### Python Functions to Add to `gitea_manager.py`

```python
# 1. Project Management
def list_projects():
    """List all projects in the repository"""
    pass

def get_project(project_id):
    """Get details of a specific project"""
    pass

def create_project(name, description, board_type="kanban"):
    """Create a new project board"""
    pass

# 2. Column/Card Operations
def list_columns(project_id):
    """List columns in a project board"""
    pass

def move_card(issue_id, column_id, position=None):
    """Move a card to a different column/position"""
    pass

def get_board_view(project_id):
    """Display Kanban board visualization"""
    pass

# 3. Batch Operations
def batch_move_issues(issue_ids, column_id):
    """Move multiple issues at once"""
    pass

def bulk_update_labels(issue_ids, labels_to_add, labels_to_remove):
    """Update labels for multiple issues"""
    pass

# 4. Webhook Integration
def create_webhook(events, url, secret=None):
    """Create webhook for automated sync"""
    pass

def list_webhooks():
    """List configured webhooks"""
    pass

# 5. Enhanced Visualization
def show_burndown_chart(milestone_id):
    """Display progress visualization"""
    pass

def show_velocity_report(days=30):
    """Calculate team velocity"""
    pass
```

### Required API Endpoints to Support
1. `/repos/{owner}/{repo}/projects` - Project management
2. `/repos/{owner}/{repo}/projects/{project_id}/columns` - Column operations
3. `/repos/{owner}/{repo}/projects/columns/{column_id}/cards` - Card movements
4. `/repos/{owner}/{repo}/hooks` - Webhook management
5. `/repos/{owner}/{repo}/issues/{index}/move` - Card moving endpoint

---

## 6. Implementation Roadmap

### Phase 1: Immediate Fixes (Week 1)
1. **Fix gitea_manager.py bugs** - Milestone listing error
2. **Create missing documentation issues** - 5 documentation tasks
3. **Audit historical data implementation** - Verify `occurrence_date` fields

### Phase 2: Manager Enhancement (Week 2)
1. **Add project board management** - Basic column/card operations
2. **Implement batch operations** - Efficiency improvements
3. **Add error handling and validation** - Robustness improvements

### Phase 3: Full Integration (Week 3-4)
1. **Implement webhook integration** - Automated code-issue sync
2. **Add visualization features** - Board views and reports
3. **Create CI/CD pipeline** - Automated testing and deployment

### Phase 4: Documentation Sync (Ongoing)
1. **Weekly documentation audits** - Ensure code-doc alignment
2. **Automated gap detection** - Script to identify discrepancies
3. **Monthly review cycles** - Stakeholder validation

---

## 7. Success Metrics

### Quantitative Metrics
1. **Issue completion rate** > 80% (currently unknown)
2. **Documentation coverage** > 90% (currently ~70%)
3. **API endpoint documentation** 100% (currently missing)
4. **Test coverage** > 75% (currently unknown)
5. **Issue definition quality** > 90% clear acceptance criteria

### Qualitative Metrics
1. **Reduced development friction** - Clear requirements
2. **Improved onboarding** - New developers can understand system
3. **Better stakeholder communication** - Clear progress visibility
4. **Reduced technical debt** - Documented vs. implemented alignment

---

## 8. Risk Assessment

### High Risk Areas
1. **API rate limiting** - Gitea API calls may hit limits with enhanced automation
2. **Network reliability** - Internal/external network switching may fail
3. **Data consistency** - Manual updates may create documentation-code drift
4. **Adoption resistance** - Team may not use enhanced features

### Mitigation Strategies
1. **Implement rate limit tracking** - Monitor and throttle API calls
2. **Add retry logic with exponential backoff** - Handle network failures
3. **Automated sync checks** - Weekly validation of code-doc alignment
4. **Training and documentation** - Clear benefits and usage guides

---

## 9. Next Steps

### Immediate Actions (Today)
1. **Create this blueprint file** ✅ **DONE**
2. **Fix `gitea_manager.py` milestone bug** - Assign to developer
3. **Create 5 documentation issues** - Add to Phase 1 milestone

### Short-term Actions (This Week)
1. **Review with project stakeholders** - Get buy-in on proposed structure
2. **Prioritize issue creation** - Based on development roadmap
3. **Assign initial implementation tasks** - Begin manager enhancements

### Long-term Vision (Quarter)
1. **Fully automated sync** - Code changes automatically update issues
2. **Comprehensive documentation** - 100% coverage of all features
3. **Predictive analytics** - Velocity-based sprint planning
4. **Integration with other tools** - CI/CD, monitoring, alerting

---

## Appendix A: Current Gitea State Snapshot

### Open Issues by Milestone
```
Phase 4: Testing & Deployment (6)
  #165 Production Deployment and Monitoring Setup
  #164 CI/CD Pipeline Setup (GitHub Actions)
  #163 Security Audit (Penetration Testing)
  #162 Accessibility Audit and Fixes
  #161 Performance Optimization
  #160 Implement Integration Tests

Phase 3: Advanced Features & Epic 11 (6)
  #158 Implement Advanced Search with Filters
  #157 Add Bulk Operations
  #156 Implement Webhook and Notification System
  #155 Add Admin Control Panels
  #154 Implement Service Booking Flow
  #153 Complete Profile Selector

Phase 2: Dashboard & Analytics Wiring (5)
  #152 Implement Historical Data (occurrence_date)
  #151 Connect User Management Table
  #150 Wire Service Map with Real Data
  #149 Implement Analytics Service (TCO/km)
  #148 Connect Gamification Components

Phase 1: Core Functionality Fixes (3)
  #146 Implement Basic Error Handling
  #145 Standardize API Base URL Usage
  #142 Implement Catalog API Endpoints

No Milestone (8)
  #140 Connect Service Moderation Map
  #139 Integrate Gamification Control Panel
  #138 Connect Financial Dashboard Tile
  #137 Implement Real-time System Health Monitor
  #136 Implement AI Researcher Logs
  #135 Connect User Management Table
```

### System Statistics
- **Total tables**: 127+ across 19 schemas
- **Active robots**: 10+ specialized workers
- **API endpoints**: ~80% implemented (estimate)
- **Frontend components**: ~60% built, ~40% wired
- **Test coverage**: Unknown (needs audit)
- **Documentation coverage**: ~70% (Masterbook 2.0.1)