264 lines
15 KiB
Markdown
264 lines
15 KiB
Markdown
# P0 Address Unification Plan
|
|
|
|
## Split-Brain Diagnosis: `marketplace.service_providers` vs. `system.addresses`
|
|
|
|
**Date:** 2026-07-07
|
|
**Author:** Rendszer-Architect
|
|
**Status:** Discovery Complete — Awaiting Approval for Implementation
|
|
|
|
---
|
|
|
|
## 1. Executive Summary
|
|
|
|
The system suffers from a **split-brain address architecture**. Two models use the centralized [`system.addresses`](backend/app/models/identity/address.py:39) table via `address_id` UUID foreign keys correctly (`fleet.organizations`, `identity.persons`), but the [`marketplace.service_providers`](backend/app/models/identity/social.py:23) table still relies entirely on **legacy flat columns** for address storage.
|
|
|
|
The [`ServiceStaging`](backend/app/models/marketplace/staged_data.py:24) model is in a **hybrid state** — it has both flat columns and an `address_id` FK — and the admin API ([`admin_providers.py`](backend/app/api/v1/endpoints/admin_providers.py:1157)) already has partial `AddressManager` integration. However, this creates a **dual-write pattern**: data flows to both the flat columns AND the normalized `system.addresses` table, with complex fallback logic.
|
|
|
|
The [`Organization`](backend/app/models/marketplace/organization.py:73) model was already fully refactored — the deprecated flat columns (`address_city`, `address_zip`, etc.) were **physically removed** from the database via [`cleanup_ghost_columns_2026_07.sql`](docs/sql/cleanup_ghost_columns_2026_07.sql). The [`ServiceProvider`](backend/app/models/identity/social.py:23) model needs the same treatment.
|
|
|
|
---
|
|
|
|
## 2. Full System Audit Results
|
|
|
|
### 2.1 Model Status Matrix
|
|
|
|
| Model | Table | Schema | Flat Columns | `address_id` FK | Status |
|
|
|---|---|---|---|---|---|
|
|
| [`Address`](backend/app/models/identity/address.py:39) | `system.addresses` | `system` | ❌ (central) | UUID PK (self) | ✅ **Source of Truth** |
|
|
| [`Organization`](backend/app/models/marketplace/organization.py:73) | `fleet.organizations` | `fleet` | ❌ (removed) | ✅ UUID FK (3x) | ✅ **Fully Refactored** |
|
|
| [`Person`](backend/app/models/identity/person.py) | `identity.persons` | `identity` | ❌ | ✅ UUID FK | ✅ **Already correct** |
|
|
| [`Branch`](backend/app/models/marketplace/organization.py) | `fleet.branches` | `fleet` | ❌ | ✅ UUID FK | ✅ **Already correct** |
|
|
| [`ServiceProfile`](backend/app/models/marketplace/service.py:21) | `marketplace.service_profiles` | `marketplace` | ❌ | ❌ (uses `location` Geometry) | ✅ **Clean design** |
|
|
| [`ServiceStaging`](backend/app/models/marketplace/staged_data.py:24) | `marketplace.service_staging` | `marketplace` | ✅ (`city`, `postal_code`, `full_address`) | ✅ UUID FK | ⚠️ **Hybrid — needs cleanup** |
|
|
| [`ServiceProvider`](backend/app/models/identity/social.py:23) | `marketplace.service_providers` | `marketplace` | ✅ (`city`, `address_zip`, `address_street_name`, `address_street_type`, `address_house_number`) | ❌ **NONE** | 🔴 **CORE PROBLEM** |
|
|
|
|
### 2.2 ServiceProvider Model — The Core Problem
|
|
|
|
File: [`backend/app/models/identity/social.py`](backend/app/models/identity/social.py:23)
|
|
|
|
The `ServiceProvider` model at lines 23-71 defines these flat address columns:
|
|
|
|
```python
|
|
city: Mapped[Optional[str]] = mapped_column(String(100), nullable=True) # line 38
|
|
address_zip: Mapped[Optional[str]] = mapped_column(String(20), nullable=True) # line 39
|
|
address_street_name: Mapped[Optional[str]] = mapped_column(String(150), nullable=True) # line 40
|
|
address_street_type: Mapped[Optional[str]] = mapped_column(String(50), nullable=True) # line 41
|
|
address_house_number: Mapped[Optional[str]] = mapped_column(String(20), nullable=True) # line 42
|
|
```
|
|
|
|
**There is NO `address_id` UUID foreign key to `system.addresses`.**
|
|
|
|
This means:
|
|
- Address data cannot be shared/deduplicated across entities
|
|
- No postal code normalization (no `GeoPostalCode` relationship)
|
|
- The `AddressManager` creates `system.addresses` records but the results are **mapped back to flat columns** instead of storing the FK
|
|
- The `_build_unified_providers_query()` in [`admin_providers.py`](backend/app/api/v1/endpoints/admin_providers.py:284) must use a complex UNION ALL with flat field extraction
|
|
|
|
### 2.3 ServiceStaging Model — Hybrid State
|
|
|
|
File: [`backend/app/models/marketplace/staged_data.py`](backend/app/models/marketplace/staged_data.py:24)
|
|
|
|
The `ServiceStaging` model at line 81 already has the FK:
|
|
|
|
```python
|
|
address_id: Mapped[Optional[uuid.UUID]] = mapped_column(
|
|
PG_UUID(as_uuid=True), ForeignKey("system.addresses.id"), nullable=True
|
|
) # line 81
|
|
```
|
|
|
|
But it ALSO retains flat columns at lines 25-27:
|
|
|
|
```python
|
|
city: Mapped[Optional[str]] = mapped_column(String(100), nullable=True)
|
|
postal_code: Mapped[Optional[str]] = mapped_column(String(20), nullable=True)
|
|
full_address: Mapped[Optional[str]] = mapped_column(String(500), nullable=True)
|
|
```
|
|
|
|
This hybrid state means the API writes to BOTH the `address_id` FK and the flat columns, creating data duplication.
|
|
|
|
### 2.4 Admin API — Partial Fix Complexity
|
|
|
|
File: [`backend/app/api/v1/endpoints/admin_providers.py`](backend/app/api/v1/endpoints/admin_providers.py:1157)
|
|
|
|
The `update_provider()` function at line 1157 has complex branching logic:
|
|
|
|
- **Lines 1242-1262**: For `ServiceStaging` (which has `address_id` FK) → uses `AddressManager.create_or_update()` then sets `provider_obj.address_id`
|
|
- **Lines 1262-1302**: For `ServiceProvider` (NO `address_id` FK) → uses `AddressManager.create_or_update()` then **maps the result back to 12+ flat columns** via `addr_mapping` (lines 1277-1302)
|
|
|
|
This dual-write pattern is fragile and error-prone. The `addr_mapping` dict at lines 1277-1302 maps each `AddressIn` field to its flat column counterpart.
|
|
|
|
Similarly, on response:
|
|
- **Lines 1507-1518**: For `ServiceProvider` → tries to resolve `address_detail` from `address_id` (which may be NULL for existing records)
|
|
- **Lines 1548-1560**: For `ServiceStaging` → resolves `address_detail` from `address_id`
|
|
|
|
The [`_build_unified_providers_query()`](backend/app/api/v1/endpoints/admin_providers.py:284) at line 284 uses UNION ALL across 3 tables with flat field extraction:
|
|
|
|
```sql
|
|
-- ServiceProvider branch (lines 306-320): selects flat columns
|
|
-- ServiceStaging branch (lines 335-352): selects flat columns
|
|
-- Organization branch (lines 369-390): JOINs with system.addresses
|
|
```
|
|
|
|
### 2.5 Frontend Status
|
|
|
|
| Page | Path | SharedAddressForm | address_detail | Flat Fallback |
|
|
|---|---|---|---|---|
|
|
| **Provider Edit** | [`providers/[id]/edit.vue`](frontend_admin/pages/providers/%5Bid%5D/edit.vue) | ✅ (line 343) | ✅ (lines 810-823) | ⚠️ (lines 1133-1134) |
|
|
| **Provider Detail** | [`providers/[id]/index.vue`](frontend_admin/pages/providers/%5Bid%5D/index.vue) | ❌ | ✅ (lines 144-166) | ✅ (lines 146, 153) |
|
|
| **Provider List** | [`providers/index.vue`](frontend_admin/pages/providers/index.vue) | ❌ | ❌ uses `provider.address` only | ✅ (line 137) |
|
|
| **Garage Edit** | [`garages/[id]/index.vue`](frontend_admin/pages/garages/%5Bid%5D/index.vue) | ✅ (line 534) | ✅ (lines 1014-1023) | ❌ |
|
|
| **Garage List** | [`garages/index.vue`](frontend_admin/pages/garages/index.vue) | ❌ | ❌ uses `garage.city` only | ✅ (line 157) |
|
|
|
|
### 2.6 Already-Refactored Endpoints (Reference)
|
|
|
|
These endpoints already use `AddressManager` correctly and serve as reference implementations:
|
|
|
|
| Endpoint | File | How |
|
|
|---|---|---|
|
|
| **Admin Organizations** | [`admin_organizations.py`](backend/app/api/v1/endpoints/admin_organizations.py) | Lines 1097-1145: `AddressOut.model_validate(org.address)` for 3 address types |
|
|
| **User Organizations** | [`organizations.py`](backend/app/api/v1/endpoints/organizations.py) | Lines 71-82: `AddressManager.create_or_update()` on org creation |
|
|
| **Admin Persons** | [`admin_persons.py`](backend/app/api/v1/endpoints/admin_persons.py) | Lines 654-661: `AddressManager.create_or_update()` for person.address |
|
|
| **User Profile** | [`users.py`](backend/app/api/v1/endpoints/users.py) | Lines 355-374: `GeoService.get_or_create_full_address()` |
|
|
|
|
---
|
|
|
|
## 3. Migration Strategy: 3-Phase Plan
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
subgraph Phase1["Phase 1: Database (Alembic)"]
|
|
A1[Add address_id UUID FK to marketplace.service_providers]
|
|
A2[Create data migration: backfill system.addresses from flat columns]
|
|
A3[Set FK constraint + add relationship to ServiceProvider model]
|
|
A4[Run sync_engine to verify schema]
|
|
end
|
|
|
|
subgraph Phase2["Phase 2: Backend API"]
|
|
B1[Simplify update_provider: remove addr_mapping logic]
|
|
B2[Refactor _build_unified_providers_query: JOIN with system.addresses]
|
|
B3[Update ProviderListItem/ProviderDetail schemas]
|
|
B4[Remove flat address field fallback in get_provider_detail]
|
|
end
|
|
|
|
subgraph Phase3["Phase 3: Frontend"]
|
|
C1[providers/index.vue: show address_detail instead of flat address]
|
|
C2[providers/[id]/index.vue: remove flat fallback]
|
|
C3[Clean up flat address fields from edit.vue]
|
|
C4[garages/index.vue: show address_detail instead of flat city]
|
|
end
|
|
|
|
Phase1 --> Phase2 --> Phase3
|
|
```
|
|
|
|
### Phase 1 — Database Migration (Model + sync_engine)
|
|
|
|
**Goal:** Add `address_id` FK to `ServiceProvider` and migrate existing data.
|
|
|
|
**Steps:**
|
|
|
|
1. **Add `address_id` column to `ServiceProvider` model** in [`social.py`](backend/app/models/identity/social.py:23):
|
|
```python
|
|
address_id: Mapped[Optional[uuid.UUID]] = mapped_column(
|
|
PG_UUID(as_uuid=True), ForeignKey("system.addresses.id"), nullable=True
|
|
)
|
|
address: Mapped[Optional["Address"]] = relationship("Address", lazy="selectin")
|
|
```
|
|
|
|
2. **Create data migration script** that:
|
|
- Iterates all `ServiceProvider` records with flat address data and NULL `address_id`
|
|
- For each record, creates a `system.addresses` record via `AddressManager.create_or_update()`
|
|
- Sets `service_providers.address_id` to the new UUID
|
|
- Logs the migration results
|
|
|
|
3. **Run sync_engine**: `docker exec sf_api python /app/backend/app/scripts/sync_engine.py`
|
|
|
|
4. **Verify**: Query `service_providers` to confirm `address_id` is populated.
|
|
|
|
5. **Deferred**: Drop flat columns in a future migration after Phase 3 is complete and verified.
|
|
|
|
### Phase 2 — Backend API Refactor
|
|
|
|
**Goal:** Simplify `admin_providers.py` to use `AddressManager` directly without flat column mapping.
|
|
|
|
**Key changes in [`admin_providers.py`](backend/app/api/v1/endpoints/admin_providers.py):**
|
|
|
|
1. **Simplify `update_provider()`** (lines 1242-1302):
|
|
- Remove the `addr_mapping` dict (lines 1277-1302) for `ServiceProvider`
|
|
- Both branches now just call `AddressManager.create_or_update()` and set `provider_obj.address_id`
|
|
- Remove flat column updates for `address_zip`, `address_street_name`, etc.
|
|
|
|
2. **Refactor `_build_unified_providers_query()`** (lines 284-407):
|
|
- For `ServiceProvider`: Add LEFT JOIN to `system.addresses` and extract fields via relationship
|
|
- Remove flat column SELECT for ServiceProvider
|
|
- Keep flat fallback temporarily for backward compatibility
|
|
|
|
3. **Update response schemas** (lines 59-105, 134-151):
|
|
- `ProviderListItem`: Add `address_detail: Optional[AddressOut]`
|
|
- `ProviderDetail`: Mark flat `address` and `city` as DEPRECATED
|
|
- `ProviderUpdateInput`: Keep `address_detail` as primary, mark flat fields deprecated
|
|
|
|
4. **Simplify `get_provider_detail()`** (lines 534-659):
|
|
- Remove flat field fallback (lines 586-587: `sp.address or ""`)
|
|
- Always resolve from `address_id` via `AddressManager.get_normalized()`
|
|
|
|
### Phase 3 — Frontend Cleanup
|
|
|
|
**Goal:** All provider/garage pages display and edit addresses via `address_detail` only.
|
|
|
|
**Detailed changes:**
|
|
|
|
1. **`providers/index.vue`** (line 137): Replace `provider.address` with `provider.address_detail?.full_address_text || [provider.address_detail?.zip, ...].filter(Boolean).join(' ')`
|
|
|
|
2. **`providers/[id]/index.vue`** (lines 144-166): Remove `|| provider.address` and `|| provider.city` fallbacks (lines 146, 153)
|
|
|
|
3. **`providers/[id]/edit.vue`** (lines 1133-1134): Remove `data.address` fallback from `data.address_detail || data.address || {}`
|
|
|
|
4. **`garages/index.vue`** (line 157): Replace `garage.city || '—'` with `garage.address_detail?.city || '—'` or similar
|
|
|
|
---
|
|
|
|
## 4. Risk Assessment
|
|
|
|
| Risk | Impact | Mitigation |
|
|
|---|---|---|
|
|
| **Data loss** during migration (flat → normalized) | High | Migration script must be transactional with rollback. Test on staging first. |
|
|
| **Existing API consumers** send flat address fields | Medium | Keep deprecated fields in schemas with `[DEPRECATED]` tags for one release cycle |
|
|
| **Frontend list view** breaks if address_detail is missing | Medium | Add fallback to flat fields in list view during transition period |
|
|
| **Service robots** (OSM Scout, OCR) write flat columns directly | Medium | Audit `service_robot_1_scout_osm.py` — it's in the modified files list |
|
|
| **Unified query** (3-way UNION ALL) breaks | High | Refactor query carefully to maintain backward compatibility |
|
|
|
|
---
|
|
|
|
## 5. Acceptance Criteria
|
|
|
|
### Phase 1 — Database Migration
|
|
- [ ] `marketplace.service_providers` has `address_id` UUID FK → `system.addresses.id`
|
|
- [ ] All existing ServiceProvider records with flat address data have corresponding `system.addresses` records linked via `address_id`
|
|
- [ ] `ServiceProvider` model has `address` relationship with `lazy="selectin"`
|
|
- [ ] `sync_engine` reports no schema drift
|
|
|
|
### Phase 2 — Backend API
|
|
- [ ] `update_provider()` no longer maps `AddressIn` fields to flat columns for `ServiceProvider`
|
|
- [ ] `get_provider_detail()` returns `address_detail` from `address_id` relationship for all providers
|
|
- [ ] `list_providers()` includes `address_detail` in each item
|
|
- [ ] `_build_unified_providers_query()` uses LEFT JOIN for ServiceProvider instead of flat column extraction
|
|
- [ ] Flat address fields (`address`, `city`) still work in API requests but are marked `[DEPRECATED]` in schemas
|
|
|
|
### Phase 3 — Frontend
|
|
- [ ] Provider list page shows addresses from `address_detail` (not flat `provider.address`)
|
|
- [ ] Provider detail page has no flat fallback for address fields
|
|
- [ ] Provider edit page sends `address_detail` only (no flat fields)
|
|
- [ ] Garage list page shows city from `address_detail`
|
|
|
|
---
|
|
|
|
## 6. References
|
|
|
|
- [AddressManager service](backend/app/services/address_manager.py:41) — 4 static methods: `resolve()`, `create_or_update()`, `get_normalized()`, `_resolve_postal_code()`
|
|
- [AddressIn/AddressOut schemas](backend/app/schemas/address.py:18) — Pydantic models for address API contracts
|
|
- [SharedAddressForm component](frontend_admin/components/SharedAddressForm.vue) — Vue form component (search for usage via `SharedAddressForm`)
|
|
- [Address model](backend/app/models/identity/address.py:39) — UUID PK, `postal_code_id` FK → `system.geo_postal_codes`
|
|
- [Organization model (REFACTORED)](backend/app/models/marketplace/organization.py:105) — Reference implementation with 3 `address_id` FKs
|
|
- [Admin Persons (REFACTORED)](backend/app/api/v1/endpoints/admin_persons.py:654) — Reference API implementation
|
|
- [Ghost columns cleanup SQL](docs/sql/cleanup_ghost_columns_2026_07.sql) — Reference for how Organization's flat columns were removed
|