Added the crontab to the repo so we can track the settings for the automation tasks

CRONTAB.md

@@ -0,0 +1,6 @@
+# The contents of the local user agent crontab
+# Used to maintain and document the automated tasks for this system
+
+PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+# m h  dom mon dow   command
+4 4-23 * * * . /home/npsagent/.config/servicem8-webhooks.env; cd /opt/webhooks && flock -n /tmp/servicem8-quote-template.lock ./poll_and_apply_quote_templates.sh --hours 6 >> /opt/webhooks/logs/cron-driver.log 2>&1

MIGRATION-NOTES.md

@@ -0,0 +1,295 @@
+Yep — I inspected the current setup. The “current bash wrapper” is the ServiceM8 plumbing quote-template pipeline under:
+
+`/opt/webhooks`
+
+In the workspace it appears as a symlink:
+
+`/home/openclaw/.openclaw/workspace/projects/plumbing -> /opt/webhooks`
+
+## Definitive migration set
+
+### Required operational scripts
+
+Copy these from `/opt/webhooks/`:
+
+```text
+poll_and_apply_quote_templates.sh
+poll_form_responses_since.py
+apply_polled_quote_template_jobmaterials.py
+servicem8_quote_template_parser.py
+servicem8_webhook_receiver.py
+servicem8_inspector.py
+```
+
+These are the active pieces:
+
+- `poll_and_apply_quote_templates.sh` — main wrapper / scheduled entry point
+- `poll_form_responses_since.py` — polls ServiceM8 form responses
+- `apply_polled_quote_template_jobmaterials.py` — applies parsed quote rows to ServiceM8 jobMaterials
+- `servicem8_quote_template_parser.py` — parsing/state logic
+- `servicem8_webhook_receiver.py` — webhook receiver/diagnostics
+- `servicem8_inspector.py` — web UI inspector
+
+### Required DB/state files
+
+Copy these, ideally while services are stopped:
+
+```text
+servicem8_formresponse_poll.db
+servicem8_quote_materials_state.db
+servicem8_webhooks.db
+```
+
+Also copy any SQLite sidecar files if present at cutover:
+
+```text
+*.db-wal
+*.db-shm
+```
+
+### Required JSONL queue/history files
+
+```text
+quote-template-jobmaterials-poll-queue.jsonl
+quote-template-jobmaterials-queue.jsonl
+```
+
+First one is the current poll-derived queue. Second is older webhook-derived queue but worth keeping for audit/history.
+
+### Useful but not strictly required
+
+```text
+PROJECT-PROGRESS.md
+docs/
+logs/
+.git/
+```
+
+I’d migrate these too unless you want a clean production-only deploy. The logs and docs are useful for future archaeology when something inevitably gets weird.
+
+## Do not blindly migrate
+
+I would **not** migrate the existing virtualenv folders directly:
+
+```text
+bin/
+lib/
+lib64/
+include/
+pyvenv.cfg
+__pycache__/
+```
+
+They are host/Python-version sensitive. Recreate the venv on the new server.
+
+## Python / pip packages
+
+Current venv freeze shows:
+
+```text
+fastapi==0.136.0
+uvicorn==0.45.0
+requests==2.33.1
+python-dotenv==1.2.2
+pydantic==2.13.3
+starlette==1.0.0
+anyio==4.13.0
+click==8.3.3
+h11==0.16.0
+httptools==0.7.1
+uvloop==0.22.1
+watchfiles==1.1.1
+websockets==16.0
+certifi==2026.4.22
+charset-normalizer==3.4.7
+idna==3.13
+urllib3==2.6.3
+PyYAML==6.0.3
+typing_extensions==4.15.0
+typing-inspection==0.4.2
+annotated-types==0.7.0
+annotated-doc==0.0.4
+pydantic_core==2.46.3
+```
+
+Practical install line:
+
+```bash
+python3.12 -m venv /opt/webhooks
+/opt/webhooks/bin/pip install --upgrade pip
+/opt/webhooks/bin/pip install fastapi 'uvicorn[standard]' requests python-dotenv
+```
+
+Or create a `requirements.txt` from the freeze if you want exact locking.
+
+System packages likely needed:
+
+```bash
+python3.12
+python3.12-venv
+sqlite3
+bash
+curl
+rsync
+systemd
+```
+
+Important: the wrapper uses GNU `date -d`, so Linux/GNU coreutils is assumed.
+
+## Environment variables required
+
+For polling/applying:
+
+```text
+SERVICEM8_ACCESS_TOKEN
+# or
+SERVICEM8_API_KEY
+```
+
+Optional/current defaults:
+
+```text
+SERVICEM8_BASE_URL=https://api.servicem8.com/api_1.0
+SERVICEM8_TIMEOUT=30
+SERVICEM8_QUOTE_TEMPLATE_FORM_UUID=3621b6be-1d19-4756-9ab4-9d5e4120f6d9
+WEBHOOK_POLL_DB_PATH=/opt/webhooks/servicem8_formresponse_poll.db
+WEBHOOK_RUN_LOG_DIR=/opt/webhooks/logs
+```
+
+For webhook receiver:
+
+```text
+WEBHOOK_HOST=0.0.0.0
+WEBHOOK_PORT=18354
+WEBHOOK_DB_PATH=/opt/webhooks/servicem8_webhooks.db
+```
+
+For inspector:
+
+```text
+WEBHOOK_DB_PATH=/opt/webhooks/servicem8_webhooks.db
+WEBHOOK_STATE_DB_PATH=/opt/webhooks/servicem8_quote_materials_state.db
+WEBHOOK_POLL_DB_PATH=/opt/webhooks/servicem8_formresponse_poll.db
+WEBHOOK_INSPECTOR_HOST=0.0.0.0
+WEBHOOK_INSPECTOR_PORT=18355
+```
+
+## Existing systemd services to recreate
+
+Current services are:
+
+```text
+servicem8-dev-webhook.service
+servicem8-dev-inspector.service
+```
+
+Current paths assume `/opt/webhooks`.
+
+Webhook:
+
+```ini
+WorkingDirectory=/opt/webhooks
+Environment="WEBHOOK_HOST=0.0.0.0"
+Environment="WEBHOOK_PORT=18354"
+Environment="WEBHOOK_DB_PATH=/opt/webhooks/servicem8_webhooks.db"
+ExecStart=/opt/webhooks/bin/uvicorn servicem8_webhook_receiver:app --host 0.0.0.0 --port 18354
+```
+
+Inspector:
+
+```ini
+WorkingDirectory=/opt/webhooks
+Environment="WEBHOOK_DB_PATH=/opt/webhooks/servicem8_webhooks.db"
+Environment="WEBHOOK_INSPECTOR_HOST=0.0.0.0"
+Environment="WEBHOOK_INSPECTOR_PORT=18355"
+ExecStart=/opt/webhooks/bin/uvicorn servicem8_inspector:app --host 0.0.0.0 --port 18355
+```
+
+On the new host, probably change:
+
+```ini
+User=michael
+Group=michael
+```
+
+to whatever the production service user is, e.g.:
+
+```ini
+User=openclaw
+Group=openclaw
+```
+
+or create a dedicated `servicem8` user.
+
+## Path/code changes for seamless migration
+
+Best option: keep the same production path:
+
+```text
+/opt/webhooks
+```
+
+If you do that, almost no code changes are needed.
+
+If you move it elsewhere, update:
+
+1. systemd `WorkingDirectory`
+2. systemd `ExecStart`
+3. env vars:
+   - `WEBHOOK_DB_PATH`
+   - `WEBHOOK_POLL_DB_PATH`
+   - `WEBHOOK_STATE_DB_PATH`
+   - `WEBHOOK_RUN_LOG_DIR`
+4. any OpenClaw cron or host cron entry that calls:
+   - `/opt/webhooks/poll_and_apply_quote_templates.sh`
+
+One gotcha: `servicem8_quote_template_parser.py` currently hardcodes the state DB beside the script:
+
+```python
+STATE_DB_PATH = Path(__file__).with_name("servicem8_quote_materials_state.db")
+```
+
+So keeping DBs beside the scripts is the safest path. If you want DBs somewhere else, I’d adjust that code to respect `WEBHOOK_STATE_DB_PATH`.
+
+## External callback / production URL
+
+Current webhook helper scripts still reference dev callback URLs like:
+
+```text
+https://nps-dev.coast2cloud.net/...
+```
+
+For the new production host, ServiceM8 webhook subscriptions need to point at the new public URL if you want inbound webhooks working.
+
+Polling does **not** depend on inbound webhooks, so the wrapper can operate without this, but the receiver/inspector diagnostics won’t capture new webhook events unless ServiceM8 is updated.
+
+## Security note
+
+There are old helper scripts with hardcoded ServiceM8 token fallbacks:
+
+```text
+servicem8-create-webhook-form-response.py
+servicem8-create-webhook-job-object.py
+servicem8-list-webhook-subscriptions-table.py
+```
+
+I would not treat those as production-safe as-is. Before migration to prod, remove hardcoded fallback tokens and require env vars only.
+
+## Recommended cutover shape
+
+1. Stop current webhook/inspector services.
+2. Copy `/opt/webhooks` excluding venv/cache, including DBs/jsonl/logs.
+3. Recreate venv on new host.
+4. Install pip packages.
+5. Add env/secrets via systemd `EnvironmentFile`, not hardcoded scripts.
+6. Recreate systemd services.
+7. Run:
+
+```bash
+/opt/webhooks/poll_and_apply_quote_templates.sh --dry-run --hours 48
+```
+
+8. If clean, run controlled live apply.
+9. Only then schedule the wrapper.
+
+At present I found no OpenClaw cron job already running this plumbing wrapper, so scheduling still appears to be a pending production decision rather than something to migrate from OpenClaw cron.

PROJECT-PROGRESS.md

@@ -1,73 +1,188 @@
 # ServiceM8 Project Progress
 
-## Current Quote Template → JobMaterials Pipeline
+## Current Direction — Soft Release Poller + Apply Pipeline
 
-### Live receiver
+We changed tune from relying mainly on ServiceM8 `form.response_created` webhooks to using a scheduled API poller as the operational safety-net/primary soft-release path.
-- `servicem8_webhook_receiver.py`
-  - receives `form.response_created`
-  - stores the raw webhook payload in `servicem8_webhooks.db`
-  - checks incoming form responses for the Quote Template `form_uuid`
-  - if matched, parses and queues the derived jobMaterials payload to:
-    - `quote-template-jobmaterials-queue.jsonl`
-  - does **not** perform live ServiceM8 writes at this stage
 
-### Parser
+Reason: ServiceM8 form webhooks are useful, but during testing some expected form completions did not reliably appear at the FastAPI listener. The ServiceM8 API endpoint `/api_1.0/formresponse.json` can be queried with a timestamp filter, so the safer operational pattern is now:
-- `servicem8-quote-template-parser.py`
+
-  - parses Quote Template `field_data`
+1. poll recent form responses with `timestamp gt 'YYYY-MM-DD HH:MM:SS'`
-  - extracts:
+2. store every returned API row locally
+3. detect Quote Template form responses
+4. parse them into desired jobMaterial rows
+5. apply only previously unapplied Quote Template responses to ServiceM8
+6. track created jobMaterial UUIDs locally to avoid duplicate applies
+
+This keeps the live webhook receiver useful as capture/diagnostics, but no longer depends on webhook delivery for completeness.
+
+## Active Soft-Release Flow
+
+### 1. Poll ServiceM8 form responses
+- Script: `poll_form_responses_since.py`
+- API endpoint: `/api_1.0/formresponse.json`
+- Confirmed filter syntax:
+  - `?$filter=timestamp gt '2026-05-04 10:00:00'`
+- Default operational DB:
+  - `servicem8_formresponse_poll.db`
+- Poll queue/output:
+  - `quote-template-jobmaterials-poll-queue.jsonl`
+- Quote Template form UUID:
+  - `3621b6be-1d19-4756-9ab4-9d5e4120f6d9`
+
+The poller stores all fetched form responses in `form_responses_raw`, then parses Quote Template matches into `quote_template_form_responses`.
+
+### 2. Parse Quote Template responses
+- Parser: `servicem8_quote_template_parser.py`
+- Extracts:
   - description of works
   - `Item 1..12` include lines
   - `Works excluded 1..4` exclude lines
-    - extra descriptive include rows such as labour/scaffolding/equipment fields
+  - extra descriptive include rows such as labour/materials/scaffolding/equipment fields
-  - builds normalized `desired_job_materials`
+- Builds normalized `desired_job_materials` rows.
 
-### Create/apply script
+### 3. Select/apply parsed Quote Template responses
-- `servicem8-create-jobmaterials-from-form-response.py`
+- Script: `apply_polled_quote_template_jobmaterials.py`
-  - standalone script
+- Safe behaviour built in:
-  - consumes a Quote Template form response JSON payload
+  - processes one `form_response_uuid` at a time when called directly
-  - builds ServiceM8 Job Material API payloads
+  - dry-run by default
-  - runs in **dry-run by default**
+  - `--apply` performs live ServiceM8 `jobmaterial.json` creates
-  - supports live create later with `--apply`
+  - refuses duplicate apply when generated material rows already exist for that form response unless `--force` is used
-  - records created/generated mappings into local state DB
+  - before any live create, checks remote ServiceM8 `/jobmaterial.json` for existing rows on the target `job_uuid`
-
+  - if remote rows already exist, records a remote-existing incident and creates nothing unless `--force-remote-existing` is explicitly used
-### Local state tracking
+- Apply/incident tracking tables in `servicem8_formresponse_poll.db`:
+  - `quote_template_apply_runs`
+  - `quote_template_apply_run_rows`
+  - `quote_template_remote_existing_incidents`
+- Created ServiceM8 job material mappings are recorded in:
   - `servicem8_quote_materials_state.db`
-  - local SQLite DB for tracking generated jobMaterials
-  - intended fields include:
-    - job UUID
-    - form response UUID
-    - created job material UUID
-    - kind/source metadata
 
-### Queue/prepared output
+Current apply payload rules:
+- All rows get:
+  - `tax_rate_uuid = 84e4dd28-06b3-452b-a796-1f58a20ac49b`
+  - `quantity = "0"`
+  - `price = ""`
+  - `displayed_amount = ""`
+  - `displayed_amount_is_tax_inclusive = ""`
+- Header material UUIDs:
+  - `include_header` → `1924893b-917f-474a-adaa-2093bd622d4b`
+  - `exclude_header` → `4947bfd7-4875-48f7-9caf-2093b9751b9b`
+- Non-header quote rows currently use:
+  - `f78b1d23-b9fa-40fe-a806-2425fe09cc0b`
+
+### 4. Wrapper for scheduled/operational use
+- Script: `poll_and_apply_quote_templates.sh`
+- Default behaviour:
+  - polls the last 24 hours from script start
+  - stores/parses results
+  - applies any parsed Quote Template responses that are not already marked/applied
+  - logs each run under `logs/poll-and-apply-YYYYMMDD-HHMMSS.log`
+- Safety options now available:
+  - `--dry-run` runs the same poll/selection flow, but previews the ServiceM8 `jobmaterial` payloads only
+  - dry-run does **not** write to ServiceM8
+  - dry-run does **not** mark quote responses as applied
+  - live apply blocks if remote ServiceM8 already has jobMaterial rows for the job, logging the captured rows/counts to `quote_template_remote_existing_incidents`
+  - `--recheck-remote-existing` revisits previously blocked rows without overriding the safety gate
+  - `--force-remote-existing` explicitly overrides the remote-existing safety gate and still records a forced incident before creating
+- Examples:
+  - `./poll_and_apply_quote_templates.sh`
+  - `./poll_and_apply_quote_templates.sh --hours 48`
+  - `./poll_and_apply_quote_templates.sh --since '2026-05-04 08:00:00'`
+  - `./poll_and_apply_quote_templates.sh --dry-run --hours 48`
+  - `./poll_and_apply_quote_templates.sh --recheck-remote-existing --hours 48`
+  - `./poll_and_apply_quote_templates.sh --force-remote-existing --hours 48`
+
+This is the proposed scheduled entry point for soft release, e.g. every 10–30 minutes. For manual confidence checks, run it with `--dry-run` first, inspect the generated payloads/log, then rerun without `--dry-run` only when ready to apply.
+
+## Live Webhook Receiver Status
+
+### Receiver
+- Script: `servicem8_webhook_receiver.py`
+- Still receives/stores:
+  - event webhooks
+  - object webhooks
+  - form-response webhooks
+- DB:
+  - `servicem8_webhooks.db`
+- Quote Template webhook queue:
   - `quote-template-jobmaterials-queue.jsonl`
-  - lightweight queue/output file written by webhook stage
-  - contains parsed/prepared `desired_job_materials` objects
-  - no live update performed yet
 
-### Inspector
+Important: webhook handling still does **not** perform live ServiceM8 writes. Heavy work stays outside the FastAPI webhook request path.
-- `servicem8_inspector.py`
+
-  - read-only browser for webhook DB
+## Inspector / Web Viewer
-  - now also includes visibility of generated-materials state DB
+
+- Script: `servicem8_inspector.py`
+- Current intended views include:
+  - webhook events
+  - webhook objects
+  - webhook form responses
+  - polled form responses
+  - parsed polled Quote Template responses
+  - poll runs
+  - dry-run/apply runs
+  - generated material state
+
+Note: the code was updated and tested on a temporary localhost port, but the existing live inspector process may need a manual/service restart before all new pages appear in the running viewer.
+
+## Required Files for Current Soft Release
+
+### Scripts
+- `poll_and_apply_quote_templates.sh` — scheduled wrapper / main operational entry point
+- `poll_form_responses_since.py` — polls ServiceM8 and populates poll DB
+- `apply_polled_quote_template_jobmaterials.py` — applies parsed responses to ServiceM8
+- `servicem8_quote_template_parser.py` — parsing logic
+- `servicem8_inspector.py` — web inspection/progress viewer
+- `servicem8_webhook_receiver.py` — still useful for webhook capture/diagnostics
+
+### Databases / state
+- `servicem8_formresponse_poll.db` — poll results, parsed quote responses, apply run status
+- `servicem8_quote_materials_state.db` — created jobMaterial mapping/state to avoid duplicates
+- `servicem8_webhooks.db` — webhook capture/archive/diagnostics
+
+### Queue/log files
+- `quote-template-jobmaterials-poll-queue.jsonl` — poll-derived parsed queue/output
+- `logs/poll-and-apply-*.log` — wrapper run logs
+- `quote-template-jobmaterials-queue.jsonl` — older webhook-derived queue; still useful but not the primary soft-release path
 
 ## Current Status
-Everything is staged and connected up to the point of:
-- webhook receive
-- UUID trigger check
-- form parsing
-- local queueing
-- dry-run jobMaterial payload generation
-- local state DB support
-- inspector visibility
 
-## Not Yet Enabled
+Operational soft-release pieces are now in place:
-- actual live POST creation of Job Materials into ServiceM8 during webhook processing
+- API polling confirmed
-- any automatic update/delete reconciliation against live ServiceM8 records
+- Quote Template detection confirmed
+- parsing confirmed
+- dry-run/apply command path tested
+- payload adjusted for current ServiceM8 requirements
+- duplicate-apply guard in place
+- wrapper script created for scheduled operation
+- wrapper now has a first-class `--dry-run` mode
+- inspector updated for progress visibility
+
+### Checkpoint — 2026-05-05
+
+Latest verified state:
+- `poll_and_apply_quote_templates.sh --dry-run` added and documented.
+- `bash -n /opt/webhooks/poll_and_apply_quote_templates.sh` passes.
+- `--help` output includes `--dry-run` usage/examples.
+- A future-since dry-run (`--dry-run --since '2099-01-01 00:00:00'`) confirmed:
+  - wrapper reports `Mode: dry-run`
+  - poll step performs no ServiceM8 writes
+  - apply step calls the Python apply tool without `--apply`
+  - payload rows are emitted as `would_create`
+  - responses are not marked applied by dry-run
+- At checkpoint time, there were still several unapplied parsed Quote Template responses available for preview/apply; this is expected while the soft release remains manual.
+
+## Not Yet Done / Next Steps
+
+- Restart/reload the live inspector process so the new poll/apply pages are available in the active web viewer.
+- Decide schedule interval for `poll_and_apply_quote_templates.sh` — likely every 10 or 30 minutes.
+- Run the wrapper manually in `--dry-run` mode against a controlled recent form response and inspect the payload/log.
+- If payload is correct, rerun the wrapper without `--dry-run` for a controlled live apply smoke test.
+- After confidence builds, wire the wrapper into cron/system scheduling.
+- Future hardening: add reconciliation/update/delete behaviour if ServiceM8 quote form responses are edited after initial apply.
 
 ## Design Notes
-- Heavy lifting is intentionally kept **out of the live webhook handler**.
+
-- The webhook handler is used only for:
+- Webhooks remain lightweight and non-mutating.
-  - capture
+- Polling is now the reliable source of completeness.
-  - UUID gate
+- Applying to ServiceM8 is tracked locally and guarded against duplicates.
-  - parse/prepare/queue
+- The wrapper defaults to live apply for scheduled soft release, but now supports `--dry-run` for manual preview/safety checks.
-- Live ServiceM8 mutation remains a separate step/script for safety.
+- The underlying apply script remains dry-run-by-default and provides the duplicate protection used by the wrapper.

apply_polled_quote_template_jobmaterials.py

@@ -0,0 +1,728 @@
+#!/usr/bin/env python3
+"""Create ServiceM8 jobMaterials for one selected polled Quote Template response.
+
+Safe-by-default:
+- Requires an explicit form response UUID.
+- Dry-run unless --apply is provided.
+- Refuses to apply twice for the same form response unless --force is used.
+- Records every dry-run/apply attempt in the poll DB for inspector visibility.
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import os
+import sqlite3
+import sys
+from contextlib import closing
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+import requests
+
+from servicem8_quote_template_parser import STATE_DB_PATH, init_state_db, record_generated_job_material
+
+SCRIPT_DIR = Path(__file__).resolve().parent
+POLL_DB_PATH = Path(os.getenv("WEBHOOK_POLL_DB_PATH", SCRIPT_DIR / "servicem8_formresponse_poll.db"))
+BASE_URL = os.getenv("SERVICEM8_BASE_URL", "https://api.servicem8.com/api_1.0")
+REQUEST_TIMEOUT = int(os.getenv("SERVICEM8_TIMEOUT", "30"))
+DEV_QUOTE_MATERIAL_UUID = "f78b1d23-b9fa-40fe-a806-2425fe09cc0b"
+QUOTE_INCLUDE_HEADER_MATERIAL_UUID = "1924893b-917f-474a-adaa-2093bd622d4b"
+QUOTE_EXCLUDE_HEADER_MATERIAL_UUID = "4947bfd7-4875-48f7-9caf-2093b9751b9b"
+DEV_QUOTE_TAX_RATE_UUID = "84e4dd28-06b3-452b-a796-1f58a20ac49b"
+QUOTE_DESCRIPTION_PREFIX = "Thank you for the opportunity to quote to "
+QUOTE_DESCRIPTION_SUFFIX = (
+    "Please find below a detailed breakdown of the proposed costs included in the quotation. "
+    "If you have any questions or concerns, please do not hesitate to contact us."
+)
+
+
+def utc_now() -> str:
+    return datetime.now(timezone.utc).isoformat()
+
+
+def escape_filter_value(value: str) -> str:
+    return value.replace("'", "''")
+
+
+def material_uuid_for_row(row: dict) -> str:
+    kind = row.get("kind", "")
+    if kind == "include_header":
+        return QUOTE_INCLUDE_HEADER_MATERIAL_UUID
+    if kind == "exclude_header":
+        return QUOTE_EXCLUDE_HEADER_MATERIAL_UUID
+    return DEV_QUOTE_MATERIAL_UUID
+
+
+def build_payload(job_uuid: str, row: dict) -> dict:
+    return {
+        "job_uuid": job_uuid,
+        "material_uuid": material_uuid_for_row(row),
+        "tax_rate_uuid": DEV_QUOTE_TAX_RATE_UUID,
+        "name": row["name"],
+        "quantity": "0",
+        "price": "",
+        "displayed_amount": "",
+        "displayed_amount_is_tax_inclusive": "",
+        "sort_order": row["sort_order"],
+    }
+
+
+def clean_text(value: Any) -> str:
+    if value is None:
+        return ""
+    return str(value).replace("\r\n", "\n").replace("\r", "\n").strip()
+
+
+def first_text(*values: Any) -> str:
+    for value in values:
+        text = clean_text(value)
+        if text:
+            return text
+    return ""
+
+
+def format_job_address(job: Dict[str, Any]) -> str:
+    direct = first_text(
+        job.get("job_address"),
+        job.get("site_address"),
+        job.get("address"),
+        job.get("location_address"),
+        job.get("billing_address"),
+    )
+    if direct:
+        return direct
+
+    parts = [
+        first_text(job.get("street"), job.get("street_address"), job.get("address_1"), job.get("address1")),
+        first_text(job.get("suburb"), job.get("city")),
+        first_text(job.get("state")),
+        first_text(job.get("postcode"), job.get("postal_code"), job.get("zip")),
+    ]
+    return " ".join(part for part in parts if part)
+
+
+def build_quote_description_text(description: str, job: Dict[str, Any]) -> str:
+    description = clean_text(description)
+    if not description:
+        return ""
+    address = format_job_address(job) or "the job address"
+    return f"{QUOTE_DESCRIPTION_PREFIX} {description} at {address}. {QUOTE_DESCRIPTION_SUFFIX}"
+
+
+def build_job_update_payload(description: str, job: Dict[str, Any]) -> dict:
+    quote_description = build_quote_description_text(description, job)
+    return {"work_done_description": quote_description} if quote_description else {}
+
+
+def retrieve_job(session: requests.Session, job_uuid: str) -> Dict[str, Any]:
+    response = session.get(f"{BASE_URL}/job/{job_uuid}.json", timeout=REQUEST_TIMEOUT)
+    if not response.ok:
+        raise RuntimeError(f"Job retrieve failed: HTTP {response.status_code} :: {response.text[:1000]}")
+    data = response.json()
+    if not isinstance(data, dict):
+        raise RuntimeError(f"Job retrieve expected object response, got {type(data).__name__}")
+    return data
+
+
+def update_job_description(session: requests.Session, job_uuid: str, payload: dict) -> None:
+    response = session.post(f"{BASE_URL}/job/{job_uuid}.json", json=payload, timeout=REQUEST_TIMEOUT)
+    if not response.ok:
+        raise RuntimeError(f"Job quote description update failed: HTTP {response.status_code} :: {response.text[:1000]}")
+
+
+def extract_company_name(job: Dict[str, Any]) -> str:
+    related = job.get("related")
+    if isinstance(related, dict):
+        company = related.get("company")
+        if isinstance(company, dict):
+            company_name = clean_text(company.get("name"))
+            if company_name:
+                return company_name
+    company = job.get("company")
+    if isinstance(company, dict):
+        company_name = clean_text(company.get("name"))
+        if company_name:
+            return company_name
+    return first_text(job.get("company_name"), job.get("customer_name"))
+
+
+def upsert_job_metadata(conn: sqlite3.Connection, *, job_uuid: str, job: Dict[str, Any], source: str) -> None:
+    job_uuid = clean_text(job_uuid or job.get("uuid"))
+    if not job_uuid:
+        return
+    now = utc_now()
+    conn.execute(
+        """
+        INSERT INTO job_metadata (
+            job_uuid, generated_job_id, job_address, company_name, raw_json,
+            first_seen_at, last_seen_at, source
+        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+        ON CONFLICT(job_uuid) DO UPDATE SET
+            generated_job_id = excluded.generated_job_id,
+            job_address = excluded.job_address,
+            company_name = excluded.company_name,
+            raw_json = excluded.raw_json,
+            last_seen_at = excluded.last_seen_at,
+            source = excluded.source
+        """,
+        (
+            job_uuid,
+            clean_text(job.get("generated_job_id")),
+            format_job_address(job),
+            extract_company_name(job),
+            json.dumps(job, ensure_ascii=False, sort_keys=True),
+            now,
+            now,
+            source,
+        ),
+    )
+    conn.commit()
+
+
+def create_job_material(session: requests.Session, payload: dict) -> str:
+    response = session.post(f"{BASE_URL}/jobmaterial.json", json=payload, timeout=REQUEST_TIMEOUT)
+    if not response.ok:
+        raise RuntimeError(f"Create failed: HTTP {response.status_code} :: {response.text}")
+    record_uuid = response.headers.get("x-record-uuid", "")
+    if not record_uuid:
+        raise RuntimeError("Create succeeded but x-record-uuid header was missing")
+    return record_uuid
+
+
+def list_remote_job_materials(session: requests.Session, job_uuid: str) -> List[Dict[str, Any]]:
+    """Return existing remote ServiceM8 jobMaterial rows for a job."""
+    filter_expr = f"job_uuid eq '{escape_filter_value(job_uuid)}'"
+    response = session.get(f"{BASE_URL}/jobmaterial.json", params={"$filter": filter_expr}, timeout=REQUEST_TIMEOUT)
+    if not response.ok:
+        raise RuntimeError(f"Remote jobMaterial preflight failed: HTTP {response.status_code} :: {response.text[:1000]}")
+    data = response.json()
+    if not isinstance(data, list):
+        raise RuntimeError(f"Remote jobMaterial preflight expected list response, got {type(data).__name__}")
+    return data
+
+
+def is_active_remote_job_material(row: Dict[str, Any]) -> bool:
+    value = row.get("active", 1)
+    return value not in (0, "0", False, "false", "False")
+
+
+def load_api_key() -> str:
+    for name in ("SERVICEM8_ACCESS_TOKEN", "SERVICEM8_API_KEY"):
+        value = os.getenv(name)
+        if value:
+            return value
+
+    fallback = SCRIPT_DIR / "servicem8-list-webhook-subscriptions-table.py"
+    if fallback.exists():
+        spec = importlib.util.spec_from_file_location("servicem8_subs", fallback)
+        if spec and spec.loader:
+            module = importlib.util.module_from_spec(spec)
+            spec.loader.exec_module(module)  # type: ignore[union-attr]
+            value = getattr(module, "ACCESS_TOKEN", None)
+            if value:
+                return str(value)
+
+    raise RuntimeError("Missing SERVICEM8_ACCESS_TOKEN or SERVICEM8_API_KEY")
+
+
+def get_conn(db_path: Path = POLL_DB_PATH) -> sqlite3.Connection:
+    conn = sqlite3.connect(db_path, timeout=30)
+    conn.row_factory = sqlite3.Row
+    return conn
+
+
+def init_apply_tables(conn: sqlite3.Connection) -> None:
+    conn.execute(
+        """
+        CREATE TABLE IF NOT EXISTS job_metadata (
+            job_uuid TEXT PRIMARY KEY,
+            generated_job_id TEXT,
+            job_address TEXT,
+            company_name TEXT,
+            raw_json TEXT NOT NULL,
+            first_seen_at TEXT NOT NULL,
+            last_seen_at TEXT NOT NULL,
+            source TEXT NOT NULL
+        )
+        """
+    )
+    conn.execute("CREATE INDEX IF NOT EXISTS idx_job_metadata_generated_job_id ON job_metadata(generated_job_id)")
+    conn.execute(
+        """
+        CREATE TABLE IF NOT EXISTS quote_template_apply_runs (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            form_response_uuid TEXT NOT NULL,
+            job_uuid TEXT NOT NULL,
+            mode TEXT NOT NULL,
+            started_at TEXT NOT NULL,
+            finished_at TEXT,
+            desired_count INTEGER NOT NULL DEFAULT 0,
+            created_count INTEGER NOT NULL DEFAULT 0,
+            status TEXT NOT NULL,
+            error TEXT
+        )
+        """
+    )
+    conn.execute(
+        """
+        CREATE TABLE IF NOT EXISTS quote_template_apply_run_rows (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            run_id INTEGER NOT NULL,
+            form_response_uuid TEXT NOT NULL,
+            job_uuid TEXT NOT NULL,
+            row_index INTEGER NOT NULL,
+            kind TEXT,
+            source_question TEXT,
+            name TEXT,
+            api_payload_json TEXT NOT NULL,
+            action TEXT NOT NULL,
+            job_material_uuid TEXT,
+            error TEXT,
+            created_at TEXT NOT NULL,
+            FOREIGN KEY(run_id) REFERENCES quote_template_apply_runs(id)
+        )
+        """
+    )
+    conn.execute(
+        """
+        CREATE TABLE IF NOT EXISTS quote_template_remote_existing_incidents (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            detected_at TEXT NOT NULL,
+            form_response_uuid TEXT NOT NULL,
+            job_uuid TEXT NOT NULL,
+            apply_run_id INTEGER,
+            desired_count INTEGER NOT NULL DEFAULT 0,
+            remote_count INTEGER NOT NULL DEFAULT 0,
+            remote_active_count INTEGER NOT NULL DEFAULT 0,
+            action TEXT NOT NULL,
+            reason TEXT,
+            remote_rows_json TEXT NOT NULL,
+            FOREIGN KEY(apply_run_id) REFERENCES quote_template_apply_runs(id)
+        )
+        """
+    )
+    conn.execute("CREATE INDEX IF NOT EXISTS idx_quote_apply_runs_form_response ON quote_template_apply_runs(form_response_uuid)")
+    conn.execute("CREATE INDEX IF NOT EXISTS idx_quote_apply_rows_run ON quote_template_apply_run_rows(run_id)")
+    conn.execute("CREATE INDEX IF NOT EXISTS idx_quote_remote_existing_form_response ON quote_template_remote_existing_incidents(form_response_uuid)")
+    conn.execute("CREATE INDEX IF NOT EXISTS idx_quote_remote_existing_job_uuid ON quote_template_remote_existing_incidents(job_uuid)")
+    conn.commit()
+
+
+def load_quote(conn: sqlite3.Connection, form_response_uuid: str) -> sqlite3.Row:
+    row = conn.execute(
+        "select * from quote_template_form_responses where form_response_uuid = ?",
+        (form_response_uuid,),
+    ).fetchone()
+    if not row:
+        raise SystemExit(f"No parsed polled Quote Template response found for UUID: {form_response_uuid}")
+    return row
+
+
+def existing_created_for_form(form_response_uuid: str) -> int:
+    if not STATE_DB_PATH.exists():
+        return 0
+    try:
+        with closing(sqlite3.connect(STATE_DB_PATH, timeout=30)) as conn:
+            row = conn.execute(
+                "select count(*) from generated_job_materials where form_response_uuid = ?",
+                (form_response_uuid,),
+            ).fetchone()
+            return int(row[0] if row else 0)
+    except sqlite3.Error:
+        return 0
+
+
+def create_apply_run(conn: sqlite3.Connection, *, form_response_uuid: str, job_uuid: str, mode: str, desired_count: int) -> int:
+    cur = conn.execute(
+        """
+        INSERT INTO quote_template_apply_runs (
+            form_response_uuid, job_uuid, mode, started_at, desired_count, status
+        ) VALUES (?, ?, ?, ?, ?, ?)
+        """,
+        (form_response_uuid, job_uuid, mode, utc_now(), desired_count, "running"),
+    )
+    conn.commit()
+    return int(cur.lastrowid)
+
+
+def finish_apply_run(conn: sqlite3.Connection, run_id: int, *, status: str, created_count: int, error: Optional[str] = None) -> None:
+    conn.execute(
+        """
+        UPDATE quote_template_apply_runs
+        SET finished_at = ?, status = ?, created_count = ?, error = ?
+        WHERE id = ?
+        """,
+        (utc_now(), status, created_count, error, run_id),
+    )
+    conn.commit()
+
+
+def record_apply_row(
+    conn: sqlite3.Connection,
+    *,
+    run_id: int,
+    form_response_uuid: str,
+    job_uuid: str,
+    row_index: int,
+    row: Dict[str, Any],
+    api_payload: Dict[str, Any],
+    action: str,
+    job_material_uuid: str = "",
+    error: Optional[str] = None,
+) -> None:
+    conn.execute(
+        """
+        INSERT INTO quote_template_apply_run_rows (
+            run_id, form_response_uuid, job_uuid, row_index, kind, source_question,
+            name, api_payload_json, action, job_material_uuid, error, created_at
+        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+        """,
+        (
+            run_id,
+            form_response_uuid,
+            job_uuid,
+            row_index,
+            row.get("kind", ""),
+            row.get("source_question", ""),
+            row.get("name", ""),
+            json.dumps(api_payload, ensure_ascii=False, sort_keys=True),
+            action,
+            job_material_uuid,
+            error,
+            utc_now(),
+        ),
+    )
+    conn.commit()
+
+
+def record_remote_existing_incident(
+    conn: sqlite3.Connection,
+    *,
+    form_response_uuid: str,
+    job_uuid: str,
+    apply_run_id: int,
+    desired_count: int,
+    remote_rows: List[Dict[str, Any]],
+    action: str,
+    reason: str,
+) -> int:
+    remote_active_count = sum(1 for row in remote_rows if is_active_remote_job_material(row))
+    cur = conn.execute(
+        """
+        INSERT INTO quote_template_remote_existing_incidents (
+            detected_at, form_response_uuid, job_uuid, apply_run_id, desired_count,
+            remote_count, remote_active_count, action, reason, remote_rows_json
+        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+        """,
+        (
+            utc_now(),
+            form_response_uuid,
+            job_uuid,
+            apply_run_id,
+            desired_count,
+            len(remote_rows),
+            remote_active_count,
+            action,
+            reason,
+            json.dumps(remote_rows, ensure_ascii=False, sort_keys=True),
+        ),
+    )
+    conn.commit()
+    return int(cur.lastrowid)
+
+
+def list_pending(conn: sqlite3.Connection) -> List[Dict[str, Any]]:
+    rows = conn.execute(
+        """
+        select form_response_uuid, discovered_at, job_uuid, description, queued_at,
+               processed_at, process_status, desired_job_materials_json
+        from quote_template_form_responses
+        order by discovered_at desc
+        """
+    ).fetchall()
+    out = []
+    for row in rows:
+        existing = existing_created_for_form(row["form_response_uuid"])
+        try:
+            desired_count = len(json.loads(row["desired_job_materials_json"] or "[]"))
+        except Exception:
+            desired_count = None
+        out.append(
+            {
+                "form_response_uuid": row["form_response_uuid"],
+                "discovered_at": row["discovered_at"],
+                "job_uuid": row["job_uuid"],
+                "description": row["description"],
+                "desired_count": desired_count,
+                "created_in_state_db": existing,
+                "processed_at": row["processed_at"],
+                "process_status": row["process_status"],
+            }
+        )
+    return out
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Dry-run or apply one selected polled Quote Template response to ServiceM8 jobMaterials")
+    parser.add_argument("--uuid", help="Polled Quote Template form_response_uuid to process")
+    parser.add_argument("--db", default=str(POLL_DB_PATH), help="Poll DB path")
+    parser.add_argument("--apply", action="store_true", help="Actually create ServiceM8 jobMaterial records. Default is dry-run.")
+    parser.add_argument("--force", action="store_true", help="Allow apply even if generated_job_materials already contains rows for this form response")
+    parser.add_argument("--force-remote-existing", action="store_true", help="Allow apply even when ServiceM8 already has jobMaterial rows for the target job; records a forced incident before creating")
+    parser.add_argument("--list", action="store_true", help="List parsed polled Quote Template responses and exit")
+    parser.add_argument("--pretty", action="store_true", help="Pretty-print JSON output")
+    args = parser.parse_args()
+
+    db_path = Path(args.db)
+    if not db_path.exists():
+        raise SystemExit(f"Poll DB not found: {db_path}")
+
+    with closing(get_conn(db_path)) as conn:
+        init_apply_tables(conn)
+
+        if args.list:
+            print(json.dumps({"ok": True, "rows": list_pending(conn)}, indent=2 if args.pretty else None, ensure_ascii=False))
+            return 0
+
+        if not args.uuid:
+            raise SystemExit("--uuid is required unless using --list")
+
+        quote = load_quote(conn, args.uuid)
+        form_response_uuid = quote["form_response_uuid"]
+        job_uuid = quote["job_uuid"]
+        desired_rows = json.loads(quote["desired_job_materials_json"] or "[]")
+        mode = "apply" if args.apply else "dry-run"
+
+        existing_created = existing_created_for_form(form_response_uuid)
+        if args.apply and existing_created and not args.force:
+            raise SystemExit(
+                f"Refusing duplicate apply: {existing_created} generated_job_materials already recorded for {form_response_uuid}. Use --force only if intentional."
+            )
+
+        run_id = create_apply_run(
+            conn,
+            form_response_uuid=form_response_uuid,
+            job_uuid=job_uuid,
+            mode=mode,
+            desired_count=len(desired_rows),
+        )
+
+        result = {
+            "ok": True,
+            "mode": mode,
+            "run_id": run_id,
+            "form_response_uuid": form_response_uuid,
+            "job_uuid": job_uuid,
+            "description": quote["description"],
+            "desired_count": len(desired_rows),
+            "created_count": 0,
+            "state_db_path": str(STATE_DB_PATH),
+            "rows": [],
+        }
+
+        try:
+            api_key = load_api_key()
+            session = requests.Session()
+            session.headers.update({"X-Api-Key": api_key, "Accept": "application/json", "Content-Type": "application/json"})
+
+            quote_description_source = clean_text(quote["description"])
+            job_details = retrieve_job(session, job_uuid) if quote_description_source else {}
+            if job_details:
+                upsert_job_metadata(conn, job_uuid=job_uuid, job=job_details, source=mode)
+            job_update_payload = build_job_update_payload(quote_description_source, job_details)
+            job_update_record_payload = {
+                "endpoint": f"/job/{job_uuid}.json",
+                "payload": job_update_payload,
+                "source_description": quote_description_source,
+                "job_address": format_job_address(job_details) if job_details else "",
+            }
+            job_update_row = {
+                "kind": "work_done_description",
+                "source_question": "Description of Works to be Quoted",
+                "name": job_update_payload.get("work_done_description", ""),
+            }
+
+            if not args.apply:
+                remote_existing_rows = list_remote_job_materials(session, job_uuid)
+                remote_existing_blocks_apply = bool(remote_existing_rows)
+                if remote_existing_blocks_apply:
+                    remote_active_count = sum(1 for remote_row in remote_existing_rows if is_active_remote_job_material(remote_row))
+                    reason = (
+                        f"Remote ServiceM8 job already has {len(remote_existing_rows)} jobMaterial row(s) "
+                        f"({remote_active_count} active); apply would be blocked before updates or creates"
+                    )
+                    incident_id = record_remote_existing_incident(
+                        conn,
+                        form_response_uuid=form_response_uuid,
+                        job_uuid=job_uuid,
+                        apply_run_id=run_id,
+                        desired_count=len(desired_rows),
+                        remote_rows=remote_existing_rows,
+                        action="dry_run_would_block",
+                        reason=reason,
+                    )
+                    result["remote_existing"] = {
+                        "incident_id": incident_id,
+                        "action": "would_block_remote_existing",
+                        "remote_count": len(remote_existing_rows),
+                        "remote_active_count": remote_active_count,
+                        "reason": reason,
+                    }
+
+                if job_update_payload:
+                    job_update_action = (
+                        "would_update_work_done_description_if_remote_empty"
+                        if remote_existing_blocks_apply
+                        else "would_update_work_done_description"
+                    )
+                    record_apply_row(
+                        conn,
+                        run_id=run_id,
+                        form_response_uuid=form_response_uuid,
+                        job_uuid=job_uuid,
+                        row_index=0,
+                        row=job_update_row,
+                        api_payload=job_update_record_payload,
+                        action=job_update_action,
+                    )
+                    result["job_update"] = {"action": job_update_action, **job_update_record_payload}
+                else:
+                    result["job_update"] = {"action": "skipped", "reason": "Quote description is empty"}
+                for idx, row in enumerate(desired_rows, start=1):
+                    api_payload = build_payload(job_uuid, row)
+                    row_action = "would_create_if_remote_empty" if remote_existing_blocks_apply else "would_create"
+                    record_apply_row(
+                        conn,
+                        run_id=run_id,
+                        form_response_uuid=form_response_uuid,
+                        job_uuid=job_uuid,
+                        row_index=idx,
+                        row=row,
+                        api_payload=api_payload,
+                        action=row_action,
+                    )
+                    result["rows"].append({"action": row_action, "kind": row.get("kind"), "payload": api_payload})
+                finish_apply_run(conn, run_id, status="dry-run", created_count=0)
+                conn.execute(
+                    "UPDATE quote_template_form_responses SET process_status = ? WHERE form_response_uuid = ?",
+                    ("dry-run", form_response_uuid),
+                )
+                conn.commit()
+                print(json.dumps(result, indent=2 if args.pretty else None, ensure_ascii=False))
+                return 0
+
+            init_state_db(STATE_DB_PATH)
+
+            remote_existing_rows = list_remote_job_materials(session, job_uuid)
+            if remote_existing_rows:
+                remote_active_count = sum(1 for remote_row in remote_existing_rows if is_active_remote_job_material(remote_row))
+                action = "forced" if args.force_remote_existing else "blocked"
+                reason = (
+                    f"Remote ServiceM8 job already has {len(remote_existing_rows)} jobMaterial row(s) "
+                    f"({remote_active_count} active); no creates attempted"
+                    if not args.force_remote_existing
+                    else f"Remote ServiceM8 job already has {len(remote_existing_rows)} jobMaterial row(s) "
+                         f"({remote_active_count} active); create forced by --force-remote-existing"
+                )
+                incident_id = record_remote_existing_incident(
+                    conn,
+                    form_response_uuid=form_response_uuid,
+                    job_uuid=job_uuid,
+                    apply_run_id=run_id,
+                    desired_count=len(desired_rows),
+                    remote_rows=remote_existing_rows,
+                    action=action,
+                    reason=reason,
+                )
+                result["remote_existing"] = {
+                    "incident_id": incident_id,
+                    "action": action,
+                    "remote_count": len(remote_existing_rows),
+                    "remote_active_count": remote_active_count,
+                    "reason": reason,
+                }
+                if not args.force_remote_existing:
+                    finish_apply_run(conn, run_id, status="blocked_remote_existing", created_count=0)
+                    conn.execute(
+                        "UPDATE quote_template_form_responses SET processed_at = ?, process_status = ?, process_error = NULL WHERE form_response_uuid = ?",
+                        (utc_now(), "blocked_remote_existing", form_response_uuid),
+                    )
+                    conn.commit()
+                    result["status"] = "blocked_remote_existing"
+                    print(json.dumps(result, indent=2 if args.pretty else None, ensure_ascii=False))
+                    return 0
+
+            if job_update_payload:
+                update_job_description(session, job_uuid, job_update_payload)
+                record_apply_row(
+                    conn,
+                    run_id=run_id,
+                    form_response_uuid=form_response_uuid,
+                    job_uuid=job_uuid,
+                    row_index=0,
+                    row=job_update_row,
+                    api_payload=job_update_record_payload,
+                    action="updated_work_done_description",
+                )
+                result["job_update"] = {"action": "updated_work_done_description", **job_update_record_payload}
+            else:
+                result["job_update"] = {"action": "skipped", "reason": "Quote description is empty"}
+
+            created_count = 0
+            for idx, row in enumerate(desired_rows, start=1):
+                api_payload = build_payload(job_uuid, row)
+                created_uuid = create_job_material(session, api_payload)
+                created_count += 1
+                record_generated_job_material(
+                    job_uuid=job_uuid,
+                    form_response_uuid=form_response_uuid,
+                    job_material_uuid=created_uuid,
+                    kind=row.get("kind", ""),
+                    source_field_uuid=row.get("source_field_uuid", ""),
+                    source_question=row.get("source_question", ""),
+                    source_text=row.get("name", ""),
+                    db_path=STATE_DB_PATH,
+                )
+                record_apply_row(
+                    conn,
+                    run_id=run_id,
+                    form_response_uuid=form_response_uuid,
+                    job_uuid=job_uuid,
+                    row_index=idx,
+                    row=row,
+                    api_payload=api_payload,
+                    action="created",
+                    job_material_uuid=created_uuid,
+                )
+                result["rows"].append({"action": "created", "kind": row.get("kind"), "job_material_uuid": created_uuid, "payload": api_payload})
+
+            result["created_count"] = created_count
+            finish_apply_run(conn, run_id, status="applied", created_count=created_count)
+            conn.execute(
+                "UPDATE quote_template_form_responses SET processed_at = ?, process_status = ?, process_error = NULL WHERE form_response_uuid = ?",
+                (utc_now(), "applied", form_response_uuid),
+            )
+            conn.commit()
+            print(json.dumps(result, indent=2 if args.pretty else None, ensure_ascii=False))
+            return 0
+
+        except Exception as exc:
+            finish_apply_run(conn, run_id, status="error", created_count=result.get("created_count", 0), error=str(exc))
+            conn.execute(
+                "UPDATE quote_template_form_responses SET process_status = ?, process_error = ? WHERE form_response_uuid = ?",
+                ("error", str(exc), form_response_uuid),
+            )
+            conn.commit()
+            raise
+
+
+if __name__ == "__main__":
+    try:
+        raise SystemExit(main())
+    except Exception as exc:
+        print(str(exc), file=sys.stderr)
+        raise SystemExit(1)

docs/list-all-form-responses.md

@@ -0,0 +1,411 @@
+# List all Form Responses
+
+
+			
+#### Filtering
+This endpoint supports result filtering. For more information on how to filter this request, [go here](/docs/filtering).
+			
+			
+#### OAuth Scope
+This endpoint requires the following OAuth scope **read_forms**.
+
+			
+
+# OpenAPI definition
+
+```json
+{
+  "openapi": "3.1.0",
+  "info": {
+    "title": "ServiceM8 API",
+    "description": "Move your app forward with the ServiceM8 API\n\n\n\n## Limits and Throttling\nTo ensure continuous quality of service, API usage can be subject to throttling. The throttle will be applied once an API consumer reaches a certain \nthreshold in terms of a maximum number of requests per minute. Most clients will never hit this threshold, but those that do, will get met by a \nHTTP 429 Too Many Requests response code. \n \nThere is a limit of 180 requests per minute, if you reach this you will receive a HTTP 429 with a text body of \"Number of allowed API requests per minute exceeded\".\nThere is a limit of 20000 requests per day, if you reach this you will receive a HTTP 429 with a text body of \"Number of allowed API requests per day exceeded\".\n\nWe encourage all API developers to anticipate this error, and take appropriate measures like e.g. using a cached value from a previous call, or passing on a message to the end user that gets subjected to this behaviour (if any).\n\nLimits are per Addon per account.\n",
+    "termsOfService": "https://www.servicem8.com/terms-of-service",
+    "version": "1.0.0"
+  },
+  "servers": [
+    {
+      "url": "https://api.servicem8.com/api_1.0"
+    }
+  ],
+  "security": [
+    {
+      "apiKey": []
+    },
+    {
+      "oauth2": []
+    }
+  ],
+  "paths": {
+    "/formresponse.json": {
+      "get": {
+        "tags": [
+          "Form Responses"
+        ],
+        "operationId": "listFormResponses",
+        "summary": "List all Form Responses",
+        "description": "\n\t\t\t\n#### Filtering\nThis endpoint supports result filtering. For more information on how to filter this request, [go here](/docs/filtering).\n\t\t\t\n\t\t\t\n#### OAuth Scope\nThis endpoint requires the following OAuth scope **read_forms**.\n\n\t\t\t",
+        "security": [
+          {
+            "apiKey": []
+          },
+          {
+            "oauth2": [
+              "read_forms"
+            ]
+          }
+        ],
+        "responses": {
+          "200": {
+            "description": "An array of Form Responses",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "array",
+                  "items": {
+                    "$ref": "#/components/schemas/FormResponse"
+                  }
+                },
+                "examples": {
+                  "success": {
+                    "value": [
+                      {
+                        "uuid": "123e4567-af16-4618-80c7-23f940cc449b",
+                        "active": 1,
+                        "edit_date": "2026-03-01 12:00:00",
+                        "form_uuid": "123e4567-f8b2-4802-a8db-23f94cf50e1b",
+                        "staff_uuid": "123e4567-5c89-4cb3-8dc0-23f94d75521b",
+                        "regarding_object": "string",
+                        "regarding_object_uuid": "123e4567-016e-482c-a3f7-23f943f6d72b",
+                        "field_data": "string",
+                        "timestamp": "2026-03-01 12:00:00",
+                        "form_by_staff_uuid": "123e4567-44e8-448d-925c-23f9435b205b",
+                        "document_attachment_uuid": "123e4567-2f15-479f-8791-23f94c0fbbbb",
+                        "asset_uuid": "123e4567-89f4-4183-ae1e-23f94832e59b"
+                      }
+                    ]
+                  }
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Bad Request - The request is malformed or contains invalid parameters",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "badRequest": {
+                    "value": {
+                      "errorCode": "1000",
+                      "message": "An error occurred completing your request"
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "401": {
+            "description": "Unauthorized - Authentication credentials are missing or invalid",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/AuthenticationError"
+                },
+                "examples": {
+                  "unauthorized": {
+                    "value": {
+                      "errorCode": "401",
+                      "message": "Authentication failed. Please check your API key or OAuth token."
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "403": {
+            "description": "Forbidden - You don't have permission to access this resource",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ForbiddenError"
+                },
+                "examples": {
+                  "forbidden": {
+                    "value": {
+                      "errorCode": "403",
+                      "message": "Access forbidden. You don't have permission to access this resource."
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "429": {
+            "description": "Too Many Requests - You have exceeded the rate limit",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/RateLimitError"
+                },
+                "examples": {
+                  "rateLimitMinute": {
+                    "value": {
+                      "errorCode": 429,
+                      "message": "Number of allowed API requests per minute exceeded"
+                    }
+                  },
+                  "rateLimitDay": {
+                    "value": {
+                      "errorCode": 429,
+                      "message": "Number of allowed API requests per day exceeded"
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "500": {
+            "description": "Internal Server Error - An unexpected error occurred on the server",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "serverError": {
+                    "value": {
+                      "errorCode": 500,
+                      "message": "An unexpected error occurred. Please try again later."
+                    }
+                  }
+                }
+              }
+            }
+          }
+        }
+      }
+    }
+  },
+  "components": {
+    "securitySchemes": {
+      "apiKey": {
+        "type": "apiKey",
+        "name": "X-Api-Key",
+        "in": "header"
+      },
+      "oauth2": {
+        "type": "oauth2",
+        "flows": {
+          "authorizationCode": {
+            "authorizationUrl": "https://api.servicem8.com/oauth/authorize",
+            "tokenUrl": "https://api.servicem8.com/oauth/access_token",
+            "scopes": {
+              "staff_locations": "Access to real-time GPS information about staff",
+              "staff_activity": "Access to clock on, lunch break and clock off information about staff",
+              "publish_sms": "Access to send SMS messages to customers and/or staff on your behalf. Note sending SMS messages will incur account charges.",
+              "publish_email": "Access to send Email messages to customers and/or staff on your behalf",
+              "vendor": "Access to basic account information",
+              "vendor_logo": "Access to account logo",
+              "vendor_email": "Access to account holder email address",
+              "read_locations": "Read-only access to Location Endpoint",
+              "manage_locations": "Full access to Location Endpoint",
+              "read_staff": "Read-only access to Staff Endpoint",
+              "manage_staff": "Full access to Staff Endpoint",
+              "read_customers": "Read-only access to Company Endpoint",
+              "manage_customers": "Full access to Company Endpoint",
+              "read_customer_contacts": "Read-only access to CompanyContact Endpoint",
+              "manage_customer_contacts": "Full access to CompanyContact Endpoint",
+              "read_jobs": "Read-only access to Job Endpoint",
+              "manage_jobs": "Full access to Job Endpoint",
+              "create_jobs": "Ability to create jobs on behalf of account. Note creating jobs may incur account charges.",
+              "read_job_contacts": "Read-only access to JobContact Endpoint",
+              "manage_job_contacts": "Full access to JobContact Endpoint",
+              "read_job_materials": "Read-only access to JobMaterials Endpoint",
+              "manage_job_materials": "Full access to JobMaterials Endpoint",
+              "read_job_categories": "Read-only access to Categories Endpoint",
+              "manage_job_categories": "Full access to Categories Endpoint",
+              "read_job_queues": "Read-only access to Job Queues Endpoint",
+              "manage_job_queues": "Full access to Job Queues Endpoint",
+              "read_tasks": "Read-only access to Tasks Endpoint",
+              "manage_tasks": "Full access to Tasks Endpoint",
+              "read_schedule": "Read-only access to JobActivity Endpoint",
+              "manage_schedule": "Full access to JobActivity Endpoint",
+              "read_inventory": "Read-only access to Materials Endpoint",
+              "manage_inventory": "Full access to Materials Endpoint",
+              "read_job_notes": "Read-only access to job notes",
+              "publish_job_notes": "Ability to add new job notes",
+              "read_job_photos": "Read-only access to job photos",
+              "publish_job_photos": "Ability to add new job photos",
+              "read_attachments": "Read-only access to Attachments Endpoint",
+              "manage_attachments": "Full access to Attachments Endpoint",
+              "read_inbox": "Read-only access to inbox messages",
+              "read_messages": "Read-only access to staff messages",
+              "manage_notifications": "Ability to read notifications and mark as read",
+              "manage_templates": "Full-access to email, sms and document templates",
+              "manage_badges": "Full-access to create/modify job badges",
+              "read_assets": "Read-only access to Assets Endpoint",
+              "manage_assets": "Full access to Assets Endpoint",
+              "read_knowledge_base": "Read-only access to Knowledge Base Endpoint",
+              "manage_knowledge_base": "Full access to Knowledge Base Endpoint"
+            }
+          }
+        }
+      }
+    },
+    "schemas": {
+      "Error": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "1000"
+          },
+          "message": {
+            "type": "string",
+            "example": "An error occurred completing your request"
+          }
+        }
+      },
+      "RateLimitError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "429"
+          },
+          "message": {
+            "type": "string",
+            "example": "Number of allowed API requests per minute exceeded"
+          }
+        }
+      },
+      "AuthenticationError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "401"
+          },
+          "message": {
+            "type": "string",
+            "example": "Authentication failed. Please check your API key or OAuth token."
+          }
+        }
+      },
+      "ForbiddenError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "403"
+          },
+          "message": {
+            "type": "string",
+            "example": "Access forbidden. You don't have permission to access this resource."
+          }
+        }
+      },
+      "FormResponse": {
+        "type": "object",
+        "properties": {
+          "form_uuid": {
+            "description": "UUID of the form used to generate this form response. Links to a specific form in the system that defines the fields to be gathered.",
+            "format": "uuid",
+            "example": "123e4567-8f59-4273-9bb2-23f94e3071eb",
+            "type": "string"
+          },
+          "staff_uuid": {
+            "description": "UUID of the staff member who completed this FormResponse.",
+            "format": "uuid",
+            "example": "123e4567-068f-433c-a2e6-23f94e815bdb",
+            "type": "string"
+          },
+          "regarding_object": {
+            "description": "The object type that this form response is associated with. Common values include 'job', 'asset', or 'company'. Works in conjunction with regarding_object_uuid to link this form response to a specific record in the system.",
+            "type": "string"
+          },
+          "regarding_object_uuid": {
+            "description": "UUID of the specific record this form response is linked to. For example, if regarding_object is 'job', this will be the UUID of the specific job. This creates a relationship between the form response and the object it refers to.",
+            "format": "uuid",
+            "example": "123e4567-41ee-4ea3-8ca2-23f94fe918eb",
+            "type": "string"
+          },
+          "field_data": {
+            "description": "JSON array of form answers captured at submission time.",
+            "type": "string"
+          },
+          "timestamp": {
+            "description": "Date and time when the form was submitted/completed. Used for sorting and displaying form responses chronologically. Format is YYYY-MM-DD HH:MM:SS in UTC timezone.",
+            "type": "string",
+            "example": "2026-03-01 12:00:00"
+          },
+          "form_by_staff_uuid": {
+            "description": "UUID of the staff member who completed or submitted this form. Identifies which user filled out the form. Used for tracking form submission history and staff accountability.",
+            "format": "uuid",
+            "example": "123e4567-0230-4fe1-83c9-23f94429362b",
+            "type": "string"
+          },
+          "document_attachment_uuid": {
+            "description": "UUID of the document attachment generated from this form response. When a form is completed, it can generate a PDF document which is stored as an attachment. This field links to that generated document attachment.",
+            "format": "uuid",
+            "example": "123e4567-ff79-499a-8ef4-23f94a981a6b",
+            "type": "string"
+          },
+          "asset_uuid": {
+            "description": "UUID of the Asset this form response is related to. Used when the FormResponsepertains to a specific asset, such as equipment inspections, maintenance checklists, or asset condition reports.",
+            "format": "uuid",
+            "example": "123e4567-fc2c-4a3d-a278-23f942d7f62b",
+            "type": "string"
+          },
+          "uuid": {
+            "format": "uuid",
+            "description": "Unique identifier for this record",
+            "example": "123e4567-faa3-4649-8ea5-23f94011885b",
+            "type": "string"
+          },
+          "active": {
+            "enum": [
+              0,
+              1
+            ],
+            "type": "integer",
+            "default": 1,
+            "description": "Record active/deleted flag.  Valid values are [0,1]"
+          },
+          "edit_date": {
+            "example": "2026-03-01 12:00:00",
+            "readOnly": true,
+            "description": "Timestamp at which record was last modified"
+          }
+        }
+      }
+    }
+  },
+  "x-speakeasy-retries": {
+    "strategy": "backoff",
+    "backoff": {
+      "initialInterval": 500,
+      "maxInterval": 60000,
+      "maxElapsedTime": 3600000,
+      "exponent": 1.5
+    },
+    "statusCodes": [
+      "5XX",
+      "429"
+    ],
+    "retryConnectionErrors": true
+  },
+  "tags": [
+    {
+      "name": "Form Responses",
+      "description": "Operations related to Form Responses"
+    }
+  ]
+}
+```

docs/list-all-job-allocations.md

@@ -0,0 +1,440 @@
+# List all Job Allocations
+
+
+			
+#### Filtering
+This endpoint supports result filtering. For more information on how to filter this request, [go here](/docs/filtering).
+			
+			
+#### OAuth Scope
+This endpoint requires the following OAuth scope **read_schedule**.
+
+			
+
+# OpenAPI definition
+
+```json
+{
+  "openapi": "3.1.0",
+  "info": {
+    "title": "ServiceM8 API",
+    "description": "Move your app forward with the ServiceM8 API\n\n\n\n## Limits and Throttling\nTo ensure continuous quality of service, API usage can be subject to throttling. The throttle will be applied once an API consumer reaches a certain \nthreshold in terms of a maximum number of requests per minute. Most clients will never hit this threshold, but those that do, will get met by a \nHTTP 429 Too Many Requests response code. \n \nThere is a limit of 180 requests per minute, if you reach this you will receive a HTTP 429 with a text body of \"Number of allowed API requests per minute exceeded\".\nThere is a limit of 20000 requests per day, if you reach this you will receive a HTTP 429 with a text body of \"Number of allowed API requests per day exceeded\".\n\nWe encourage all API developers to anticipate this error, and take appropriate measures like e.g. using a cached value from a previous call, or passing on a message to the end user that gets subjected to this behaviour (if any).\n\nLimits are per Addon per account.\n",
+    "termsOfService": "https://www.servicem8.com/terms-of-service",
+    "version": "1.0.0"
+  },
+  "servers": [
+    {
+      "url": "https://api.servicem8.com/api_1.0"
+    }
+  ],
+  "security": [
+    {
+      "apiKey": []
+    },
+    {
+      "oauth2": []
+    }
+  ],
+  "paths": {
+    "/joballocation.json": {
+      "get": {
+        "tags": [
+          "Job Allocations"
+        ],
+        "operationId": "listJobAllocations",
+        "summary": "List all Job Allocations",
+        "description": "\n\t\t\t\n#### Filtering\nThis endpoint supports result filtering. For more information on how to filter this request, [go here](/docs/filtering).\n\t\t\t\n\t\t\t\n#### OAuth Scope\nThis endpoint requires the following OAuth scope **read_schedule**.\n\n\t\t\t",
+        "security": [
+          {
+            "apiKey": []
+          },
+          {
+            "oauth2": [
+              "read_schedule"
+            ]
+          }
+        ],
+        "responses": {
+          "200": {
+            "description": "An array of Job Allocations",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "array",
+                  "items": {
+                    "$ref": "#/components/schemas/JobAllocation"
+                  }
+                },
+                "examples": {
+                  "success": {
+                    "value": [
+                      {
+                        "uuid": "123e4567-aa11-4415-ac38-23f94bc904db",
+                        "active": 1,
+                        "edit_date": "2026-03-01 12:00:00",
+                        "job_uuid": "123e4567-ef33-4570-9929-23f945326d1b",
+                        "queue_uuid": "123e4567-e495-4a8d-a253-23f9408ba0db",
+                        "staff_uuid": "123e4567-e0d8-481f-b4bb-23f944e9c86b",
+                        "allocation_date": "2026-03-01 12:00:00",
+                        "allocation_window_uuid": "123e4567-0f0c-48b9-8570-23f94fcc00db",
+                        "allocated_by_staff_uuid": "123e4567-ab70-4071-8066-23f94537066b",
+                        "allocated_timestamp": "2026-03-01 12:00:00",
+                        "expiry_timestamp": "2026-03-01 12:00:00",
+                        "read_timestamp": "2026-03-01 12:00:00",
+                        "completion_timestamp": "2026-03-01 12:00:00",
+                        "estimated_duration": "string",
+                        "revised_duration": "string",
+                        "sort_priority": "string",
+                        "requires_acceptance": "string",
+                        "acceptance_status": "string",
+                        "acceptance_timestamp": "2026-03-01 12:00:00"
+                      }
+                    ]
+                  }
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Bad Request - The request is malformed or contains invalid parameters",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "badRequest": {
+                    "value": {
+                      "errorCode": "1000",
+                      "message": "An error occurred completing your request"
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "401": {
+            "description": "Unauthorized - Authentication credentials are missing or invalid",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/AuthenticationError"
+                },
+                "examples": {
+                  "unauthorized": {
+                    "value": {
+                      "errorCode": "401",
+                      "message": "Authentication failed. Please check your API key or OAuth token."
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "403": {
+            "description": "Forbidden - You don't have permission to access this resource",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ForbiddenError"
+                },
+                "examples": {
+                  "forbidden": {
+                    "value": {
+                      "errorCode": "403",
+                      "message": "Access forbidden. You don't have permission to access this resource."
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "429": {
+            "description": "Too Many Requests - You have exceeded the rate limit",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/RateLimitError"
+                },
+                "examples": {
+                  "rateLimitMinute": {
+                    "value": {
+                      "errorCode": 429,
+                      "message": "Number of allowed API requests per minute exceeded"
+                    }
+                  },
+                  "rateLimitDay": {
+                    "value": {
+                      "errorCode": 429,
+                      "message": "Number of allowed API requests per day exceeded"
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "500": {
+            "description": "Internal Server Error - An unexpected error occurred on the server",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "serverError": {
+                    "value": {
+                      "errorCode": 500,
+                      "message": "An unexpected error occurred. Please try again later."
+                    }
+                  }
+                }
+              }
+            }
+          }
+        }
+      }
+    }
+  },
+  "components": {
+    "securitySchemes": {
+      "apiKey": {
+        "type": "apiKey",
+        "name": "X-Api-Key",
+        "in": "header"
+      },
+      "oauth2": {
+        "type": "oauth2",
+        "flows": {
+          "authorizationCode": {
+            "authorizationUrl": "https://api.servicem8.com/oauth/authorize",
+            "tokenUrl": "https://api.servicem8.com/oauth/access_token",
+            "scopes": {
+              "staff_locations": "Access to real-time GPS information about staff",
+              "staff_activity": "Access to clock on, lunch break and clock off information about staff",
+              "publish_sms": "Access to send SMS messages to customers and/or staff on your behalf. Note sending SMS messages will incur account charges.",
+              "publish_email": "Access to send Email messages to customers and/or staff on your behalf",
+              "vendor": "Access to basic account information",
+              "vendor_logo": "Access to account logo",
+              "vendor_email": "Access to account holder email address",
+              "read_locations": "Read-only access to Location Endpoint",
+              "manage_locations": "Full access to Location Endpoint",
+              "read_staff": "Read-only access to Staff Endpoint",
+              "manage_staff": "Full access to Staff Endpoint",
+              "read_customers": "Read-only access to Company Endpoint",
+              "manage_customers": "Full access to Company Endpoint",
+              "read_customer_contacts": "Read-only access to CompanyContact Endpoint",
+              "manage_customer_contacts": "Full access to CompanyContact Endpoint",
+              "read_jobs": "Read-only access to Job Endpoint",
+              "manage_jobs": "Full access to Job Endpoint",
+              "create_jobs": "Ability to create jobs on behalf of account. Note creating jobs may incur account charges.",
+              "read_job_contacts": "Read-only access to JobContact Endpoint",
+              "manage_job_contacts": "Full access to JobContact Endpoint",
+              "read_job_materials": "Read-only access to JobMaterials Endpoint",
+              "manage_job_materials": "Full access to JobMaterials Endpoint",
+              "read_job_categories": "Read-only access to Categories Endpoint",
+              "manage_job_categories": "Full access to Categories Endpoint",
+              "read_job_queues": "Read-only access to Job Queues Endpoint",
+              "manage_job_queues": "Full access to Job Queues Endpoint",
+              "read_tasks": "Read-only access to Tasks Endpoint",
+              "manage_tasks": "Full access to Tasks Endpoint",
+              "read_schedule": "Read-only access to JobActivity Endpoint",
+              "manage_schedule": "Full access to JobActivity Endpoint",
+              "read_inventory": "Read-only access to Materials Endpoint",
+              "manage_inventory": "Full access to Materials Endpoint",
+              "read_job_notes": "Read-only access to job notes",
+              "publish_job_notes": "Ability to add new job notes",
+              "read_job_photos": "Read-only access to job photos",
+              "publish_job_photos": "Ability to add new job photos",
+              "read_attachments": "Read-only access to Attachments Endpoint",
+              "manage_attachments": "Full access to Attachments Endpoint",
+              "read_inbox": "Read-only access to inbox messages",
+              "read_messages": "Read-only access to staff messages",
+              "manage_notifications": "Ability to read notifications and mark as read",
+              "manage_templates": "Full-access to email, sms and document templates",
+              "manage_badges": "Full-access to create/modify job badges",
+              "read_assets": "Read-only access to Assets Endpoint",
+              "manage_assets": "Full access to Assets Endpoint",
+              "read_knowledge_base": "Read-only access to Knowledge Base Endpoint",
+              "manage_knowledge_base": "Full access to Knowledge Base Endpoint"
+            }
+          }
+        }
+      }
+    },
+    "schemas": {
+      "Error": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "1000"
+          },
+          "message": {
+            "type": "string",
+            "example": "An error occurred completing your request"
+          }
+        }
+      },
+      "RateLimitError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "429"
+          },
+          "message": {
+            "type": "string",
+            "example": "Number of allowed API requests per minute exceeded"
+          }
+        }
+      },
+      "AuthenticationError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "401"
+          },
+          "message": {
+            "type": "string",
+            "example": "Authentication failed. Please check your API key or OAuth token."
+          }
+        }
+      },
+      "ForbiddenError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "403"
+          },
+          "message": {
+            "type": "string",
+            "example": "Access forbidden. You don't have permission to access this resource."
+          }
+        }
+      },
+      "JobAllocation": {
+        "type": "object",
+        "properties": {
+          "job_uuid": {
+            "description": "The UUID of the job that this allocation relates to.",
+            "format": "uuid",
+            "example": "123e4567-796e-43d3-89b8-23f942b7f0ab",
+            "type": "string"
+          },
+          "queue_uuid": {
+            "description": "DEPRECATED"
+          },
+          "staff_uuid": {
+            "description": "The UUID of the staff member this job is allocated to.",
+            "format": "uuid",
+            "example": "123e4567-4525-400b-ab3e-23f94473c62b",
+            "type": "string"
+          },
+          "allocation_date": {
+            "description": "The minimum start date for a job allocation to be completed by a staff member. Setting this date will ensure the job allocation appears in the future on staff schedules.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "allocation_window_uuid": {
+            "description": "The UUID of the allocation window that defines when the job should be completed (e.g. Urgent, Early Morning, During Business Hours).",
+            "format": "uuid",
+            "example": "123e4567-7fd1-45f3-8068-23f945565e9b",
+            "type": "string"
+          },
+          "allocated_by_staff_uuid": {
+            "description": "The UUID of the staff member who allocated the job.",
+            "format": "uuid",
+            "example": "123e4567-c873-4cca-91e7-23f943de293b",
+            "type": "string"
+          },
+          "allocated_timestamp": {
+            "description": "The timestamp when the job was allocated.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "expiry_timestamp": {
+            "description": "The timestamp when the job allocation expires.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "read_timestamp": {
+            "description": "The timestamp when the job allocation was read by the staff member.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "completion_timestamp": {
+            "description": "The timestamp when the job allocation was marked as completed.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "estimated_duration": {
+            "description": "DEPRECATED"
+          },
+          "revised_duration": {
+            "description": "DEPRECATED"
+          },
+          "sort_priority": {
+            "description": "The sort priority for displaying this job allocation.",
+            "type": "string"
+          },
+          "requires_acceptance": {
+            "description": "DEPRECATED"
+          },
+          "acceptance_status": {
+            "description": "DEPRECATED"
+          },
+          "acceptance_timestamp": {
+            "description": "DEPRECATED"
+          },
+          "uuid": {
+            "format": "uuid",
+            "description": "Unique identifier for this record",
+            "example": "123e4567-8160-4f2d-951a-23f94c5382cb",
+            "type": "string"
+          },
+          "active": {
+            "enum": [
+              0,
+              1
+            ],
+            "type": "integer",
+            "default": 1,
+            "description": "Record active/deleted flag.  Valid values are [0,1]"
+          },
+          "edit_date": {
+            "example": "2026-03-01 12:00:00",
+            "readOnly": true,
+            "description": "Timestamp at which record was last modified"
+          }
+        }
+      }
+    }
+  },
+  "x-speakeasy-retries": {
+    "strategy": "backoff",
+    "backoff": {
+      "initialInterval": 500,
+      "maxInterval": 60000,
+      "maxElapsedTime": 3600000,
+      "exponent": 1.5
+    },
+    "statusCodes": [
+      "5XX",
+      "429"
+    ],
+    "retryConnectionErrors": true
+  },
+  "tags": [
+    {
+      "name": "Job Allocations",
+      "description": "Operations related to Job Allocations"
+    }
+  ]
+}
+```

docs/list-all-staff-members.md

@@ -0,0 +1,468 @@
+# List all Staff Members
+
+
+			
+#### Filtering
+This endpoint supports result filtering. For more information on how to filter this request, [go here](/docs/filtering).
+			
+			
+#### OAuth Scope
+This endpoint requires the following OAuth scope **read_staff**.
+
+			
+
+# OpenAPI definition
+
+```json
+{
+  "openapi": "3.1.0",
+  "info": {
+    "title": "ServiceM8 API",
+    "description": "Move your app forward with the ServiceM8 API\n\n\n\n## Limits and Throttling\nTo ensure continuous quality of service, API usage can be subject to throttling. The throttle will be applied once an API consumer reaches a certain \nthreshold in terms of a maximum number of requests per minute. Most clients will never hit this threshold, but those that do, will get met by a \nHTTP 429 Too Many Requests response code. \n \nThere is a limit of 180 requests per minute, if you reach this you will receive a HTTP 429 with a text body of \"Number of allowed API requests per minute exceeded\".\nThere is a limit of 20000 requests per day, if you reach this you will receive a HTTP 429 with a text body of \"Number of allowed API requests per day exceeded\".\n\nWe encourage all API developers to anticipate this error, and take appropriate measures like e.g. using a cached value from a previous call, or passing on a message to the end user that gets subjected to this behaviour (if any).\n\nLimits are per Addon per account.\n",
+    "termsOfService": "https://www.servicem8.com/terms-of-service",
+    "version": "1.0.0"
+  },
+  "servers": [
+    {
+      "url": "https://api.servicem8.com/api_1.0"
+    }
+  ],
+  "security": [
+    {
+      "apiKey": []
+    },
+    {
+      "oauth2": []
+    }
+  ],
+  "paths": {
+    "/staff.json": {
+      "get": {
+        "tags": [
+          "Staff Members"
+        ],
+        "operationId": "listStaffMembers",
+        "summary": "List all Staff Members",
+        "description": "\n\t\t\t\n#### Filtering\nThis endpoint supports result filtering. For more information on how to filter this request, [go here](/docs/filtering).\n\t\t\t\n\t\t\t\n#### OAuth Scope\nThis endpoint requires the following OAuth scope **read_staff**.\n\n\t\t\t",
+        "security": [
+          {
+            "apiKey": []
+          },
+          {
+            "oauth2": [
+              "read_staff"
+            ]
+          }
+        ],
+        "responses": {
+          "200": {
+            "description": "An array of Staff Members",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "array",
+                  "items": {
+                    "$ref": "#/components/schemas/Staff"
+                  }
+                },
+                "examples": {
+                  "success": {
+                    "value": [
+                      {
+                        "first": "string",
+                        "last": "string",
+                        "email": "string",
+                        "mobile": "string",
+                        "lng": "number",
+                        "lat": "number",
+                        "geo_timestamp": "2026-03-01 12:00:00",
+                        "job_title": "string",
+                        "navigating_to_job_uuid": "123e4567-ed86-4242-b22f-23f949021d4b",
+                        "navigating_timestamp": "2026-03-01 12:00:00",
+                        "navigating_expiry_timestamp": "2026-03-01 12:00:00",
+                        "color": "string",
+                        "custom_icon_url": "string",
+                        "status_message": "string",
+                        "status_message_timestamp": "2026-03-01 12:00:00",
+                        "hide_from_schedule": "string",
+                        "uuid": "123e4567-ce52-4b1f-8564-23f94e82d3cb",
+                        "active": 1,
+                        "edit_date": "2026-03-01 12:00:00",
+                        "can_receive_push_notification": "string",
+                        "security_role_uuid": "123e4567-873a-46dc-b21b-23f9497ac49b",
+                        "labour_material_uuid": "123e4567-bd3b-4d18-af43-23f941752a7b"
+                      }
+                    ]
+                  }
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Bad Request - The request is malformed or contains invalid parameters",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "badRequest": {
+                    "value": {
+                      "errorCode": "1000",
+                      "message": "An error occurred completing your request"
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "401": {
+            "description": "Unauthorized - Authentication credentials are missing or invalid",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/AuthenticationError"
+                },
+                "examples": {
+                  "unauthorized": {
+                    "value": {
+                      "errorCode": "401",
+                      "message": "Authentication failed. Please check your API key or OAuth token."
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "403": {
+            "description": "Forbidden - You don't have permission to access this resource",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ForbiddenError"
+                },
+                "examples": {
+                  "forbidden": {
+                    "value": {
+                      "errorCode": "403",
+                      "message": "Access forbidden. You don't have permission to access this resource."
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "429": {
+            "description": "Too Many Requests - You have exceeded the rate limit",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/RateLimitError"
+                },
+                "examples": {
+                  "rateLimitMinute": {
+                    "value": {
+                      "errorCode": 429,
+                      "message": "Number of allowed API requests per minute exceeded"
+                    }
+                  },
+                  "rateLimitDay": {
+                    "value": {
+                      "errorCode": 429,
+                      "message": "Number of allowed API requests per day exceeded"
+                    }
+                  }
+                }
+              }
+            }
+          },
+          "500": {
+            "description": "Internal Server Error - An unexpected error occurred on the server",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/Error"
+                },
+                "examples": {
+                  "serverError": {
+                    "value": {
+                      "errorCode": 500,
+                      "message": "An unexpected error occurred. Please try again later."
+                    }
+                  }
+                }
+              }
+            }
+          }
+        }
+      }
+    }
+  },
+  "components": {
+    "securitySchemes": {
+      "apiKey": {
+        "type": "apiKey",
+        "name": "X-Api-Key",
+        "in": "header"
+      },
+      "oauth2": {
+        "type": "oauth2",
+        "flows": {
+          "authorizationCode": {
+            "authorizationUrl": "https://api.servicem8.com/oauth/authorize",
+            "tokenUrl": "https://api.servicem8.com/oauth/access_token",
+            "scopes": {
+              "staff_locations": "Access to real-time GPS information about staff",
+              "staff_activity": "Access to clock on, lunch break and clock off information about staff",
+              "publish_sms": "Access to send SMS messages to customers and/or staff on your behalf. Note sending SMS messages will incur account charges.",
+              "publish_email": "Access to send Email messages to customers and/or staff on your behalf",
+              "vendor": "Access to basic account information",
+              "vendor_logo": "Access to account logo",
+              "vendor_email": "Access to account holder email address",
+              "read_locations": "Read-only access to Location Endpoint",
+              "manage_locations": "Full access to Location Endpoint",
+              "read_staff": "Read-only access to Staff Endpoint",
+              "manage_staff": "Full access to Staff Endpoint",
+              "read_customers": "Read-only access to Company Endpoint",
+              "manage_customers": "Full access to Company Endpoint",
+              "read_customer_contacts": "Read-only access to CompanyContact Endpoint",
+              "manage_customer_contacts": "Full access to CompanyContact Endpoint",
+              "read_jobs": "Read-only access to Job Endpoint",
+              "manage_jobs": "Full access to Job Endpoint",
+              "create_jobs": "Ability to create jobs on behalf of account. Note creating jobs may incur account charges.",
+              "read_job_contacts": "Read-only access to JobContact Endpoint",
+              "manage_job_contacts": "Full access to JobContact Endpoint",
+              "read_job_materials": "Read-only access to JobMaterials Endpoint",
+              "manage_job_materials": "Full access to JobMaterials Endpoint",
+              "read_job_categories": "Read-only access to Categories Endpoint",
+              "manage_job_categories": "Full access to Categories Endpoint",
+              "read_job_queues": "Read-only access to Job Queues Endpoint",
+              "manage_job_queues": "Full access to Job Queues Endpoint",
+              "read_tasks": "Read-only access to Tasks Endpoint",
+              "manage_tasks": "Full access to Tasks Endpoint",
+              "read_schedule": "Read-only access to JobActivity Endpoint",
+              "manage_schedule": "Full access to JobActivity Endpoint",
+              "read_inventory": "Read-only access to Materials Endpoint",
+              "manage_inventory": "Full access to Materials Endpoint",
+              "read_job_notes": "Read-only access to job notes",
+              "publish_job_notes": "Ability to add new job notes",
+              "read_job_photos": "Read-only access to job photos",
+              "publish_job_photos": "Ability to add new job photos",
+              "read_attachments": "Read-only access to Attachments Endpoint",
+              "manage_attachments": "Full access to Attachments Endpoint",
+              "read_inbox": "Read-only access to inbox messages",
+              "read_messages": "Read-only access to staff messages",
+              "manage_notifications": "Ability to read notifications and mark as read",
+              "manage_templates": "Full-access to email, sms and document templates",
+              "manage_badges": "Full-access to create/modify job badges",
+              "read_assets": "Read-only access to Assets Endpoint",
+              "manage_assets": "Full access to Assets Endpoint",
+              "read_knowledge_base": "Read-only access to Knowledge Base Endpoint",
+              "manage_knowledge_base": "Full access to Knowledge Base Endpoint"
+            }
+          }
+        }
+      }
+    },
+    "schemas": {
+      "Error": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "1000"
+          },
+          "message": {
+            "type": "string",
+            "example": "An error occurred completing your request"
+          }
+        }
+      },
+      "RateLimitError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "429"
+          },
+          "message": {
+            "type": "string",
+            "example": "Number of allowed API requests per minute exceeded"
+          }
+        }
+      },
+      "AuthenticationError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "401"
+          },
+          "message": {
+            "type": "string",
+            "example": "Authentication failed. Please check your API key or OAuth token."
+          }
+        }
+      },
+      "ForbiddenError": {
+        "type": "object",
+        "properties": {
+          "errorCode": {
+            "type": "number",
+            "format": "int32",
+            "example": "403"
+          },
+          "message": {
+            "type": "string",
+            "example": "Access forbidden. You don't have permission to access this resource."
+          }
+        }
+      },
+      "Staff": {
+        "type": "object",
+        "properties": {
+          "first": {
+            "description": "Staff First Name",
+            "type": "string",
+            "maxLength": 30
+          },
+          "last": {
+            "description": "Staff Last Name",
+            "type": "string",
+            "maxLength": 30
+          },
+          "email": {
+            "description": "Staff Email Address. This is also your login name.",
+            "format": "email",
+            "type": "string"
+          },
+          "mobile": {
+            "description": "Mobile phone number of the staff member. Used for SMS communications and identification when calling.",
+            "type": "string"
+          },
+          "lng": {
+            "description": "Longitude coordinate of the staff member's current or last known location. Used for tracking staff locations and calculating routes and travel distances.",
+            "type": "number",
+            "format": "float"
+          },
+          "lat": {
+            "description": "Latitude coordinate of the staff member's current or last known location. Used for tracking staff locations and calculating routes and travel distances.",
+            "type": "number",
+            "format": "float"
+          },
+          "geo_timestamp": {
+            "description": "The date and time when the staff member's geographic location (lat/lng) was last updated. Format is YYYY-MM-DD HH:MM:SS. Used to determine how recent the location data is.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "job_title": {
+            "description": "The staff member's job title or role within the organization. Used for organizational purposes and displayed in various places throughout the system.",
+            "type": "string"
+          },
+          "navigating_to_job_uuid": {
+            "description": "UUID of the job the staff member is currently navigating to. Used to track which job a staff member is traveling toward.",
+            "format": "uuid",
+            "example": "123e4567-11b0-4089-8ab2-23f9484d54ab",
+            "type": "string"
+          },
+          "navigating_timestamp": {
+            "description": "The date and time when the staff member started navigating to a job. Format is YYYY-MM-DD HH:MM:SS. Used to track when navigation began.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "navigating_expiry_timestamp": {
+            "description": "The date and time when navigation to a job is expected to complete or expire. Format is YYYY-MM-DD HH:MM:SS. Used to determine if navigation is still active.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "color": {
+            "description": "The color assigned to this staff member, represented as a hex color code. Used for visual identification in the schedule, dispatch board, and other interfaces.",
+            "type": "string"
+          },
+          "custom_icon_url": {
+            "description": "DEPRECATED"
+          },
+          "status_message": {
+            "description": "Short message summarising the staff's current status.",
+            "type": "string"
+          },
+          "status_message_timestamp": {
+            "description": "The date and time when the staff member's status message was last updated. Format is YYYY-MM-DD HH:MM:SS. Used to determine how recent the status message is.",
+            "example": "2026-03-01 12:00:00",
+            "type": "string"
+          },
+          "hide_from_schedule": {
+            "description": "Boolean flag controlling whether this staff member appears in the schedule view. When true (1), the staff member is hidden from the schedule. When false (0), they appear normally in scheduling interfaces..  Valid values are [0,1]",
+            "type": "integer",
+            "enum": [
+              0,
+              1
+            ]
+          },
+          "uuid": {
+            "format": "uuid",
+            "description": "Unique identifier for this record",
+            "example": "123e4567-9fa9-46c2-9e1d-23f94b3418fb",
+            "type": "string"
+          },
+          "active": {
+            "enum": [
+              0,
+              1
+            ],
+            "type": "integer",
+            "default": 1,
+            "description": "Record active/deleted flag.  Valid values are [0,1]"
+          },
+          "edit_date": {
+            "example": "2026-03-01 12:00:00",
+            "readOnly": true,
+            "description": "Timestamp at which record was last modified"
+          },
+          "can_receive_push_notification": {
+            "type": "string"
+          },
+          "security_role_uuid": {
+            "format": "uuid",
+            "example": "123e4567-4ff4-4f95-8521-23f9452e9a2b",
+            "type": "string"
+          },
+          "labour_material_uuid": {
+            "format": "uuid",
+            "example": "123e4567-5c47-4d54-b8e7-23f94882c18b",
+            "type": "string"
+          }
+        },
+        "required": [
+          "first",
+          "last",
+          "email"
+        ]
+      }
+    }
+  },
+  "x-speakeasy-retries": {
+    "strategy": "backoff",
+    "backoff": {
+      "initialInterval": 500,
+      "maxInterval": 60000,
+      "maxElapsedTime": 3600000,
+      "exponent": 1.5
+    },
+    "statusCodes": [
+      "5XX",
+      "429"
+    ],
+    "retryConnectionErrors": true
+  },
+  "tags": [
+    {
+      "name": "Staff Members",
+      "description": "Operations related to Staff Members"
+    }
+  ]
+}
+```

docs/servicem8-dev-inspector.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=ServiceM8 FastAPI Inspector
+After=network.target
+
+[Service]
+User=michael
+Group=michael
+WorkingDirectory=/opt/webhooks
+Environment="WEBHOOK_DB_PATH=/opt/webhooks/servicem8_webhooks.db"
+Environment="WEBHOOK_INSPECTOR_HOST=0.0.0.0"
+Environment="WEBHOOK_INSPECTOR_PORT=18355"
+ExecStart=/opt/webhooks/bin/uvicorn servicem8_inspector:app --host 0.0.0.0 --port 18355
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target

docs/servicem8-dev-webhook.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=ServiceM8 FastAPI Webhook Receiver
+After=network.target
+
+[Service]
+User=michael
+Group=michael
+WorkingDirectory=/opt/webhooks
+Environment="WEBHOOK_HOST=0.0.0.0"
+Environment="WEBHOOK_PORT=18354"
+Environment="WEBHOOK_DB_PATH=/opt/webhooks/servicem8_webhooks.db"
+ExecStart=/opt/webhooks/bin/uvicorn servicem8_webhook_receiver:app --host 0.0.0.0 --port 18354
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target

poll_and_apply_quote_templates.sh

@@ -0,0 +1,186 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Poll ServiceM8 form responses, parse Quote Template responses, then apply any
+# newly/unapplied parsed quote responses to ServiceM8 jobMaterials.
+#
+# Default window: last 24 hours from script start, in local system time.
+# Override with:
+#   --since 'YYYY-MM-DD HH:MM:SS'
+#   --hours 48
+#
+# By default this wrapper applies unapplied parsed responses. Use --dry-run to
+# run the poll and preview each pending apply without writing to ServiceM8.
+# The apply script still refuses duplicate applies unless --force is explicitly
+# passed through. It also checks ServiceM8 for existing jobMaterial rows on
+# the target job and blocks the apply unless --force-remote-existing is passed.
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+POLL_SCRIPT="$SCRIPT_DIR/poll_form_responses_since.py"
+APPLY_SCRIPT="$SCRIPT_DIR/apply_polled_quote_template_jobmaterials.py"
+DB_PATH="${WEBHOOK_POLL_DB_PATH:-$SCRIPT_DIR/servicem8_formresponse_poll.db}"
+LOG_DIR="${WEBHOOK_RUN_LOG_DIR:-$SCRIPT_DIR/logs}"
+QUOTE_TEMPLATE_FORM_UUID="${SERVICEM8_QUOTE_TEMPLATE_FORM_UUID:-3621b6be-1d19-4756-9ab4-9d5e4120f6d9}"
+
+SINCE=""
+HOURS="24"
+FORCE="0"
+FORCE_REMOTE_EXISTING="0"
+RECHECK_REMOTE_EXISTING="0"
+DRY_RUN="0"
+
+usage() {
+  cat <<EOF
+Usage: $0 [--since 'YYYY-MM-DD HH:MM:SS'] [--hours N] [--dry-run] [--force] [--force-remote-existing] [--recheck-remote-existing]
+
+Examples:
+  $0
+  $0 --hours 48
+  $0 --since '2026-05-04 08:00:00'
+  $0 --dry-run --hours 48
+
+This will:
+  1. Poll /formresponse.json using timestamp gt SINCE
+  2. Store/parse Quote Template responses into $DB_PATH
+  3. Apply parsed responses that do not already have generated materials recorded
+
+With --dry-run, step 3 previews the ServiceM8 jobMaterial payloads only; it does
+not write to ServiceM8 or mark responses as applied.
+
+Safety:
+  --force keeps the existing local duplicate override behaviour.
+  --force-remote-existing allows creating rows even when the remote ServiceM8
+    job already has jobMaterial rows; the incident is still recorded.
+  --recheck-remote-existing revisits rows previously blocked because remote
+    jobMaterial rows existed. If the remote rows are gone, the apply can proceed.
+EOF
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --since)
+      SINCE="${2:-}"
+      shift 2
+      ;;
+    --hours)
+      HOURS="${2:-}"
+      shift 2
+      ;;
+    --force)
+      FORCE="1"
+      shift
+      ;;
+    --force-remote-existing)
+      FORCE_REMOTE_EXISTING="1"
+      shift
+      ;;
+    --recheck-remote-existing)
+      RECHECK_REMOTE_EXISTING="1"
+      shift
+      ;;
+    --dry-run)
+      DRY_RUN="1"
+      shift
+      ;;
+    -h|--help)
+      usage
+      exit 0
+      ;;
+    *)
+      echo "Unknown argument: $1" >&2
+      usage >&2
+      exit 2
+      ;;
+  esac
+done
+
+if [[ -z "$SINCE" ]]; then
+  SINCE="$(date -d "-${HOURS} hours" '+%Y-%m-%d %H:%M:%S')"
+fi
+
+mkdir -p "$LOG_DIR"
+RUN_ID="$(date '+%Y%m%d-%H%M%S')"
+LOG_FILE="$LOG_DIR/poll-and-apply-$RUN_ID.log"
+
+exec > >(tee -a "$LOG_FILE") 2>&1
+
+echo "== ServiceM8 Quote Template poll/apply run =="
+echo "Started: $(date --iso-8601=seconds)"
+echo "Since:   $SINCE"
+echo "Mode:    $([[ "$DRY_RUN" == "1" ]] && echo "dry-run" || echo "apply")"
+echo "Remote existing: $([[ "$FORCE_REMOTE_EXISTING" == "1" ]] && echo "force" || ([[ "$RECHECK_REMOTE_EXISTING" == "1" ]] && echo "recheck" || echo "block"))"
+echo "DB:      $DB_PATH"
+echo "Log:     $LOG_FILE"
+echo
+
+echo "== Polling form responses =="
+"$POLL_SCRIPT" --since "$SINCE" --summary-limit 20
+
+echo
+echo "== Finding unapplied parsed Quote Template responses =="
+mapfile -t FORM_RESPONSE_UUIDS < <(
+  python3 - "$DB_PATH" "$QUOTE_TEMPLATE_FORM_UUID" "$RECHECK_REMOTE_EXISTING" "$FORCE_REMOTE_EXISTING" <<'PY'
+import sqlite3
+import sys
+
+poll_db, quote_form_uuid, recheck_remote_existing, force_remote_existing = sys.argv[1:5]
+include_remote_blocked = recheck_remote_existing == "1" or force_remote_existing == "1"
+status_clause = "coalesce(q.process_status, '') != 'applied'"
+if not include_remote_blocked:
+    status_clause += " and coalesce(q.process_status, '') != 'blocked_remote_existing'"
+
+conn = sqlite3.connect(poll_db)
+conn.row_factory = sqlite3.Row
+rows = conn.execute(
+    f"""
+    select q.form_response_uuid
+    from quote_template_form_responses q
+    left join form_responses_raw r on r.uuid = q.form_response_uuid
+    where q.form_uuid = ?
+      and {status_clause}
+    order by coalesce(r.timestamp, q.discovered_at) asc, q.form_response_uuid asc
+    """,
+    (quote_form_uuid,),
+).fetchall()
+for row in rows:
+    print(row["form_response_uuid"])
+PY
+)
+
+if [[ ${#FORM_RESPONSE_UUIDS[@]} -eq 0 ]]; then
+  echo "No unapplied Quote Template responses found."
+  echo "Finished: $(date --iso-8601=seconds)"
+  exit 0
+fi
+
+printf 'Found %d unapplied Quote Template response(s):\n' "${#FORM_RESPONSE_UUIDS[@]}"
+printf ' - %s\n' "${FORM_RESPONSE_UUIDS[@]}"
+
+echo
+if [[ "$DRY_RUN" == "1" ]]; then
+  echo "== Dry-run preview only; no ServiceM8 writes =="
+  APPLY_ARGS=(--pretty)
+else
+  echo "== Applying to ServiceM8 =="
+  APPLY_ARGS=(--apply --pretty)
+fi
+if [[ "$FORCE" == "1" ]]; then
+  APPLY_ARGS+=(--force)
+fi
+if [[ "$FORCE_REMOTE_EXISTING" == "1" ]]; then
+  APPLY_ARGS+=(--force-remote-existing)
+fi
+
+for uuid in "${FORM_RESPONSE_UUIDS[@]}"; do
+  echo
+  if [[ "$DRY_RUN" == "1" ]]; then
+    echo "-- Dry-run form_response_uuid=$uuid --"
+  else
+    echo "-- Applying form_response_uuid=$uuid --"
+  fi
+  "$APPLY_SCRIPT" --uuid "$uuid" "${APPLY_ARGS[@]}"
+done
+
+echo
+echo "Finished: $(date --iso-8601=seconds)"
+echo "Log: $LOG_FILE"

poll_form_responses_since.py

@@ -144,6 +144,21 @@ def init_db(db_path: Path) -> None:
             )
             """
         )
+        conn.execute(
+            """
+            CREATE TABLE IF NOT EXISTS job_metadata (
+                job_uuid TEXT PRIMARY KEY,
+                generated_job_id TEXT,
+                job_address TEXT,
+                company_name TEXT,
+                raw_json TEXT NOT NULL,
+                first_seen_at TEXT NOT NULL,
+                last_seen_at TEXT NOT NULL,
+                source TEXT NOT NULL
+            )
+            """
+        )
+        conn.execute("CREATE INDEX IF NOT EXISTS idx_job_metadata_generated_job_id ON job_metadata(generated_job_id)")
         conn.commit()
 
 
@@ -205,6 +220,109 @@ def fetch_form_responses(
     return response.status_code, data, filter_expr
 
 
+def retrieve_job(
+    *,
+    api_key: str,
+    base_url: str,
+    job_uuid: str,
+    timeout: int,
+) -> Dict[str, Any]:
+    response = requests.get(
+        f"{base_url.rstrip('/')}/job/{job_uuid}.json",
+        headers={"X-Api-Key": api_key, "Accept": "application/json"},
+        timeout=timeout,
+    )
+    if not response.ok:
+        raise RuntimeError(f"Job retrieve failed for {job_uuid}: HTTP {response.status_code}: {response.text[:1000]}")
+    data = response.json()
+    if not isinstance(data, dict):
+        raise RuntimeError(f"Job retrieve expected object response, got {type(data).__name__}")
+    return data
+
+
+def clean_text(value: Any) -> str:
+    if value is None:
+        return ""
+    return str(value).replace("\r\n", "\n").replace("\r", "\n").strip()
+
+
+def first_text(*values: Any) -> str:
+    for value in values:
+        text = clean_text(value)
+        if text:
+            return text
+    return ""
+
+
+def format_job_address(job: Dict[str, Any]) -> str:
+    direct = first_text(
+        job.get("job_address"),
+        job.get("site_address"),
+        job.get("address"),
+        job.get("location_address"),
+        job.get("billing_address"),
+    )
+    if direct:
+        return direct
+
+    parts = [
+        first_text(job.get("street"), job.get("street_address"), job.get("address_1"), job.get("address1")),
+        first_text(job.get("suburb"), job.get("city")),
+        first_text(job.get("state")),
+        first_text(job.get("postcode"), job.get("postal_code"), job.get("zip")),
+    ]
+    return " ".join(part for part in parts if part)
+
+
+def extract_company_name(job: Dict[str, Any]) -> str:
+    related = job.get("related")
+    if isinstance(related, dict):
+        company = related.get("company")
+        if isinstance(company, dict):
+            company_name = clean_text(company.get("name"))
+            if company_name:
+                return company_name
+    company = job.get("company")
+    if isinstance(company, dict):
+        company_name = clean_text(company.get("name"))
+        if company_name:
+            return company_name
+    return first_text(job.get("company_name"), job.get("customer_name"))
+
+
+def upsert_job_metadata(conn: sqlite3.Connection, *, job_uuid: str, job: Dict[str, Any], now: str, source: str) -> None:
+    job_uuid = clean_text(job_uuid or job.get("uuid"))
+    if not job_uuid:
+        return
+
+    values = (
+        job_uuid,
+        clean_text(job.get("generated_job_id")),
+        format_job_address(job),
+        extract_company_name(job),
+        json.dumps(job, ensure_ascii=False, sort_keys=True),
+        now,
+        now,
+        source,
+    )
+    conn.execute(
+        """
+        INSERT INTO job_metadata (
+            job_uuid, generated_job_id, job_address, company_name, raw_json,
+            first_seen_at, last_seen_at, source
+        ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+        ON CONFLICT(job_uuid) DO UPDATE SET
+            generated_job_id = excluded.generated_job_id,
+            job_address = excluded.job_address,
+            company_name = excluded.company_name,
+            raw_json = excluded.raw_json,
+            last_seen_at = excluded.last_seen_at,
+            source = excluded.source
+        """,
+        values,
+    )
+
+
 def insert_or_update_raw(
     conn: sqlite3.Connection,
     row: Dict[str, Any],
@@ -457,6 +575,7 @@ def main() -> int:
 
         inserted = updated = quote_matches = newly_queued = 0
         now = utc_now()
+        fetched_job_uuids = set()
         if conn is not None:
             for row in rows:
                 was_inserted, is_quote = insert_or_update_raw(
@@ -469,6 +588,26 @@ def main() -> int:
                 updated += 0 if was_inserted else 1
                 if is_quote:
                     quote_matches += 1
+                    job_uuid = clean_text(row.get("regarding_object_uuid"))
+                    if job_uuid and job_uuid not in fetched_job_uuids:
+                        try:
+                            job = retrieve_job(
+                                api_key=api_key,
+                                base_url=args.base_url,
+                                job_uuid=job_uuid,
+                                timeout=args.timeout,
+                            )
+                            upsert_job_metadata(conn, job_uuid=job_uuid, job=job, now=now, source="formresponse_poll")
+                            fetched_job_uuids.add(job_uuid)
+                        except Exception as exc:
+                            # Polling/parsing should still proceed if job metadata enrichment fails.
+                            print(
+                                json.dumps(
+                                    {"warning": "job_metadata_fetch_failed", "job_uuid": job_uuid, "error": str(exc)},
+                                    ensure_ascii=False,
+                                ),
+                                file=sys.stderr,
+                            )
                     if parse_and_store_quote_response(
                         conn,
                         row,

pyvenv.cfg

@@ -1,5 +0,0 @@
-home = /usr/bin
-include-system-site-packages = false
-version = 3.12.3
-executable = /usr/bin/python3.12
-command = /usr/bin/python3 -m venv /opt/webhooks

servicem8-create-webhook-form-response.py

@@ -7,8 +7,8 @@ BASE_URL = os.getenv('SERVICEM8_BASE_URL', 'https://api.servicem8.com')
 ENDPOINT = '/webhook_subscriptions/event'
 ACCESS_TOKEN = os.getenv('SERVICEM8_ACCESS_TOKEN', 'smk-ac525b-99c4b96305a49c7c-fe4dd3e705b647ea')
 EVENT_NAME = os.getenv('SERVICEM8_EVENT', 'form.response_created')
-CALLBACK_URL = os.getenv('SERVICEM8_CALLBACK_URL', 'https://nps-dev.coast2cloud.net/webhooks/servicem8/form-response')
+CALLBACK_URL = os.getenv('SERVICEM8_CALLBACK_URL', 'https://webhook.naroomaplumbing.au/webhooks/servicem8/form-response')
-UNIQUE_ID = os.getenv('SERVICEM8_UNIQUE_ID', 'dev-form-response')
+UNIQUE_ID = os.getenv('SERVICEM8_UNIQUE_ID', 'au-dev-form-response')
 
 
 def pretty_print_json(data):

servicem8-create-webhook-job-object.py

@@ -8,8 +8,8 @@ ENDPOINT = '/webhook_subscriptions'
 ACCESS_TOKEN = os.getenv('SERVICEM8_ACCESS_TOKEN', 'smk-ac525b-99c4b96305a49c7c-fe4dd3e705b647ea')
 OBJECT_NAME = os.getenv('SERVICEM8_OBJECT', 'job')
 FIELDS = os.getenv('SERVICEM8_FIELDS', 'uuid,status,date,queue_uuid,queue_expiry_date,work_order_date,active,edit_date,generated_job_id')
-CALLBACK_URL = os.getenv('SERVICEM8_CALLBACK_URL', 'https://nps-dev.coast2cloud.net/webhooks/servicem8-object')
+CALLBACK_URL = os.getenv('SERVICEM8_CALLBACK_URL', 'https://webhook.naroomaplumbing.au/webhooks/servicem8-object')
-UNIQUE_ID = os.getenv('SERVICEM8_UNIQUE_ID', 'dev-job-object')
+UNIQUE_ID = os.getenv('SERVICEM8_UNIQUE_ID', 'au-dev-job-object')
 
 
 def pretty_print_json(data):

servicem8-create-webhook-job-updated.py

@@ -5,9 +5,9 @@ import requests
 
 BASE_URL = os.getenv('SERVICEM8_BASE_URL', 'https://api.servicem8.com')
 ENDPOINT = '/webhook_subscriptions/event'
-ACCESS_TOKEN = os.getenv('SERVICEM8_ACCESS_TOKEN', '')
+ACCESS_TOKEN = os.getenv('SERVICEM8_ACCESS_TOKEN', 'smk-ac525b-99c4b96305a49c7c-fe4dd3e705b647ea')
 EVENT_NAME = os.getenv('SERVICEM8_EVENT', 'job.updated')
-CALLBACK_URL = os.getenv('SERVICEM8_CALLBACK_URL', 'https://nps-dev.coast2cloud.net/webhooks/servicem8-job-updated')
+CALLBACK_URL = os.getenv('SERVICEM8_CALLBACK_URL', 'https://webhook.naroomaplumbing.au/webhooks/servicem8-job-updated')
 UNIQUE_ID = os.getenv('SERVICEM8_UNIQUE_ID', 'dev-job-updated')
 
 

servicem8_inspector.py

@@ -47,6 +47,7 @@ def html_page(title: str, body: str) -> HTMLResponse:
       <a href='/poll/form-responses'>Polled form responses</a>
       <a href='/poll/quote-template'>Polled quote templates</a>
       <a href='/poll/apply-runs'>Apply runs</a>
+      <a href='/poll/remote-existing-incidents'>Remote existing incidents</a>
       <a href='/generated-materials'>Generated materials</a>
     </nav>
     """
@@ -67,6 +68,7 @@ def html_page(title: str, body: str) -> HTMLResponse:
       code, pre { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
       pre { white-space: pre-wrap; word-break: break-word; background: #0f172a; color: #e2e8f0; padding: 14px; border-radius: 10px; overflow-x: auto; }
       .pill { display: inline-block; padding: 2px 8px; border-radius: 999px; background: #e0f2fe; color: #075985; font-size: 0.85rem; margin: 2px 4px 2px 0; }
+      .job-id { font-weight: 700; color: #111827; }
       .section { margin: 24px 0; }
       .toolbar { background: white; border: 1px solid #e5e7eb; border-radius: 10px; padding: 12px; margin-bottom: 16px; }
       input[type='text'] { padding: 8px; width: 280px; max-width: 100%; }
@@ -92,6 +94,14 @@ def pretty_json(value):
         return str(value)
 
 
+def html_value(value):
+    if value is None:
+        return ""
+    if isinstance(value, (dict, list)):
+        return pretty_json(value)
+    return str(value)
+
+
 def parse_json_field(value, default=None):
     if value is None:
         return default
@@ -130,6 +140,63 @@ def link_with_params(path, **params):
     return f"{path}?{urlencode(filtered)}" if filtered else path
 
 
+def resolve_generated_job_id(job_uuid: str) -> str:
+    job_uuid = str(job_uuid or "").strip()
+    if not job_uuid:
+        return ""
+
+    try:
+        with closing(get_poll_conn()) as conn:
+            row = conn.execute(
+                "select generated_job_id from job_metadata where job_uuid = ?",
+                (job_uuid,),
+            ).fetchone()
+            if row and row["generated_job_id"]:
+                return str(row["generated_job_id"])
+    except sqlite3.Error:
+        pass
+
+    try:
+        with closing(get_conn()) as conn:
+            row = conn.execute(
+                """
+                with jobs as (
+                    select
+                        json_extract(payload_json, '$.data.uuid') as job_uuid,
+                        json_extract(payload_json, '$.data.generated_job_id') as generated_job_id,
+                        received_at
+                    from webhook_events
+                    where json_extract(payload_json, '$.data.generated_job_id') is not null
+                    union all
+                    select
+                        json_extract(payload_json, '$.related.job.uuid') as job_uuid,
+                        json_extract(payload_json, '$.related.job.generated_job_id') as generated_job_id,
+                        received_at
+                    from webhook_form_responses
+                    where json_extract(payload_json, '$.related.job.generated_job_id') is not null
+                )
+                select generated_job_id
+                from jobs
+                where job_uuid = ?
+                  and generated_job_id is not null
+                order by received_at desc
+                limit 1
+                """,
+                (job_uuid,),
+            ).fetchone()
+            if row and row["generated_job_id"]:
+                return str(row["generated_job_id"])
+    except sqlite3.Error:
+        pass
+
+    return ""
+
+
+def job_id_html(job_uuid: str) -> str:
+    generated_job_id = resolve_generated_job_id(job_uuid)
+    return f"<span class='job-id'>{escape(generated_job_id)}</span>" if generated_job_id else ""
+
+
 @app.get("/health")
 def health():
     return {"ok": True, "db_path": DB_PATH, "state_db_path": STATE_DB_PATH, "poll_db_path": POLL_DB_PATH}
@@ -160,15 +227,22 @@ def dashboard():
         "poll_runs": 0,
         "polled_form_responses": 0,
         "polled_quote_templates": 0,
+        "remote_existing_incidents": 0,
     }
     latest_poll_run = None
     latest_polled_form = None
     latest_polled_quote = None
+    latest_remote_existing_incident = None
     try:
         with closing(get_poll_conn()) as conn:
             poll_counts["poll_runs"] = conn.execute("select count(*) from poll_runs").fetchone()[0]
             poll_counts["polled_form_responses"] = conn.execute("select count(*) from form_responses_raw").fetchone()[0]
             poll_counts["polled_quote_templates"] = conn.execute("select count(*) from quote_template_form_responses").fetchone()[0]
+            try:
+                poll_counts["remote_existing_incidents"] = conn.execute("select count(*) from quote_template_remote_existing_incidents").fetchone()[0]
+                latest_remote_existing_incident = conn.execute("select detected_at from quote_template_remote_existing_incidents order by id desc limit 1").fetchone()
+            except sqlite3.Error:
+                pass
             latest_poll_run = conn.execute("select finished_at from poll_runs order by id desc limit 1").fetchone()
             latest_polled_form = conn.execute("select timestamp from form_responses_raw order by timestamp desc limit 1").fetchone()
             latest_polled_quote = conn.execute("select discovered_at from quote_template_form_responses order by discovered_at desc limit 1").fetchone()
@@ -196,6 +270,7 @@ def dashboard():
         <div><strong>Latest poll run</strong></div><div>{escape(latest_poll_run[0] if latest_poll_run else '—')}</div>
         <div><strong>Latest polled form timestamp</strong></div><div>{escape(latest_polled_form[0] if latest_polled_form else '—')}</div>
         <div><strong>Latest polled quote discovered</strong></div><div>{escape(latest_polled_quote[0] if latest_polled_quote else '—')}</div>
+        <div><strong>Latest remote-existing incident</strong></div><div>{escape(latest_remote_existing_incident[0] if latest_remote_existing_incident else '—')}</div>
       </div>
     </div>
     <div class='section'>
@@ -208,6 +283,7 @@ def dashboard():
         <li><a href='/poll/quote-template'>Browse parsed polled Quote Template responses</a></li>
         <li><a href='/poll/runs'>Browse poll runs</a></li>
         <li><a href='/poll/apply-runs'>Browse dry-run/apply runs</a></li>
+        <li><a href='/poll/remote-existing-incidents'>Browse remote-existing incidents</a></li>
         <li><a href='/generated-materials'>Browse generated job-material state</a></li>
       </ul>
     </div>
@@ -441,6 +517,7 @@ def list_generated_materials(page: int = Query(1, ge=1)):
         table_rows.append(
             f"<tr>"
             f"<td><a href='/generated-materials/{row['id']}'>{row['id']}</a></td>"
+            f"<td>{job_id_html(row['job_uuid'])}</td>"
             f"<td>{escape(row['job_uuid'] or '')}</td>"
             f"<td>{escape(row['form_response_uuid'] or '')}</td>"
             f"<td>{escape(row['job_material_uuid'] or '')}</td>"
@@ -452,8 +529,8 @@ def list_generated_materials(page: int = Query(1, ge=1)):
 
     body = f"""
     <table>
-      <thead><tr><th>ID</th><th>Job UUID</th><th>Form response UUID</th><th>Job material UUID</th><th>Kind</th><th>Source question</th><th>Updated</th></tr></thead>
+      <thead><tr><th>ID</th><th>Job ID</th><th>Job UUID</th><th>Form response UUID</th><th>Job material UUID</th><th>Kind</th><th>Source question</th><th>Updated</th></tr></thead>
-      <tbody>{''.join(table_rows) or "<tr><td colspan='7'>No rows found.</td></tr>"}</tbody>
+      <tbody>{''.join(table_rows) or "<tr><td colspan='8'>No rows found.</td></tr>"}</tbody>
     </table>
     <div class='pagination'>
       {f"<a href='{link_with_params('/generated-materials', page=page-1)}'>← Prev</a>" if page > 1 else ''}
@@ -477,6 +554,7 @@ def generated_material_detail(row_id: int):
     body = f"""
     <div class='card summary-grid'>
       <div><strong>ID</strong></div><div>{row['id']}</div>
+      <div><strong>Job ID</strong></div><div>{job_id_html(row['job_uuid'])}</div>
       <div><strong>Job UUID</strong></div><div>{escape(row['job_uuid'] or '')}</div>
       <div><strong>Form response UUID</strong></div><div>{escape(row['form_response_uuid'] or '')}</div>
       <div><strong>Job material UUID</strong></div><div>{escape(row['job_material_uuid'] or '')}</div>
@@ -510,10 +588,10 @@ def form_response_detail(row_id: int):
     for item in sorted(field_data, key=lambda x: x.get("SortOrder", 0)):
         field_rows.append(
             f"<tr>"
-            f"<td>{escape(str(item.get('SortOrder', '')))}</td>"
+            f"<td>{escape(html_value(item.get('SortOrder')))}</td>"
-            f"<td>{escape(item.get('FieldType', ''))}</td>"
+            f"<td>{escape(html_value(item.get('FieldType')))}</td>"
-            f"<td>{escape(item.get('Question', ''))}</td>"
+            f"<td>{escape(html_value(item.get('Question')))}</td>"
-            f"<td>{escape(item.get('Response', ''))}</td>"
+            f"<td>{escape(html_value(item.get('Response')))}</td>"
             f"</tr>"
         )
 
@@ -542,6 +620,91 @@ def form_response_detail(row_id: int):
     return html_page(f"Form response {row_id}", body)
 
 
+@app.get("/poll/remote-existing-incidents", response_class=HTMLResponse)
+def list_remote_existing_incidents(page: int = Query(1, ge=1)):
+    offset = (page - 1) * PAGE_SIZE
+    try:
+        with closing(get_poll_conn()) as conn:
+            rows = conn.execute(
+                """
+                select id, detected_at, form_response_uuid, job_uuid, apply_run_id,
+                       desired_count, remote_count, remote_active_count, action, reason
+                from quote_template_remote_existing_incidents
+                order by id desc
+                limit ? offset ?
+                """,
+                (PAGE_SIZE, offset),
+            ).fetchall()
+    except sqlite3.Error as e:
+        return html_page("Remote existing incidents", f"<div class='card'>Incident table unavailable: {escape(str(e))}</div>")
+
+    table_rows = []
+    for row in rows:
+        run_link = f"<a href='/poll/apply-runs/{row['apply_run_id']}'>{row['apply_run_id']}</a>" if row['apply_run_id'] else ""
+        table_rows.append(
+            f"<tr><td><a href='/poll/remote-existing-incidents/{row['id']}'>{row['id']}</a></td>"
+            f"<td>{escape(row['detected_at'] or '')}</td><td>{escape(row['action'] or '')}</td>"
+            f"<td><a href='/poll/quote-template/{escape(row['form_response_uuid'])}'>{escape(row['form_response_uuid'])}</a></td>"
+            f"<td>{job_id_html(row['job_uuid'])}</td><td>{escape(row['job_uuid'] or '')}</td><td>{run_link}</td>"
+            f"<td>{row['desired_count']}</td><td>{row['remote_count']}</td><td>{row['remote_active_count']}</td>"
+            f"<td>{escape((row['reason'] or '')[:180])}</td></tr>"
+        )
+
+    body = f"""
+    <table>
+      <thead><tr><th>ID</th><th>Detected</th><th>Action</th><th>Form response UUID</th><th>Job ID</th><th>Job UUID</th><th>Apply run</th><th>Desired</th><th>Remote rows</th><th>Active</th><th>Reason</th></tr></thead>
+      <tbody>{''.join(table_rows) or "<tr><td colspan='11'>No remote-existing incidents found.</td></tr>"}</tbody>
+    </table>
+    <div class='pagination'>
+      {f"<a href='{link_with_params('/poll/remote-existing-incidents', page=page-1)}'>← Prev</a>" if page > 1 else ''}
+      <a href='{link_with_params('/poll/remote-existing-incidents', page=page+1)}'>Next →</a>
+    </div>
+    """
+    return html_page("Remote existing incidents", body)
+
+
+@app.get("/poll/remote-existing-incidents/{incident_id}", response_class=HTMLResponse)
+def remote_existing_incident_detail(incident_id: int):
+    try:
+        with closing(get_poll_conn()) as conn:
+            row = conn.execute("select * from quote_template_remote_existing_incidents where id = ?", (incident_id,)).fetchone()
+    except sqlite3.Error as e:
+        return html_page("Remote existing incident", f"<div class='card'>Incident table unavailable: {escape(str(e))}</div>")
+    if not row:
+        raise HTTPException(status_code=404, detail="Remote existing incident not found")
+
+    remote_rows = parse_json_field(row["remote_rows_json"], []) or []
+    material_rows = []
+    if isinstance(remote_rows, list):
+        for item in remote_rows:
+            if isinstance(item, dict):
+                material_rows.append(
+                    f"<tr><td>{escape(str(item.get('uuid', '')))}</td><td>{escape(str(item.get('active', '')))}</td>"
+                    f"<td>{escape(str(item.get('name', '')))}</td><td>{escape(str(item.get('material_uuid', '')))}</td>"
+                    f"<td>{escape(str(item.get('quantity', '')))}</td><td>{escape(str(item.get('price', '')))}</td>"
+                    f"<td>{escape(str(item.get('sort_order', '')))}</td></tr>"
+                )
+    run_link = f"<a href='/poll/apply-runs/{row['apply_run_id']}'>{row['apply_run_id']}</a>" if row['apply_run_id'] else ""
+    body = f"""
+    <div class='card summary-grid'>
+      <div><strong>ID</strong></div><div>{row['id']}</div>
+      <div><strong>Detected</strong></div><div>{escape(row['detected_at'] or '')}</div>
+      <div><strong>Action</strong></div><div>{escape(row['action'] or '')}</div>
+      <div><strong>Form response UUID</strong></div><div><a href='/poll/quote-template/{escape(row['form_response_uuid'])}'>{escape(row['form_response_uuid'])}</a></div>
+      <div><strong>Job ID</strong></div><div>{job_id_html(row['job_uuid'])}</div>
+      <div><strong>Job UUID</strong></div><div>{escape(row['job_uuid'] or '')}</div>
+      <div><strong>Apply run</strong></div><div>{run_link}</div>
+      <div><strong>Desired rows</strong></div><div>{row['desired_count']}</div>
+      <div><strong>Remote rows</strong></div><div>{row['remote_count']}</div>
+      <div><strong>Remote active rows</strong></div><div>{row['remote_active_count']}</div>
+      <div><strong>Reason</strong></div><div>{escape(row['reason'] or '')}</div>
+    </div>
+    <div class='section'><h2>Remote jobMaterial rows</h2><table><thead><tr><th>UUID</th><th>Active</th><th>Name</th><th>Material UUID</th><th>Qty</th><th>Price</th><th>Sort</th></tr></thead><tbody>{''.join(material_rows) or "<tr><td colspan='7'>No remote rows captured.</td></tr>"}</tbody></table></div>
+    <div class='section'><h2>Raw remote rows JSON</h2><pre>{escape(pretty_json(remote_rows))}</pre></div>
+    """
+    return html_page(f"Remote existing incident {incident_id}", body)
+
+
 @app.get("/poll/apply-runs", response_class=HTMLResponse)
 def list_apply_runs(page: int = Query(1, ge=1)):
     offset = (page - 1) * PAGE_SIZE
@@ -566,14 +729,14 @@ def list_apply_runs(page: int = Query(1, ge=1)):
             f"<tr><td><a href='/poll/apply-runs/{row['id']}'>{row['id']}</a></td>"
             f"<td>{escape(row['mode'] or '')}</td><td>{escape(row['status'] or '')}</td>"
             f"<td><a href='/poll/quote-template/{escape(row['form_response_uuid'])}'>{escape(row['form_response_uuid'])}</a></td>"
-            f"<td>{escape(row['job_uuid'] or '')}</td><td>{escape(row['started_at'] or '')}</td><td>{escape(row['finished_at'] or '')}</td>"
+            f"<td>{job_id_html(row['job_uuid'])}</td><td>{escape(row['job_uuid'] or '')}</td><td>{escape(row['started_at'] or '')}</td><td>{escape(row['finished_at'] or '')}</td>"
             f"<td>{row['desired_count']}</td><td>{row['created_count']}</td><td>{escape((row['error'] or '')[:160])}</td></tr>"
         )
 
     body = f"""
     <table>
-      <thead><tr><th>ID</th><th>Mode</th><th>Status</th><th>Form response UUID</th><th>Job UUID</th><th>Started</th><th>Finished</th><th>Desired</th><th>Created</th><th>Error</th></tr></thead>
+      <thead><tr><th>ID</th><th>Mode</th><th>Status</th><th>Form response UUID</th><th>Job ID</th><th>Job UUID</th><th>Started</th><th>Finished</th><th>Desired</th><th>Created</th><th>Error</th></tr></thead>
-      <tbody>{''.join(table_rows) or "<tr><td colspan='10'>No apply runs found yet.</td></tr>"}</tbody>
+      <tbody>{''.join(table_rows) or "<tr><td colspan='11'>No apply runs found yet.</td></tr>"}</tbody>
     </table>
     <div class='pagination'>
       {f"<a href='{link_with_params('/poll/apply-runs', page=page-1)}'>← Prev</a>" if page > 1 else ''}
@@ -592,11 +755,23 @@ def apply_run_detail(run_id: int):
                 "select * from quote_template_apply_run_rows where run_id = ? order by row_index asc",
                 (run_id,),
             ).fetchall()
+            incidents = conn.execute(
+                "select id, detected_at, action, remote_count, remote_active_count, reason from quote_template_remote_existing_incidents where apply_run_id = ? order by id desc",
+                (run_id,),
+            ).fetchall()
     except sqlite3.Error as e:
         return html_page("Apply run", f"<div class='card'>Apply-run table unavailable: {escape(str(e))}</div>")
     if not run:
         raise HTTPException(status_code=404, detail="Apply run not found")
 
+    incident_rows = []
+    for incident in incidents:
+        incident_rows.append(
+            f"<tr><td><a href='/poll/remote-existing-incidents/{incident['id']}'>{incident['id']}</a></td>"
+            f"<td>{escape(incident['detected_at'] or '')}</td><td>{escape(incident['action'] or '')}</td>"
+            f"<td>{incident['remote_count']}</td><td>{incident['remote_active_count']}</td><td>{escape((incident['reason'] or '')[:180])}</td></tr>"
+        )
+
     table_rows = []
     for row in rows:
         payload = parse_json_field(row['api_payload_json'], {}) or {}
@@ -613,6 +788,7 @@ def apply_run_detail(run_id: int):
       <div><strong>Mode</strong></div><div>{escape(run['mode'] or '')}</div>
       <div><strong>Status</strong></div><div>{escape(run['status'] or '')}</div>
       <div><strong>Form response UUID</strong></div><div><a href='/poll/quote-template/{escape(run['form_response_uuid'])}'>{escape(run['form_response_uuid'])}</a></div>
+      <div><strong>Job ID</strong></div><div>{job_id_html(run['job_uuid'])}</div>
       <div><strong>Job UUID</strong></div><div>{escape(run['job_uuid'] or '')}</div>
       <div><strong>Started</strong></div><div>{escape(run['started_at'] or '')}</div>
       <div><strong>Finished</strong></div><div>{escape(run['finished_at'] or '')}</div>
@@ -620,6 +796,7 @@ def apply_run_detail(run_id: int):
       <div><strong>Created</strong></div><div>{run['created_count']}</div>
       <div><strong>Error</strong></div><div>{escape(run['error'] or '')}</div>
     </div>
+    <div class='section'><h2>Remote-existing incidents</h2><table><thead><tr><th>ID</th><th>Detected</th><th>Action</th><th>Remote rows</th><th>Active</th><th>Reason</th></tr></thead><tbody>{''.join(incident_rows) or "<tr><td colspan='6'>No remote-existing incidents for this run.</td></tr>"}</tbody></table></div>
     <div class='section'><h2>Rows</h2><table><thead><tr><th>#</th><th>Action</th><th>Kind</th><th>Name</th><th>Created UUID</th><th>Source question</th><th>Error</th></tr></thead><tbody>{''.join(table_rows) or "<tr><td colspan='7'>No rows recorded.</td></tr>"}</tbody></table></div>
     """
     return html_page(f"Apply run {run_id}", body)
@@ -701,7 +878,7 @@ def list_polled_form_responses(q: str = Query(""), quote_only: int = Query(0), p
             f"<tr><td><a href='/poll/form-responses/{escape(row['uuid'])}'>{escape(row['uuid'])}</a></td>"
             f"<td>{escape(row['timestamp'] or '')}</td><td>{escape(row['edit_date'] or '')}</td>"
             f"<td>{escape(row['form_uuid'] or '')}<br>{quote_pill}</td>"
-            f"<td>{escape(row['regarding_object'] or '')}</td><td>{escape(row['regarding_object_uuid'] or '')}</td>"
+            f"<td>{escape(row['regarding_object'] or '')}</td><td>{job_id_html(row['regarding_object_uuid'])}</td><td>{escape(row['regarding_object_uuid'] or '')}</td>"
             f"<td>{escape(row['parse_status'] or '')}</td><td>{row['seen_count']}</td><td>{escape(row['last_seen_at'] or '')}</td></tr>"
         )
 
@@ -715,8 +892,8 @@ def list_polled_form_responses(q: str = Query(""), quote_only: int = Query(0), p
       </form>
     </div>
     <table>
-      <thead><tr><th>UUID</th><th>Timestamp</th><th>Edit date</th><th>Form UUID</th><th>Regarding</th><th>Object UUID</th><th>Parse</th><th>Seen</th><th>Last seen</th></tr></thead>
+      <thead><tr><th>UUID</th><th>Timestamp</th><th>Edit date</th><th>Form UUID</th><th>Regarding</th><th>Job ID</th><th>Object UUID</th><th>Parse</th><th>Seen</th><th>Last seen</th></tr></thead>
-      <tbody>{''.join(table_rows) or "<tr><td colspan='9'>No rows found.</td></tr>"}</tbody>
+      <tbody>{''.join(table_rows) or "<tr><td colspan='10'>No rows found.</td></tr>"}</tbody>
     </table>
     <div class='pagination'>
       {f"<a href='{link_with_params('/poll/form-responses', q=q, quote_only=quote_only, page=page-1)}'>← Prev</a>" if page > 1 else ''}
@@ -744,8 +921,8 @@ def polled_form_response_detail(form_response_uuid: str):
         for item in sorted(field_data, key=lambda x: x.get("SortOrder", 0) if isinstance(x, dict) else 0):
             if isinstance(item, dict):
                 field_rows.append(
-                    f"<tr><td>{escape(str(item.get('SortOrder', '')))}</td><td>{escape(item.get('FieldType', ''))}</td>"
+                    f"<tr><td>{escape(html_value(item.get('SortOrder')))}</td><td>{escape(html_value(item.get('FieldType')))}</td>"
-                    f"<td>{escape(item.get('Question', ''))}</td><td>{escape(item.get('Response', ''))}</td></tr>"
+                    f"<td>{escape(html_value(item.get('Question')))}</td><td>{escape(html_value(item.get('Response')))}</td></tr>"
                 )
     quote_link = f"<div><strong>Parsed quote</strong></div><div><a href='/poll/quote-template/{escape(form_response_uuid)}'>Open parsed Quote Template view</a></div>" if quote else ""
 
@@ -756,6 +933,7 @@ def polled_form_response_detail(form_response_uuid: str):
       <div><strong>Edit date</strong></div><div>{escape(row['edit_date'] or '')}</div>
       <div><strong>Form UUID</strong></div><div>{escape(row['form_uuid'] or '')}</div>
       <div><strong>Regarding object</strong></div><div>{escape(row['regarding_object'] or '')}</div>
+      <div><strong>Job ID</strong></div><div>{job_id_html(row['regarding_object_uuid'])}</div>
       <div><strong>Regarding UUID</strong></div><div>{escape(row['regarding_object_uuid'] or '')}</div>
       <div><strong>First seen</strong></div><div>{escape(row['first_seen_at'] or '')}</div>
       <div><strong>Last seen</strong></div><div>{escape(row['last_seen_at'] or '')}</div>
@@ -796,15 +974,15 @@ def list_polled_quote_templates(page: int = Query(1, ge=1)):
             material_count = "?"
         table_rows.append(
             f"<tr><td><a href='/poll/quote-template/{escape(row['form_response_uuid'])}'>{escape(row['form_response_uuid'])}</a></td>"
-            f"<td>{escape(row['discovered_at'] or '')}</td><td>{escape(row['job_uuid'] or '')}</td>"
+            f"<td>{escape(row['discovered_at'] or '')}</td><td>{job_id_html(row['job_uuid'])}</td><td>{escape(row['job_uuid'] or '')}</td>"
             f"<td>{escape(row['description'] or '')}</td><td>{material_count}</td>"
             f"<td>{escape(row['queued_at'] or '')}</td><td>{escape(row['process_status'] or '')}</td></tr>"
         )
 
     body = f"""
     <table>
-      <thead><tr><th>Form response UUID</th><th>Discovered</th><th>Job UUID</th><th>Description</th><th>Rows</th><th>Queued</th><th>Status</th></tr></thead>
+      <thead><tr><th>Form response UUID</th><th>Discovered</th><th>Job ID</th><th>Job UUID</th><th>Description</th><th>Rows</th><th>Queued</th><th>Status</th></tr></thead>
-      <tbody>{''.join(table_rows) or "<tr><td colspan='7'>No quote template rows found.</td></tr>"}</tbody>
+      <tbody>{''.join(table_rows) or "<tr><td colspan='8'>No quote template rows found.</td></tr>"}</tbody>
     </table>
     <div class='pagination'>
       {f"<a href='{link_with_params('/poll/quote-template', page=page-1)}'>← Prev</a>" if page > 1 else ''}
@@ -842,18 +1020,29 @@ def polled_quote_template_detail(form_response_uuid: str):
                 "select id, mode, status, started_at, finished_at, desired_count, created_count, error from quote_template_apply_runs where form_response_uuid = ? order by id desc limit 8",
                 (row['form_response_uuid'],),
             ).fetchall()
+            recent_incidents = conn.execute(
+                "select id, detected_at, action, remote_count, remote_active_count, reason from quote_template_remote_existing_incidents where form_response_uuid = ? order by id desc limit 8",
+                (row['form_response_uuid'],),
+            ).fetchall()
     except sqlite3.Error:
         recent_runs = []
+        recent_incidents = []
     recent_run_rows = []
     for run in recent_runs:
         recent_run_rows.append(
             f"<tr><td><a href='/poll/apply-runs/{run['id']}'>{run['id']}</a></td><td>{escape(run['mode'] or '')}</td><td>{escape(run['status'] or '')}</td><td>{escape(run['started_at'] or '')}</td><td>{escape(run['finished_at'] or '')}</td><td>{run['desired_count']}</td><td>{run['created_count']}</td><td>{escape((run['error'] or '')[:120])}</td></tr>"
         )
+    recent_incident_rows = []
+    for incident in recent_incidents:
+        recent_incident_rows.append(
+            f"<tr><td><a href='/poll/remote-existing-incidents/{incident['id']}'>{incident['id']}</a></td><td>{escape(incident['detected_at'] or '')}</td><td>{escape(incident['action'] or '')}</td><td>{incident['remote_count']}</td><td>{incident['remote_active_count']}</td><td>{escape((incident['reason'] or '')[:120])}</td></tr>"
+        )
 
     body = f"""
     <div class='card summary-grid'>
       <div><strong>Form response UUID</strong></div><div>{escape(row['form_response_uuid'])}</div>
       <div><strong>Discovered</strong></div><div>{escape(row['discovered_at'] or '')}</div>
+      <div><strong>Job ID</strong></div><div>{job_id_html(row['job_uuid'])}</div>
       <div><strong>Job UUID</strong></div><div>{escape(row['job_uuid'] or '')}</div>
       <div><strong>Form UUID</strong></div><div>{escape(row['form_uuid'] or '')}</div>
       <div><strong>Author</strong></div><div>{escape(row['author_name'] or '')}</div>
@@ -877,6 +1066,10 @@ def polled_quote_template_detail(form_response_uuid: str):
       <h2>Recent dry-run/apply runs for this response</h2>
       <table><thead><tr><th>ID</th><th>Mode</th><th>Status</th><th>Started</th><th>Finished</th><th>Desired</th><th>Created</th><th>Error</th></tr></thead><tbody>{''.join(recent_run_rows) or "<tr><td colspan='8'>No dry-run/apply runs yet.</td></tr>"}</tbody></table>
     </div>
+    <div class='section'>
+      <h2>Remote-existing incidents for this response</h2>
+      <table><thead><tr><th>ID</th><th>Detected</th><th>Action</th><th>Remote rows</th><th>Active</th><th>Reason</th></tr></thead><tbody>{''.join(recent_incident_rows) or "<tr><td colspan='6'>No remote-existing incidents yet.</td></tr>"}</tbody></table>
+    </div>
     <div class='section'><h2>Desired jobMaterial rows</h2><table><thead><tr><th>Sort</th><th>Kind</th><th>Name</th><th>Material UUID</th><th>Source question</th></tr></thead><tbody>{''.join(material_rows) or "<tr><td colspan='5'>No desired jobMaterial rows.</td></tr>"}</tbody></table></div>
     <div class='section'><h2>Parsed JSON</h2><pre>{escape(pretty_json(parsed))}</pre></div>
     """

servicem8_quote_template_parser.py

@@ -34,7 +34,7 @@ def clean_text(value: Any) -> str:
 
 
 def get_state_conn(db_path: Path = STATE_DB_PATH):
-    conn = sqlite3.connect(db_path)
+    conn = sqlite3.connect(db_path, timeout=30)
     conn.row_factory = sqlite3.Row
     return conn