tts-workshop/docs/implementation-plan.md

# TTS Workshop — Implementation Plan

> **Scope:** The concrete build plan — files, endpoints, dependencies, build
> order. For the system's architecture and dependency graph, see
> [`architecture.md`](./architecture.md). For the rationale behind key decisions,
> see [`decisions.md`](./decisions.md).

A lightweight, client-only pnpm monorepo for searching the Tabletop Simulator
Steam Workshop, fetching full TTS save files, and analyzing their contents.

## Goals

- Search the TTS Workshop to discover item IDs (Steam has no official search API,
  so this scrapes the browse page).
- Fetch a full TTS save (`TTSMod`) for a given item ID via the Steam Web API +
  BSON deserialization.
- Extract and inspect objects and asset references from a save, in an isomorphic
  package reusable from a future frontend.
- Client-only, lightweight, no caching layer.

## Non-goals (for now)

- Backend traversal endpoints (deferred by design).
- Caching / Redis / multi-instance concerns.

## Architecture

```
apps/web ──► apps/proxy ──► packages/tts ──► packages/shared
      │            │            │
      │            │            └──► (fetchMod → TTSMod)
      │            ▼
      ├──► packages/mesh ──► packages/shared (types)
      └──► packages/extract ──► packages/shared (types)
           (isomorphic, used by the frontend)
```

- **`apps/web`** — React frontend (search + mod pages).
- **`apps/proxy`** — Hono server. Search + fetch only. No traversal endpoints.
- **`packages/tts`** — low-level fetcher: Steam API call, BSON parse, filename
  derivation. Returns raw parsed `TTSMod`.
- **`packages/extract`** — analysis: flatten/filter objects, extract asset refs,
  download assets. Isomorphic (browser + Node).
- **`packages/mesh`** — 2D shapes + extrusion into 3D geometry for the
  frontend viewers. Isomorphic (browser + Node).
- **`packages/shared`** — shared types + zod schemas.

## Repository layout

```
tts-workshop/
├── pnpm-workspace.yaml
├── package.json                # root scripts (dev, build, lint)
├── tsconfig.base.json
├── .npmrc
├── .env.example                # STEAM_API_KEY, PORT
├── docs/
│   └── implementation-plan.md  # this file
├── apps/
│   ├── proxy/
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── src/
│   │   ├── index.ts        # Hono app + @hono/node-server
│   │   ├── routes/
│   │   │   ├── search.ts   # GET /search?q=...&page=1
│   │   │   ├── items.ts    # GET /items/:id, /items/:id/file
│   │   │   ├── asset.ts    # GET /asset?url=... (CORS-safe asset proxy)
│   │   │   ├── trace.ts    # GET /trace?url=... (image → vector shape)
│   │   │   ├── svgShape.ts # parse vtracer SVG into { outline, holes }
│   │   │   └── health.ts   # GET /health
│   │   └── env.ts          # zod env validation
│   └── web/
│       ├── package.json
│       ├── tsconfig.json
│       ├── vite.config.ts      # dev proxy → localhost:3000
│       ├── index.html
│       └── src/
│           ├── main.tsx        # React root + router
│           ├── App.tsx         # layout + routes
│           ├── api.ts          # fetch wrappers for /search, /items
│           ├── index.css       # tailwind v4 entry
│           ├── pages/
│           │   ├── SearchPage.tsx
│           │   └── ModPage.tsx
│           ├── components/
│           │   ├── SearchResults.tsx
│           │   ├── ObjectTree.tsx     # containment-tree sidebar
│           │   ├── viewers.tsx        # viewer registry + default viewer
│           │   ├── viewers/           # 3D viewers (r3f/drei/postprocessing)
│           │   │   ├── register.ts    # registers per-class 3D viewers (lazy)
│           │   │   ├── Scene.tsx      # shared Canvas: lights, controls, post
│           │   │   ├── TileViewer.tsx
│           │   │   ├── TokenViewer.tsx
│           │   │   ├── CardViewer.tsx
│           │   │   ├── CustomModelViewer.tsx
│           │   │   └── assetUrl.ts    # route asset URLs through the proxy
│           │   ├── objectIcons.tsx    # class → icon mapping
│           │   ├── objectIconsData.ts # generated icon subset (do not edit)
│           │   └── objectIcons.test.ts
│           └── stores/
│               ├── searchStore.ts  # zustand
│               └── modStore.ts     # zustand
└── packages/
    ├── shared/
    │   └── src/
    │       ├── types.ts        # TTSMod, TTSObject, WorkshopItem, SearchResult
    │       └── schemas.ts
    ├── tts/
    │   └── src/
    │       ├── index.ts        # fetchMod, getFileName
    │       └── errors.ts
    ├── mesh/
    │   └── src/
    │       ├── index.ts        # public API barrel
    │       ├── types.ts        # FaceGeometry, ExtrudedGeometry, UVBounds
    │       ├── shapes.ts       # Shape + shape generators (rect, circle, ...)
    │       ├── tessellate.ts   # triangulate + cap faces
    │       ├── walls.ts        # side walls
    │       └── extrude.ts      # extrudeShape / extrudeShapeParts
    └── extract/
        ├── package.json
        ├── tsconfig.json
        └── src/
            ├── index.ts        # public API barrel
            ├── objects.ts      # flatten/filter/traverse helpers
            ├── refs.ts         # extract asset references
            ├── download.ts     # fetch + decode referenced assets
            └── types.ts        # extracted-object / asset types
```

## Packages

### `packages/shared`

Shared types and validation schemas used across the monorepo.

- `types.ts`
  - `TTSMod`, `TTSObject` (from the existing scraper code)
  - `WorkshopItem` — metadata from the Steam API (`id`, `title`, `author`,
    `previewImageUrl`, `fileUrl`, ...)
  - `SearchResult` — `{ items: WorkshopItem[], page, hasMore }`
- `schemas.ts`
  - zod schemas for env vars, query params, and response shapes.

### `packages/tts`

Low-level fetcher, extracted from the existing scraper.

- `index.ts`
  - `fetchMod(id: string): Promise<TTSMod>` — Steam API call to
    `ISteamRemoteStorage/GetPublishedFileDetails/v1` to get `file_url`, then
    download + BSON-deserialize into `TTSMod`.
  - `fetchModFromUrl(fileUrl)` — download + BSON-deserialize from a direct URL
    (no Steam API call).
  - `fetchModFile(id, apiKey)` / `fetchModFileFromUrl(fileUrl)` — raw save
    bytes + derived filename, with and without the Steam API.
  - `getFileName(url: string): Promise<string>` — derive filename from the
    `content-disposition` header.
- `errors.ts`
  - Typed errors: missing `file_url`, Steam API failure, rate limit, invalid key.
- Notes
  - Swap the browser `BSON` global for the `bson` npm package.
  - `traverseMod` / `markParent` move to `packages/extract` (traversal is
    analysis, not fetching).

### `packages/extract`

Isomorphic analysis of a parsed `TTSMod`. No Node-specific APIs.

- `objects.ts`
  - `flattenObjects(mod)` — all objects in the tree.
  - `filterObjects(mod, predicate)` — filter by name, GUID, type, etc.
  - `findObject(mod, guid)` — lookup by GUID.
  - `buildTree(mod)` — containment tree (`{ object, label, children }`),
    mirroring how TTS nests objects; `label` is `Nickname` when present,
    else the class `Name`.
  - `traverseMod` / `markParent` (moved from `tts`).
  - Returns lightweight graph shapes: `{ guid, name, type, parentGuid,
    childrenGuids, refs }`.
- `refs.ts`
  - `extractRefs(object)` — walk one object, pull every external URL:
    `CustomPDF.PDFUrl`, `CustomDeck[*].FaceURL`/`BackURL`,
    `CustomImage.ImageURL`/`ImageSecondaryURL`.
  - `collectRefs(mod)` — all refs across the save, deduped by URL.
  - `AssetRef` type: `{ kind, url, ownerGuid }`.
- `download.ts`
  - `downloadAsset(url)` — `fetch` → `Blob`.
  - `downloadAll(refs, { concurrency, onProgress })` — batched downloads with
    progress callback.
  - `guessMimeType(url)` — infer mime from extension.
- Design constraints
  - No `Buffer` — use `ArrayBuffer` / `Uint8Array` / `Blob` (Node 18+).
  - No Node-only packages (`cheerio` stays in the backend search only).
  - Pure, deterministic functions where possible.

### `packages/mesh`

2D shape + extrusion library used by the frontend viewers to build 3D geometry.

- `shapes.ts`
  - `Shape` — `{ outline, holes }`, the minimal interface the tessellator and
    wall generator need. Shape generators: `rectShape`, `polygonShape`,
    `hexShape`, `circleShape`, `roundedRectShape`, `frameShape`, plus
    `scaleShape` and `signedArea`.
- `tessellate.ts`
  - `triangulate(shape)` — earcut triangulation (same as three.js).
  - `capFaces(shape, height, uvScale, uvBounds)` — top/bottom faces. UVs map
    the shape's bounding box (or `uvBounds` framing) to the unit square; the
    bottom face uses the same planar xy mapping as the top (no mirror).
- `walls.ts`
  - `wallFaces(shape, height, uvScale, uvBounds)` — side walls with outward
    normals and planar xy UVs (z-independent).
- `extrude.ts`
  - `extrudeShape(shape, options)` — merged caps + walls as one geometry.
  - `extrudeShapeParts(shape, options)` — caps and walls as separate
    geometries (for distinct materials).
  - `ExtrudeOptions` — `height`, `capUvScale`, `wallUvScale`, `uvBounds`.
- `types.ts`
  - `FaceGeometry`, `ExtrudedGeometry`, `UVBounds`.

### `apps/proxy`

Hono server exposing search + fetch.

- `index.ts` — Hono app + `@hono/node-server` bootstrap, CORS via `@hono/cors`.
- `routes/search.ts`
  - `GET /search?q=...&page=1` — scrape
    `steamcommunity.com/workshop/browse/?appid=286160&searchtext=...` with
    `cheerio`, extract `{ id, title, author, previewImageUrl }` from the result
    grid. Supports `pagenum` pagination. No API key required.
- `routes/items.ts`
  - `GET /items/:id` — full parsed `TTSMod`. Accepts an optional `fileUrl`
    query param to download the save directly (no Steam API key needed);
    otherwise resolves via the Steam API.
  - `GET /items/:id/file` — raw save bytes, filename from `getFileName`. Also
    accepts `fileUrl`.
- `routes/asset.ts`
  - `GET /asset?url=...` — fetch an external asset (texture, model) and stream
    it back with a `Content-Type` header. Workshop hosts often omit CORS
    headers, which would block three.js loaders in the browser; routing through
    the proxy makes those assets loadable. Only `http(s)` URLs are allowed.
- `routes/trace.ts`
  - `GET /trace?url=...&mode=alpha&threshold=128&format=shape&offset=...` —
    fetch an image, trace it into a vector shape, and return the result
    BSON-encoded. `mode` is `alpha` (default), `bw`, or `color`; `format` is
    `shape` (default) or `svg`; `offset` (optional) insets (negative) or
    outsets (positive) the shape in pixels. `shape` returns a parsed
    `{ outline, holes }` polygon matching `@tts/mesh`'s `Shape` interface,
    ready to extrude.
- `routes/svgShape.ts`
  - `parseSvgShape(svg)` — parse a vtracer SVG into a `TracedShape`
    (`{ outline, holes }`): flatten beziers to polylines, split subpaths into
    rings, classify by winding (CCW outline / CW hole), and assign holes to
    their containing outline.
  - `offsetShape(shape, delta)` — inset/outset a `TracedShape` via
    `clipper-lib` (Clipper miter joins); outline and holes offset in opposite
    directions and are recombined with a boolean difference, so holes grow on
    inset and shrink on outset. Collapsed shapes return an empty outline; a
    split outline keeps the largest ring.
- `routes/health.ts`
  - `GET /health` — liveness.
- `env.ts` — zod validation of `STEAM_API_KEY`, `PORT`.

### `apps/web`

React frontend (Vite + React Router + Tailwind v4 + Zustand). Consumes the
proxy API and `packages/extract` directly for analysis.

- `main.tsx` — React root, `BrowserRouter`.
- `App.tsx` — app shell (header) + routes: `/` (search), `/mod/:id` (mod).
- `api.ts` — typed `fetch` wrappers for `/search` and `/items/:id`, plus a
  `modFileUrl` helper for the raw-file download.
- `pages/SearchPage.tsx` — search form, drives `searchStore`.
- `pages/ModPage.tsx` — loads the mod via `modStore`, uses `@tts/extract`
  (`buildTree`, `collectRefs`) to render a containment-tree sidebar and an
  inspector pane for the selected object, and links to the raw save file.
- `components/SearchResults.tsx` — result grid + pagination.
- `components/ObjectTree.tsx` — recursive tree sidebar; each entry shows a
  class icon (hover for the class name) + display label, indented by depth.
  Clicking selects an object.
- `components/viewers.tsx` — viewer registry (`registerViewer` /
  `resolveViewer`) plus a `DefaultViewer` that renders an object's fields;
  custom per-class viewers can be registered later.
- `components/viewers/` — 3D viewers built on `@react-three/fiber`,
  `@react-three/drei`, and `@react-three/postprocessing`. `register.ts`
  registers lazy-loaded viewers for `Tile`/`Custom_Tile` (flat box),
  `Custom_Token` (shape traced from the image's alpha channel via `/trace`,
  extruded with `@tts/mesh`), `Card`/`Deck`/`Custom_Deck` (thin box with
  face/back textures), and `Custom_Model`/`Custom_Model_Bag`/
  `Custom_Model_Infinite_Bag` (GLTF/OBJ/FBX from `CustomMesh.MeshURL`).
  `Scene.tsx` is a shared canvas with lighting, orbit controls, contact
  shadows, and subtle bloom/vignette. `assetUrl.ts` routes asset URLs through
  the proxy for CORS-safe loading. The viewers are lazy-loaded so the three.js
  stack is code-split out of the main bundle.
- `components/objectIcons.tsx` — maps TTS object classes to one or more
  Iconify icons (`iconsForObject`); unknown classes fall back to a help icon.
  Icons may come from multiple sets (mdi, material-symbols, file-icons, ...);
  names not bundled are fetched from the Iconify CDN at runtime.
- `components/objectIconsData.ts` — generated by
  `scripts/generate-object-icons.mjs`; registers a small mdi subset with
  Iconify's offline storage so those render without a network round-trip.
- `stores/searchStore.ts` / `stores/modStore.ts` — Zustand stores for search
  and mod state.
- `vite.config.ts` — dev proxy for `/search`, `/items`, `/health` →
  `http://localhost:3000`.
- Styling: Tailwind v4 via `@tailwindcss/vite`; `index.css` imports
  `tailwindcss`.

## Endpoints

| Method | Path                | Description                                  | Auth |
| ------ | ------------------- | -------------------------------------------- | ---- |
| GET    | `/health`           | Liveness                                     | —    |
| GET    | `/search?q=&page=`  | Scrape Workshop browse, return item list     | —    |
| GET    | `/items/:id`        | Full parsed `TTSMod` (`?fileUrl=` skips key) | key* |
| GET    | `/items/:id/file`   | Raw save bytes, filename from header         | key* |
| GET    | `/asset?url=`       | CORS-safe proxy for external assets          | —    |
| GET    | `/trace?url=&mode=&format=&offset=` | Trace an image into a vector shape (BSON) | —    |

\* `STEAM_API_KEY` is optional; `/items/*` works without it when a `fileUrl`
query param is supplied.

## Data flow

```
/search?q=wingspan  →  scrape browse page  →  list of {id, title, fileUrl, ...}
        ↓
/items/:id          →  Steam API (needs key)  →  file_url
        ↓
        └─ /items/:id?fileUrl=...  →  download directly (no key)
        ↓
                      download BSON → parse → TTSMod
        ↓
packages/extract    →  flatten objects / extract refs / download assets
        ↓
/trace?url=...      →  sharp decode → vtracer → parse SVG → BSON shape
```

## Dependencies

- `hono`, `@hono/node-server`, `@hono/cors` — server.
- `bson` — BSON deserialization (save files + trace results).
- `@visioncortex/vtracer` — raster-to-SVG vectorization (wasm).
- `sharp` — image decoding to RGBA for tracing.
- `svgpath` — SVG path parsing for traced shapes.
- `cheerio` — Workshop browse page scraping (backend only).
- `zod` — validation.
- `react`, `react-dom`, `react-router-dom`, `zustand` — frontend.
- `three`, `@react-three/fiber`, `@react-three/drei`,
  `@react-three/postprocessing` — 3D object viewers.
- `@iconify/react`, `@iconify-json/mdi`, `@iconify/utils` — iconify icons for
  object class tags (subset bundled via `scripts/generate-object-icons.mjs`).
- `vite`, `@vitejs/plugin-react`, `tailwindcss`, `@tailwindcss/vite` — frontend tooling.
- `tsx`, `typescript`, `eslint`, `prettier` — tooling.

## Tooling

- TypeScript strict mode.
- `tsx` for dev, `tsc` for build; `vite` for the frontend dev server/build.
- `vitest` for unit tests, colocated as `*.test.ts` next to sources.
- Root scripts: `pnpm dev`, `pnpm dev:web`, `pnpm build`, `pnpm test`, `pnpm lint`.

## Build order

1. Scaffold workspace (`pnpm-workspace.yaml`, root `package.json`, base tsconfig,
   `.npmrc`).
2. `packages/shared` — types + zod schemas.
3. `packages/tts` — fetch + parse (existing scraper code).
4. `packages/extract` — objects, refs, download.
5. `apps/proxy` — search + items + health routes, env validation.
6. `apps/web` — React frontend (search + mod pages), consuming the proxy and
   `packages/extract`.
7. Wire up root scripts, `.env.example`, README.

## Risks / caveats

- **Search scraping is fragile** — Steam can change their HTML or rate-limit.
  No official search API exists, so this is the standard approach.
- **Members-only items** won't appear in scraped search results.
- **`file_url` may be missing/expired** — handle cleanly with a 404-style error.
- **BSON parsing is the expensive part** — large saves; no caching by design
  (client-only).
- **`STEAM_API_KEY` is exposed in the client process** — acceptable for a
  personal tool; the key is only needed for the metadata call that yields
  `file_url`.

## Open decisions (defaults in bold)

- **`packages/extract` vs folding into `packages/tts`** — **separate package**
  (clean fetch vs analyze boundary).
- **Download output type** — **`Blob`** (easier for `<img>`/`<object>` in a
  frontend) vs raw `ArrayBuffer`.
- **Move `traverseMod`/`markParent` into `extract`** — **yes** (traversal is
  analysis, not fetching).