# TTS Workshop — Implementation Plan

> **Scope:** The concrete build plan — files, endpoints, dependencies, build
> order. For the system's architecture and dependency graph, see
> [`architecture.md`](./architecture.md). For the rationale behind key decisions,
> see [`decisions.md`](./decisions.md).

A lightweight, client-only pnpm monorepo for searching the Tabletop Simulator
Steam Workshop, fetching full TTS save files, and analyzing their contents.

## Goals

- Search the TTS Workshop to discover item IDs (Steam has no official search API,
  so this scrapes the browse page).
- Fetch a full TTS save (`TTSMod`) for a given item ID via the Steam Web API +
  BSON deserialization.
- Extract and inspect objects and asset references from a save, in an isomorphic
  package reusable from a future frontend.
- Client-only, lightweight, no caching layer.

## Non-goals (for now)

- Backend traversal endpoints (deferred by design).
- Caching / Redis / multi-instance concerns.

## Architecture

```
apps/web ──► apps/proxy ──► packages/tts ──► packages/shared
      │            │            │
      │            │            └──► (fetchMod → TTSMod)
      │            ▼
      └──► packages/extract ──► packages/shared (types)
           (isomorphic, used by the frontend)
```

- **`apps/web`** — React frontend (search + mod pages).
- **`apps/proxy`** — Hono server. Search + fetch only. No traversal endpoints.
- **`packages/tts`** — low-level fetcher: Steam API call, BSON parse, filename
  derivation. Returns raw parsed `TTSMod`.
- **`packages/extract`** — analysis: flatten/filter objects, extract asset refs,
  download assets. Isomorphic (browser + Node).
- **`packages/shared`** — shared types + zod schemas.

## Repository layout

```
tts-workshop/
├── pnpm-workspace.yaml
├── package.json                # root scripts (dev, build, lint)
├── tsconfig.base.json
├── .npmrc
├── .env.example                # STEAM_API_KEY, PORT
├── docs/
│   └── implementation-plan.md  # this file
├── apps/
│   ├── proxy/
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── src/
│   │       ├── index.ts        # Hono app + @hono/node-server
│   │       ├── routes/
│   │       │   ├── search.ts   # GET /search?q=...&page=1
│   │       │   ├── items.ts    # GET /items/:id, /items/:id/file
│   │       │   └── health.ts   # GET /health
│   │       └── env.ts          # zod env validation
│   └── web/
│       ├── package.json
│       ├── tsconfig.json
│       ├── vite.config.ts      # dev proxy → localhost:3000
│       ├── index.html
│       └── src/
│           ├── main.tsx        # React root + router
│           ├── App.tsx         # layout + routes
│           ├── api.ts          # fetch wrappers for /search, /items
│           ├── index.css       # tailwind v4 entry
│           ├── pages/
│           │   ├── SearchPage.tsx
│           │   └── ModPage.tsx
│           ├── components/
│           │   ├── SearchResults.tsx
│           │   ├── ObjectTree.tsx     # containment-tree sidebar
│           │   ├── viewers.tsx        # viewer registry + default viewer
│           │   ├── objectIcons.tsx    # class → icon mapping
│           │   ├── objectIconsData.ts # generated icon subset (do not edit)
│           │   └── objectIcons.test.ts
│           └── stores/
│               ├── searchStore.ts  # zustand
│               └── modStore.ts     # zustand
└── packages/
    ├── shared/
    │   └── src/
    │       ├── types.ts        # TTSMod, TTSObject, WorkshopItem, SearchResult
    │       └── schemas.ts
    ├── tts/
    │   └── src/
    │       ├── index.ts        # fetchMod, getFileName
    │       └── errors.ts
    └── extract/
        ├── package.json
        ├── tsconfig.json
        └── src/
            ├── index.ts        # public API barrel
            ├── objects.ts      # flatten/filter/traverse helpers
            ├── refs.ts         # extract asset references
            ├── download.ts     # fetch + decode referenced assets
            └── types.ts        # extracted-object / asset types
```

## Packages

### `packages/shared`

Shared types and validation schemas used across the monorepo.

- `types.ts`
  - `TTSMod`, `TTSObject` (from the existing scraper code)
  - `WorkshopItem` — metadata from the Steam API (`id`, `title`, `author`,
    `previewImageUrl`, `fileUrl`, ...)
  - `SearchResult` — `{ items: WorkshopItem[], page, hasMore }`
- `schemas.ts`
  - zod schemas for env vars, query params, and response shapes.

### `packages/tts`

Low-level fetcher, extracted from the existing scraper.

- `index.ts`
  - `fetchMod(id: string): Promise<TTSMod>` — Steam API call to
    `ISteamRemoteStorage/GetPublishedFileDetails/v1` to get `file_url`, then
    download + BSON-deserialize into `TTSMod`.
  - `fetchModFromUrl(fileUrl)` — download + BSON-deserialize from a direct URL
    (no Steam API call).
  - `fetchModFile(id, apiKey)` / `fetchModFileFromUrl(fileUrl)` — raw save
    bytes + derived filename, with and without the Steam API.
  - `getFileName(url: string): Promise<string>` — derive filename from the
    `content-disposition` header.
- `errors.ts`
  - Typed errors: missing `file_url`, Steam API failure, rate limit, invalid key.
- Notes
  - Swap the browser `BSON` global for the `bson` npm package.
  - `traverseMod` / `markParent` move to `packages/extract` (traversal is
    analysis, not fetching).

### `packages/extract`

Isomorphic analysis of a parsed `TTSMod`. No Node-specific APIs.

- `objects.ts`
  - `flattenObjects(mod)` — all objects in the tree.
  - `filterObjects(mod, predicate)` — filter by name, GUID, type, etc.
  - `findObject(mod, guid)` — lookup by GUID.
  - `buildTree(mod)` — containment tree (`{ object, label, children }`),
    mirroring how TTS nests objects; `label` is `Nickname` when present,
    else the class `Name`.
  - `traverseMod` / `markParent` (moved from `tts`).
  - Returns lightweight graph shapes: `{ guid, name, type, parentGuid,
    childrenGuids, refs }`.
- `refs.ts`
  - `extractRefs(object)` — walk one object, pull every external URL:
    `CustomPDF.PDFUrl`, `CustomDeck[*].FaceURL`/`BackURL`,
    `CustomImage.ImageURL`/`ImageSecondaryURL`.
  - `collectRefs(mod)` — all refs across the save, deduped by URL.
  - `AssetRef` type: `{ kind, url, ownerGuid }`.
- `download.ts`
  - `downloadAsset(url)` — `fetch` → `Blob`.
  - `downloadAll(refs, { concurrency, onProgress })` — batched downloads with
    progress callback.
  - `guessMimeType(url)` — infer mime from extension.
- Design constraints
  - No `Buffer` — use `ArrayBuffer` / `Uint8Array` / `Blob` (Node 18+).
  - No Node-only packages (`cheerio` stays in the backend search only).
  - Pure, deterministic functions where possible.

### `apps/proxy`

Hono server exposing search + fetch.

- `index.ts` — Hono app + `@hono/node-server` bootstrap, CORS via `@hono/cors`.
- `routes/search.ts`
  - `GET /search?q=...&page=1` — scrape
    `steamcommunity.com/workshop/browse/?appid=286160&searchtext=...` with
    `cheerio`, extract `{ id, title, author, previewImageUrl }` from the result
    grid. Supports `pagenum` pagination. No API key required.
- `routes/items.ts`
  - `GET /items/:id` — full parsed `TTSMod`. Accepts an optional `fileUrl`
    query param to download the save directly (no Steam API key needed);
    otherwise resolves via the Steam API.
  - `GET /items/:id/file` — raw save bytes, filename from `getFileName`. Also
    accepts `fileUrl`.
- `routes/health.ts`
  - `GET /health` — liveness.
- `env.ts` — zod validation of `STEAM_API_KEY`, `PORT`.

### `apps/web`

React frontend (Vite + React Router + Tailwind v4 + Zustand). Consumes the
proxy API and `packages/extract` directly for analysis.

- `main.tsx` — React root, `BrowserRouter`.
- `App.tsx` — app shell (header) + routes: `/` (search), `/mod/:id` (mod).
- `api.ts` — typed `fetch` wrappers for `/search` and `/items/:id`, plus a
  `modFileUrl` helper for the raw-file download.
- `pages/SearchPage.tsx` — search form, drives `searchStore`.
- `pages/ModPage.tsx` — loads the mod via `modStore`, uses `@tts/extract`
  (`buildTree`, `collectRefs`) to render a containment-tree sidebar and an
  inspector pane for the selected object, and links to the raw save file.
- `components/SearchResults.tsx` — result grid + pagination.
- `components/ObjectTree.tsx` — recursive tree sidebar; each entry shows a
  class icon (hover for the class name) + display label, indented by depth.
  Clicking selects an object.
- `components/viewers.tsx` — viewer registry (`registerViewer` /
  `resolveViewer`) plus a `DefaultViewer` that renders an object's fields;
  custom per-class viewers can be registered later.
- `components/objectIcons.tsx` — maps TTS object classes to one or more MDI
  icons (`iconsForObject`); unknown classes fall back to a help icon.
- `components/objectIconsData.ts` — generated by
  `scripts/generate-object-icons.mjs`; registers only the used MDI icons with
  Iconify's offline storage so the full set isn't bundled.
- `stores/searchStore.ts` / `stores/modStore.ts` — Zustand stores for search
  and mod state.
- `vite.config.ts` — dev proxy for `/search`, `/items`, `/health` →
  `http://localhost:3000`.
- Styling: Tailwind v4 via `@tailwindcss/vite`; `index.css` imports
  `tailwindcss`.

## Endpoints

| Method | Path                | Description                                  | Auth |
| ------ | ------------------- | -------------------------------------------- | ---- |
| GET    | `/health`           | Liveness                                     | —    |
| GET    | `/search?q=&page=`  | Scrape Workshop browse, return item list     | —    |
| GET    | `/items/:id`        | Full parsed `TTSMod` (`?fileUrl=` skips key) | key* |
| GET    | `/items/:id/file`   | Raw save bytes, filename from header         | key* |

\* `STEAM_API_KEY` is optional; `/items/*` works without it when a `fileUrl`
query param is supplied.

## Data flow

```
/search?q=wingspan  →  scrape browse page  →  list of {id, title, fileUrl, ...}
        ↓
/items/:id          →  Steam API (needs key)  →  file_url
        ↓
        └─ /items/:id?fileUrl=...  →  download directly (no key)
        ↓
                      download BSON → parse → TTSMod
        ↓
packages/extract    →  flatten objects / extract refs / download assets
```

## Dependencies

- `hono`, `@hono/node-server`, `@hono/cors` — server.
- `bson` — BSON deserialization.
- `cheerio` — Workshop browse page scraping (backend only).
- `zod` — validation.
- `react`, `react-dom`, `react-router-dom`, `zustand` — frontend.
- `@iconify/react`, `@iconify-json/mdi`, `@iconify/utils` — iconify icons for
  object class tags (subset bundled via `scripts/generate-object-icons.mjs`).
- `vite`, `@vitejs/plugin-react`, `tailwindcss`, `@tailwindcss/vite` — frontend tooling.
- `tsx`, `typescript`, `eslint`, `prettier` — tooling.

## Tooling

- TypeScript strict mode.
- `tsx` for dev, `tsc` for build; `vite` for the frontend dev server/build.
- `vitest` for unit tests, colocated as `*.test.ts` next to sources.
- Root scripts: `pnpm dev`, `pnpm dev:web`, `pnpm build`, `pnpm test`, `pnpm lint`.

## Build order

1. Scaffold workspace (`pnpm-workspace.yaml`, root `package.json`, base tsconfig,
   `.npmrc`).
2. `packages/shared` — types + zod schemas.
3. `packages/tts` — fetch + parse (existing scraper code).
4. `packages/extract` — objects, refs, download.
5. `apps/proxy` — search + items + health routes, env validation.
6. `apps/web` — React frontend (search + mod pages), consuming the proxy and
   `packages/extract`.
7. Wire up root scripts, `.env.example`, README.

## Risks / caveats

- **Search scraping is fragile** — Steam can change their HTML or rate-limit.
  No official search API exists, so this is the standard approach.
- **Members-only items** won't appear in scraped search results.
- **`file_url` may be missing/expired** — handle cleanly with a 404-style error.
- **BSON parsing is the expensive part** — large saves; no caching by design
  (client-only).
- **`STEAM_API_KEY` is exposed in the client process** — acceptable for a
  personal tool; the key is only needed for the metadata call that yields
  `file_url`.

## Open decisions (defaults in bold)

- **`packages/extract` vs folding into `packages/tts`** — **separate package**
  (clean fetch vs analyze boundary).
- **Download output type** — **`Blob`** (easier for `<img>`/`<object>` in a
  frontend) vs raw `ArrayBuffer`.
- **Move `traverseMod`/`markParent` into `extract`** — **yes** (traversal is
  analysis, not fetching).