tts-workshop/docs/implementation-plan.md

TTS Workshop — Implementation Plan

Scope: The concrete build plan — files, endpoints, dependencies, build order. For the system's architecture and dependency graph, see architecture.md. For the rationale behind key decisions, see decisions.md.

A lightweight, client-only pnpm monorepo for searching the Tabletop Simulator Steam Workshop, fetching full TTS save files, and analyzing their contents.

Goals

• Search the TTS Workshop to discover item IDs (Steam has no official search API, so this scrapes the browse page).
• Fetch a full TTS save (TTSMod) for a given item ID via the Steam Web API + BSON deserialization.
• Extract and inspect objects and asset references from a save, in an isomorphic package reusable from a future frontend.
• Client-only, lightweight, no caching layer.

Non-goals (for now)

• Backend traversal endpoints (deferred by design).
• Caching / Redis / multi-instance concerns.

Architecture

apps/web ──► apps/proxy ──► packages/tts ──► packages/shared
      │            │            │
      │            │            └──► (fetchMod → TTSMod)
      │            ▼
      └──► packages/extract ──► packages/shared (types)
           (isomorphic, used by the frontend)

• apps/web — React frontend (search + mod pages).
• apps/proxy — Hono server. Search + fetch only. No traversal endpoints.
• packages/tts — low-level fetcher: Steam API call, BSON parse, filename derivation. Returns raw parsed TTSMod.
• packages/extract — analysis: flatten/filter objects, extract asset refs, download assets. Isomorphic (browser + Node).
• packages/shared — shared types + zod schemas.

Repository layout

tts-workshop/
├── pnpm-workspace.yaml
├── package.json                # root scripts (dev, build, lint)
├── tsconfig.base.json
├── .npmrc
├── .env.example                # STEAM_API_KEY, PORT
├── docs/
│   └── implementation-plan.md  # this file
├── apps/
│   ├── proxy/
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── src/
│   │       ├── index.ts        # Hono app + @hono/node-server
│   │       ├── routes/
│   │       │   ├── search.ts   # GET /search?q=...&page=1
│   │       │   ├── items.ts    # GET /items/:id, /items/:id/file
│   │       │   └── health.ts   # GET /health
│   │       └── env.ts          # zod env validation
│   └── web/
│       ├── package.json
│       ├── tsconfig.json
│       ├── vite.config.ts      # dev proxy → localhost:3000
│       ├── index.html
│       └── src/
│           ├── main.tsx        # React root + router
│           ├── App.tsx         # layout + routes
│           ├── api.ts          # fetch wrappers for /search, /items
│           ├── index.css       # tailwind v4 entry
│           ├── pages/
│           │   ├── SearchPage.tsx
│           │   └── ModPage.tsx
│           ├── components/
│           │   ├── SearchResults.tsx
│           │   ├── ObjectTree.tsx     # containment-tree sidebar
│           │   ├── viewers.tsx        # viewer registry + default viewer
│           │   ├── objectIcons.tsx    # class → icon mapping
│           │   ├── objectIconsData.ts # generated icon subset (do not edit)
│           │   └── objectIcons.test.ts
│           └── stores/
│               ├── searchStore.ts  # zustand
│               └── modStore.ts     # zustand
└── packages/
    ├── shared/
    │   └── src/
    │       ├── types.ts        # TTSMod, TTSObject, WorkshopItem, SearchResult
    │       └── schemas.ts
    ├── tts/
    │   └── src/
    │       ├── index.ts        # fetchMod, getFileName
    │       └── errors.ts
    └── extract/
        ├── package.json
        ├── tsconfig.json
        └── src/
            ├── index.ts        # public API barrel
            ├── objects.ts      # flatten/filter/traverse helpers
            ├── refs.ts         # extract asset references
            ├── download.ts     # fetch + decode referenced assets
            └── types.ts        # extracted-object / asset types

Packages

packages/shared

Shared types and validation schemas used across the monorepo.

• types.ts
  • TTSMod, TTSObject (from the existing scraper code)
  • WorkshopItem — metadata from the Steam API (id, title, author, previewImageUrl, fileUrl, ...)
  • SearchResult — { items: WorkshopItem[], page, hasMore }
• schemas.ts
  • zod schemas for env vars, query params, and response shapes.

packages/tts

Low-level fetcher, extracted from the existing scraper.

• index.ts
  • fetchMod(id: string): Promise<TTSMod> — Steam API call to ISteamRemoteStorage/GetPublishedFileDetails/v1 to get file_url, then download + BSON-deserialize into TTSMod.
  • fetchModFromUrl(fileUrl) — download + BSON-deserialize from a direct URL (no Steam API call).
  • fetchModFile(id, apiKey) / fetchModFileFromUrl(fileUrl) — raw save bytes + derived filename, with and without the Steam API.
  • getFileName(url: string): Promise<string> — derive filename from the content-disposition header.
• errors.ts
  • Typed errors: missing file_url, Steam API failure, rate limit, invalid key.
• Notes
  • Swap the browser BSON global for the bson npm package.
  • traverseMod / markParent move to packages/extract (traversal is analysis, not fetching).

packages/extract

Isomorphic analysis of a parsed TTSMod. No Node-specific APIs.

• objects.ts
  • flattenObjects(mod) — all objects in the tree.
  • filterObjects(mod, predicate) — filter by name, GUID, type, etc.
  • findObject(mod, guid) — lookup by GUID.
  • buildTree(mod) — containment tree ({ object, label, children }), mirroring how TTS nests objects; label is Nickname when present, else the class Name.
  • traverseMod / markParent (moved from tts).
  • Returns lightweight graph shapes: { guid, name, type, parentGuid, childrenGuids, refs }.
• refs.ts
  • extractRefs(object) — walk one object, pull every external URL: CustomPDF.PDFUrl, CustomDeck[*].FaceURL/BackURL, CustomImage.ImageURL/ImageSecondaryURL.
  • collectRefs(mod) — all refs across the save, deduped by URL.
  • AssetRef type: { kind, url, ownerGuid }.
• download.ts
  • downloadAsset(url) — fetch → Blob.
  • downloadAll(refs, { concurrency, onProgress }) — batched downloads with progress callback.
  • guessMimeType(url) — infer mime from extension.
• Design constraints
  • No Buffer — use ArrayBuffer / Uint8Array / Blob (Node 18+).
  • No Node-only packages (cheerio stays in the backend search only).
  • Pure, deterministic functions where possible.

apps/proxy

Hono server exposing search + fetch.

• index.ts — Hono app + @hono/node-server bootstrap, CORS via @hono/cors.
• routes/search.ts
  • GET /search?q=...&page=1 — scrape steamcommunity.com/workshop/browse/?appid=286160&searchtext=... with cheerio, extract { id, title, author, previewImageUrl } from the result grid. Supports pagenum pagination. No API key required.
• routes/items.ts
  • GET /items/:id — full parsed TTSMod. Accepts an optional fileUrl query param to download the save directly (no Steam API key needed); otherwise resolves via the Steam API.
  • GET /items/:id/file — raw save bytes, filename from getFileName. Also accepts fileUrl.
• routes/health.ts
  • GET /health — liveness.
• env.ts — zod validation of STEAM_API_KEY, PORT.

apps/web

React frontend (Vite + React Router + Tailwind v4 + Zustand). Consumes the proxy API and packages/extract directly for analysis.

• main.tsx — React root, BrowserRouter.
• App.tsx — app shell (header) + routes: / (search), /mod/:id (mod).
• api.ts — typed fetch wrappers for /search and /items/:id, plus a modFileUrl helper for the raw-file download.
• pages/SearchPage.tsx — search form, drives searchStore.
• pages/ModPage.tsx — loads the mod via modStore, uses @tts/extract (buildTree, collectRefs) to render a containment-tree sidebar and an inspector pane for the selected object, and links to the raw save file.
• components/SearchResults.tsx — result grid + pagination.
• components/ObjectTree.tsx — recursive tree sidebar; each entry shows a class icon (hover for the class name) + display label, indented by depth. Clicking selects an object.
• components/viewers.tsx — viewer registry (registerViewer / resolveViewer) plus a DefaultViewer that renders an object's fields; custom per-class viewers can be registered later.
• components/objectIcons.tsx — maps TTS object classes to one or more Iconify icons (iconsForObject); unknown classes fall back to a help icon. Icons may come from multiple sets (mdi, material-symbols, file-icons, ...); names not bundled are fetched from the Iconify CDN at runtime.
• components/objectIconsData.ts — generated by scripts/generate-object-icons.mjs; registers a small mdi subset with Iconify's offline storage so those render without a network round-trip.
• stores/searchStore.ts / stores/modStore.ts — Zustand stores for search and mod state.
• vite.config.ts — dev proxy for /search, /items, /health → http://localhost:3000.
• Styling: Tailwind v4 via @tailwindcss/vite; index.css imports tailwindcss.

Endpoints

Method	Path	Description	Auth
GET	/health	Liveness	—
GET	/search?q=&page=	Scrape Workshop browse, return item list	—
GET	/items/:id	Full parsed TTSMod (?fileUrl= skips key)	key*
GET	/items/:id/file	Raw save bytes, filename from header	key*

* STEAM_API_KEY is optional; /items/* works without it when a fileUrl query param is supplied.

Data flow

/search?q=wingspan  →  scrape browse page  →  list of {id, title, fileUrl, ...}
        ↓
/items/:id          →  Steam API (needs key)  →  file_url
        ↓
        └─ /items/:id?fileUrl=...  →  download directly (no key)
        ↓
                      download BSON → parse → TTSMod
        ↓
packages/extract    →  flatten objects / extract refs / download assets

Dependencies

• hono, @hono/node-server, @hono/cors — server.
• bson — BSON deserialization.
• cheerio — Workshop browse page scraping (backend only).
• zod — validation.
• react, react-dom, react-router-dom, zustand — frontend.
• @iconify/react, @iconify-json/mdi, @iconify/utils — iconify icons for object class tags (subset bundled via scripts/generate-object-icons.mjs).
• vite, @vitejs/plugin-react, tailwindcss, @tailwindcss/vite — frontend tooling.
• tsx, typescript, eslint, prettier — tooling.

Tooling

• TypeScript strict mode.
• tsx for dev, tsc for build; vite for the frontend dev server/build.
• vitest for unit tests, colocated as *.test.ts next to sources.
• Root scripts: pnpm dev, pnpm dev:web, pnpm build, pnpm test, pnpm lint.

Build order

1. Scaffold workspace (pnpm-workspace.yaml, root package.json, base tsconfig, .npmrc).
2. packages/shared — types + zod schemas.
3. packages/tts — fetch + parse (existing scraper code).
4. packages/extract — objects, refs, download.
5. apps/proxy — search + items + health routes, env validation.
6. apps/web — React frontend (search + mod pages), consuming the proxy and packages/extract.
7. Wire up root scripts, .env.example, README.

Risks / caveats

• Search scraping is fragile — Steam can change their HTML or rate-limit. No official search API exists, so this is the standard approach.
• Members-only items won't appear in scraped search results.
• file_url may be missing/expired — handle cleanly with a 404-style error.
• BSON parsing is the expensive part — large saves; no caching by design (client-only).
• STEAM_API_KEY is exposed in the client process — acceptable for a personal tool; the key is only needed for the metadata call that yields file_url.

Open decisions (defaults in bold)

• packages/extract vs folding into packages/tts — separate package (clean fetch vs analyze boundary).
• Download output type — Blob (easier for <img>/<object> in a frontend) vs raw ArrayBuffer.
• Move traverseMod/markParent into extract — yes (traversal is analysis, not fetching).