tts-workshop/docs/implementation-plan.md

TTS Workshop — Implementation Plan

Scope: The concrete build plan — files, endpoints, dependencies, build order. For the system's architecture and dependency graph, see architecture.md. For the rationale behind key decisions, see decisions.md.

A lightweight, client-only pnpm monorepo for searching the Tabletop Simulator Steam Workshop, fetching full TTS save files, and analyzing their contents.

Goals

• Search the TTS Workshop to discover item IDs (Steam has no official search API, so this scrapes the browse page).
• Fetch a full TTS save (TTSMod) for a given item ID via the Steam Web API + BSON deserialization.
• Extract and inspect objects and asset references from a save, in an isomorphic package reusable from a future frontend.
• Client-only, lightweight, no caching layer.

Non-goals (for now)

• Backend traversal endpoints (deferred by design).
• Caching / Redis / multi-instance concerns.

Architecture

apps/web ──► apps/proxy ──► packages/tts ──► packages/shared
      │            │            │
      │            │            └──► (fetchMod → TTSMod)
      │            ▼
      ├──► packages/mesh ──► packages/shared (types)
      └──► packages/extract ──► packages/shared (types)
           (isomorphic, used by the frontend)

• apps/web — React frontend (search + mod pages).
• apps/proxy — Hono server. Search + fetch only. No traversal endpoints.
• packages/tts — low-level fetcher: Steam API call, BSON parse, filename derivation. Returns raw parsed TTSMod.
• packages/extract — analysis: flatten/filter objects, extract asset refs, download assets. Isomorphic (browser + Node).
• packages/mesh — 2D shapes + extrusion into 3D geometry for the frontend viewers. Isomorphic (browser + Node).
• packages/shared — shared types + zod schemas.

Repository layout

tts-workshop/
├── pnpm-workspace.yaml
├── package.json                # root scripts (dev, build, lint)
├── tsconfig.base.json
├── .npmrc
├── .env.example                # STEAM_API_KEY, PORT
├── docs/
│   └── implementation-plan.md  # this file
├── apps/
│   ├── proxy/
│   │   ├── package.json
│   │   ├── tsconfig.json
│   │   └── src/
│   │   ├── index.ts        # Hono app + @hono/node-server
│   │   ├── routes/
│   │   │   ├── search.ts   # GET /search?q=...&page=1
│   │   │   ├── items.ts    # GET /items/:id, /items/:id/file
│   │   │   ├── asset.ts    # GET /asset?url=... (CORS-safe asset proxy)
│   │   │   ├── trace.ts    # GET /trace?url=... (image → vector shape)
│   │   │   ├── svgShape.ts # parse vtracer SVG into { outline, holes }
│   │   │   └── health.ts   # GET /health
│   │   └── env.ts          # zod env validation
│   └── web/
│       ├── package.json
│       ├── tsconfig.json
│       ├── vite.config.ts      # dev proxy → localhost:3000
│       ├── index.html
│       └── src/
│           ├── main.tsx        # React root + router
│           ├── App.tsx         # layout + routes
│           ├── api.ts          # fetch wrappers for /search, /items
│           ├── index.css       # tailwind v4 entry
│           ├── pages/
│           │   ├── SearchPage.tsx
│           │   └── ModPage.tsx
│           ├── components/
│           │   ├── SearchResults.tsx
│           │   ├── ObjectTree.tsx     # containment-tree sidebar
│           │   ├── viewers.tsx        # viewer registry + default viewer
│           │   ├── viewers/           # 3D viewers (r3f/drei/postprocessing)
│           │   │   ├── register.ts    # registers per-class 3D viewers (lazy)
│           │   │   ├── Scene.tsx      # shared Canvas: lights, controls, post
│           │   │   ├── TileViewer.tsx
│           │   │   ├── TokenViewer.tsx
│           │   │   ├── CardViewer.tsx
│           │   │   ├── CustomModelViewer.tsx
│           │   │   └── assetUrl.ts    # route asset URLs through the proxy
│           │   ├── objectIcons.tsx    # class → icon mapping
│           │   ├── objectIconsData.ts # generated icon subset (do not edit)
│           │   └── objectIcons.test.ts
│           └── stores/
│               ├── searchStore.ts  # zustand
│               └── modStore.ts     # zustand
└── packages/
    ├── shared/
    │   └── src/
    │       ├── types.ts        # TTSMod, TTSObject, WorkshopItem, SearchResult
    │       └── schemas.ts
    ├── tts/
    │   └── src/
    │       ├── index.ts        # fetchMod, getFileName
    │       └── errors.ts
    ├── mesh/
    │   └── src/
    │       ├── index.ts        # public API barrel
    │       ├── types.ts        # FaceGeometry, ExtrudedGeometry, UVBounds
    │       ├── shapes.ts       # Shape + shape generators (rect, circle, ...)
    │       ├── tessellate.ts   # triangulate + cap faces
    │       ├── walls.ts        # side walls
    │       └── extrude.ts      # extrudeShape / extrudeShapeParts
    └── extract/
        ├── package.json
        ├── tsconfig.json
        └── src/
            ├── index.ts        # public API barrel
            ├── objects.ts      # flatten/filter/traverse helpers
            ├── refs.ts         # extract asset references
            ├── download.ts     # fetch + decode referenced assets
            └── types.ts        # extracted-object / asset types

Packages

packages/shared

Shared types and validation schemas used across the monorepo.

• types.ts
  • TTSMod, TTSObject (from the existing scraper code)
  • WorkshopItem — metadata from the Steam API (id, title, author, previewImageUrl, fileUrl, ...)
  • SearchResult — { items: WorkshopItem[], page, hasMore }
• schemas.ts
  • zod schemas for env vars, query params, and response shapes.

packages/tts

Low-level fetcher, extracted from the existing scraper.

• index.ts
  • fetchMod(id: string): Promise<TTSMod> — Steam API call to ISteamRemoteStorage/GetPublishedFileDetails/v1 to get file_url, then download + BSON-deserialize into TTSMod.
  • fetchModFromUrl(fileUrl) — download + BSON-deserialize from a direct URL (no Steam API call).
  • fetchModFile(id, apiKey) / fetchModFileFromUrl(fileUrl) — raw save bytes + derived filename, with and without the Steam API.
  • getFileName(url: string): Promise<string> — derive filename from the content-disposition header.
• errors.ts
  • Typed errors: missing file_url, Steam API failure, rate limit, invalid key.
• Notes
  • Swap the browser BSON global for the bson npm package.
  • traverseMod / markParent move to packages/extract (traversal is analysis, not fetching).

packages/extract

Isomorphic analysis of a parsed TTSMod. No Node-specific APIs.

• objects.ts
  • flattenObjects(mod) — all objects in the tree.
  • filterObjects(mod, predicate) — filter by name, GUID, type, etc.
  • findObject(mod, guid) — lookup by GUID.
  • buildTree(mod) — containment tree ({ object, label, children }), mirroring how TTS nests objects; label is Nickname when present, else the class Name.
  • traverseMod / markParent (moved from tts).
  • Returns lightweight graph shapes: { guid, name, type, parentGuid, childrenGuids, refs }.
• refs.ts
  • extractRefs(object) — walk one object, pull every external URL: CustomPDF.PDFUrl, CustomDeck[*].FaceURL/BackURL, CustomImage.ImageURL/ImageSecondaryURL.
  • collectRefs(mod) — all refs across the save, deduped by URL.
  • AssetRef type: { kind, url, ownerGuid }.
• download.ts
  • downloadAsset(url) — fetch → Blob.
  • downloadAll(refs, { concurrency, onProgress }) — batched downloads with progress callback.
  • guessMimeType(url) — infer mime from extension.
• Design constraints
  • No Buffer — use ArrayBuffer / Uint8Array / Blob (Node 18+).
  • No Node-only packages (cheerio stays in the backend search only).
  • Pure, deterministic functions where possible.

packages/mesh

2D shape + extrusion library used by the frontend viewers to build 3D geometry.

• shapes.ts
  • Shape — { outline, holes }, the minimal interface the tessellator and wall generator need. Shape generators: rectShape, polygonShape, hexShape, circleShape, roundedRectShape, frameShape, plus scaleShape and signedArea.
• tessellate.ts
  • triangulate(shape) — earcut triangulation (same as three.js).
  • frontFaces(shape, height, uvScale, uvBounds) — top face, normal +Z.
  • backFaces(shape, height, uvScale, uvBounds) — bottom face, normal -Z. UVs map the shape's bounding box (or uvBounds framing) to the unit square; the back uses the same planar xy mapping as the front (no mirror).
  • capFaces(...) — front + back merged into one geometry (front vertices first, then back).
• walls.ts
  • wallFaces(shape, height, uvScale, uvBounds) — side walls with outward normals and planar xy UVs (z-independent).
• extrude.ts
  • extrudeShape(shape, options) — merged front + back + walls as one geometry.
  • extrudeShapeParts(shape, options) — { front, back, walls } as separate geometries, so each face can carry its own material.
  • ExtrudeOptions — height, capUvScale, wallUvScale, uvBounds.
• types.ts
  • FaceGeometry, ExtrudedGeometry, UVBounds.

apps/proxy

Hono server exposing search + fetch.

• index.ts — Hono app + @hono/node-server bootstrap, CORS via @hono/cors.
• routes/search.ts
  • GET /search?q=...&page=1 — scrape steamcommunity.com/workshop/browse/?appid=286160&searchtext=... with cheerio, extract { id, title, author, previewImageUrl } from the result grid. Supports pagenum pagination. No API key required.
• routes/items.ts
  • GET /items/:id — full parsed TTSMod. Accepts an optional fileUrl query param to download the save directly (no Steam API key needed); otherwise resolves via the Steam API.
  • GET /items/:id/file — raw save bytes, filename from getFileName. Also accepts fileUrl.
• routes/asset.ts
  • GET /asset?url=... — fetch an external asset (texture, model) and stream it back with a Content-Type header. Workshop hosts often omit CORS headers, which would block three.js loaders in the browser; routing through the proxy makes those assets loadable. Only http(s) URLs are allowed.
• routes/trace.ts
  • GET /trace?url=...&mode=alpha&threshold=128&format=shape&offset=... — fetch an image, trace it into a vector shape, and return the result BSON-encoded. mode is alpha (default), bw, or color; format is shape (default) or svg; offset (optional) insets (negative) or outsets (positive) the shape in pixels. shape returns a parsed { outline, holes } polygon matching @tts/mesh's Shape interface, ready to extrude.
• routes/svgShape.ts
  • parseSvgShape(svg) — parse a vtracer SVG into a TracedShape ({ outline, holes }): flatten beziers to polylines, split subpaths into rings, classify by winding (CCW outline / CW hole), and assign holes to their containing outline.
  • offsetShape(shape, delta) — inset/outset a TracedShape via clipper-lib (Clipper miter joins); outline and holes offset in opposite directions and are recombined with a boolean difference, so holes grow on inset and shrink on outset. Collapsed shapes return an empty outline; a split outline keeps the largest ring.
• routes/health.ts
  • GET /health — liveness.
• env.ts — zod validation of STEAM_API_KEY, PORT.

apps/web

React frontend (Vite + React Router + Tailwind v4 + Zustand). Consumes the proxy API and packages/extract directly for analysis.

• main.tsx — React root, BrowserRouter.
• App.tsx — app shell (header) + routes: / (search), /mod/:id (mod).
• api.ts — typed fetch wrappers for /search and /items/:id, plus a modFileUrl helper for the raw-file download.
• pages/SearchPage.tsx — search form, drives searchStore.
• pages/ModPage.tsx — loads the mod via modStore, uses @tts/extract (buildTree, collectRefs) to render a containment-tree sidebar and an inspector pane for the selected object, and links to the raw save file.
• components/SearchResults.tsx — result grid + pagination.
• components/ObjectTree.tsx — recursive tree sidebar; each entry shows a class icon (hover for the class name) + display label, indented by depth. Clicking selects an object by its unique index path (not GUID — cards in a deck share the deck's GUID).
• components/viewers.tsx — viewer registry (registerViewer / resolveViewer) plus a DefaultViewer that renders an object's fields; custom per-class viewers can be registered later.
• components/viewers/ — 3D viewers built on @react-three/fiber, @react-three/drei, and @react-three/postprocessing. register.ts registers lazy-loaded viewers for Tile/Custom_Tile (flat box), Custom_Token (shape traced from the image's alpha channel via /trace, extruded with @tts/mesh), Card/CardCustom/Deck/DeckCustom/ Custom_Deck (thin rounded rect with face/back textures), and Custom_Model/Custom_Model_Bag/Custom_Model_Infinite_Bag (GLTF/OBJ/FBX from CustomMesh.MeshURL). Viewers build { front, back, walls } geometry via extrudeShapeParts; back faces are flipped left/right on the material (flipTexture.ts) so they aren't mirrored, and card faces slice the deck sprite sheet via CardID (cardResolution.ts). Scene.tsx is a shared canvas with lighting, orbit controls, contact shadows, and subtle bloom/vignette; it fits the camera to the object's bounds via drei's Bounds inside the Suspense boundary, so it frames the loaded content. assetUrl.ts routes asset URLs through the proxy for CORS-safe loading. The viewers are lazy-loaded so the three.js stack is code-split out of the main bundle.
• components/objectIcons.tsx — maps TTS object classes to one or more Iconify icons (iconsForObject); unknown classes fall back to a help icon. Icons may come from multiple sets (mdi, material-symbols, file-icons, ...); names not bundled are fetched from the Iconify CDN at runtime.
• components/objectIconsData.ts — generated by scripts/generate-object-icons.mjs; registers a small mdi subset with Iconify's offline storage so those render without a network round-trip.
• stores/searchStore.ts / stores/modStore.ts — Zustand stores for search and mod state.
• vite.config.ts — dev proxy for /search, /items, /health → http://localhost:3000.
• Styling: Tailwind v4 via @tailwindcss/vite; index.css imports tailwindcss.

Endpoints

Method	Path	Description	Auth
GET	/health	Liveness	—
GET	/search?q=&page=	Scrape Workshop browse, return item list	—
GET	/items/:id	Full parsed TTSMod (?fileUrl= skips key)	key*
GET	/items/:id/file	Raw save bytes, filename from header	key*
GET	/asset?url=	CORS-safe proxy for external assets	—
GET	/trace?url=&mode=&format=&offset=	Trace an image into a vector shape (BSON)	—

* STEAM_API_KEY is optional; /items/* works without it when a fileUrl query param is supplied.

Data flow

/search?q=wingspan  →  scrape browse page  →  list of {id, title, fileUrl, ...}
        ↓
/items/:id          →  Steam API (needs key)  →  file_url
        ↓
        └─ /items/:id?fileUrl=...  →  download directly (no key)
        ↓
                      download BSON → parse → TTSMod
        ↓
packages/extract    →  flatten objects / extract refs / download assets
        ↓
/trace?url=...      →  sharp decode → vtracer → parse SVG → BSON shape

Dependencies

• hono, @hono/node-server, @hono/cors — server.
• bson — BSON deserialization (save files + trace results).
• @visioncortex/vtracer — raster-to-SVG vectorization (wasm).
• sharp — image decoding to RGBA for tracing.
• svgpath — SVG path parsing for traced shapes.
• cheerio — Workshop browse page scraping (backend only).
• zod — validation.
• react, react-dom, react-router-dom, zustand — frontend.
• three, @react-three/fiber, @react-three/drei, @react-three/postprocessing — 3D object viewers.
• @iconify/react, @iconify-json/mdi, @iconify/utils — iconify icons for object class tags (subset bundled via scripts/generate-object-icons.mjs).
• vite, @vitejs/plugin-react, tailwindcss, @tailwindcss/vite — frontend tooling.
• tsx, typescript, eslint, prettier — tooling.

Tooling

• TypeScript strict mode.
• tsx for dev, tsc for build; vite for the frontend dev server/build.
• vitest for unit tests, colocated as *.test.ts next to sources.
• Root scripts: pnpm dev, pnpm dev:web, pnpm build, pnpm test, pnpm lint.

Build order

1. Scaffold workspace (pnpm-workspace.yaml, root package.json, base tsconfig, .npmrc).
2. packages/shared — types + zod schemas.
3. packages/tts — fetch + parse (existing scraper code).
4. packages/extract — objects, refs, download.
5. apps/proxy — search + items + health routes, env validation.
6. apps/web — React frontend (search + mod pages), consuming the proxy and packages/extract.
7. Wire up root scripts, .env.example, README.

Risks / caveats

• Search scraping is fragile — Steam can change their HTML or rate-limit. No official search API exists, so this is the standard approach.
• Members-only items won't appear in scraped search results.
• file_url may be missing/expired — handle cleanly with a 404-style error.
• BSON parsing is the expensive part — large saves; no caching by design (client-only).
• STEAM_API_KEY is exposed in the client process — acceptable for a personal tool; the key is only needed for the metadata call that yields file_url.

Open decisions (defaults in bold)

• packages/extract vs folding into packages/tts — separate package (clean fetch vs analyze boundary).
• Download output type — Blob (easier for <img>/<object> in a frontend) vs raw ArrayBuffer.
• Move traverseMod/markParent into extract — yes (traversal is analysis, not fetching).