fix(csv-loader): escape quotes in schema rows

Implement loader-side escaping for the schema row to prevent
`csv-parse` from misinterpreting double-quoted string literals (e.g.,
`"active" | "inactive"`) as field delimiters.
This commit is contained in:
2026-08-06 10:25:24 +08:00
parent cdff31c126
commit 37e3514c0c
4 changed files with 82 additions and 8 deletions
+1
View File
@@ -156,6 +156,7 @@ The following TypeScript types are exported:
- In a union, **reference members are tried before non-reference members** (e.g. `@users | string` resolves `1` to the user object, falling back to a plain string when the reference doesn't match) - In a union, **reference members are tried before non-reference members** (e.g. `@users | string` resolves `1` to the user object, falling back to a plain string when the reference doesn't match)
- Special characters can be escaped with backslash: `\;`, `\[`, `\]`, `\\` - Special characters can be escaped with backslash: `\;`, `\[`, `\]`, `\\`
- Empty arrays/tuples are not allowed - Empty arrays/tuples are not allowed
- In CSV schema rows, double-quoted string literals like `"active" | "inactive"` are handled automatically by the loader; avoid commas inside string literals (use single-quoted literals like `'a,b'` for those)
- For CSV loading with reference resolution, see [csv-loader.md](./csv-loader.md) - For CSV loading with reference resolution, see [csv-loader.md](./csv-loader.md)
```javascript ```javascript
+10 -8
View File
@@ -1,6 +1,6 @@
# Implementation Plan: Syntax Clarity & Parser Robustness # Implementation Plan: Syntax Clarity & Parser Robustness
Status: **In progress — Phases 1, 2 & 3 applied** (Phases 46 not yet done) Status: **In progress — Phases 1, 2, 3 & 4 applied** (Phases 56 not yet done)
## Goals ## Goals
@@ -53,16 +53,18 @@ All changes are **breaking** to the DSL → bump to `2.0.0`, update README + `cs
--- ---
## Phase 4 — Quote conflict resolution ## Phase 4 — Quote conflict resolution ✅ Applied
**Files:** `src/parser.ts`, `src/csv-loader/loader.ts`, README, `csv-loader.md` **Files:** `src/csv-loader/loader.ts`, `src/csv-loader/tests/parseCsv-basic.test.ts`, README, `csv-loader.md`
**Options (pick one):** - Added `escapeSchemaRowQuotes()` in `loader.ts`: the schema row (2nd non-empty line) is split on the delimiter, and any cell containing the quote char has its inner quotes escaped and is wrapped in quotes before `csv-parse` runs.
- **(a) Standardize on single-quoted literals only** — minimal change, but leaves a footgun. - This lets users write `"active" | "inactive"` naturally in a schema row — previously csv-parse threw `Invalid Closing Quote`.
- **(b) Quote the entire schema cell** so `csv-parse` treats it as one field — requires loader-side handling to strip the outer quotes before parsing. - Single-quoted literals (`'on' | 'off'`) are untouched (no `"` present).
- **(c) Drop quotes entirely** — use bare-token literals (e.g. `on` / `off` as identifiers). Most robust, largest change. - Added a test in `parseCsv-basic.test.ts`.
**Recommendation:** (b) — it's the only option that fixes the root cause (schema cells containing `"` break `csv-parse`) without redesigning the literal syntax. Requires: in `loader.ts`, detect schema-row cells that are fully quoted and unwrap before `parseSchema`. **Decision:** Implemented option (b) — loader-side escaping of the schema row. This fixes the root cause (schema cells containing `"` break csv-parse) without redesigning the literal syntax.
**Known limitation (pre-existing, out of scope):** a comma inside a string literal (e.g. `"a,b"`) is ambiguous with the CSV delimiter and still fails — use single-quoted literals (`'a,b'`) for those, or a non-comma delimiter.
--- ---
+56
View File
@@ -33,6 +33,53 @@ import {
parseReferenceValue, parseReferenceValue,
} from "./reference-resolver.js"; } from "./reference-resolver.js";
import { generateTypeDefinition } from "./type-gen.js"; import { generateTypeDefinition } from "./type-gen.js";
/**
* Escape double-quote characters in the schema row so csv-parse doesn't
* misinterpret them as CSV field delimiters.
*
* The schema row is the 2nd non-empty line. Cells that contain the quote
* character (e.g. `"active" | "inactive"`) have their inner quotes escaped
* and are wrapped in quotes, so csv-parse treats each as a single field.
*/
function escapeSchemaRowQuotes(
content: string,
delimiter: string,
quote: string,
escape: string,
): string {
if (!quote || !content.includes(quote)) {
return content;
}
const lines = content.split(/\r?\n/);
let schemaRowIndex = -1;
let nonEmptyCount = 0;
for (let i = 0; i < lines.length; i++) {
if (lines[i].trim() === "") continue;
nonEmptyCount++;
if (nonEmptyCount === 2) {
schemaRowIndex = i;
break;
}
}
if (schemaRowIndex === -1) {
return content;
}
const cells = lines[schemaRowIndex].split(delimiter);
const transformed = cells
.map((cell) => {
if (!cell.includes(quote)) {
return cell;
}
const escaped = cell.split(quote).join(escape + quote);
return quote + escaped + quote;
})
.join(delimiter);
lines[schemaRowIndex] = transformed;
return lines.join("\n");
}
import { csvToModule } from "./module-gen.js"; import { csvToModule } from "./module-gen.js";
import { import {
parseTypeDeclaration, parseTypeDeclaration,
@@ -96,6 +143,15 @@ export function parseCsv(
filteredContent = nonCommentLines.join("\n"); filteredContent = nonCommentLines.join("\n");
} }
// Escape double-quotes in the schema row so csv-parse doesn't treat them
// as CSV field delimiters (e.g. `"active" | "inactive"` in a schema cell).
filteredContent = escapeSchemaRowQuotes(
filteredContent,
delimiter,
quote,
escape,
);
const records = parse(filteredContent, { const records = parse(filteredContent, {
delimiter, delimiter,
quote, quote,
@@ -65,6 +65,21 @@ describe("parseCsv - basic parsing", () => {
expect(result.data[1]).toEqual({ name: "Bob", status: "off" }); expect(result.data[1]).toEqual({ name: "Bob", status: "off" });
}); });
it("should parse CSV with double-quoted string literal columns", () => {
const csv = [
"name,status",
'string,"active" | "inactive"',
"Alice,active",
"Bob,inactive",
].join("\n");
const result = parseCsv(csv, { emitTypes: false });
expect(result.data).toHaveLength(2);
expect(result.data[0]).toEqual({ name: "Alice", status: "active" });
expect(result.data[1]).toEqual({ name: "Bob", status: "inactive" });
});
it("should parse CSV with array columns", () => { it("should parse CSV with array columns", () => {
const csv = [ const csv = [
"name,tags", "name,tags",