oecs-sharp/docs/implementation-plan.md

Implementation Plan: OECS (Observable ECS)

Overview

This document lays out the phased implementation of OECS as a .NET class library (DLL). Each phase produces a shippable increment; later phases build on earlier ones. Architecture decisions are captured in docs/architecture.md.

Target

• .NET 8 (LTS), net8.0 (C# 12)
• Dependencies: MessagePack (serialization), R3 (reactivity)
• Output: OECS.dll
• Source generator: OECS.SourceGen.dll (component registry for serialization)

---

Phase 1 — Core Foundation (Week 1)

Goal: Create/destroy entities, add/remove components, iterate sparse sets.

1.1 Project Scaffold

OECS.sln
├── OECS/OECS.csproj
├── OECS.SourceGen/OECS.SourceGen.csproj
├── OECS.Tests/OECS.Tests.csproj
├── Game.Blackjack/Blackjack.csproj
├── Game.Blackjack.Tests/Game.Blackjack.Tests.csproj
└── Game.TicTacToe/...

• OECS.csproj targets net8.0, references MessagePack and R3.
• OECS.Tests.csproj references OECS + xunit + FluentAssertions.

1.2 Entity

readonly struct Entity : IEquatable<Entity>

Decision	Rationale
32-bit uint backing	Keeps size small; 24-bit ID (16.7M) + 8-bit version (256 gens) is ample for observable-first use cases.
Entity.Null sentinel (value 0)	Entity ID 0 is reserved; version 0 means "never alive."
Id (24-bit) and Version (8-bit) properties	Expose for debugging; opaque otherwise.
ToString() → "Entity(42:v3)"	Debuggability.

Internal constructor; World is the only factory.

1.3 SparseSet<T>

class SparseSet<T> where T : struct

Field	Purpose
T[] dense	Packed component values (no holes).
Entity[] denseEntities	Parallel array: which entity owns each dense slot.
int[] sparse	Maps entity ID → dense index; -1 = absent.

Operations: Add(Entity, T), Remove(Entity), ref T Get(Entity), bool Contains(Entity), int Count, void Clear().

dense and sparse arrays grow geometrically (×2) on overflow.

Why sparse sets over archetypes? The design prioritizes cheap add/remove (for reactivity) over raw iteration throughput. Archetypes would require moving entities between archetypes on component change, which complicates change tracking.

1.4 ComponentStore

class ComponentStore

Holds a Dictionary<Type, object> mapping component types to their SparseSet<T>. Provides typed generic methods:

• void Add<T>(Entity, T)
• void Remove<T>(Entity)
• ref T Get<T>(Entity)
• T Read<T>(Entity) — copy without auto-mark
• bool TryGet<T>(Entity, out T)
• bool Has<T>(Entity)
• void RemoveAll(Entity) — called on entity destruction

1.5 World

class World

Responsibility	Detail
Entity allocation	Free-list of recycled IDs; bump allocator for new IDs.
Component access	Delegates to ComponentStore.
Entity destruction	Returns ID to free list, increments version, removes all components.

API surface:

Entity CreateEntity();
void DestroyEntity(Entity entity);
void AddComponent<T>(Entity entity, T component) where T : struct;
void RemoveComponent<T>(Entity entity) where T : struct;
ref T GetComponent<T>(Entity entity) where T : struct;
T ReadComponent<T>(Entity entity) where T : struct;
bool TryGetComponent<T>(Entity entity, out T value) where T : struct;
bool HasComponent<T>(Entity entity) where T : struct;
bool IsAlive(Entity entity);

1.6 Tests

• Entity creation returns unique IDs.
• Entity destruction recycles IDs with incremented version.
• IsAlive returns false for destroyed entities.
• Add/remove/has component round-trips correctly.
• Sparse set iteration visits all added components.
• Removing a component mid-iteration is safe (deferred or swap-remove).

---

Phase 2 — Queries & Systems (Week 2)

Goal: Define queries, register systems, run ticks.

2.1 Query Description

A query is defined by:

• A set of "with" component types.
• A set of "without" component types.

class QueryDescriptor
{
    IReadOnlySet<Type> With { get; }
    IReadOnlySet<Type> Without { get; }
}

2.2 QueryBuilder

Fluent API returned by World.Query():

world.Query()
    .With<Position>()
    .With<Velocity>()
    .Without<Frozen>()
    .Build()  // → QueryDescriptor

2.3 Query Execution

World provides two iteration styles:

ForEach callbacks (1–6 component types):

void ForEach<T1, T2>(QueryDescriptor query, ForEachAction<T1, T2> action);

Ref struct iterators via EntityIterator.Select<T>() extensions (1–3 component types):

using var iter = world.Select<Position, Velocity>(query);
while (iter.MoveNext()) { ... }

Both accept an optional QueryDescriptor. When omitted, all entities with the given component types are iterated (no Without filter).

The smallest "with" sparse set is used as the driver; other sets are probed for membership. The singleton entity (ID 1) is always skipped.

2.4 ISystem

interface ISystem
{
    void Run(World world);
}

Systems do not declare a query on the interface. Instead, they either read singletons directly or build queries internally and iterate with ForEach / Select. This keeps the interface minimal and gives systems full flexibility.

Why no auto-injection? The explicit API makes the iteration cost visible and avoids the need for source generators or reflection for system dispatch.

2.5 ITickedSystem

interface ITickedSystem : ISystem
{
    void Run(World world, Tick tick);
}

Systems that need tick metadata (delta time or logical tick marker) implement this. Both Run overloads must be implemented; SystemGroup dispatches to the appropriate one at runtime.

2.6 System Registration & Ordering

class SystemGroup
{
    void Add(ISystem system);          // registration order = execution order
    void Remove(ISystem system);
    void RunTimed(float deltaTime);
    void RunLogical();
    int Count { get; }
}

Systems run in registration order. SystemGroup automatically:

• Drains commands before the tick, after each system, and after the full tick.
• Posts changes after each system and after the full tick.

2.7 Tick

readonly struct Tick
{
    TickType Type { get; }   // Timed or Logical
    float DeltaTime { get; } // 0 for logical ticks
}

2.8 Tests

• Query with single component returns matching entities.
• Query with multiple components returns intersection.
• Without<T> excludes entities with T.
• Adding/removing components updates query results.
• Systems run in registration order.
• Destroyed entities don't appear in queries.
• ITickedSystem receives correct tick data.

---

Phase 3 — Commands (Week 3)

Goal: Serializable command queue with deferred execution.

3.1 ICommand

interface ICommand
{
    void Execute(World world);
}

Commands are structs implementing ICommand. They are typically [MessagePackObject] for serialization. Not stored in ECS sparse sets — they live in a queue.

3.2 CommandQueue

class CommandQueue
{
    void Enqueue<T>(T command) where T : struct, ICommand;
    void ExecuteAll(World world);  // FIFO, fully drains queue
    void Clear();
    void ClearErrors();
    int Count { get; }
    IReadOnlyList<Exception> Errors { get; }
}

Enqueue<T> uses a constrained generic to avoid boxing at the call site. Commands enqueued during ExecuteAll are processed in the same drain cycle.

3.3 Integration with World

World owns a CommandQueue. After each system runs (or after the full tick), the queue is drained automatically by SystemGroup. Manual drain is also available:

world.ExecuteCommands(); // manual drain

Systems enqueue commands via world.Commands.Enqueue(...).

3.4 Error Handling

If a command's Execute throws, the exception is caught and stored. The queue continues processing remaining commands. After ExecuteAll, any errors are available via CommandQueue.Errors.

3.5 Tests

• Commands execute in FIFO order.
• A command can read/write ECS state.
• A command enqueued during ExecuteAll runs in the same drain cycle.
• Exceptions are collected, not lost.
• Serialization round-trip preserves command data.

---

Phase 4 — Relationships (Week 4)

Goal: Relationship components with auto-managed source/target and reverse lookup.

4.1 IRelationship

interface IRelationship
{
    Entity Source { get; }
    Entity Target { get; }
}

4.2 Relationship<TSelf, TTarget>

[MessagePackObject]
struct Relationship<TSelf, TTarget> : IRelationship
    where TSelf : struct where TTarget : struct
{
    [Key(0)] Entity Source { get; set; }
    [Key(1)] Entity Target { get; set; }
}

The IRelationship marker interface lets ComponentStore detect relationships and maintain the reverse index. The TSelf/TTarget phantom types differentiate relationship kinds at the type level.

Alternatively, implement IRelationship directly on your own struct (see the Blackjack Holds and InDeck types).

4.3 Reverse Index

World maintains a reverse index per relationship type, mapping target → set of source entities. Updated automatically when relationship components are added/removed.

4.4 Reverse Lookup API

IReadOnlyCollection<Entity> GetSources<T>(Entity target) where T : struct, IRelationship;

4.5 Cascading Behavior

When an entity is destroyed:

• All relationships where it is the source are removed (components dropped).
• All relationships where it is the target are removed (components dropped from source entities).
• The reverse index is cleaned up.

4.6 Tests

• Adding a relationship updates the reverse index.
• Removing a relationship updates the reverse index.
• Destroying a source entity cleans up its relationships.
• Destroying a target entity cleans up incoming relationships.
• Reverse lookup returns correct sources.

---

Phase 5 — Reactivity (Week 5–6)

Goal: Change tracking, marking, posting, and R3 observable queries.

5.1 Change Kinds

enum ChangeKind
{
    EntityAdded,
    EntityRemoved,
    ComponentAdded,
    ComponentRemoved,
    ComponentModified
}

struct EntityChange
{
    Entity Entity { get; }
    ChangeKind Kind { get; }
    Type? ComponentType { get; } // null for entity-level changes
}

5.2 ChangeBuffer

class ChangeBuffer
{
    void Mark<...>(...);
    void Post();  // pushes to R3 subjects, then clears
}

5.3 Automatic vs. Manual Marking

Change Type	Marking
Entity created	Auto
Entity destroyed	Auto
Component added	Auto
Component removed	Auto
Component modified	Manual via world.MarkModified<T>(entity)

Rationale: structural changes are always detectable. Value mutations inside a ref T are not — the sparse set has no way to know the caller changed the value. Requiring an explicit MarkModified call is the simplest correct approach.

ReadComponent<T> (returns a copy) and ReadSingleton<T> never auto-mark. Use these when you only need to inspect state without signaling changes.

5.4 Posting Model

• During a system's Run, changes accumulate in a pending buffer.
• After each system's Run, Post() is called automatically by SystemGroup.
• After the full tick, Post() is called once more.
• When not in a system run, changes are not posted automatically — the caller must call world.PostChanges().

5.5 R3 Integration

World exposes observables:

Observable<EntityChange> ObserveEntityChanges();
Observable<EntityChange> ObserveComponentChanges<T>() where T : struct;
Observable<EntityChange> ObserveQuery(QueryDescriptor query);

These are backed by Subject<EntityChange> instances. Subscribers receive batched changes after each Post().

5.6 Subscription Lifecycle

Subscriptions return IDisposable. UI code ties this to component lifecycle:

world.ObserveComponentChanges<Health>()
    .Subscribe(change => UpdateHealthBar(change))
    .AddTo(componentDisposables); // R3's AddTo

5.7 Tests

• Entity creation posts EntityAdded.
• Entity destruction posts EntityRemoved.
• Component add/remove posts corresponding changes.
• MarkModified posts ComponentModified.
• Changes are batched per Post() call.
• Subscribers receive changes in order.
• Disposing a subscription stops notifications.

---

Phase 6 — Singletons (Week 6)

Goal: Singleton entity and ergonomic accessors.

6.1 Singleton Entity

World reserves entity ID 1 as the singleton entity. It is never destroyed and is excluded from normal queries (all iterators skip ID 1).

6.2 Singleton Accessors

void SetSingleton<T>(T component) where T : struct;
ref T GetSingleton<T>() where T : struct;
T ReadSingleton<T>() where T : struct;
bool HasSingleton<T>() where T : struct;
void RemoveSingleton<T>() where T : struct;

These are convenience wrappers around AddComponent/GetComponent on the singleton entity. GetSingleton returns a ref (auto-marks during iteration); ReadSingleton returns a copy (never auto-marks).

6.3 Query Exclusion

Queries automatically exclude the singleton entity. If a user genuinely wants to include it, they can query it by its entity ID directly.

6.4 Tests

• SetSingleton/GetSingleton round-trips.
• Singleton entity does not appear in normal queries.
• Removing a singleton works.
• Singleton survives tick execution.

---

Phase 7 — Source Generator & Serialization (Week 7)

Goal: Compile-time component registry for serialization without reflection.

7.1 OECS.SourceGen

A Roslyn incremental source generator that scans for all types used as generic arguments to World methods (AddComponent<T>, GetComponent<T>, SetSingleton<T>, etc.) and generates a ComponentRegistry class with per-type serialize/deserialize callbacks via MessagePack.

7.2 WorldSerializer

static class WorldSerializer
{
    static void Save(World world, Stream stream);
    static void Load(World world, Stream stream);
}

Uses ComponentRegistry to serialize/deserialize all entities and their components to/from a MessagePack stream. IRelationship.Source is fixed up to the owning entity on load.

7.3 Snapshot Types

WorldSnapshot, EntitySnapshot, and ComponentEntry are the public serialization DTOs.

---

Phase 8 — Polish & Documentation (Week 8)

8.1 XML Docs

All public API surface gets <summary> XML documentation comments.

8.2 README

Quick-start guide with a minimal example: create world, register system, run tick, observe changes.

8.3 NuGet Packaging

OECS.csproj includes package metadata:

• PackageId: OECS
• Description: "Observable ECS for C# — an entity component system focused on a clean reactive API surface."
• PackageTags: ecs;reactive;observable;gamedev
• Bundles OECS.SourceGen.dll as an analyzer for consumers.

8.4 Testing Games

Test games (Game.Blackjack, Game.TicTacToe) serve as integration tests and design validation. See docs/testing-games.md for the testing strategy.

---

Dependency Graph

Phase 1 (Core)
  └─→ Phase 2 (Queries & Systems)
        └─→ Phase 3 (Commands)
        └─→ Phase 4 (Relationships)
              └─→ Phase 5 (Reactivity)
                    └─→ Phase 6 (Singletons)
                          └─→ Phase 7 (Source Gen & Serialization)
                                └─→ Phase 8 (Polish)

Phases 3 and 4 can be done in parallel; Phase 5 depends on both.

---

Open Questions

1. Parallel system execution? Deferred. The design is single-threaded by default. If needed, systems could declare read/write component access for automatic parallel scheduling — but this adds significant complexity.

2. Multiple worlds? The design supports it naturally — World is a class, you can instantiate multiple. No cross-world references are supported.

3. Component import from CSV/MasterMemory? The design.md mentions this as a potential feature. Not yet implemented.