oecs-sharp/docs/implementation-plan.md

Implementation Plan: OECS (Observable ECS)

Overview

This document lays out the phased implementation of OECS as a .NET class library (DLL). Each phase produces a shippable increment; later phases build on earlier ones. Architecture decisions are captured in docs/architecture.md.

Target

• .NET 8 (LTS), net8.0 (C# 12)
• Dependencies: MessagePack (serialization), R3 (reactivity)
• Output: OECS.dll
• Source generator: OECS.SourceGen.dll (component registry for serialization)

---

Phase 1 — Core Foundation (Week 1)

Goal: Create/destroy entities, add/remove components, iterate sparse sets.

1.1 Project Scaffold

OECS.sln
├── OECS/OECS.csproj
├── OECS.SourceGen/OECS.SourceGen.csproj
├── OECS.Tests/OECS.Tests.csproj
├── Game.Blackjack/Blackjack.csproj
├── Game.Blackjack.Tests/Game.Blackjack.Tests.csproj
└── Game.TicTacToe/...

• OECS.csproj targets net8.0, references MessagePack and R3.
• OECS.Tests.csproj references OECS + xunit + FluentAssertions.

1.2 Entity

readonly struct Entity : IEquatable<Entity>

Decision	Rationale
32-bit uint backing	Keeps size small; 24-bit ID (16.7M) + 8-bit version (256 gens) is ample for observable-first use cases.
Entity.Null sentinel (value 0)	Entity ID 0 is reserved; version 0 means "never alive."
Id (24-bit) and Version (8-bit) properties	Expose for debugging; opaque otherwise.
ToString() → "Entity(42:v3)"	Debuggability.

Internal constructor; World is the only factory.

1.3 SparseSet<T>

class SparseSet<T> where T : struct

Field	Purpose
T[] dense	Packed component values (no holes).
Entity[] denseEntities	Parallel array: which entity owns each dense slot.
int[] sparse	Maps entity ID → dense index; -1 = absent.

Operations: Add(Entity, T), Remove(Entity), ref T Get(Entity), bool Contains(Entity), int Count, void Clear().

dense and sparse arrays grow geometrically (×2) on overflow.

Why sparse sets over archetypes? The design prioritizes cheap add/remove (for reactivity) over raw iteration throughput. Archetypes would require moving entities between archetypes on component change, which complicates change tracking.

1.4 ComponentStore

class ComponentStore

Holds a Dictionary<Type, object> mapping component types to their SparseSet<T>. Provides typed generic methods:

• void Add<T>(Entity, T)
• void Remove<T>(Entity)
• ref T Get<T>(Entity)
• T Read<T>(Entity) — copy without auto-mark
• bool TryGet<T>(Entity, out T)
• bool Has<T>(Entity)
• void RemoveAll(Entity) — called on entity destruction

1.5 World

class World

Responsibility	Detail
Entity allocation	Free-list of recycled IDs; bump allocator for new IDs.
Component access	Delegates to ComponentStore.
Entity destruction	Returns ID to free list, increments version, removes all components.

API surface:

Entity CreateEntity();
void DestroyEntity(Entity entity);
void AddComponent<T>(Entity entity, T component) where T : struct;
void RemoveComponent<T>(Entity entity) where T : struct;
ref T GetComponent<T>(Entity entity) where T : struct;
T ReadComponent<T>(Entity entity) where T : struct;
bool TryGetComponent<T>(Entity entity, out T value) where T : struct;
bool HasComponent<T>(Entity entity) where T : struct;
bool IsAlive(Entity entity);

1.6 Tests

• Entity creation returns unique IDs.
• Entity destruction recycles IDs with incremented version.
• IsAlive returns false for destroyed entities.
• Add/remove/has component round-trips correctly.
• Sparse set iteration visits all added components.
• Removing a component mid-iteration is safe (deferred or swap-remove).

---

Phase 2 — Queries & Systems (Week 2)

Goal: Define queries, register systems, run ticks.

2.1 Query Description

A query is defined by:

• A set of "with" component types.
• A set of "without" component types.

class QueryDescriptor
{
    IReadOnlySet<Type> With { get; }
    IReadOnlySet<Type> Without { get; }
}

2.2 QueryBuilder

Fluent API returned by World.Query():

world.Query()
    .With<Position>()
    .With<Velocity>()
    .Without<Frozen>()
    .Build()  // → QueryDescriptor

2.3 Query Execution

World provides two iteration styles:

ForEach callbacks (1–6 component types):

void ForEach<T1, T2>(QueryDescriptor query, ForEachAction<T1, T2> action);

Ref struct iterators via EntityIterator.Select<T>() extensions (1–3 component types):

using var iter = world.Select<Position, Velocity>(query);
while (iter.MoveNext()) { ... }

Both accept an optional QueryDescriptor. When omitted, all entities with the given component types are iterated (no Without filter).

The smallest "with" sparse set is used as the driver; other sets are probed for membership. The singleton entity (ID 1) is always skipped.

2.4 ISystem

interface ISystem
{
    void Run(World world);
}

Systems do not declare a query on the interface. Instead, they either read singletons directly or build queries internally and iterate with ForEach / Select. This keeps the interface minimal and gives systems full flexibility.

Why no auto-injection? The explicit API makes the iteration cost visible and avoids the need for source generators or reflection for system dispatch.

2.5 ITickedSystem

interface ITickedSystem : ISystem
{
    void Run(World world, Tick tick);
}

Systems that need tick metadata (delta time or logical tick marker) implement this. Both Run overloads must be implemented; SystemGroup dispatches to the appropriate one at runtime.

2.6 System Registration & Ordering

class SystemGroup
{
    void Add(ISystem system);          // registration order = execution order
    void Remove(ISystem system);
    void RunTimed(float deltaTime);
    void RunLogical();
    int Count { get; }
}

Systems run in registration order. SystemGroup automatically:

• Drains commands before the tick, after each system, and after the full tick.
• Posts changes after each system and after the full tick.

2.7 Tick

readonly struct Tick
{
    TickType Type { get; }   // Timed or Logical
    float DeltaTime { get; } // 0 for logical ticks
}

2.8 Tests

• Query with single component returns matching entities.
• Query with multiple components returns intersection.
• Without<T> excludes entities with T.
• Adding/removing components updates query results.
• Systems run in registration order.
• Destroyed entities don't appear in queries.
• ITickedSystem receives correct tick data.

---

Phase 3 — Commands (Week 3)

Goal: Serializable command queue with deferred execution.

3.1 ICommand

interface ICommand
{
    void Execute(World world);
}

Commands are structs implementing ICommand. They are typically [MessagePackObject] for serialization. Not stored in ECS sparse sets — they live in a queue.

3.2 CommandQueue

class CommandQueue
{
    void Enqueue<T>(T command) where T : struct, ICommand;
    void ExecuteAll(World world);  // FIFO, fully drains queue
    void Clear();
    void ClearErrors();
    int Count { get; }
    IReadOnlyList<Exception> Errors { get; }
}

Enqueue<T> uses a constrained generic to avoid boxing at the call site. Commands enqueued during ExecuteAll are processed in the same drain cycle.

3.3 Integration with World

World owns a CommandQueue. After each system runs (or after the full tick), the queue is drained automatically by SystemGroup. Manual drain is also available:

world.ExecuteCommands(); // manual drain

Systems enqueue commands via world.Commands.Enqueue(...).

3.4 Error Handling

If a command's Execute throws, the exception is caught and stored. The queue continues processing remaining commands. After ExecuteAll, any errors are available via CommandQueue.Errors.

3.5 Tests

• Commands execute in FIFO order.
• A command can read/write ECS state.
• A command enqueued during ExecuteAll runs in the same drain cycle.
• Exceptions are collected, not lost.
• Serialization round-trip preserves command data.

---

Phase 4 — Relationships (Week 4)

Goal: Relationship components with auto-managed source/target and reverse lookup.

4.1 IRelationship

interface IRelationship
{
    Entity Source { get; }
    Entity Target { get; }
}

4.2 Relationship<TSelf, TTarget>

[MessagePackObject]
struct Relationship<TSelf, TTarget> : IRelationship
    where TSelf : struct where TTarget : struct
{
    [Key(0)] Entity Source { get; set; }
    [Key(1)] Entity Target { get; set; }
}

The IRelationship marker interface lets ComponentStore detect relationships and maintain the reverse index. The TSelf/TTarget phantom types differentiate relationship kinds at the type level.

Alternatively, implement IRelationship directly on your own struct (see the Blackjack Holds and InDeck types).

4.3 Reverse Index

World maintains a reverse index per relationship type, mapping target → set of source entities. Updated automatically when relationship components are added/removed.

4.4 Reverse Lookup API

IReadOnlyCollection<Entity> GetSources<T>(Entity target) where T : struct, IRelationship;

4.5 Cascading Behavior

When an entity is destroyed:

• All relationships where it is the source are removed (components dropped).
• All relationships where it is the target are removed (components dropped from source entities).
• The reverse index is cleaned up.

4.6 Tests

• Adding a relationship updates the reverse index.
• Removing a relationship updates the reverse index.
• Destroying a source entity cleans up its relationships.
• Destroying a target entity cleans up incoming relationships.
• Reverse lookup returns correct sources.

---

Phase 5 — Reactivity (Week 5–6)

Goal: Change tracking, marking, posting, and R3 observable queries.

5.1 Change Kinds

enum ChangeKind
{
    EntityAdded,
    EntityRemoved,
    ComponentAdded,
    ComponentRemoved,
    ComponentModified
}

struct EntityChange
{
    Entity Entity { get; }
    ChangeKind Kind { get; }
    Type? ComponentType { get; } // null for entity-level changes
}

5.2 ChangeBuffer

class ChangeBuffer
{
    void Mark<...>(...);
    void Post();  // pushes to R3 subjects, then clears
}

5.3 Automatic vs. Manual Marking

Change Type	Marking
Entity created	Auto
Entity destroyed	Auto
Component added	Auto
Component removed	Auto
Component modified	Manual via world.MarkModified<T>(entity)

Rationale: structural changes are always detectable. Value mutations inside a ref T are not — the sparse set has no way to know the caller changed the value. Requiring an explicit MarkModified call is the simplest correct approach.

ReadComponent<T> (returns a copy) and ReadSingleton<T> never auto-mark. Use these when you only need to inspect state without signaling changes.

5.4 Posting Model

• During a system's Run, changes accumulate in a pending buffer.
• After each system's Run, Post() is called automatically by SystemGroup.
• After the full tick, Post() is called once more.
• When not in a system run, changes are not posted automatically — the caller must call world.PostChanges().

5.5 R3 Integration

World exposes observables:

Observable<EntityChange> ObserveEntityChanges();
Observable<EntityChange> ObserveComponentChanges<T>() where T : struct;
Observable<EntityChange> ObserveQuery(QueryDescriptor query);

These are backed by Subject<EntityChange> instances. Subscribers receive batched changes after each Post().

5.6 Subscription Lifecycle

Subscriptions return IDisposable. UI code ties this to component lifecycle:

world.ObserveComponentChanges<Health>()
    .Subscribe(change => UpdateHealthBar(change))
    .AddTo(componentDisposables); // R3's AddTo

5.7 Tests

• Entity creation posts EntityAdded.
• Entity destruction posts EntityRemoved.
• Component add/remove posts corresponding changes.
• MarkModified posts ComponentModified.
• Changes are batched per Post() call.
• Subscribers receive changes in order.
• Disposing a subscription stops notifications.

---

Phase 6 — Singletons (Week 6)

Goal: Singleton entity and ergonomic accessors.

6.1 Singleton Entity

World reserves entity ID 1 as the singleton entity. It is never destroyed and is excluded from normal queries (all iterators skip ID 1).

6.2 Singleton Accessors

void SetSingleton<T>(T component) where T : struct;
ref T GetSingleton<T>() where T : struct;
T ReadSingleton<T>() where T : struct;
bool HasSingleton<T>() where T : struct;
void RemoveSingleton<T>() where T : struct;

These are convenience wrappers around AddComponent/GetComponent on the singleton entity. GetSingleton returns a ref (auto-marks during iteration); ReadSingleton returns a copy (never auto-marks).

6.3 Query Exclusion

Queries automatically exclude the singleton entity. If a user genuinely wants to include it, they can query it by its entity ID directly.

6.4 Tests

• SetSingleton/GetSingleton round-trips.
• Singleton entity does not appear in normal queries.
• Removing a singleton works.
• Singleton survives tick execution.

---

Phase 7 — Source Generator & Serialization (Week 7)

Goal: Compile-time component registry for serialization without reflection.

7.1 OECS.SourceGen

A Roslyn incremental source generator that scans for all types used as generic arguments to World methods (AddComponent<T>, GetComponent<T>, SetSingleton<T>, etc.) and generates a ComponentRegistry class with per-type serialize/deserialize callbacks via MessagePack.

7.2 WorldSerializer

static class WorldSerializer
{
    static void Save(World world, Stream stream);
    static void Load(World world, Stream stream);
}

Uses ComponentRegistry to serialize/deserialize all entities and their components to/from a MessagePack stream. IRelationship.Source is fixed up to the owning entity on load.

7.3 Snapshot Types

WorldSnapshot, EntitySnapshot, and ComponentEntry are the public serialization DTOs.

---

Phase 8 — Polish & Documentation (Week 8)

8.1 XML Docs

All public API surface gets <summary> XML documentation comments.

8.2 README

Quick-start guide with a minimal example: create world, register system, run tick, observe changes.

8.3 NuGet Packaging

OECS.csproj includes package metadata:

• PackageId: OECS
• Description: "Observable ECS for C# — an entity component system focused on a clean reactive API surface."
• PackageTags: ecs;reactive;observable;gamedev
• Bundles OECS.SourceGen.dll as an analyzer for consumers.

8.4 Testing Games

Test games (Game.Blackjack, Game.TicTacToe) serve as integration tests and design validation. See docs/testing-games.md for the testing strategy.

---

Phase 9 — Interrupts (Week 9)

Goal: System-issued blocking signals resolved by handler commands.

9.1 IInterrupt

Marker interface for interrupt struct types. Interrupts are not components — they're transient signals stored in the InterruptStore.

9.2 IInterruptHandlerCommand<T>

interface IInterruptHandlerCommand<TInterrupt> : ICommand
    where TInterrupt : struct, IInterrupt
{
    bool TryResolve(TInterrupt interrupt);
}

Extends ICommand with a default Execute that looks up the pending interrupt of type TInterrupt, calls TryResolve, and resolves the interrupt if the handler returns true.

9.3 InterruptStore

Internal class owned by SystemGroup. Maps Type → boxed IInterrupt. Supports Add, TryGet, Remove, HasAny, Has<T>, Clear.

9.4 World API

void Interrupt<T>(T interrupt) where T : struct, IInterrupt;
bool HasInterrupt<T>() where T : struct, IInterrupt;

Lightweight pass-through to the InterruptStore injected by SystemGroup. When no SystemGroup manages the world, interrupts are no-ops.

9.5 SystemGroup Integration

At the start of each tick, SystemGroup.RunAll drains commands and checks _interrupts.HasAny. If pending, the tick skips all systems but still posts changes. Interrupts block the next tick, not the current one.

9.6 Tests

• Interrupt blocks the next tick.
• Handler resolves interrupt and unblocks.
• Handler rejection leaves interrupt pending.
• HasInterrupt query works.
• Duplicate interrupt type throws.
• Multiple interrupt types are independent.
• Interrupt is a no-op without SystemGroup.
• Commands still execute when blocked.

---

Dependency Graph

Phase 1 (Core)
  └─→ Phase 2 (Queries & Systems)
        └─→ Phase 3 (Commands)
              └─→ Phase 9 (Interrupts)
        └─→ Phase 4 (Relationships)
              └─→ Phase 5 (Reactivity)
                    └─→ Phase 6 (Singletons)
                          └─→ Phase 7 (Source Gen & Serialization)
                                └─→ Phase 8 (Polish)

Interrupts depend on Phase 3 (Commands) because they use ICommand for handler commands. They're independent of Phases 4–8.

---

Open Questions

1. Parallel system execution? Deferred. The design is single-threaded by default. If needed, systems could declare read/write component access for automatic parallel scheduling — but this adds significant complexity.

2. Multiple worlds? The design supports it naturally — World is a class, you can instantiate multiple. No cross-world references are supported.

3. Component import from CSV/MasterMemory? The design.md mentions this as a potential feature. Not yet implemented.