How to Apply Advanced Core-Engine Concepts

Use this guide to configure CPU affinity, tune the OS, build lockless pipelines, enforce memory rules, schedule multi-rate tasks, and preserve determinism in the Sim Engine.

1. Pin the Simulation Thread with CPU Affinity

CPU affinity determines which physical core(s) the Sim Engine runs on. Pinning avoids cross-core migration latency, cache invalidation, and jitter, and gives predictable frame start times.

1. Windows: assign the simulation loop thread to a dedicated core, set the highest real-time priority, and use the Multimedia Class Scheduler Service (MMCSS) if needed.
2. Linux: use a low-latency or RT kernel, pin the thread with sched_setaffinity(), and apply a real-time scheduling policy such as SCHED_FIFO.

Recommended layout for a 4-core CPU:

This layout eliminates interference and minimizes jitter.

2. Tune the Operating System

Real-time performance improves significantly once the OS is tuned to stop interfering with the pinned thread.

Windows

• Benchmark both timer modes and pick the best one: bcdedit /set useplatformclock yes or no.
• Use MMCSS for soft real-time (AVStream, Pro Audio settings, high-priority thread groups).
• Disable core parking so Windows cannot migrate your thread to a waking core.
• Disable dynamic frequency scaling: use the High Performance power plan and enable 100% minimum CPU frequency.

Linux

1. Use a PREEMPT_RT kernel for deterministic execution.
2. Disable power-saving features (C-states beyond C1, P-states) and use the performance governor:

   sudo cpupower frequency-set -g performance

3. Move IRQs off the dedicated core via /proc/irq/*/smp_affinity.
4. Isolate the core at boot:

   isolcpus=2 nohz_full=2 rcu_nocbs=2

5. Lock memory to avoid page faults:

   mlockall(MCL_CURRENT | MCL_FUTURE);

3. Build a Lockless Pipeline

A lockless design minimizes thread stalls and avoids unpredictable mutex behavior — no lock/unlock overhead, no priority inversion, no scheduler interference.

• Use single-producer/single-consumer (SPSC) ring buffers, atomic flags, double-buffering, atomic sequence counters, or lockless queues (MPSC, SPSC).

Example recording pipeline:

Frame Loop (Producer)
   → Lockless ring buffer
        → Recording Thread (Consumer)

The producer writes one frame of data at a time; the consumer flushes whenever possible.

Double Buffer Technique

1. Maintain a Write Buffer (real-time thread) and a Read Buffer (background thread).
2. Swap pointers atomically every frame:

   if (!busy.load()) {
       swapBuffers();
       busy.store(true);
   }

Result: zero locks, zero stalls.

4. Follow Memory Management Rules

Instead, do this outside or before the loop:

• Pre-allocate all buffers
• Pre-load configurations
• Reserve container capacity
• Pre-generate lookup tables
• Use object pools or fixed-size memory blocks

Lock all pages so the OS cannot page memory to disk:

mlockall(MCL_CURRENT | MCL_FUTURE);

5. Configure Multi-Rate Scheduling

Not every task needs to run at the main frame rate. The Sim Engine can run tasks at 1x rate (e.g., 1000 Hz), sub-rate (e.g., every 10 frames = 100 Hz), or super-rate if needed.

if (frameId % 10 == 0)
    run_100Hz_task();
if (frameId % 2 == 0)
    run_500Hz_task();
run_1000Hz_task(); // always runs

• Assign the most time-critical code to the highest rate.
• Move heavy, non-critical logic to lower rates (logging, UI updates, communications).
• Use mid-rate for physics models and PID controllers.
• Avoid mixing I/O and compute-heavy tasks in the same rate unless necessary.

6. Preserve Simulation Determinism

Determinism means the same input sequence always produces the same outputs. Achieve it by ensuring:

• Fixed time steps — no variable dt; all updates run at constant intervals.
• Deterministic execution order — tasks run in the same order every frame.
• No race conditions — avoid shared mutable state unless lockless or protected.
• No nondeterministic sources — seed RNGs, avoid reading OS time inside the loop, avoid async callbacks modifying simulation state.
• Consistent floating-point behavior — avoid FMA differences across compilers, avoid multi-thread FP races, use SIMD deterministically, and watch for cross-platform FP rounding differences.
• Hardware-independent behavior — set CPU affinity, use real-time scheduling policies, and stabilize frequency scaling.