Performance and profiling

Fast code starts with measuring. This chapter is the loop to follow when a game is slow, or before you’ve decided whether it is: measure, read the report, fix the one biggest thing, measure again. Everything here is built in; you need no outside tools. The performance chapter has the engine’s own benchmark numbers and the frame-level tools (the overlay, GPU passes, traces).

1. Measure

A frame takes about as long as the slower of the CPU (your systems, then recording the draws) and the GPU. Three things are on offer, from cheapest to most detailed:

The overlay and CRABBY_BENCH switch the deeper profiling on by themselves. In your own code, ctx.set_profiling(true) does. While it is off, a scope costs a single atomic load and the allocation counter a single load per allocation, so leaving scopes in shipped code is fine.

Time your own code: scopes

Every system is timed automatically. To see inside one, wrap the parts you suspect:

#![allow(unused)]
fn main() {
/// Time the parts of your own code you suspect.
fn steering(ctx: &mut Ctx) {
    profile_scope!("steering");
    {
        profile_scope!("steering/neighbors");
        std::hint::black_box((0..2000).map(|i| (i as f32).sqrt()).sum::<f32>());
    }
    ctx.profile("steering/avoid", |ctx| {
        ctx.world.query::<&mut Transform2D>().for_each(|t| t.position.x += 1.0);
    });
}
}

// In one .cpp file, at global scope: count C++ `new` calls in the reports.
CRABBY_COUNT_CPP_ALLOCATIONS

/// Time the parts of your own code you suspect.
static void steering(crabby::ctx& ctx) {
    CRABBY_PROFILE_SCOPE("steering");
    {
        CRABBY_PROFILE_SCOPE("steering/neighbors");
        volatile float x = 0;
        for (int i = 0; i < 2000; ++i) x = x + std::sqrt(float(i));
    }
    (void)ctx;
}

profile_scope!("name") (C++: CRABBY_PROFILE_SCOPE("name")) times the rest of the block. ctx.profile("name", |ctx| ...) times a closure. Scopes nest (times are inclusive), they work on any thread, and times from several threads add up. A scope named "steering/neighbors" sorts next to its parent in the report.

Count allocations

An allocation in code that runs every frame is a common reason for stutter. To count them, install the counting allocator once, in your main.rs:

#![allow(unused)]
fn main() {
#[global_allocator]
static ALLOCATOR: CountingAllocator = CountingAllocator;
}

// Put this once at global scope in one .cpp file (see the scopes example above):
CRABBY_COUNT_CPP_ALLOCATIONS

Counting happens only while profiling is on. The report then says how many allocations each frame and each system makes. In C and C++ the engine’s own allocations are counted too; CRABBY_COUNT_CPP_ALLOCATIONS adds your new calls. (A C game can call crabby_note_allocation(bytes) from its own allocator.)

In a running game

#![allow(unused)]
fn main() {
/// F5 starts timing your scopes and counting allocations; F6 prints where
/// the last half minute went.
fn perf_keys(ctx: &mut Ctx) {
    if ctx.key_pressed(Key::F5) {
        let on = !ctx.profiling();
        ctx.set_profiling(on);
    }
    if ctx.key_pressed(Key::F6) {
        println!("{}", ctx.perf_report());
    }
}
}

/// F5 starts timing your scopes and counting allocations; F6 prints where
/// the last half minute went.
static void perf_report_keys(crabby::ctx& ctx) {
    static bool on = false;
    if (ctx.key_pressed(CRABBY_KEY_F5)) ctx.set_profiling(on = !on);
    if (ctx.key_pressed(CRABBY_KEY_F6)) std::printf("%s", ctx.perf_report().text().c_str());
}

Or without changing the game: CRABBY_BENCH=600 cargo run --release plays a fixed run offscreen and prints the same report at the end (CRABBY_BENCH_REPORT=now.json also saves it); the MCP server’s perf_report tool takes profiling=true for an agent to do the same on a running game.

In a test or CI, with no window

#![allow(unused)]
fn main() {
/// Run it headless for 200 ticks (after 20 warm-up ticks), then read the
/// report and fail the test if it's too slow. Keep time limits generous:
/// machines differ.
#[test]
fn steering_stays_within_budget() {
    let (_world, report) = Game::new("bench", 1, 1)
        .sim_system(steering)
        .profile_headless(20, 200, |ctx| {
            for _ in 0..100 {
                ctx.world.spawn((Transform2D::default(),));
            }
        });
    println!("{report}");
    report.assert_within(
        &Budget::new()
            .frame_p99_ms(50.0)
            .system("steering", 10.0)
            .scope("steering/neighbors", 10.0),
    );
}
}

/// Run the game headless for 200 ticks (after 20 warm-up ticks) and keep it
/// within budget. Call it from a test or a CI job; keep time limits generous.
static void steering_stays_within_budget() {
    auto run = crabby::game("bench", 1, 1)
        .sim_system(steering, "steering")
        .profile_headless(20, 200, [](crabby::ctx&) {});
    std::printf("%s", run.report.text().c_str());
    std::string broken = run.report.check(R"({
        "frame_p99_ms": 50,
        "systems": {"steering": 10},
        "scopes": {"steering/neighbors": 10}
    })");
    if (!broken.empty()) throw std::runtime_error("over budget:\n" + broken);
}

profile_headless(warmup, ticks, setup) runs the simulation like run_headless, but times every tick and returns the report with the world. The first warmup ticks aren’t counted: buffers grow and caches fill.

2. Read the report

println!("{report}") prints this (shortened):

== performance report: 200 frames, 100 entities ==
frame ms: avg 0.052  p50 0.049  p95 0.061  p99 0.090  max 0.412
allocations per frame: 3.0 (0.2 KB), at most 3
-- systems (avg ms, max ms, allocs/frame) --
   0.047    0.390      3.0  my_game::steering
-- scopes (avg ms, max ms, calls/frame) --
   0.040    0.380      1.0  steering
   0.030    0.350      1.0  steering/neighbors
-- slowest frames --
frame 143: 0.41 ms, 3 allocations  [my_game::steering 0.39, steering 0.38, steering/neighbors 0.35]
-- warnings --
! `my_game::steering` allocates 3 times per frame
    -> Keep the buffer between frames instead: a Vec in a resource ...

• frame ms: the average says how fast it is, p99 and max say how smooth. A p99 much higher than p50 is stutter: the average hides it.
• systems: where the CPU time goes, slowest first. A system’s time covers everything it did, scopes and allocations included. Scopes sit inside systems; don’t add them up with the systems.
• allocs/frame: anything above zero in per-frame code deserves a look. Counts, unlike times, are the same on every machine.
• slowest frames: what was running in the worst frames. If these share a system or scope, that’s your hitch.
• render (windowed games): CPU time in systems and drawing, GPU time, draw calls and triangles for the camera and the shadows, and GPU time per pass. Display wait is idle time, not work.
• warnings: the report spots common problems (an allocating system, one system taking over half the time, spikes, GPU or CPU bound, heavy shadows, thousands of draw calls) and says what to try.

The same report is data: report.systems, report.scopes, report.frame_ms.p99, report.allocs and so on (C++: parse report.json()).

3. Budgets and comparing runs

A budget turns “it should stay fast” into a failing test. Limits are optional; anything you leave out isn’t checked:

#![allow(unused)]
fn main() {
use ponko::*;
let report = PerfReport::default();
let budget = Budget::new()
    .frame_p99_ms(4.0)
    .allocs_per_frame(0.0)
    .system("steering", 0.5)
    .scope("steering/neighbors", 0.3);
report.assert_within(&budget); // panics, listing what broke
}

std::string broken = report.check(R"({
    "frame_p99_ms": 4, "allocs_per_frame": 0,
    "systems": {"steering": 0.5},
    "scopes": {"steering/neighbors": 0.3}
})");
if (!broken.empty()) throw std::runtime_error(broken);

Timings differ between machines and runs, so give time limits headroom (twice what you see on your machine), or use them to catch only big regressions. Allocation counts are exact: allocs_per_frame(0.0) on a system that should never allocate is the budget to rely on.

To catch regressions without picking numbers, keep a report from a good run and compare against it:

#![allow(unused)]
fn main() {
use ponko::*;
let (_w, report) = Game::new("t", 1, 1).profile_headless(0, 5, |_| {});
report.save("target/perf-now.json").unwrap();
if let Ok(before) = PerfReport::load("perf-baseline.json") {
    for slower in report.regressions(&before, 0.25) {
        println!("slower: {slower}"); // e.g. system `my_game::steering` (ms): 0.047 -> 0.081
    }
}
}

(C++: perf_report::save, load and regressions.) A value counts when it grew by more than the tolerance and by a little in absolute terms, so noise in tiny numbers is ignored.

4. Common fixes

Allocates every frame. Keep buffers between frames: a Vec held in a resource (or captured by the system), clear()ed and refilled. Avoid format!, to_string, collect, Box and Vec::new + push in code that runs each tick. C++: reuse a std::vector and reserve.

One system dominates. Wrap its halves in scopes to see which. Then, in order of effort: iterate with for_each / par_for_each instead of iter() (ECS); do the work for a quarter of the entities per tick with ctx.turns(4); run it less often with sim_system_every (The simulation clock); use a spatial grid or flow field instead of checking every pair (Spatial queries).

Spikes. Look at slowest frames: the same system each time usually means work that sometimes does a lot (a path search, a rebuild, spawning a wave). Spread it over ticks, or do it ahead of time. Allocation bursts, asset loads and spawning thousands of entities at once show up the same way. ctx.record_trace(path, frames) shows a slow frame on a timeline in Perfetto.

GPU bound (GPU time close to the frame time, display wait large). The pass list says which pass. Shadows: fewer casters, a simpler ShadowMesh, set_mesh_shadows(false) for small things. Pixels: Game::msaa(2), fewer or smaller things drawn. Triangles: simpler meshes far away. Many draw calls: pack sprites into an atlas, share meshes and materials.

CPU bound in drawing. Compare draw calls and models with what’s really visible; cull and merge what you can.

When something is still slow after this: sample <pid> 5 on macOS or a profiler for the native code, or --features profiling with puffin_viewer for a live flame graph of the systems.

What isn’t measured yet

• GPU memory use and per-mesh memory.
• Rust allocations on other threads are counted for the whole process, so a system is charged for what rayon workers allocate during it.
• Scopes are process-wide: two games running side by side in one process (only tests do that) would see each other’s scopes.