Add extensible free-space motion benchmark

[yuecideng]

Description

This PR implements the first three benchmark steps discussed in #437:

• refactors the monolithic NeuralPlanner benchmark into configuration, adapter registry, runner, scenario, metric, artifact, aggregation, and reporting modules;
• adds free-space-common smoke and coverage suites with deterministic Franka case manifests;
• uses cuRobo as the primary baseline and keeps IK interpolation and TOPPRA as optional diagnostic baselines;
• leaves NMG as an explicit unsupported adapter stub while making its internal pos_eps and rot_eps independently configurable from suite-owned external validation thresholds;
• recomputes ordered waypoint, motion-validity, joint-limit, path-quality, latency, throughput, and memory metrics outside the planner timing boundary;
• writes resolved configuration, environment metadata, case manifest, numeric JSONL, aggregates, and one Markdown report containing exactly three tables;
• adds an explicit cuRobo backend preparation hook so construct, prepare, cold-plan, and warm-plan costs are measured separately.

Atomic-task remains out of scope for this PR and can be added through the extension points in a follow-up.

Dependencies: adds psutil>=5.9 for process RSS measurement.

Partially addresses #437.

Type of change

• New feature (non-breaking change which adds functionality)

Screenshots

Not applicable; this change provides a CLI benchmark and generated Markdown/JSON artifacts.

Validation

• black . — 580 files unchanged
• black --check --diff --color ./
• python -m compileall -q embodichain/lab/sim/planners/curobo/curobo_planner.py scripts/benchmark/__main__.py scripts/benchmark/planners/neural_planner tests/benchmark/planners/test_motion_generation_benchmark.py
• pytest -q tests/benchmark tests/sim/planners/test_curobo_planner.py -m 'not gpu and not slow' — 51 passed, 3 deselected
• manual minimal end-to-end runs: cuRobo on CUDA, IK interpolation on CPU with B=1/2, and TOPPRA on CPU

The nightly-scale coverage matrix was not run locally.

Checklist

• I have run the black . command to format the code base.
• I have made corresponding changes to the documentation
• I have added tests that prove the feature works.
• Dependencies have been updated, if applicable.

[Copilot]

Pull request overview

This PR replaces the previous monolithic NeuralPlanner benchmark with a new, extensible motion-generation benchmark framework centered on free-space cases and cuRobo as the primary baseline. It introduces suite-driven configuration, a registry-based adapter/scenario architecture, a generic runner with lifecycle timing phases, external validation metrics (ordered waypoint matching + motion-validity gate), and standardized artifacts/reporting (including a single Markdown report with exactly three tables).

Changes:

• Refactor benchmark implementation into modular motion-generation components (config/registry/runner/scenarios/metrics/aggregation/artifacts/reporting) and wire a new embodichain benchmark motion-generation CLI subcommand.
• Add deterministic free-space-common smoke/coverage suite YAMLs and a free_space scenario provider that generates stable manifests and start-state bins.
• Extend planners to support the benchmark lifecycle and cleanup: add CuroboPlanner.prepare_backend() for explicit backend warmup timing, and add ToppraPlanner.close() so adapters can reliably release worker pools.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/benchmark/planners/test_neural_planner_benchmark.py	Removes tests tied to the deleted monolithic NeuralPlanner benchmark implementation.
tests/benchmark/motion_generation/test_motion_generation_benchmark.py	Adds comprehensive unit tests covering ordered waypoint matching, outcome/aggregation semantics, reporting constraints (3 tables), adapter lifecycle, and runner gating.
tests/benchmark/motion_generation/init.py	Introduces test package for the new motion-generation benchmark suite.
scripts/benchmark/planners/neural_planner/run_benchmark.py	Removes the legacy monolithic NeuralPlanner benchmark runner.
scripts/benchmark/motion_generation/init.py	Adds package marker for the new motion-generation benchmark modules.
scripts/benchmark/motion_generation/run_benchmark.py	Adds the new CLI-compatible entry point supporting suite selection and overrides.
scripts/benchmark/motion_generation/config.py	Adds suite/config models, YAML loading, hashing utilities, and validation for free-space track parameters.
scripts/benchmark/motion_generation/registry.py	Adds registries for planner adapters and scenario providers.
scripts/benchmark/motion_generation/runner.py	Adds a generic benchmark runner with lifecycle timing, capability gating, artifact writing, aggregation, and report generation.
scripts/benchmark/motion_generation/aggregation.py	Adds aggregation logic for Time & Memory, Success & Metrics, and Leaderboard tables with case-macro averaging semantics.
scripts/benchmark/motion_generation/reporting.py	Adds Markdown reporting that always renders exactly three tables with stable schemas and formatted values.
scripts/benchmark/motion_generation/artifacts.py	Adds reproducibility artifacts: resolved suite YAML, environment metadata, case manifest, JSON aggregates, and JSONL trial records.
scripts/benchmark/motion_generation/metrics/init.py	Exposes metric helpers for performance timing and trajectory validation.
scripts/benchmark/motion_generation/metrics/performance.py	Adds timed-call measurement including CPU RSS (psutil), GPU allocator deltas, and optional peak GPU tracking.
scripts/benchmark/motion_generation/metrics/stats.py	Adds shared nearest-rank percentile helper.
scripts/benchmark/motion_generation/metrics/trajectory.py	Adds ordered waypoint matching, external motion-validity metrics, joint-limit checks, and path metrics.
scripts/benchmark/motion_generation/scenarios/init.py	Registers built-in scenarios for benchmark discovery.
scripts/benchmark/motion_generation/scenarios/base.py	Defines the ScenarioProvider contract for track case generation.
scripts/benchmark/motion_generation/scenarios/free_space.py	Adds deterministic free-space case generation for free-space-common (seed-stable manifests, bins, shapes).
scripts/benchmark/motion_generation/planners/init.py	Registers built-in planner adapters for benchmark discovery.
scripts/benchmark/motion_generation/planners/base.py	Defines PlannerAdapter lifecycle contract and metadata hashing/export.
scripts/benchmark/motion_generation/planners/curobo.py	Adds cuRobo adapter with explicit prepare phase and empty-world configuration.
scripts/benchmark/motion_generation/planners/ik_interpolate.py	Adds sequential IK + interpolation diagnostic adapter with per-env failure handling.
scripts/benchmark/motion_generation/planners/toppra.py	Adds IK+TOPPRA diagnostic adapter with optional dependency gating and explicit planner cleanup.
scripts/benchmark/motion_generation/planners/neural.py	Adds explicit unsupported NMG adapter stub with configurable internal eps values.
scripts/benchmark/motion_generation/suites/smoke.yaml	Adds smoke suite YAML for free-space-common with deterministic Franka manifests.
scripts/benchmark/motion_generation/suites/coverage.yaml	Adds coverage suite YAML expanding batch/path/bin/seed matrix.
scripts/benchmark/motion_generation/README.md	Adds usage and scope documentation for the new benchmark suite.
scripts/benchmark/motion_generation/BENCHMARK_DESIGN.md	Updates benchmark design doc references to the new module layout and CLI entry point.
scripts/benchmark/curobo_extraction/run_benchmark.py	Updates module run instruction to the new package path.
scripts/benchmark/curobo_extraction/init.py	Adds package marker for cuRobo extraction microbenchmark.
scripts/benchmark/main.py	Replaces the old planners-neural-planner subcommand with motion-generation and wires parser arguments.
pyproject.toml	Adds psutil>=5.9 dependency for process RSS measurement used in benchmark tooling.
embodichain/lab/sim/planners/toppra_planner.py	Adds explicit close() method and routes __del__ through it for reliable pool teardown.
embodichain/lab/sim/planners/curobo/curobo_planner.py	Adds prepare_backend() API to separate backend materialization/warmup from first plan timing.
docs/source/guides/cli.md	Updates CLI docs to reference the new motion-generation benchmark command.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.