Tags

Tags give the ability to mark specific points in history as being important

  • prereg-apollo-v1.2-freeze

    6ed3a3c5 · style: ruff format analysis + merge scripts (whitespace only) · 
    Pre-specification anchor chain for the Apollo 11 confirmatory replication.

Preserves the granular freeze commits cited by docs/prereg-apollo-confirmatory.md
(squash-merged to main as 1a031f6 via MR !27, which collapsed the chain):
  a650aa2 instrument + frozen analysis (pre-outcome)
  1a6ace7 prereg v1.1 committed alongside (pre-outcome)
  aff2646 plumbing pilot + shard-merge tooling (pre-outcome)
  9814194 prereg v1.2 model deviation (pre-outcome)
  efe99fb + 6ed3a3c post-collection lint/format only (verified zero behavioral delta)
Collection ran from 9814194; verdicts at sysml-bench-results 3fdd21a.

  • v0.1.1

    protected
    10ef7731 · fix: bump v0.1.1, decouple publish jobs, buildah for container · 
    sysml-bench v0.1.1: CI fix — buildah container build, decoupled publish jobs

Fixes:
- Replace docker-in-docker with buildah for podman runner compatibility
- Decouple publish stages so PyPI, container, and HF dataset run in parallel
- Container: registry.gitlab.com/nomograph/sysml-bench:v0.1.1
- pip install sysml-bench==0.1.1

  • v0.1.0

    protected
    7b021579 · fix: replace docker-in-docker with buildah for container build · 
    sysml-bench v0.1.0: Reproducible benchmark for SysML v2 model comprehension

Initial public release:
- 132 tasks across 13 categories (O1-O14)
- 5 models evaluated (Claude 3.5/3.7, GPT-4o, Gemini 2.0 Flash, DeepSeek-V3)
- Tool-augmented evaluation with sysml-cli (tree-sitter parser)
- pip install sysml-bench
- Container: registry.gitlab.com/nomograph/sysml-bench:v0.1.0
- HuggingFace: nomograph/sysml-v2-reasoning-benchmark