Transpile the Hack conformance fixtures into a Psalm test suite

[danog]

Instead of pointing each Hack conformance fixture to an "equivalent" Psalm test (//// psalm-test:), the fixtures are now transpiled automatically into a pure PHP test suite, so every fixture is checked against both HHVM and Psalm.

Changes

• Subfolders: fixtures are split by topic into bin/hack-conformance/fixtures/{type_variables,contexts,polymorphic_contexts}/. run.php, HackConformanceTest and the HHVM CI job find them recursively.

• Transpiler (bin/hack-conformance/Transpiler.php): converts the subset of Hack the fixtures use into PHP with Psalm docblocks. It does not parse Hack itself: it walks the parse tree HHVM's own parser prints (hh_parse --full-fidelity-json-parse-tree, run in the pinned image and cached in .hh-parse-cache/). The mapping table is at the top of the file:

  • Hack types become a native type plus a docblock type.
  • Contexts become @psalm-capabilities. A missing context is Hack's [defaults], so it becomes @psalm-impure. Every context list includes read-props, because Hack's [] may read properties.
  • Abstract context constants become purity templates (as is the lower bound, super the upper), and a subclass binding one becomes @extends Parent[...]. Concrete context constants become @psalm-type / @psalm-import-type aliases.
  • [ctx $f] with (function()[_]: T) becomes Closure[_]. A [_] anywhere else is an error.
  • A lambda without a context gets the context of the function around it, as in Hack.
  • Also converted: lambdas, Hack arrays and shapes, inout (as by-reference plus write-refs), readonly, is, and function pointers.

  An unknown parse-tree node kind is an error naming it and its line, never a silent mistranslation.

• Generator (bin/hack-conformance/transpile.php): writes tests/HackConformanceTranspiledTest.php. A fixture HHVM accepts becomes a valid-code case, and one it rejects becomes an invalid-code case.

• Staleness check: transpile.php --check transpiles again and compares the result with the committed suite.

  • HackConformanceTranspiledTest::setUpBeforeClass runs it, so every transpiled case fails on a stale suite, even when one case is run alone.
  • Where HHVM cannot run, --check exits 3, and the committed cases run as they are.
  • HackConformanceTest::testTranspiledSuiteIsUpToDate skips in that case. The HHVM CI job runs it with --fail-on-skipped, so there it must pass.

• New fixture headers:

  • //// psalm-error: — the issue an error fixture must produce in Psalm.
  • //// psalm-ignore: , ... — issues beside the point of the fixture.
  • //// psalm-divergence: — a known disagreement with Hack, generated as a SKIPPED- case.

Divergences the transpiled suite shows

• Covariant callable parameters (type_variables/multiple_return_types, type_variables/empty_construction_union_return): Psalm reports InvalidTemplateParam for @param Closure(TKey): mixed in a @template-covariant TKey class unless the method is mutation-free. That position is covariant, and Hack accepts it. The hand-written tests suppressed this; the fixtures now use psalm-ignore.
• type_variables/typed_closure_param_empty_construction (skipped): Psalm types an empty new ArrayCollection() as ArrayCollection and keeps that after the assignment to an ArrayCollection property. It then reports ParadoxicalCondition on the typed lambda parameter, which Hack accepts.

Verification

• HackConformanceTranspiledTest: 28 cases pass, 1 is skipped (the divergence above).
• php bin/hack-conformance/run.php: HHVM agrees with all 29 fixtures.
• phpcs and Psalm self-analysis are clean on the changed files.

🤖 Generated with Claude Code