CREXX

REXX Language implementation

View the Project on GitHub adesutherland/CREXX

CREXX Roadmap

Status: project direction. This document is not a release contract.

The current release documentation describes what CREXX implements today. This roadmap is the repository home for future direction, research themes, and discussion clean-up decisions that should not be mistaken for current language or toolchain behaviour.

Use this rule of thumb:

Release 1 Plan

The fixed-date path to Release 1 is tracked in release-1-plan.md. That plan carries the current Release 1 gates, scope tiers, provisional owners, and issue candidates. This roadmap remains the home for broader future direction and research themes.

Beta 3 Milestone

Beta 3 is the next Release 1 foundation milestone after the tagged beta 2 baseline. It is work in progress on develop until v1.0.0-beta.3 exists. The beta 3 planning note is releases/v1.0.0-beta.3.md. Candidate issues and working team guidance are tracked in planning/beta-3/issue-candidates.md until GitHub issues are created.

Date Milestone Direction
2026-06-17 Beta 3 opens Move live branch docs and version strings to beta 3 WIP while preserving beta 2 release notes as the latest completed beta baseline.
2026-07-03 Design lock Approve Level B/G split, plugin policy, UTF ownership, Level C MVP, GPU/threading scope, and issue ownership.
2026-07-31 Beta 3 foundation target Land or defer high-risk VM/compiler foundations, decide beta 3 package shape, and make the release note match actual tag assets before publication.
2026-08-14 Feature complete after beta 3 Freeze user-facing Release 1 surface except for release defects, documentation, QA, examples, packaging, and measured performance work.

Packaging Roadmap

Beta 2 shipped the package formats that were already in the release close-out path: portable ZIPs, signed Windows ZIPs through the maintainer signing flow, macOS .pkg installers when Apple signing and notarization are configured, and prototype Linux .deb packaging through the moving dev snapshot.

For beta 3, improve the end-user install experience while keeping portable ZIP assets available for CI, testing, and users who do not want a system install:

Platform Direction
macOS Treat the signed, notarized, stapled .pkg as the preferred user install. Keep ZIPs as portable developer/CI archives.
Linux Keep the .deb as adequate for Debian/Ubuntu-style users, but harden it before promoting it from prototype: install/uninstall smoke tests, dependency review, and later .rpm packaging.
Windows Add a signed NSIS setup.exe for beta 3 if the signing and upload flow is reliable. It should be built from the signed Windows payload, install CREXX into a normal Windows location, add the tools to PATH, register an uninstaller, and keep the signed ZIP available for portable use.

Longer-term Windows packaging may include WiX/MSI and winget publication, but the simple click-through NSIS installer is the next practical user-experience step.

Release 1 / Level B Quality Issues

These items are specific enough to track as GitHub issues because they affect the Release 1 or Level B quality bar.

Area Issue Source discussions Notes
Unicode and text semantics #583 #155, #162, #194, #231, #470 Define and verify Level B behaviour for case conversion, TRANSLATE, comparison, and byte/codepoint conversion decisions.

Closed Release 1 quality issues remain useful as evidence rather than active roadmap work:

Area Issue Outcome
Tool output paths #584 Closed with normalized -o behavior and regression coverage.
RXAS float literal precision #585 Closed with stored binary64 precision coverage separated from display formatting.
RXAS instruction coverage #586 Closed after the instruction inventory and regression surface were extended.

Platform Roadmap

Mainframe support is a long-term platform direction, not a Release 1 desktop release gate.

Theme Source discussions Direction
Legacy 32-bit platform validation Release 1 .int contract Keep .int signed 64-bit even when revisiting a 32-bit host ABI. Audit pointer-sized handles, RXBIN compatibility, compiler/toolchain availability, memory limits, and performance as separate platform work; do not restore a host-sized .int typedef.
z/VM CMS support #278 Keep the CMS interest and contact history as roadmap context. Current CMS work is best expressed as deterministic demos and ADDRESS environment compatibility rather than a full platform promise.
VM/370 build recovery #294, #322 Investigate cross-compilation and source-structure constraints after the Level B desktop release line is stable.
MVS/370 porting #379 Treat as a future platform project. Likely needs a dedicated maintainer, toolchain notes, and a clear cross-build strategy.

Language And Compatibility Roadmap

These items are useful direction but should not become Release 1 commitments until they are narrowed.

Theme Source discussions Direction
Classic compatibility BIFs #129 BITAND, BITOR, and BITXOR belong with broader Classic/Level C compatibility planning, not as isolated Level B promises.
RexxScript positioning Beta 3 planning Treat RexxScript as a modern interpreted-only Rexx-family surface that primarily uses strings. It is not the Level C compiler path, though it may later migrate toward a light Classic Rexx subset and should share BIF implementations where that is clean.
Level C canonical AST lowering Beta 3 planning, compiler/docs/levelc_working_architecture.md Level C is compiled Classic Rexx. The intended implementation path is to transform the Level C parse/AST shape into the canonical compiler AST shape, then use the normal validation, optimization, and emission pipeline where possible.
Source provenance through generated code Beta 3 planning Preprocessor output, RexxScript integration, parser-mode diagnostics, Level C lowering, TRACE, and debug metadata need a shared source-line/provenance strategy so errors map to the user source rather than generated text.
Classic Rexx value model Beta 3 planning Create a dedicated Classic Rexx value class with canonical string storage and cached derived forms such as integer or numeric values. This supports Level C and shared BIFs without leaking Level B typed semantics.
Classic Rexx variable pool Beta 3 planning, #424 Create a variable-pool abstraction for Classic Rexx lookup, assignment, stems, host-variable access, SAA, and ADDRESS integration. BIFs and Level C work should use this instead of ad hoc maps.
Shared Rexx BIF surface Beta 3 planning, #129 Curate BIFs that can serve RexxScript string-first use and later Level C Classic value/pool use. Prefer shared implementation strategy over per-BIF conversion islands.
Rexx-style loose comparison #233 and closed issue #150 Keep under future compatibility review. Current Level B comparison policy should remain explicit in the language reference.
Argument count and optional parameters #219 Current arg(), arg[], ..., and ?name behaviour is documented. Reopen only if a concrete unsupported case appears.
StringIterable loop sugar and callbacks #591 Level B should keep the minimal collection contract as StringIterable.iterator() returning a StringIterator with hasNext() / next(). Object-valued collections use the same explicit tagging style through ObjectIterator / ObjectIterable; string-key object maps use StringObject... names. A no-argument forEach() method is not useful without callable/reference support, so richer Java-style callback iteration belongs with future Level G facilities. Possible Level B syntax sugar such as loop item over collection should lower to the current iterator loop and validate against the relevant iterator interface contract. Bare collection names remain reserved for future Level G generic or generic-like surfaces. Object-key collections stay deferred until object equality/hash/ordering semantics are defined.
Compile-time build metadata #454 A _build_date()-style virtual function is a possible convenience, but it needs a naming and semantics decision before issue tracking.
Unused imports #467 and closed issue #441 Already tracked previously. Reopen only if the compiler policy is still wanted and not implemented.

Runtime, Backend, And Performance Roadmap

Theme Source discussions Direction
JIT / MIR / LLVM-style backend research #331 Research only. Keep separate from the interpreter and bytecode release contract.
File-I/O instruction migration Transferred former RCC-6; runtime-capability-composition-roadmap.md; Release 1 backlog candidate 42 Post-Release-1 candidate. Design a typed low-level rx_io call surface, replace raw FILE * integer exposure with context-owned handles, inventory producers/consumers, and measure call, code-size and startup effects before deciding dual lowering or opcode tombstones. Preserve byte/text/UTF behavior, signals, standard streams, close-on-exec, both VMs, packaging and late load. This row is planning authority, not implementation approval.
Measured RXAS instruction-family review Transferred former RCC-7; #288, #338, #357; Release 1 backlog candidate 43 Post-Release-1 candidate. Review the existing FNV-1a rxhash opcode, partitioned clock/environment/version/random utilities, then sockets and reflection in that order as separate decisions. Keep an instruction unless use, performance, ownership, size, compatibility and cross-platform evidence justify conversion. Do not reclaim numeric IDs without an approved RXBIN/ISA boundary.
Optimizer and loop super-instructions #339 BCTP / IGTBR optimizer work has landed. Future optimizer work should be benchmark-driven and covered by RXAS optimizer tests.
Performance benchmark portfolio expansion current results report POSTPERF-05 is complete and no further production performance stage is automatically authorized. Portfolio v3 retains all 14 AWFY lanes plus Base64, RexxCPS and separate capability controls. The Apple Stage 5 scorecard is complete. cREXX is 1.214718x/1.250737x faster than CPython on the equivalent common four, while the full control set splits seven median wins each; CPython’s strong object/container rows are therefore product evidence, not an assumed slow-language baseline. CD, DeltaBlue, Towers and Havlak receive bounded release reviews, with a production change only if current evidence supports a safe, general and material improvement. Storage/List ownership, NBody, Permute and any unselected release-review finding carry into the next release. Direct Java and CPython remain controls, primitive-Java work is not NetRexx capability, and exact-commit formal Linux QA-C remains the Stage 5 exit item. Any test revision prompted by a finding must preserve algorithm, work and observable result and repeat the affected qualification.
Generic final/concrete scalar access transferred PERF3-04 Complete. The accepted exact type-generic G1 lane plus four-family RXAS guard proof covers integer and float reads/writes, receiver/copyback/signal/TRACE/import semantics and both concrete VMs without new syntax, opcode, RXBIN or ABI. All 266 verdict processes and 2,249/2,249 final Debug tests pass; float writes improve by 32.75-35.41% and the selected RXBIN grows 2.07%. The bounded signal-policy worklist fixes exponential resolution. Adrian accepted the remaining +8.104172% (about 17.5 ms) real httpcodec assembler lifecycle cost for closure. Current packed i64/f32 forms remain controls; the typed-binary redesign stays in BINARY-01.
Native packed numeric storage and byte interoperability BINARY-01 (transferred from POSTPERF-04-PACKED-01) Release 1 Level B and explicit Level G owner verdicts accepted. .binary remains the sole Level B container and is guaranteed native-aligned. Zero-based <packed..int>(item) and <packed..float>(item) reads/writes expose contiguous host-native rxinteger/rxfloat items without endian or width conversion, a packed type/flag, or new RXBIN metadata. The symmetric .packedint and .packedfloat Level G classes own the same bytes directly in register.0.binary; their accepted exact-getter rework reaches direct Level B read parity while retaining the uninitialized-receiver signal boundary. Raw <at..type> access retains byte positions, encoded widths, endianness, conversion and unaligned interchange. rxstats and the now-implemented exact CPU rxvector provider are the first bulk consumers. Automatically packing ordinary .int[] and .float[] arrays is explicitly a post-release Level G roadmap item, including custom indexed-reference and boxed-array compatibility work.
Exact packed vector CPU provider RXVECTOR-01 Accepted and sanitizer-qualified. The process-reentrant Level G rxvector provider exposes explicit portable f32le conversion and exact deterministic cosine/top-k over BINARY-01 owners. The accepted profiling-off Release verdict reaches the shared direct-kernel ceiling and 14.68x-15.62x paired speedup over the Level B oracle. Installed dynamic/static selection, public documentation, bounded crexx-rag integration, proportional normal-Debug QA, and supported Linux sanitizer qualification are complete; GitHub supplies final-head cross-platform coverage.
RXAS control-flow and bounded compiler optimization PERF3-11/12 performance programme The reusable immutable CFG, signal policy, sparse component SSA/use index and transactional proof service are implemented. Current consumers include branch threading, conversion/copy placement, capability-lazy loop-scoped joined-key reuse and the completed exit-owned PARSE lowering. Continue only with evidence-selected bounded hoisting, register finalisation and late-inlining consumers, one production candidate and first Release verdict at a time; keep exact local normalizations in the cheap peephole and do not recreate dense whole-procedure scans.
UNICODE-NORM-02-F1: exact scalar accessor through a class-attribute receiver copies the whole object LEVEL-L-INLINE-01 and UNICODE-NORM-02-F1 performance activities Published to origin/hotfix and origin/develop at code SHA c0ac864d59428a807102a7b266933967f3b2e294; local QA complete and exact-SHA hosted QA pending. Exact call-free read-only scalar getters now borrow proved receiver storage without a whole-value COPY_REG_REG; local/imported methods share the same proof, indexed receivers are evaluated once through a live locator, mutation writes through to the original storage, and normal plus signal cleanup unlinks the borrowed receiver. References, escapes, whole-value replacement, dynamic/interface dispatch, unproved shapes and deliberate residual calls remain ordinary calls. The cumulative final-tree evidence is: prescribed focused Debug 10/10; full Debug 2,397/2,397 before the teardown-only SAN-007 repair plus post-repair normal-Debug SAN-006/SAN-007 2/2; complete post-repair Apple-ASan build and 2,398/2,398 CTests; ordinary profiling-off Release build with focused 10/10 and additional root cases 7/7; explicit optimized/no-opt dual-VM matrix 16/16; zero-conflict Lemon regeneration; and install, native package plus metadata smoke. This is published development content, not a release or tag.
RXC-CALL-REG-01: discarded call temporaries inflate procedure local counts RXC-CALL-REG-01 performance activity Published to origin/hotfix and origin/develop at code SHA c0ac864d59428a807102a7b266933967f3b2e294; local QA complete and exact-SHA hosted QA pending. RXC recycles discarded standalone-call results, temporary member receivers outside the reserved call window, and statement-call node registers before deferred cleanup. The permanent 32-call regression compiles, assembles and runs under both VMs in optimized and no-opt modes; the final dispatch procedure uses 6 optimized and 39 no-opt locals, within the 7/39 limits, while the retained pre-fix no-opt image required 101. The Unicode scale diagnostic fell from 6,455 to 361 locals in scratch but did not materially improve normalization wall time: this remains a compiler resource repair, not a claim that the generated Unicode normalizer is faster or selected.
CRI-18: disjoint block-local and enclosing values did not warn in a mixed if/else Downstream crexx-rag Codex extraction plus an independent 28-line Level-G reproducer; re-audited on origin/develop at 273c7b46 Reclassified as a source defect plus missing diagnostic coverage; there is no value-merge or register-allocation defect. A grouped DO arm creates a local scope, while a single-statement arm executes in the enclosing scope. The two result assignments therefore create different variables, and the later read legally selects the enclosing variable; taking the grouped arm leaves that object invalid and the expected method access raises SIGNAL OUT_OF_RANGE. The validator now reports the deterministic #NOT_IN_SAME_SCOPE warning when an implicit assignment creates the disjoint enclosing binding; an explicit name = .type declaration suppresses it, and a visible parent is reused quietly. Review of the first published fix found that its impact analysis was incomplete: compiler-exit-generated private assignments also triggered the new rule, and the TRACE scratch warning changed a syntax-highlight token from severity 48 to warning severity 50, failing the hosted Build workflow. The follow-up retains the rule and instead makes ADDRESS, EXECIO, QUERY, SIGNAL, SORT, SORTX and TRACE scratch bindings explicit. Permanent tests assert exact warning symbols and source locations, explicit suppression, repeated exit expansion without internal warnings, the TRACE severity contract, and crexx progress from fresh RXAS through fresh RXBIN for rexxcps_levelb, including more than 64 KiB of valid captured warnings. Focused Debug panels pass locally; issues #646 and #669 remain open pending reporter/cross-platform confirmation, and this hotfix is not closure evidence until its exact commit passes both hosted Build and Sanitizer workflows.
String performance follow-up #470 Performance results are useful, but regressions or semantic fallout should be tracked through concrete bugs such as #583.

The post-PERF3 performance order is therefore benchmark foundations, generic scalar-access proof, then broader bounded optimizer consumers. BINARY-01 is a separate future language/runtime design and does not interrupt that sequence. VM handler layout is frozen until release-candidate finalisation, when the current portfolio and platform evidence must be regenerated before any low-level shape decision.

Concurrency Roadmap

This is the sole project roadmap entry that orders concurrency work. The concurrency/WORKLIST.md and concurrency/QA-CLOSEOUT.md files are detailed status and evidence ledgers; they do not create a separate product roadmap.

The initial concurrency implementation is feature-complete on develop for its frozen Release 1 surface: local and isolated-process structured tasks, bounded channels and endpoints, child-process redirection, the Level B control classes, Level G task/parallel syntax, and the concurrent HTTP client/server and LLM transports over one private protocol backend.

Release 1 concurrency work is now a bounded QA and publication programme. The target-host commands and evidence rules are frozen in concurrency/qa/, and the independent continuation is defined in INDEPENDENT-REVIEW-PROMPT.md.

Gate Status Work Exit condition
QA-A: test readiness complete Independently review every solution point, close direct test gaps, maintain the labelled matrix in concurrency/TEST-MANIFEST.md, and prepare exact platform commands. The frozen candidate is test-ready on Mac and every solution point has a source/test/risk disposition.
QA-B: Mac closeout complete Focused Debug/Release, both applicable VM modes, optimized/unoptimized toolchain paths, sanitizer, stress, broad regression, install and package proof pass. The retained performance run remains battery-only diagnostic evidence and is not relabelled. It found no confirmed adverse guard; earlier AC baselines remain valid. Adrian waived an unchanged AC replay on 2026-08-18 unless a relevant source change, evidence inconsistency or new performance question reopens it.
QA-C: Linux qualification delegated to GitHub final-head closeout Build CREXX runs the Linux MinSizeRel build, CTest and package path; Sanitizer QA runs Linux x64 ASan/LSan. Adrian selected those workflows as the closeout authority on 2026-08-23. The final-head GitHub workflows pass and retain their logs/artifacts.
QA-D: Windows qualification complete Exact clean commit 2b793c81e passed the frozen native Windows matrix under MSVC, Clang and GCC, including full CTest, TLS, stress, install and ZIP-package proof. The retained runner evidence, inventories and digests satisfy the Windows defect-return discipline; no replay is needed without a concrete inconsistency or new source candidate.
QA-E: publication decision complete: published as initial Reconcile packages, release notes, compatibility boundary and residual risks. Adrian selected publication as initial on 2026-08-16. The corrected publication commit 53b3de77a passed Build CREXX on Windows x64, Linux x64, macOS arm64 and macOS x86_64, published the development snapshot, and passed CodeQL. Native Windows qualification is complete; Linux/final-head closeout follows the selected GitHub authority in QA-C.

Feature development remains frozen during this programme. Concrete services and .taskscope.ask(), provider type 3, a public provider-plugin ABI, pool telemetry, server TLS/readiness/background lifecycle, HTTP/2 and WebSockets are post-Release-1 candidates requiring separate design approval. Shared writable VM state, detached ordinary tasks and public worker identities remain outside the model. These candidates are recorded here so they are not lost; they are not an approved automatic sequence after QA-E.

The publication choice does not waive QA-B through QA-D and does not declare the surface stable. CI failures receive bounded tactical repairs and replay; native Linux and Windows failures return to Mac under the same defect-return policy.

Library, Plugin, And Host Integration Roadmap

Theme Source discussions Direction
Runtime capability composition planning/release-1/runtime-capability-composition-roadmap.md, PERF3-07 CAP-05, CREXXRAG-SHA256 RCC-1 through RCC-5F are published complete. They provide declarative provider dependencies, static-first trusted dynamic resolution, automatic native-package selection, explicit per-mutable-module-instance initialization, rx_hash, the rxfloat/rxint/rxdecimal family, packed-float rxstats, and the rxid/rxfs/rxplatform split. The broad pre-release system, mixed rxmath, and boxed statistics surfaces are retired; only direct scalar rxmath compatibility names remain in rxfloat. Existing metadata homes stay RXBIN 007; a file-structure change moves to 008. Consolidated Release, Debug, install/package, documentation, Apple-ASan, and supported Linux ASan/LSan QA is green. GitHub Build CREXX and Sanitizer QA supply final-head cross-platform closeout. The former RCC-6/RCC-7 instruction-to-call candidates are transferred to the broader runtime/backend roadmap and post-Release-1 backlog; they are not outstanding gates in this composition roadmap.
Math library expansion #384 Extend the separated rxfloat, rxint, rxdecimal, and rxstats families according to their numeric contracts. The historical mixed rxmath provider is retired; its remaining names are direct scalar compatibility registrations supplied by rxfloat, not a future bundle home.
Regex support #399 and closed issue #414 RxLite now provides a pure-Rexx regex surface in rxfnsb. External/native regex dependencies remain a future packaging decision, not an open Release 1 blocker.
System plugin portability #398 Keep platform coverage under normal plugin test hardening. Open a fresh issue only for a failing platform-specific test.
REXX/SAA compatibility #424 Continue through the crexxsaa and RXPA host-integration path. Variable-pool emulation needs explicit design before new commitments.
IO endpoints, process pipes, and native handles ai-context/CREXX_IO_PIPE_WORKING.md, #491 Bounded provider type 4 endpoints and type 5 structured child processes now underpin ADDRESS redirection and concurrent HTTP streaming. Broader rxio.* stream classes, reusable pipeline helpers and any public native-handle surface remain future work.
Mixed Rexx/native libraries #432 Combining Rexx scripts and native plugin functions into one library remains an architecture direction for plugin packaging.
Threads and subtasks #491, concurrency/WORKLIST.md Local-thread and isolated-process structured tasks form the published initial receiver-owned transfer surface with no shared writable VM state. Native Windows qualification is complete; final-head Linux/build/package and sanitizer closeout uses the selected GitHub workflow authority. Durable single-owner services and open-host/provider extension are separately approved later work.
CRI-17: native RXPA providers in attached tasks Downstream crexx-rag CRI-17; RXVM_INTERPRETER.md Open CREXX attached-task provider-resolution defect/capability gap; not a SQLite/plugin defect and not a process-worker blocker. A separate process starts a fresh VM and resolves its native providers normally. The current local-thread task path requires a sealed bytecode-only generation: a generation containing an already loaded native module returns RXVM_PROGRAM_NATIVE_EXCLUDED, while a provider loaded after sealing is not propagated to the attached task VM. Design a supported declarative provider-resolution path for each attached task VM, create and destroy its normal RXPA V2 per-VM session, preserve provider identity/lifetime and fail-closed diagnostics, and prove the result with a generic session-aware provider through local tasks. Until then attached tasks exchange transferable values and their controller owns native-provider calls. This row records direction and closure evidence; it is not automatic implementation approval or a Release 1 commitment.

Closed As Already Handled Or Stale

The following discussion topics were not converted into issues because the current tree already handles them, they are covered by existing closed issues, or the information was too stale to carry forward as a commitment: