Repository navigation
nist_sync: split ControlCatalogs by NIST family - #15115
Merged
Mab879 merged 1 commit intoOct 29, 2026
Merged
Conversation
ggbecker
marked this pull request as draft
September 10, 2026 19:13
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
ggbecker
force-pushed
the
split-gemara-format-into-groups
branch
from
September 11, 2026 09:41
b1a6a65 to
1f5094a
Compare
ggbecker
marked this pull request as ready for review
September 11, 2026 09:45
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
Member
Author
|
This PR complytime/complytime-policies#53 has contents from this PR |
Member
Author
|
I'm self-merging this pull request as it has been a long standing PR and it has been already used to submit changes to complytime/complytime-policies#53 This is to allow us to proceed with related work with regards to NIST 800-53 and Gemara/Complytime. Rebasing the PR before merging it. |
…iases
Match the layout complytime-policies uses (see complytime/complytime-
policies#55): one ControlCatalog per NIST 800-53 family (AC, AU, SI, ...)
alongside the existing combined catalog, so files stay reviewable and
each catalog's 'groups' scope is a single family.
- gemara/catalog.py: add GemaraCatalogBuilder.build_by_group(), returning
one catalog dict per family with a family-scoped metadata id/description
and title suffix.
- export_to_gemara.py: write the per-family catalogs to
build/gemara/{product}/by_group/nist-800-53-rev5-{product}-{family}-catalog.yaml.
- generate_policies_pr.py: stage those into
governance/catalogs/nist-800-53/{product}/nist-800-53-{product}/, matching
the complytime-policies directory structure.
Also fix a pre-existing bug hit while generating these: ruamel.yaml aliases
list/dict objects by id(), and CPython can reuse the same id() for distinct
short-lived lists of equal length (e.g. repeated ['fedora-low'] built fresh
in a loop). This silently collapsed unrelated 'applicability' lists into
YAML anchors/aliases (&id001 / *id001) in the generated catalogs. Set
representer.ignore_aliases = lambda data: True in every script that dumps
these structures (export_to_gemara.py, generate_policies_pr.py,
generate_complyctl_bundle.py) to disable aliasing entirely.
ggbecker
force-pushed
the
split-gemara-format-into-groups
branch
from
October 29, 2026 14:15
1f5094a to
8b37e9d
Compare
Mab879
approved these changes
Oct 29, 2026
ggbecker
added a commit
to ggbecker/content
that referenced
this pull request
Nov 5, 2026
The export-to-gemara workflow only runs on push to master and on a schedule, so a broken nist_sync script (like the missing gemara/yaml_output.py in ComplianceAsCode#15115) is invisible until after merge. Add a PR-triggered gate that, on changes under utils/nist_sync/ or the NIST 800-53 control files: - imports every script (--help) so a missing module fails fast, - runs the full export with --validate and fails on any [WARN]/[ERROR]/ FAIL marker (the export itself only warns and exits 0), - runs the export test suite, - runs generate_complyctl_bundle.py and generate_policies_pr.py. The gate uses the in-Python structural validation rather than CUE, so it needs no external schema/binary pins; authoritative CUE validation stays in export-to-gemara.yml.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
ControlCataloggeneration for NIST SP 800-53.complytime-policiesrepository layout.Rationale:
complytime-policiesand keeps each catalog scoped to a single NIST 800-53 family.Review Hints:
Review the changes together across:
utils/nist_sync/export_to_gemara.pyutils/nist_sync/gemara/catalog.pyutils/nist_sync/generate_policies_pr.pyutils/nist_sync/generate_complyctl_bundle.pyGenerate artifacts for all supported products:
python3 utils/nist_sync/export_to_gemara.py \ --products rhel8,rhel9,rhel10,fedora \ --output-dir build/gemara \ --validate