Skip to content

test(lint): cover three caption rules that had no tests - #5436

Merged
miguel-heygen merged 2 commits into
heygen-com:mainfrom
aron-intframe:test-cover-three-caption-rules
Oct 26, 2026
Merged

miguel-heygen merged 2 commits into
heygen-com:mainfrom
aron-intframe:test-cover-three-caption-rules

Conversation

@aron-intframe

Copy link
Copy Markdown
Contributor

What

Tests for caption_transcript_not_inline, caption_textshadow_on_group_container and caption_fittext_scale_mismatch. No rule behaviour changes.

Why

No test mentioned any of the three codes.

Related work

Refs #5388.

How

Four cases per code: a trigger asserting code and severity, plus negatives pinning each gate. The caption_fittext_scale_mismatch block asserts the figures the rule reports about itself (1600 x 1.3 -> 2080px, hint 1307px) and that applying the hint's own maxWidth clears the warning, as the existing "its fix must not create an error" block does.

Noted rather than changed, per the issue:

  • 1600 * 1.1 is 1760.0000000000002, so with the threshold at effectiveWidth > 1760 a composition exactly at the limit reports. The negative case uses 1600 x 1.05.
  • caption_textshadow_on_group_container gates on caption CSS only; its two siblings also accept options.filePath.

Test plan

bun run --cwd packages/lint test -> 20 files, 919 passed (907 before). oxlint, oxfmt and tsc clean.

Every test was re-run against a deliberately broken rule (inverted gate, regex targets dropped, threshold moved, hint constant changed); each break turned exactly one red. The three codes report nothing across the 874 HTML files in registry/ and skills/.

  • Unit tests added/updated

The commit is SSH-signed but our key is not on the account yet, so GitHub shows unknown_key; registering it flips the commit with no re-push.

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at b04a7c94. Thanks!

captions.test.ts passes 20/20, and 10 of 11 rule mutations were caught. The fitText tests check the exact sizes in the message and in the suggested fix, and confirm that applying the fix clears the warning. Merges cleanly.

Optional: removing the caption-file check on caption_textshadow_on_group_container is not caught. A case with a non-caption composition would cover it.

— Rames

@jrusso1020

Copy link
Copy Markdown
Collaborator

Thanks for this! It's approved and CI is green, but main only accepts verified commits. Your commit b04a7c94 is signed, but GitHub doesn't recognize the key (unknown_key), so it can't merge yet.

To fix it:

  1. Add the public key you signed with to GitHub as a Signing key (Settings → SSH and GPG keys; for SSH keys pick Key type: Signing key).
  2. Make sure the commit email (hello@intframe.com) is a verified email on your GitHub account.
  3. Re-sign and push so GitHub checks it again:
    git commit --amend --no-edit -S
    git push --force-with-lease

Docs: About commit signature verification.

Once the commit shows Verified, we'll re-approve and merge.

— Rames

auto-merge was automatically disabled October 13, 2026 02:31

Head branch was pushed to by a user without write access

@aron-intframe

Copy link
Copy Markdown
Contributor Author

Checked the signing side before writing back, and your read holds: the signature itself is fine, the key is the problem. I ran GitHub's own copy of the signature and payload for b04a7c94 (from /commits/b04a7c94) through ssh-keygen, then looked at what the account actually has:

$ ssh-keygen -Y verify -f allowed_signers -I hello@intframe.com -n git -s commit.sig < payload.bin
Good "git" signature for hello@intframe.com with ED25519 key SHA256:hiiSV13V5UKmkhtE1G7UgQCF9in1tb/uuLAyuT7iSzU
$ curl -s https://api.github.com/users/aron-intframe/ssh_signing_keys   # and /gpg_keys
[]

One part of step 3 does not hold on its own, though. Re-signing and force-pushing changes nothing while the key is unregistered. I pushed a freshly signed commit a few minutes ago and GitHub reports the same reason for both:

b04a7c94 False unknown_key
bf802b50 False unknown_key

So step 1 is the whole fix, and it needs a person in account settings. Our automation token (scopes repo, workflow) is refused:

$ curl -s -o /dev/null -w "HTTP %{http_code}" -X POST https://api.github.com/user/ssh_signing_keys -d '{"title":"INTFRAME oss signing","key":"ssh-ed25519 AAAA..."}'
HTTP 404

That registration is queued with the account holder. The key is ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMU8Jg6BZk9qnWQuwtutIK+NyBj4vabunmoiYTI2E8Pi (SHA256:hiiSV13V5UKmkhtE1G7UgQCF9in1tb/uuLAyuT7iSzU), and the commit email is hello@intframe.com as you noted.

Not verified by me: whether adding the key re-verifies the commits already pushed, with no further push. I have no way to test that until the key is on the account, so if they still read unknown_key afterwards I will amend and force-push exactly as you wrote.

Separately, your optional review note was right, and it is covered now. Deleting the isCaptionFile gate from caption_textshadow_on_group_container left the suite fully green, so nothing pinned it:

 Test Files  20 passed (20)
      Tests  919 passed (919)

bf802b5 adds a non-caption composition that tweens textShadow on a container the rule's pattern does match (tl.to(el, { textShadow: ... }) with only .title-group in the CSS). With the gate deleted it is the only red test:

 FAIL  src/rules/captions.test.ts > caption rules > caption_textshadow_on_group_container > does not warn for a glowing title group outside a caption composition
      Tests  1 failed | 919 passed (920)

With the rule restored, and tsc, oxlint and oxfmt clean on the file:

      Tests  920 passed (920)

Two things the push caused, both yours to decide: it dismissed your approval at b04a7c94, and I am happy to squash the two commits into one if you prefer a single commit under the verified-commits rule (they are signed with the same key, so they flip together). Also the new commit's workflow runs are sitting at action_required for the fork, so CI needs your click before it reads green again.

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still approved at bf802b50. The only change since my approval at b04a7c94 is one new test in packages/lint/src/rules/captions.test.ts:329-350: a glowing title group with a textShadow tween in a file that has no caption classes. That covers the gap I mentioned last time.

What I ran:

  • captions.test.ts: 21/21 pass.
  • Mutation: I removed the caption-CSS gate (if (!isCaptionFile) return findings;) in captions.ts. The new test fails, and so does the existing transcript negative, so both caption-file gates are now pinned.

The branch is a plain fast-forward from the approved commit with no rebase, and it still merges cleanly. CI workflows haven't run on this fork yet and need a maintainer to approve them.

— Rames

@aron-intframe

Copy link
Copy Markdown
Contributor Author

Re-ran your checks on bf802b50 before answering. Your mutation result holds, and I split it per gate so each test is tied to the gate it covers.

Baseline matches your 21/21:

$ bun run --cwd packages/lint test src/rules/captions.test.ts
      Tests  21 passed (21)

Worth noting for the record: captions.ts carries that exact line twice, at 80 (caption_transcript_not_inline) and 160 (caption_textshadow_on_group_container), so deleting "the caption-CSS gate" by text removes both at once, which is why you saw two reds. Deleting them one at a time:

$ sed -i '80d' packages/lint/src/rules/captions.ts
 FAIL  src/rules/captions.test.ts > caption rules > caption_transcript_not_inline > does not error for a non-caption composition that fetches a transcript
      Tests  1 failed | 20 passed (21)

$ sed -i '160d' packages/lint/src/rules/captions.ts
 FAIL  src/rules/captions.test.ts > caption rules > caption_textshadow_on_group_container > does not warn for a glowing title group outside a caption composition
      Tests  1 failed | 20 passed (21)

Each gate turns exactly one test red and nothing else, so "both caption-file gates are now pinned" is true per gate, not just in aggregate. Full package is 20 passed (20) files / 920 passed (920) tests, and git diff --stat b04a7c94 bf802b50 is one file, +23 lines, which matches your reading of captions.test.ts:329-350.

One line of your review was stale by the time it posted. CI did run on this fork and it is green, 65 success and 14 skipped with no failures:

$ curl .../commits/bf802b50/check-runs?per_page=100 | jq '[.check_runs[].conclusion] | group_by(.) ...'
{"skipped":14,"success":65}
2026-10-15T08:42:31Z

That last timestamp is the final check completing, 46 seconds before your review at 2026-10-15T08:43:17Z. So the workflows had already been approved and had finished, no click needed there.

That leaves the signature as the only blocker, and it is now confirmed from the ruleset rather than inferred:

$ curl .../repos/heygen-com/hyperframes/rules/branches/main | jq -r '.[].type'
required_signatures
$ graphql pullRequest(number:5436){mergeable mergeStateStatus reviewDecision}
{"mergeable":"MERGEABLE","mergeStateStatus":"BLOCKED","reviewDecision":"APPROVED"}
b04a7c94 false unknown_key
bf802b50 false unknown_key

New, and not in our favour: I tried to sidestep the unregistered key by having GitHub author the commit itself through the Contents API, so it would carry GitHub's own web-flow signature. I ran it on a throwaway branch in our fork, not on this PR, and GitHub did not sign it:

$ curl -X PUT .../contents/.sigtest-probe -d '{"message":"chore: signature probe","branch":"sigtest-scratch",...}' | jq -c '.commit.verification'
{"verified":false,"reason":"unsigned","signature":null,"payload":null,"verified_at":null}

That branch is deleted. So there is no route from our side that avoids registering the key, and /users/aron-intframe/ssh_signing_keys is still []. The registration is queued with our account holder and has been for three days.

What I am deliberately not doing: not force-pushing. A re-sign cannot change the state while the key is unregistered, which the b04a7c94 / bf802b50 pair above already demonstrates, and require_last_push_approval is on, so a push would only throw away the approval you just gave for no gain. The branch stays where you approved it.

Still not verified by me: whether registering the key re-verifies these two commits with no further push. I cannot test that until it is on the account. If they still read unknown_key afterwards, I will amend and force-push as you first described, and squash the two into one commit at the same time unless you would rather keep them separate.

If you would rather not wait on us, taking the 23 lines over to a branch on your side would merge today. Your call, and no objection from me either way.

@jrusso1020

Copy link
Copy Markdown
Collaborator

This is approved, but GitHub doesn't recognize the key these commits are signed with, so they show as unverified and main won't accept them. Adding that signing key to your GitHub account (Settings, SSH and GPG keys, as a signing key) usually fixes it without a new push. If not, a re-signed force-push works too.

— Rames

@aron-intframe
aron-intframe force-pushed the test-cover-three-caption-rules branch from bf802b5 to faa4913 Compare October 26, 2026 02:29
@aron-intframe aron-intframe reopened this Oct 26, 2026
@aron-intframe

Copy link
Copy Markdown
Contributor Author

The signature blocker is gone, and not the way either of us described. The key is still not on the account, but both commits on this branch now read Verified:

c3c72873 True valid | test(lint): cover three caption rules that had no tests
226fc531 True valid | test(lint): pin the caption CSS gate on the group textShadow rule

I have to correct myself first. On 10-16 I told you there was no route from our side that avoided registering the key. That was wrong, and I had tested only one of the two. The REST contents API does not sign ("reason": "unsigned", as I reported). The GraphQL createCommitOnBranch mutation does: GitHub signs the commit with its own web-flow key, wasSignedByGitHub: true, committer GitHub <noreply@github.com>. I probed it on a throwaway branch in our fork first, confirmed verified: true, reason: valid, then rebuilt this branch's two commits through it. That branch is deleted.

So of your two sentences, the second does not hold for us and the first I still cannot test. Our key is unregistered as of today, and our token is still refused:

POST /user/ssh_signing_keys -> HTTP 404

A re-signed force-push with that key changes nothing, which b04a7c94 and bf802b50 already showed, both unknown_key. Whether registering the key would have re-verified them with no push is still not verified by me, and now moot.

What I pushed is the same content you approved, not a rewrite. Same two commit messages, same order, same base, and the tree and blob are identical to bf802b50:

bf802b50 tree=a18decafcd0c73ae7fd04ed8137d6618a6e98d92 blob=39694bb26a87029b5d5e8001e1605b4b3dd37205
226fc531 tree=a18decafcd0c73ae7fd04ed8137d6618a6e98d92 blob=39694bb26a87029b5d5e8001e1605b4b3dd37205

Re-ran everything on that tree, packages/lint:

      Tests  920 passed (920)

and the per-gate mutations still each turn exactly one test red, sed -i '80d' then sed -i '160d' on captions.ts:

 FAIL  src/rules/captions.test.ts > caption rules > caption_transcript_not_inline > does not error for a non-caption composition that fetches a transcript
 FAIL  src/rules/captions.test.ts > caption rules > caption_textshadow_on_group_container > does not warn for a glowing title group outside a caption composition
      Tests  1 failed | 20 passed (21)

A mess of my own making, before you find it in the timeline. I reset the branch ref to the merge-base as a separate step before creating the two commits, which left the PR at 0 commits for about 90 seconds, and GitHub auto-closed it. The timeline reads closed 02:29:01Z, head_ref_force_pushed 02:29:02Z, then reopened 02:30:34Z, all by us. I reopened it and your two approval records survived. I should have built the commits on a temporary ref and moved this one once. No content was lost, but the close is on me, not on you.

Where it stands now:

{'state': 'OPEN', 'mergeable': 'MERGEABLE', 'mergeStateStatus': 'BLOCKED', 'reviewDecision': 'APPROVED'}

Two things still need you, neither of which I can do. The fork's workflow runs for the new head are waiting on your approval click:

Counter({('completed', 'action_required'): 8})

and require_last_push_approval is on in the main ruleset, so our push means it needs your re-approve before it can go in.

One honest side effect: these commits are authored as 41572471+aron-intframe@users.noreply.github.com with GitHub as committer, so the commit email is no longer hello@intframe.com and your step 2 about verified emails no longer applies. If you would rather have one commit than two under the verified-commits rule, say so and I will squash them the same way, which is one more signed commit and no change to the diff.

— Aron

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked at 226fc53. Test-only, as the issue asks: positive and negative cases for caption_transcript_not_inline, caption_textshadow_on_group_container and caption_fittext_scale_mismatch. I ran captions.test.ts: 21 passed. With the three rules' finding codes changed in captions.ts, 4 cases fail, at least one per rule, so each new test really reads its rule.

Verdict: APPROVE
Reasoning: closes #5388 with tests that fail when the rules break; no production change.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 26, 2026
Merged via the queue into heygen-com:main with commit 95aba51 Oct 26, 2026
70 of 72 checks passed
SanHsien added a commit to SanHsien/hyperframes that referenced this pull request Oct 26, 2026
Merge 20 more upstream commits (acd0bc3..ad4d3fa, v0.8.205: heygen-com#6403,
heygen-com#6363, heygen-com#6209, heygen-com#6395, heygen-com#6424, heygen-com#6426, heygen-com#6442, heygen-com#6430, heygen-com#6402, heygen-com#6435, heygen-com#6439,
heygen-com#6449, heygen-com#5436, heygen-com#6455 and docs). No conflicts. Record the 11 open PRs
heygen-com#6440-heygen-com#6458 and issue heygen-com#6443 in docs/DECISIONS.md (none adopted), re-check
the earlier open PRs, and advance the commit watermark to ad4d3fa, the
PR watermark to heygen-com#6458 and the issue watermark to heygen-com#6443.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@aron-intframe

Copy link
Copy Markdown
Contributor Author

Re-ran your check before answering, and your mutation result holds exactly as you reported it: 4 cases fail, at least one per rule.

Baseline on the merged content (captions.test.ts is blob 39694bb2 at 226fc531, at the squash commit 95aba51e and at today's main tip 736f38ff4, so these are the same tests you approved):

      Tests  21 passed (21)

Then the rename you described, the three finding codes in captions.ts (code: "caption_transcript_not_inline" and its two siblings) changed to ..._MUT:

 FAIL  src/rules/captions.test.ts > caption rules > caption_transcript_not_inline > errors when the transcript is fetched instead of inlined
 FAIL  src/rules/captions.test.ts > caption rules > caption_textshadow_on_group_container > warns when textShadow is tweened on the group element
 FAIL  src/rules/captions.test.ts > caption rules > caption_textshadow_on_group_container > warns when textShadow is tweened on a group selector
 FAIL  src/rules/captions.test.ts > caption rules > caption_fittext_scale_mismatch > warns when maxWidth times the largest word scale overflows the frame
      Tests  4 failed | 17 passed (21)

One transcript case, two textShadow cases, one fitText case. Worth one line for the record: a code rename can only kill the positives, the negatives stay green by construction, so the gates themselves are pinned by the per-gate deletions from earlier in this thread (sed -i '80d' then sed -i '160d'), not by this mutation.

Whole packages/lint on the squash commit 95aba51e:

      Tests  1213 passed (1213)

Two things from the merged state that need you, both checked here.

#5388 did not close. The PR body said "Refs #5388" rather than "Closes", so the squash merge left it open:

number 5388 | state open | state_reason None | closed_at None

Your reasoning line read "closes #5388", so I think it just wants a manual close.

The float edge from the PR body is still live on main, and it is slightly worse than "reports at the limit". The rule quotes the threshold value back to the author:

maxWidth=1600 scale=1.1 -> WARNS  Effective width 1760px may overflow the composition (1920px minus margins).
1600 * 1.1 = 1760.0000000000002 | > 1760 ? true | Math.round = 1760

So maxWidth: 1600 with scale: 1.1 warns while telling the author the effective width is 1760px, which is not greater than the 1760px bound the message itself cites. Out of scope for a test-only PR and I left it alone deliberately, but I am happy to send a one-line follow-up (compare rounded, or > 1760 to >= 1761) with a test pinning 1600 x 1.1, if you would rather not carry it.

Checked but not conclusive: the one red check on 95aba51e is "Require all Windows test lanes", and it failed because the three Windows lanes came back cancelled while the core lane passed. The run's own conclusion is cancelled, and the preceding pushes to main show the same pattern, each run cancelled by the next push, so I read it as the workflow's concurrency rule rather than this change. I did not read the lane logs, so if a Windows lane really is flaking on main that is still worth a look, just not from this diff, which is test-only inside packages/lint.

Thanks for the merge, and for the mutation check rather than a glance at the diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants