Skip to content

fix: report malformed ignore comments as invalid-ignore-comment - #5583

Open
Aayushyaash wants to merge 2 commits into
pyrefly-org:mainfrom
Aayushyaash:fix-invalid-ignore-comment
Open

Aayushyaash wants to merge 2 commits into
pyrefly-org:mainfrom
Aayushyaash:fix-invalid-ignore-comment

Conversation

@Aayushyaash

Copy link
Copy Markdown
Contributor

Summary

Malformed Pyrefly ignore comments were previously silently discarded without suppressing errors or producing a diagnostic. As a result, users had no indication that their ignore directives contained typos or syntax errors.

This PR introduces the invalid-ignore-comment lint rule (default severity: warn), enabled in the default, strict, and all presets.

It detects three malformed directive patterns:

x: int = "oops"  # pyrefly: ignoree             # Typo in directive keyword
x: int = "oops"  # pyrefly: ignore[bad-code     # Unterminated bracket
x: int = "oops"  # pyrefly: ignore[]            # Empty error-code list

Background and additional changes

This PR builds on the implementation by @mikeleppane in closed PR #3832, splitting the work into two PRs as requested by maintainers.

Additional changes and improvements:

  1. Rebased on latest main: Removed stale diffs in binding/expr.rs.
  2. Fixed comma-only bracket lists: Comments such as # pyrefly: ignore[,] and # pyrefly: ignore[ , ] are now correctly recognized as MalformedReason::EmptyBracket.
  3. Aligned Pyre suggestions: Updated suggestions for unrecognized Pyre directives to use # pyre-ignore and # pyre-fixme.
  4. Expanded unit tests: Added coverage for multi-code unterminated brackets (ignore[code1, code2) and whitespace/comma-only lists (ignore[,], ignore[ , ]).
  5. Preserved existing suppression behavior:
    • Typos: Continue to suppress nothing.
    • Unterminated brackets: Retain the existing lenient parsing behavior while emitting a warning.
    • Empty brackets: Preserve vec![""] blanket suppression for non-Pyrefly tools using directives such as # type: ignore[].
  6. Made the lint unsuppressable: Centralized the behavior in ErrorKind::is_unsuppressable(), preventing the diagnostic from being hidden by same-line ignore comments, file-level # pyrefly: ignore-errors, or automated --suppress-errors comment generators.

Closes #3752

  • I am a human, and I fully understand the code I am submitting.

Test Plan

  • Unit tests in crates/pyrefly_python/src/ignore.rs:

    • test_parse_ignore_comment
    • test_malformed_ignores_are_additive
    • Cover typo, single-code and multi-code unterminated brackets, empty brackets, and comma-only lists.
    • Verify that regular comments such as # type: list[int] and # pyrefly: see ticket are not flagged.
  • State tests in pyrefly/lib/state/errors.rs:

    • test_invalid_ignore_comment_emitted: Verifies the diagnostic is emitted while the underlying type error remains.
    • test_invalid_ignore_comment_variants: Covers unterminated and empty brackets.
    • test_empty_bracket_ignore_is_additive: Verifies that # type: ignore[] retains its existing suppression behavior.
    • test_invalid_ignore_comment_is_unsuppressable: Verifies that same-line ignore comments cannot suppress the lint.
    • test_ignore_comment_lints_survive_file_level_ignore_errors: Verifies that file-level ignore directives cannot suppress the lint.
  • End-to-end tests: Added Scrut coverage in test/ignores.md.

  • Documentation and preset tests: test_doc_headers, test_doc_severities, and test_preset_doc.

Validation

cargo check
cargo test -p pyrefly_python
cargo test -p pyrefly --lib test_invalid_ignore_comment
cargo test -p pyrefly --lib test_empty_bracket_ignore_is_additive
cargo test -p pyrefly --lib test_ignore_comment_lints_survive_file_level_ignore_errors

Aayushyaash and others added 2 commits November 8, 2026 15:37
Co-authored-by: Mikko Leppänen <mleppan23@gmail.com>
Co-authored-by: Mikko Leppänen <mleppan23@gmail.com>
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

✅ This PR meets all requirements from our pull request checklist.

Optional recommendations:

  • This PR is very large, which makes it slow and difficult to review. We strongly recommend splitting it into smaller PRs that can be merged independently. If that isn't possible, please structure it as a series of small commits. If you've already split it as far as is practical, no action is needed.

Note: this comment updates automatically after changes to the PR. Updates may take up to half an hour.

@meta-codesync

meta-codesync Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D130518155. (Because this pull request was imported automatically, there will not be any future comments.)

@Aayushyaash

Copy link
Copy Markdown
Contributor Author

Pr targeting #3450 builds on top of this pr. Once this pr is merged, I'll rebase the changes onto main and open the follow up pr.

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@connernilsen connernilsen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Aayushyaash, thanks for working on this. A few comments on things to look at. Let me know if you have any questions or disagree with any of the comments I've made

// before (a blanket for tools whose codes we ignore, a
// no-op for Pyrefly) so the diagnostic stays additive.
ParsedIgnore::Malformed {
suppression: Some(suppression),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An empty list suppresses [""], which suppresses nothing. This causes two diagnostics to be produced here, such as:

x = 1  # pyrefly: ignore[]
# ERROR 1:8-27: Unused `# pyrefly: ignore` comment for code(s):   [unused-ignore]
#  WARN 1:8-27: Empty error-code list in `# pyrefly: ignore[]`     [invalid-ignore-comment]

Can we update this to skip unused ignore errors when they come from bad content?

Also, suppression.kind.is_empty() can't be true here, because parse_error_codes always returns at least one element. The .all(..) check is enough on its own, but up to you if you want to remove it or not

Comment on lines +448 to +451
fn collect_invalid_ignore_comment_errors_for_module(
module: &Module,
enabled_ignores: &SmallSet<Tool>,
) -> Vec<Error> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This collects diagnostics outside ErrorCollector::collect_sorted, so it skips the ignore-errors-in-generated-code filter applied there. With that option on, a generated file still gets a warning:

# @generated
x: int = 1  # type: ignore1   # WARN invalid-ignore-comment; expected: nothing

Please also return early when malformed() is empty, so the common case does no extra work.


/// Collects `invalid-ignore-comment` diagnostics for all loaded modules.
/// Skips files we are told never to report errors on.
pub fn collect_invalid_ignore_comment_errors(&self) -> Vec<Error> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entry point skips ErrorStyle::Never modules, but the per-module call in collect_error_refs_with_ignore_diagnostics (around line 432, used by the CLI check and LSP diagnostics) doesn't. Could we have a single per-module function with the filters (error style, generated code) in one place, and have both paths call it?

Also, the other function calls config.enabled_ignores(path) a second time for the same module. That does glob matching, so please reuse the first result.

Comment on lines +492 to +494
if err.error_kind().is_unsuppressable() {
return SuppressionEffect::None;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invalid-ignore-comment errors are synthesized after collection (state/errors.rs:448). They never pass through this function or through Error::suppression_effect, so for the new kind these checks never run. Only the is_suppressable change, which keeps --suppress-errors from writing these comments, has any effect. Item 6 of the summary claims more than that.

Could you drop the checks that don't take effect, or explain what they protect?

// so the fix is to move or remove the comment rather than to suppress
// the error. `suppression_effect` already returns `None` for them, but
// keep the explicit short-circuit so their intent stays local.
let effect = if error.error_kind().is_unsuppressable() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the comment says, suppression_effect already returns None for these kinds. Please keep only one of the two checks.

}

#[test]
fn test_invalid_ignore_comment_is_unsuppressable() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test and test_ignore_comment_lints_survive_file_level_ignore_errors call collect_invalid_ignore_comment_errors(), which applies no suppression at all. They pass even with every is_unsuppressable change reverted. To test what users see, go through the display path, e.g. collect_display_errors_with_ignore_diagnostics().

self.is_unused_ignore()
|| matches!(
self,
ErrorKind::MisplacedIgnore | ErrorKind::InvalidIgnoreComment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding MisplacedIgnore here changes how misplaced-ignore behaves, which the summary doesn't mention and no test covers:

  • A # pyrefly: ignore[misplaced-ignore] no longer counts as used, so it becomes an unused-ignore under strict.
  • File-level ignore-errors no longer hides misplaced-ignore.
  • --suppress-errors stops writing comments for it.

That may be the right call, but could you split it into its own PR with a test? If you think it's necessary to keep as part of this diff, please let me know here and update the summary

// suppression. (A trailing identifier char, e.g. `ignore-errorss`,
// falls through to the typo branch below.)
ParsedIgnore::NotADirective
} else {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With mypy ignores enabled (permissive-ignores = true), mypy inline config is flagged:

# mypy: ignore-missing-imports   # WARN: did you mean `# mypy: ignore`?

Consider not treating a - or _ right after ignore as a typo for tools other than Pyrefly. Also, # mypy: ignore isn't a real mypy per-line form, so that suggestion is misleading.

}

/// The lowercase name used in ignore comments, e.g. `pyrefly`.
pub fn as_str(self) -> &'static str {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This copies the from_comment function a few lines above. Tool already derives ValueEnum and kebab-case serde, so could one direction be derived from the other, or both come from a single table?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new rule - invalid ignore comment

3 participants