Repository navigation
GROOVY-12568: SC: report a protected or package-private member of another run-time package - #3097
Conversation
…h new directly only within the compilation unit Under static compilation, new A(...) of a protected or package-private constructor of a class with the same package name was compiled to NEW and INVOKESPECIAL. The JVM allows that call only within the same run-time package, which is the package name and the class loader, so it failed with IllegalAccessError when A came from another class loader: a class of an earlier parse of the same GroovyShell, which GroovyClassLoader defines with another InnerLoader, or a precompiled class used by a script. Dynamic and type-checked code create such an object through the meta-class and work. Such a constructor is now called directly only when its class is compiled in the same compilation unit, which defines all its classes with one class loader, as for a constructor reference. Otherwise the object is created through the meta-class, which selects the constructor at run time. This also applies to the no-argument constructor that a map-style construction such as new A(name: 'x') calls before it sets the properties. Public constructors, and constructors of classes of the same compilation unit, are still called directly. The constructor reference writer uses the same check.
… only within the compilation unit Under static compilation, a call to a protected or package-private method of a class with the same package name was compiled to a direct INVOKEVIRTUAL, INVOKESTATIC or INVOKESPECIAL. As for a constructor, the JVM allows that call only within the same run-time package, so it failed with IllegalAccessError when the class came from another class loader, for example a class of an earlier parse of the same GroovyShell. This also hit a protected or package-private getter or setter used as a property, such as a.name or a.name = value. Such a method is now called directly only when its class is compiled in the same compilation unit, or when the JVM allows the call from any package: a protected method called from a subclass on this, super or a receiver of the subclass type, or a protected static method called from a subclass. Otherwise the method is selected at run time through the meta-class, as dynamic code does. Public methods, and methods of classes of the same compilation unit, are still called directly.
…nother compilation unit through a synthetic method A statically compiled method reference such as m::pm, M::pm or M::spm gives LambdaMetafactory a method handle for the method. For a protected or package-private method of a class with the same package name, the JVM resolves that handle only within the same run-time package, so the reference failed with IllegalAccessError when the class came from another class loader, for example a class of an earlier parse of the same GroovyShell. For such a method of a class of another compilation unit, the reference now targets a synthetic static method of the current class that calls the method. The invocation writer compiles that call directly when the JVM allows it from any package (a protected method called through the subclass), or else selects the method at run time. Method references to methods of classes of the same compilation unit, and to public methods, still use a method handle for the method itself.
…ly within the compilation unit AsmClassGenerator.isFieldDirectlyAccessible decided by the package name if a protected or package-private field can be read or written with GETFIELD, PUTFIELD, GETSTATIC or PUTSTATIC. As for methods and constructors, the JVM allows that only within the same run-time package, so the access failed with IllegalAccessError when the class came from another class loader, for example a class of an earlier parse of the same GroovyShell. Under static compilation this hit a.field, a.@field, Type.field, an implicit field of a superclass, compound assignments and increments. Dynamic code hit it for super.field, super.@field and this.@field of a superclass. A protected field of a superclass with the same package name but of another compilation unit was also read through a receiver of the superclass type, which fails verification ("Bad access to protected data in getfield") when the classes are in different run-time packages. Such a field is now accessed directly only when its class is compiled in the same compilation unit, or through the subclass rule for a protected field (with a receiver of the current class). Otherwise it is read and written through the meta-class, as dynamic code does. Public fields, and fields of classes of the same compilation unit, are still accessed directly.
…ckage instead of failing verification
Outside its package, a protected constructor can only be called by
super(...) or by creating an anonymous inner class (JLS 6.6.2.2). The type
checker accepts new A(...) of such a constructor in a subclass of A, as it
accepts other protected members there, and type-checked code creates the
object through the meta-class. Under static compilation, the call was
compiled to NEW and INVOKESPECIAL, and the class failed verification when
it was loaded: "VerifyError: Bad access to protected <init> method". The
same happened for a map-style construction such as new A([:]), which calls
the protected no-argument constructor.
The static compiler now reports "Cannot call protected constructor for A
from class B", as it reports a protected method called on a receiver that
is not the subclass (GROOVY-7325). super(...), this(...) and
new A(...) {} still call the constructor directly. A constructor reference
A::new to such a constructor still selects it at run time.
… through a superclass receiver Outside its package, a protected instance method or field can be used only through a receiver of the subclass (JLS 6.6.2.1, JVMS 4.10.1.8). For a method, the static compiler meant to report such a call (GROOVY-7325), but its check compared the types the wrong way round: it tested whether the current class is a subtype of the receiver type instead of the reverse. So a.m() with a receiver of the superclass type, in a subclass in another package, was compiled to a direct call and the class failed verification: "VerifyError: Bad access to protected data in invokevirtual". This also hit a protected getter or setter used as a property (a.name, a.name = value). A method reference such as a::m or A::m to such a method failed with BootstrapMethodError, and an attribute assignment a.@field = value failed verification in putfield. The check now requires the receiver to be of the current class (or this, super or a static method), so such a call or method reference is reported as "Cannot access method", as GROOVY-7325 intended and as javac does. An attribute assignment through such a receiver is now made through the meta-class, as the attribute read a.@field already was. Calls through this, super or a receiver of the subclass type are still direct.
…amic cannot access the constructor A dynamic new of a protected constructor of another package, in a subclass, is accepted by Groovy and by the type checker. With invokedynamic, the constructor selector unreflected the selected constructor with the lookup of the calling class and turned an IllegalAccessException into GroovyBugError. For a constructor that the JVM refuses to the caller, such as the protected ObjectInputStream() of the encapsulated java.io package called by new in a subclass, the call failed with "BUG! UNCAUGHT EXCEPTION: member is protected". As the property selectors do since GROOVY-12314, the constructor selector now treats the refusal as an access-control outcome and leaves the call to the meta-class, which reports "could not access constructor", as the call does without invokedynamic. A constructor that reflection can reach, such as a protected constructor of a class of another package in the unnamed module, is still called.
…ethod of another run-time package For a dynamic super.m() call, the class gets a synthetic super$N$m method that calls the superclass method with INVOKESPECIAL. For a package-private method of a superclass with the same package name but of another compilation unit, the classes may be in different run-time packages, for example when the superclass comes from an earlier parse of the same GroovyShell, and the JVM refuses the call: super.m() failed with IllegalAccessError. Such a method, and a package-private method of a superclass of another package, no longer gets a super$ method, and the meta-class calls the superclass method reflectively, as it does for a statically compiled super.m() of such a method.
…ugh the meta-class only for classes defined in memory The previous commits accessed a protected or package-private member of a class with the same package name through the meta-class whenever that class was compiled by another compilation unit. That also hit class files: a Java class of a joint compilation, a main class used by a test, a class left over from an incremental compilation. Such classes are loaded by one class loader at run time, as javac assumes, so the direct access worked; the meta-class path was slower, followed meta-class semantics, needed reflection metadata in a native image, and made an incremental build give other bytecode than a full one. The run-time package of such a class only differs when the classes are defined in memory by a GroovyClassLoader, which defines each compilation (a GroovyShell script, parseClass, JSR-223) with an InnerLoader of its own. GroovyClassLoader now records that on the compilation unit, and AsmClassGenerator.isInSameRuntimePackage treats a class of another compilation unit as another run-time package only then. Constructors, map-style construction, methods, getters and setters, method references, fields, dynamic super and this.@ access and super$ methods all use this check. Compiling to class files keeps direct access, as before these changes and as javac does. (The rest of the change: new, map-style construction, methods, getters and setters, method references to methods, fields, dynamic super and this.@ access and super$ methods. The GroovyClassLoader marker, isInSameRuntimePackage and constructor references come first.)
The tests for protected and package-private members of a class of another compilation now compile it through a GroovyClassLoader, which defines it with another class loader; classes compiled to class files keep direct access. Name the tests accordingly.
…rules of constructor references - isInSameRuntimePackage: isPrimaryClassNode() stands for "defined by the class loader of the accessing class", not for "compiled from source". - StaticInvocationWriter.isDirectlyAccessible: unlike isMemberDirectlyAccessible it has no subclass rule, since the verifier rejects NEW and INVOKESPECIAL of a protected constructor of another package in a subclass too. - findConstructorReferenceTarget: a selected constructor is called directly, so a meta-class constructor override is not used, as for a statically compiled new. The guard comment now says why enums are left to the meta-class: the type checker accepts an enum constructor reference whose arguments match the generated constructor, but Java code cannot instantiate an enum. - addSyntheticMethod: the ACC_FINAL rule of JVMS 4.6 applies to every helper method of a method reference in an interface. (The part on statically compiled new; the rest comes with constructor references.)
… interface All helper methods that a method reference adds to the current class are created by addSyntheticMethod, which leaves out ACC_FINAL in an interface. The interface tests covered the constructor-reference and variadic helpers; they now also cover the array constructor reference, the static extension method helper (Thread::startDaemon), the Class method helper (String::getName) and the helper that calls a package-private method of a class of another class loader (M::spm, M::pm), in static and default methods, and check that every helper is private and not final. With ACC_FINAL put back, both tests fail with ClassFormatError (illegal modifiers 0x101A). (The case of a method of another run-time package; the other helper methods are tested with constructor references.)
…AbstractBytecodeTestCase MethodReferenceTest and StaticCompileRuntimePackageTest each had a copy of the helper that compiles a script against the classes of an earlier parse of the same GroovyClassLoader, and of the code that keeps only the instructions of one method. AbstractBytecodeTestCase now provides them as compileAgainstLibrary, extractMethodSequence and selectMethod, and both tests use them. Test code only; the tests are unchanged. (Here StaticCompileRuntimePackageTest; MethodReferenceTest was moved to the helpers with constructor references.)
…errors COMPATIBILITY.md describes how statically compiled code of a compilation that a GroovyClassLoader defines in memory reaches protected and package-private members of classes of other compilations, and the new compile errors for new of a protected constructor of another package and for a protected method of another package called through a superclass receiver. StaticInvocationWriter notes that the constructor error is reported during class generation, so @TypeChecked code is not checked for it and works dynamically.
…ass of another class loader Since GROOVY-12454, new A(map) with a Map variable, under static compilation, calls the no-argument constructor and then sets the properties of the map at run time. For a class of an earlier parse of the same GroovyShell whose no-argument constructor is package-private, the constructor is called through the meta-class, as for a map literal.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3097 +/- ##
==================================================
+ Coverage 74.3385% 74.4209% +0.0824%
- Complexity 44310 44424 +114
==================================================
Files 1699 1700 +1
Lines 150860 151131 +271
Branches 28755 28812 +57
==================================================
+ Hits 112147 112473 +326
+ Misses 28439 28381 -58
- Partials 10274 10277 +3
🚀 New features to boost your workflow:
|
daniellansun
left a comment
There was a problem hiding this comment.
LGTM for GROOVY-12568 (and the bundled GROOVY-12488 constructor-reference work this branch carries).
JVM run-time package = package name + ClassLoader. Marking each GroovyClassLoader in-memory compilation (InnerLoader metadata on the CompileUnit) and then refusing direct bytecode access to protected / package-private members across that boundary — falling back to meta-class / synthetic helpers, and emitting the new compile errors for illegal new of a protected ctor / protected method via a foreign receiver — matches JLS/JVMS and dynamic Groovy.
AsmClassGenerator.isInSameRuntimePackage / isFieldDirectlyAccessible, StaticInvocationWriter.isDirectlyAccessible + writeDynamicConstructorCall, MopWriter skipping impossible super$ bridges, and the indy ctor path are coherent. StaticCompileRuntimePackageTest + Groovy12568 cover GCL isolation, map-style construction, method/ctor refs, and fields. COMPATIBILITY.md / operator docs spell the Groovy 7 behavior change clearly.
CI fully green. Codecov ~84% patch with mostly branch partials (no large missing blocks) — acceptable for this surface.
Merge notes: branch is ~30 commits ahead / a few behind master; open #3022 (GROOVY-12488) overlaps — prefer one landing path. #3098 / #3099 are siblings on top of this tip; merge this first.
|
Friendly ping: this PR is approved. If there are no objections within 72 hours, this PR will be merged. Thanks! |
|
@daniellansun Thanks. #3097 is stacked on #3022 (GROOVY-12488), so merging it also lands #3022's commits. Either order works: merge #3022 first and #3097 then shows only its own commits, or merge #3097 and close #3022. #3098 and #3099 go after #3097. I merged master into all four to clear a COMPATIBILITY.md conflict (both sides added an entry; kept both). |
# Conflicts: # src/main/java/org/codehaus/groovy/classgen/asm/sc/StaticTypesMethodReferenceExpressionWriter.java
…to the language rule For a public or private field the language rule of isMemberDirectlyAccessible decides. For a protected or package-private field, JVMS 5.4.4 allows access from the run-time package of the declaring class, or to a protected field from a subclass. That condition implies isMemberDirectlyAccessible, so it replaces it there instead of being checked after it.
|
@blackdrag Simplified in e9c86b5: if (field.isPublic() || field.isPrivate()) { // nestmates can access a private field
return isMemberDirectlyAccessible(field.getModifiers(), declaringClass, accessingClass);
}
// JVMS 5.4.4
return isInSameRuntimePackage(declaringClass, accessingClass)
|| (field.isProtected() && accessingClass.isDerivedFrom(declaringClass));
The result is the same as before for every field, so no test changed. Also merged master now that #3022 has landed, so this PR shows only GROOVY-12568. |
|
Now that #3019 is merged I think the idea of this PR is actually in conflict. |
|
@blackdrag I see the conflict. For the same failure, What this PR does now:
The alternative in line with #3019: the type checker treats such a member or class as inaccessible and reports "Cannot access ...". This applies to If that is what you mean, I'll rework #3097 this way, and #3098 and #3099 with it. I also merged master in (93d5776); |
|
@codeconsole I think the dynamic path does not make sense here for static compilation because:
I see a few paths forward here:
For (2) I would use the compiler configuration to set if want to check this, since this is a very specific scenario and GroovyShell could then for example set it by default. Unchecked means path (1) path 3 makes this of course a much bigger implementation since we do not have that so far. Well maybe the LambdaFactory utilities can help here, since we really actually want a constant call site for this. And path (1) is pretty much a "won´t fix" I would not use ScriptByteCodeAdapter#invokeNew for this. What variant makes most sense for you? |
|
@blackdrag Agreed: the meta-class route goes. I'd take path 2.
Rework:
Does that split and name work for you? |
# Conflicts: # COMPATIBILITY.md
# Conflicts: # COMPATIBILITY.md # src/main/java/org/codehaus/groovy/classgen/asm/sc/StaticTypesMethodReferenceExpressionWriter.java
# Conflicts: # COMPATIBILITY.md
The note is now in the pull request description. An entry carried on this branch conflicted with every later addition to the end of that file.
…2568 The note is now in the description of apache#3097, which this builds on. An entry carried on this branch conflicted with every later addition to the end of that file.
…2568 The note is now in the description of apache#3097, which this builds on. An entry carried on this branch conflicted with every later addition to the end of that file.
…me package Static compilation should not select such a member at run time: compile time and run time may see different classes, the meta-class can influence the call, and the call may still fail at run time. The writers are back to what master emits. What is not about the run-time package moves to issues of its own: the protected members of another package, the super$ method of a package-private method, dynamic super and this access to such a field, and the invokedynamic failure for a constructor the JVM refuses.
…r run-time package The JVM allows access to a protected or package-private member only within the same run-time package, which is the package name and the class loader, apart from what a subclass can do with a protected member. A GroovyClassLoader defines each compilation in memory with a class loader of its own, so such code compiles and then fails with IllegalAccessError. A @CompileStatic script in a GroovyShell that calls a package-private method of a class of an earlier parse, or of a library class in the same package, is enough. The new option CompilerConfiguration.RUNTIME_PACKAGE_CHECK reports such a constructor, method or field when the class is generated. It is off by default, with the same bytecode as before, and GroovyShell turns it on for the class loader it creates, unless the configuration sets it. It has no effect on code compiled to class files, and on dynamic and @TypeChecked code, which reach such members through the meta-class.
|
@blackdrag This is now path 2, as proposed. The meta-class route is gone,
The option is Your comment on #3098 is the same check for a package-private class, on the instructions that name the class. The rest moved as proposed:
Your objection also applies to something already on master: since #3022 (GROOVY-12488), a constructor reference to such a constructor is selected at run time. I left that as it is, so |
daniellansun
left a comment
There was a problem hiding this comment.
Re-reviewed tip b4a1589ab2fd after @blackdrag’s CHANGES_REQUESTED and the path-2 rework.
Jochen’s concerns appear addressed. The meta-class / invokeNew fallback is gone; writers emit the same bytecode as master when the option is off. AsmClassGenerator.isFieldDirectlyAccessible / isMemberDirectlyAccessible are back to master, so the earlier inline comment on that helper no longer applies. The check now lives in RuntimePackageCheck and only fires when:
- the member is protected or package-private,
- the accessing class shares the package name but not the run-time package (
isInSameRuntimePackage), - outside the JVMS 5.4.4 / 4.10.1.8 protected-subclass rule,
- and
CompilerConfiguration.RUNTIME_PACKAGE_CHECKis on (explicitly, or via theGroovyClassLoader/GroovyShellin-memory marker).
That matches path 2 as discussed: default off (same as today’s IllegalAccessError at run time for class files); GroovyShell opts in for its loader without mutating CompilerConfiguration.DEFAULT; @CompileStatic reports at class-generation time; dynamic / @TypeChecked code keeps the meta-class path.
The option wiring, anonymous-ctor selection, and StaticCompileRuntimePackageTest matrix look solid. Splitting the unrelated pieces into #3251 / #3252 / #3253 / #3098 / #3099 keeps this PR focused.
Non-blocking note (as you already called out): since #3022, a constructor reference to such a constructor can still be selected at run time and is not reported under this option. Leaving that alone to avoid breaking working code is reasonable; if we later want it gated too, a follow-up on top of this tip is fine.
LGTM for GROOVY-12568 path 2. Please land before #3098 / #3099. @blackdrag — if anything in the new shape still looks off, happy to revisit.
…or generated by @InheritConstructors @InheritConstructors adds a constructor for each constructor of the superclass. When one of them is a package-private constructor of another run-time package, the generated constructor fails only if it is called, as on master; reporting its super(...) call broke a class that works when the other constructors are used. A super(...) call without a source position in a generated constructor is no longer reported.
Dynamic code calls a super constructor with INVOKESPECIAL too, not through the meta-class, so super(...), the implicit super() and the constructor of an anonymous inner class fail with IllegalAccessError in dynamic code as well. The report of such a call now says so instead of advising @CompileDynamic, which stays the advice for the other members.
Its methods are called by the class generator, the static invocation writer and the constructor call transformer, which are in three packages, so the class stays public; it is not API for users.
The tests of members of the same compilation unit compiled with a plain CompilationUnit, where the check is off and every class counts as being in the same run-time package, so they passed whatever the check decided. They now compile in memory with a GroovyClassLoader and the option on, as the class loader of a GroovyShell does, and the joint compilation test sets the option, as the tests of class files do.
✅ All tests passed ✅🏷️ Commit: a6ab62c Learn more about TestLens at testlens.app/docs. |
Under
@CompileStatic, code that uses a protected or package-private member of a class in another run-time package compiles and then fails withIllegalAccessError. The run-time package is the package name and the class loader. AGroovyClassLoaderdefines each compilation in memory with a class loader of its own (aGroovyShellscript,parseClass, a script run withgroovy), so a class from an earlier parse, or a precompiled class used by such a script, is in another run-time package even with the same package name:The new option
CompilerConfiguration.RUNTIME_PACKAGE_CHECKreports such an access when the class is generated, which is where the access is emitted:GroovyShellturns it on for the class loader it creates, unless the configuration sets the option. The option is in the optimization options, likeERROR_RECOVERY, and can also be set with thegroovy.runtime.package.checksystem property.GroovyClassLoader, throughAsmClassGenerator.isInSameRuntimePackage.new, map-style construction,super(...), an implicitsuper()and an anonymous class), methods (calls, a getter or setter used as a property, method references) and fields (reads, attribute reads and writes,++and+=).super(...)or an anonymous class, a protected static member, and a protected instance member throughthis,superor a receiver of the subclass.@CompileStaticcode is affected. Dynamic and@TypeCheckedcode reach such members through the meta-class.A constructor reference such as
A::newis not reported: since GROOVY-12488 it selects such a constructor at run time. A write through property syntax such asnew F().pf = 'w'is not reported either, as it already goes through the meta-class and works.What this PR used to include beyond that is separate now: protected members of another package are #3251 (GROOVY-12717),
super$for a package-private method of another run-time package is #3099, dynamicsuper.f,super.@fandthis.@fof such a field are #3252 (GROOVY-12718), and the invokedynamicGroovyBugErrorfor a constructor the JVM refuses is #3253 (GROOVY-12719).A call of a package-private super constructor of another run-time package, explicit, implicit (
super()) or from an anonymous inner class, is reported without the@CompileDynamicadvice, since dynamic code calls a super constructor without the meta-class too; the message ends "A super constructor call fails at run time in dynamic code too". Thesuper(...)call of a constructor that@InheritConstructorsgenerates is not checked: as before, such a constructor fails only if it is called.RuntimePackageCheckis marked@Internal.Compatibility
GroovyShellturns the check on, so a@CompileStaticscript or class in a shell that uses such a member no longer compiles. If it ran that code it failed anyway, withIllegalAccessError,VerifyErrororBootstrapMethodError. What changes is code that compiled and never reached the access. Use@CompileDynamic, or set the option to false to get the old behaviour. A shell given aGroovyClassLoaderwith the same configuration uses that class loader and does not change it, so the option decides there. AGroovyClassLoadersubclass whose collector defines all compilations with one class loader should leave the option off, as its classes really are in one run-time package.Tests
StaticCompileRuntimePackageTestcovers each form above reported with the option on, in a subclass and in an unrelated class, in closures and lambdas, what the subclass rule still allows, theIllegalAccessErrorwith the option off, the default of a shell and of a plainGroovyClassLoader, the system property,@CompileDynamic, and dynamic and@TypeCheckedcode. 11 of its 25 tests fail without the check. The others pin what must not change, among them class files, joint compilation with Java, and classes of the same compilation. 15,542 tests inorg.codehaus.groovy.classgen,groovy.transform.stc,groovy,gls,bugs,org.codehaus.groovy.transformandorg.apache.groovypass with the check on for everyGroovyShellthey use, so it reports nothing in them. The Checkstyle and CodeNarc gates pass.