Repository navigation
GROOVY-12569: SC: report a package-private class of another run-time package that an instruction names - #3098
codeconsole wants to merge 54 commits into
Conversation
…h new directly only within the compilation unit Under static compilation, new A(...) of a protected or package-private constructor of a class with the same package name was compiled to NEW and INVOKESPECIAL. The JVM allows that call only within the same run-time package, which is the package name and the class loader, so it failed with IllegalAccessError when A came from another class loader: a class of an earlier parse of the same GroovyShell, which GroovyClassLoader defines with another InnerLoader, or a precompiled class used by a script. Dynamic and type-checked code create such an object through the meta-class and work. Such a constructor is now called directly only when its class is compiled in the same compilation unit, which defines all its classes with one class loader, as for a constructor reference. Otherwise the object is created through the meta-class, which selects the constructor at run time. This also applies to the no-argument constructor that a map-style construction such as new A(name: 'x') calls before it sets the properties. Public constructors, and constructors of classes of the same compilation unit, are still called directly. The constructor reference writer uses the same check.
… only within the compilation unit Under static compilation, a call to a protected or package-private method of a class with the same package name was compiled to a direct INVOKEVIRTUAL, INVOKESTATIC or INVOKESPECIAL. As for a constructor, the JVM allows that call only within the same run-time package, so it failed with IllegalAccessError when the class came from another class loader, for example a class of an earlier parse of the same GroovyShell. This also hit a protected or package-private getter or setter used as a property, such as a.name or a.name = value. Such a method is now called directly only when its class is compiled in the same compilation unit, or when the JVM allows the call from any package: a protected method called from a subclass on this, super or a receiver of the subclass type, or a protected static method called from a subclass. Otherwise the method is selected at run time through the meta-class, as dynamic code does. Public methods, and methods of classes of the same compilation unit, are still called directly.
…nother compilation unit through a synthetic method A statically compiled method reference such as m::pm, M::pm or M::spm gives LambdaMetafactory a method handle for the method. For a protected or package-private method of a class with the same package name, the JVM resolves that handle only within the same run-time package, so the reference failed with IllegalAccessError when the class came from another class loader, for example a class of an earlier parse of the same GroovyShell. For such a method of a class of another compilation unit, the reference now targets a synthetic static method of the current class that calls the method. The invocation writer compiles that call directly when the JVM allows it from any package (a protected method called through the subclass), or else selects the method at run time. Method references to methods of classes of the same compilation unit, and to public methods, still use a method handle for the method itself.
…ly within the compilation unit AsmClassGenerator.isFieldDirectlyAccessible decided by the package name if a protected or package-private field can be read or written with GETFIELD, PUTFIELD, GETSTATIC or PUTSTATIC. As for methods and constructors, the JVM allows that only within the same run-time package, so the access failed with IllegalAccessError when the class came from another class loader, for example a class of an earlier parse of the same GroovyShell. Under static compilation this hit a.field, a.@field, Type.field, an implicit field of a superclass, compound assignments and increments. Dynamic code hit it for super.field, super.@field and this.@field of a superclass. A protected field of a superclass with the same package name but of another compilation unit was also read through a receiver of the superclass type, which fails verification ("Bad access to protected data in getfield") when the classes are in different run-time packages. Such a field is now accessed directly only when its class is compiled in the same compilation unit, or through the subclass rule for a protected field (with a receiver of the current class). Otherwise it is read and written through the meta-class, as dynamic code does. Public fields, and fields of classes of the same compilation unit, are still accessed directly.
…ckage instead of failing verification
Outside its package, a protected constructor can only be called by
super(...) or by creating an anonymous inner class (JLS 6.6.2.2). The type
checker accepts new A(...) of such a constructor in a subclass of A, as it
accepts other protected members there, and type-checked code creates the
object through the meta-class. Under static compilation, the call was
compiled to NEW and INVOKESPECIAL, and the class failed verification when
it was loaded: "VerifyError: Bad access to protected <init> method". The
same happened for a map-style construction such as new A([:]), which calls
the protected no-argument constructor.
The static compiler now reports "Cannot call protected constructor for A
from class B", as it reports a protected method called on a receiver that
is not the subclass (GROOVY-7325). super(...), this(...) and
new A(...) {} still call the constructor directly. A constructor reference
A::new to such a constructor still selects it at run time.
… through a superclass receiver Outside its package, a protected instance method or field can be used only through a receiver of the subclass (JLS 6.6.2.1, JVMS 4.10.1.8). For a method, the static compiler meant to report such a call (GROOVY-7325), but its check compared the types the wrong way round: it tested whether the current class is a subtype of the receiver type instead of the reverse. So a.m() with a receiver of the superclass type, in a subclass in another package, was compiled to a direct call and the class failed verification: "VerifyError: Bad access to protected data in invokevirtual". This also hit a protected getter or setter used as a property (a.name, a.name = value). A method reference such as a::m or A::m to such a method failed with BootstrapMethodError, and an attribute assignment a.@field = value failed verification in putfield. The check now requires the receiver to be of the current class (or this, super or a static method), so such a call or method reference is reported as "Cannot access method", as GROOVY-7325 intended and as javac does. An attribute assignment through such a receiver is now made through the meta-class, as the attribute read a.@field already was. Calls through this, super or a receiver of the subclass type are still direct.
…amic cannot access the constructor A dynamic new of a protected constructor of another package, in a subclass, is accepted by Groovy and by the type checker. With invokedynamic, the constructor selector unreflected the selected constructor with the lookup of the calling class and turned an IllegalAccessException into GroovyBugError. For a constructor that the JVM refuses to the caller, such as the protected ObjectInputStream() of the encapsulated java.io package called by new in a subclass, the call failed with "BUG! UNCAUGHT EXCEPTION: member is protected". As the property selectors do since GROOVY-12314, the constructor selector now treats the refusal as an access-control outcome and leaves the call to the meta-class, which reports "could not access constructor", as the call does without invokedynamic. A constructor that reflection can reach, such as a protected constructor of a class of another package in the unnamed module, is still called.
…ethod of another run-time package For a dynamic super.m() call, the class gets a synthetic super$N$m method that calls the superclass method with INVOKESPECIAL. For a package-private method of a superclass with the same package name but of another compilation unit, the classes may be in different run-time packages, for example when the superclass comes from an earlier parse of the same GroovyShell, and the JVM refuses the call: super.m() failed with IllegalAccessError. Such a method, and a package-private method of a superclass of another package, no longer gets a super$ method, and the meta-class calls the superclass method reflectively, as it does for a statically compiled super.m() of such a method.
…ugh the meta-class only for classes defined in memory The previous commits accessed a protected or package-private member of a class with the same package name through the meta-class whenever that class was compiled by another compilation unit. That also hit class files: a Java class of a joint compilation, a main class used by a test, a class left over from an incremental compilation. Such classes are loaded by one class loader at run time, as javac assumes, so the direct access worked; the meta-class path was slower, followed meta-class semantics, needed reflection metadata in a native image, and made an incremental build give other bytecode than a full one. The run-time package of such a class only differs when the classes are defined in memory by a GroovyClassLoader, which defines each compilation (a GroovyShell script, parseClass, JSR-223) with an InnerLoader of its own. GroovyClassLoader now records that on the compilation unit, and AsmClassGenerator.isInSameRuntimePackage treats a class of another compilation unit as another run-time package only then. Constructors, map-style construction, methods, getters and setters, method references, fields, dynamic super and this.@ access and super$ methods all use this check. Compiling to class files keeps direct access, as before these changes and as javac does. (The rest of the change: new, map-style construction, methods, getters and setters, method references to methods, fields, dynamic super and this.@ access and super$ methods. The GroovyClassLoader marker, isInSameRuntimePackage and constructor references come first.)
The tests for protected and package-private members of a class of another compilation now compile it through a GroovyClassLoader, which defines it with another class loader; classes compiled to class files keep direct access. Name the tests accordingly.
…rules of constructor references - isInSameRuntimePackage: isPrimaryClassNode() stands for "defined by the class loader of the accessing class", not for "compiled from source". - StaticInvocationWriter.isDirectlyAccessible: unlike isMemberDirectlyAccessible it has no subclass rule, since the verifier rejects NEW and INVOKESPECIAL of a protected constructor of another package in a subclass too. - findConstructorReferenceTarget: a selected constructor is called directly, so a meta-class constructor override is not used, as for a statically compiled new. The guard comment now says why enums are left to the meta-class: the type checker accepts an enum constructor reference whose arguments match the generated constructor, but Java code cannot instantiate an enum. - addSyntheticMethod: the ACC_FINAL rule of JVMS 4.6 applies to every helper method of a method reference in an interface. (The part on statically compiled new; the rest comes with constructor references.)
… interface All helper methods that a method reference adds to the current class are created by addSyntheticMethod, which leaves out ACC_FINAL in an interface. The interface tests covered the constructor-reference and variadic helpers; they now also cover the array constructor reference, the static extension method helper (Thread::startDaemon), the Class method helper (String::getName) and the helper that calls a package-private method of a class of another class loader (M::spm, M::pm), in static and default methods, and check that every helper is private and not final. With ACC_FINAL put back, both tests fail with ClassFormatError (illegal modifiers 0x101A). (The case of a method of another run-time package; the other helper methods are tested with constructor references.)
…AbstractBytecodeTestCase MethodReferenceTest and StaticCompileRuntimePackageTest each had a copy of the helper that compiles a script against the classes of an earlier parse of the same GroovyClassLoader, and of the code that keeps only the instructions of one method. AbstractBytecodeTestCase now provides them as compileAgainstLibrary, extractMethodSequence and selectMethod, and both tests use them. Test code only; the tests are unchanged. (Here StaticCompileRuntimePackageTest; MethodReferenceTest was moved to the helpers with constructor references.)
…errors COMPATIBILITY.md describes how statically compiled code of a compilation that a GroovyClassLoader defines in memory reaches protected and package-private members of classes of other compilations, and the new compile errors for new of a protected constructor of another package and for a protected method of another package called through a superclass receiver. StaticInvocationWriter notes that the constructor error is reported during class generation, so @TypeChecked code is not checked for it and works dynamically.
…ass of another class loader Since GROOVY-12454, new A(map) with a Map variable, under static compilation, calls the no-argument constructor and then sets the properties of the map at run time. For a class of an earlier parse of the same GroovyShell whose no-argument constructor is package-private, the constructor is called through the meta-class, as for a map literal.
…s loader in bytecode A class that is not public is accessible only within its run-time package (JVMS 5.4.4), and the JVM checks that whenever an instruction names the class: NEW, CHECKCAST, INSTANCEOF, the owner of a method or field, and the classes of a method handle or invokedynamic descriptor. Dynamic code uses a package-private class of an earlier parse of the same GroovyShell, or a precompiled class used by a script, without naming it, so new Hidden(), Hidden.sm(), Hidden.FIELD, def h = new Hidden(); h.run(); h.name and Runnable r = new Hidden() all work. Statically compiled code named the class in each of them and failed with IllegalAccessError. For a class that AsmClassGenerator.isClassDirectlyAccessible rejects (not public, same package name, but another run-time package, which is only known for classes defined in memory by a GroovyClassLoader), the static writers now avoid naming it: - new Hidden(...), also map-style, creates the object through the meta-class on the Class object, which is loaded by name; - a static method, property or field of the class is used through the meta-class; - a value of the class is typed for bytecode as its nearest superclass that can be named (or Object): local variables declared with def, method results, field values, the elements of a spread or a list property, so storing, passing or assigning it to an interface type needs no cast to the class; - a method of the class is called directly through a public superclass or interface that declares it, otherwise through the meta-class, and a method with a parameter of the class through the meta-class; - a method pointer keeps the class for method selection, and a method reference that would need a method handle naming the class is a method closure, as in dynamic code. What names the class in the source still names it in bytecode and fails as in dynamic code: a variable declared with the class, a cast to it and instanceof. Classes compiled to class files, classes of the same compilation and public classes are named directly, as before.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3098 +/- ##
==================================================
+ Coverage 74.4569% 74.4715% +0.0146%
- Complexity 44841 44860 +19
==================================================
Files 1702 1702
Lines 151994 152073 +79
Branches 29119 29143 +24
==================================================
+ Hits 113170 113251 +81
- Misses 28415 28417 +2
+ Partials 10409 10405 -4
🚀 New features to boost your workflow:
|
daniellansun
left a comment
There was a problem hiding this comment.
GROOVY-12569 is the right companion to #3097: even when the member is public, bytecode must not name a non-public class of another run-time package (NEW / CHECKCAST / owner type).
isClassDirectlyAccessible + getDirectlyAccessibleSuperClass, rewriting ctor / method / field / method-pointer / type-chooser paths to a namable superclass (or meta-class construction), and the Hidden usage matrix in StaticCompileRuntimePackageTest look solid. Map-style construction via DYNAMIC_RESOLUTION is covered.
Blocking for approve right now:
- CI:
lts (17, windows-latest, 1)60m timeout — please re-run. - Codecov patch ~79% with 53 uncovered/partial lines across
StaticTypesMethodReferenceExpressionWriter,StaticInvocationWriter,StaticTypesCallSiteWriter,AsmClassGenerator,StaticTypesTypeChooser,MethodPointerExpressionWriter. That is a lot of new control flow for a class-naming edge; please add cases that hit the missing branches (or point at existing tests Codecov missed).
Depends on #3097 tip; land after #3097. Sibling of #3099 (12570), not a linear stack.
…ed through public classes
|
# Conflicts: # src/main/java/org/codehaus/groovy/classgen/asm/sc/StaticTypesMethodReferenceExpressionWriter.java
…to the language rule For a public or private field the language rule of isMemberDirectlyAccessible decides. For a protected or package-private field, JVMS 5.4.4 allows access from the run-time package of the declaring class, or to a protected field from a subclass. That condition implies isMemberDirectlyAccessible, so it replaces it there instead of being checked after it.
# Conflicts: # src/main/java/org/codehaus/groovy/classgen/AsmClassGenerator.java
# Conflicts: # COMPATIBILITY.md
…class # Conflicts: # COMPATIBILITY.md
# Conflicts: # COMPATIBILITY.md # src/main/java/org/codehaus/groovy/classgen/asm/sc/StaticTypesMethodReferenceExpressionWriter.java
…class # Conflicts: # COMPATIBILITY.md # src/main/java/org/codehaus/groovy/classgen/asm/sc/StaticTypesMethodReferenceExpressionWriter.java
…class # Conflicts: # COMPATIBILITY.md
# Conflicts: # COMPATIBILITY.md
The note is now in the pull request description. An entry carried on this branch conflicted with every later addition to the end of that file.
…2568 The note is now in the description of apache#3097, which this builds on. An entry carried on this branch conflicted with every later addition to the end of that file.
…me package Static compilation should not select such a member at run time: compile time and run time may see different classes, the meta-class can influence the call, and the call may still fail at run time. The writers are back to what master emits. What is not about the run-time package moves to issues of its own: the protected members of another package, the super$ method of a package-private method, dynamic super and this access to such a field, and the invokedynamic failure for a constructor the JVM refuses.
…r run-time package The JVM allows access to a protected or package-private member only within the same run-time package, which is the package name and the class loader, apart from what a subclass can do with a protected member. A GroovyClassLoader defines each compilation in memory with a class loader of its own, so such code compiles and then fails with IllegalAccessError. A @CompileStatic script in a GroovyShell that calls a package-private method of a class of an earlier parse, or of a library class in the same package, is enough. The new option CompilerConfiguration.RUNTIME_PACKAGE_CHECK reports such a constructor, method or field when the class is generated. It is off by default, with the same bytecode as before, and GroovyShell turns it on for the class loader it creates, unless the configuration sets it. It has no effect on code compiled to class files, and on dynamic and @TypeChecked code, which reach such members through the meta-class.
The meta-class route for a package-private class goes with it, as static compilation should not select at run time: this takes the writers of the reworked GROOVY-12568 and leaves the check of the class for the next commit.
…age that an instruction names A class that is not public is accessible only within its run-time package (JVMS 5.4.4), and the JVM checks that whenever an instruction names the class: NEW, CHECKCAST, INSTANCEOF, the owner of a method or field, and the classes of a method handle or invokedynamic descriptor. Statically compiled code names it in new Hidden(), Hidden.sm(), Hidden.FIELD, and a call on a value of the class, and fails with IllegalAccessError when the class comes from an earlier parse of the same GroovyShell or from a library used by a script. Under CompilerConfiguration.RUNTIME_PACKAGE_CHECK, such an instruction is now reported when the class is generated, as is a superclass or interface that the JVM refuses. A value of the class that no instruction names is not reported, as it works: it can be stored, passed on and assigned to a public type.
…e check The helper class of an annotation collector is generated that way, and the check of every method of a compilation must not fail for it.
daniellansun
left a comment
There was a problem hiding this comment.
Re-reviewed tip 8303d97efd63 as the GROOVY-12569 companion on the path-2 design (no longer the earlier “rewrite to a namable superclass / meta-class construction” approach).
RuntimePackageCheck.checkSuperTypes (refuse a non-public superclass / interface from another run-time package) and checkNamedClasses / NamedClassChecker (ASM visitor reporting instructions that name such a class — owners, descriptors, handles, constants) are the right layer: they mirror what the JVM verifies, leave values that never name the class alone, and stay behind the same RUNTIME_PACKAGE_CHECK option as #3097. The Hidden usage matrix in StaticCompileRuntimePackageTest exercises the interesting instruction shapes.
Earlier coverage concerns about getDirectlyAccessibleSuperClass / rewrite branches are obsolete on this tip. LGTM — please land after #3097 (this tip already merges that work).
…or generated by @InheritConstructors @InheritConstructors adds a constructor for each constructor of the superclass. When one of them is a package-private constructor of another run-time package, the generated constructor fails only if it is called, as on master; reporting its super(...) call broke a class that works when the other constructors are used. A super(...) call without a source position in a generated constructor is no longer reported.
Dynamic code calls a super constructor with INVOKESPECIAL too, not through the meta-class, so super(...), the implicit super() and the constructor of an anonymous inner class fail with IllegalAccessError in dynamic code as well. The report of such a call now says so instead of advising @CompileDynamic, which stays the advice for the other members.
Its methods are called by the class generator, the static invocation writer and the constructor call transformer, which are in three packages, so the class stays public; it is not API for users.
The tests of members of the same compilation unit compiled with a plain CompilationUnit, where the check is off and every class counts as being in the same run-time package, so they passed whatever the check decided. They now compile in memory with a GroovyClassLoader and the option on, as the class loader of a GroovyShell does, and the joint compilation test sets the option, as the tests of class files do.
# Conflicts: # src/main/java/org/codehaus/groovy/classgen/asm/sc/RuntimePackageCheck.java
…face of another run-time package The JVM refuses to load a class whose superclass or interface it cannot access, in dynamic code too, so @CompileDynamic does not help there. The report of such a supertype now says so, and the instructions of the class that name the supertype, such as the super constructor call of a generated constructor, are no longer reported a second time with the advice.
…by @InheritConstructors Like its super(...) call, the code that @InheritConstructors generates for a constructor of the superclass, such as the cast of a parameter of a package-private class of another run-time package, fails only if that constructor is called. Instructions without a source position in a generated constructor are no longer reported; code with a source position there, such as a field initializer, still is.
…class # Conflicts: # src/main/java/org/codehaus/groovy/classgen/asm/sc/RuntimePackageCheck.java # src/main/java/org/codehaus/groovy/control/CompilerConfiguration.java # src/test/groovy/org/codehaus/groovy/classgen/asm/sc/StaticCompileRuntimePackageTest.groovy
🚨 All tests passed but jobs failed 🚨Failed Jobs without Test Failures🏷️ Commit: 6137d02 Rerun ControlsClick the checkbox to trigger a rerun:
Learn more about TestLens at testlens.app/docs. |
This builds on #3097 (GROOVY-12568), whose option it uses; the commits of #3097 are included until it is merged.
A class that is not public is accessible only within its run-time package (JVMS 5.4.4), and the JVM checks that whenever an instruction names the class:
NEW,CHECKCAST,INSTANCEOF, the owner of a method or field, and the classes of a method handle or invokedynamic descriptor. A package-private class of an earlier parse of the sameGroovyShell, or of a library used by a script, is in another run-time package, so statically compiled code that names it fails withIllegalAccessError:Under
CompilerConfiguration.RUNTIME_PACKAGE_CHECK, such an instruction is now reported when the class is generated:The check looks at the instructions of a statically compiled method, not at the source, so it reports exactly what the JVM refuses. A value of the class that no instruction names is not reported, because it works:
def h = Pub.make(),Hidden h = Pub.make(), assigning it to a public type, passing it on,h.toString(),h.&own. A cast,instanceofand a method reference that names the class are reported. A statically compiled class whose superclass or interface is such a class is reported too, as the JVM refuses to load it.A class of the same compilation, a public class, and a class compiled to a class file are not affected. With the option off, the bytecode is the same as before.
A package-private superclass or interface of another run-time package is reported once, at the class, noting that such a class fails to load in dynamic code too, instead of advising
@CompileDynamic. Code without a source position in a generated constructor, such as the parameter cast in one that@InheritConstructorsadds, is not checked.Compatibility
As for #3097: a
@CompileStaticscript or class in aGroovyShellthat names such a class no longer compiles. If it ran that code it failed anyway, withIllegalAccessError. What changes is code that compiled and never reached the instruction. Use@CompileDynamic, or set the option to false. Dynamic and@TypeCheckedcode are not affected, and keep failing where they name the class themselves, as for a cast.Tests
StaticCompileRuntimePackageTest:testInstructionsThatNamePackagePrivateClassOfAnotherRuntimePackage(30 forms),testMessageOfPackagePrivateClassOfAnotherRuntimePackageandtestSuperTypesOfAnotherRuntimePackagefail without the change.testValueOfPackagePrivateClassOfAnotherRuntimePackageThatNoInstructionNames,testPackagePrivateClassInDynamicAndTypeCheckedCode,testPackagePrivateClassOfTheSameScriptOrWithTheOptionOffandtestPackagePrivateClassOfClassCompiledBeforeToClassFilespin what must not change. 15,549 tests inorg.codehaus.groovy.classgen,groovy.transform.stc,groovy,gls,bugs,org.codehaus.groovy.transformandorg.apache.groovypass with the check on for everyGroovyShellthey use. The Checkstyle and CodeNarc gates pass.