Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Safe Composition of Metadata for ShardingSphere Proxy Native #33179

Merged
merged 1 commit into from
Oct 11, 2024

Conversation

linghengqian
Copy link
Member

@linghengqian linghengqian commented Oct 9, 2024

For #29052.

Changes proposed in this pull request:

To make usage of third-party reachability metadata safe we must guarantee that metadata addition can't break existing programs. This is currently not the case

  • Adds GraalVM Reachability Metadata for ch.qos.logback:logback-core:1.2.13

  • Adds GraalVM Reachability Metadata for com.alibaba:druid:1.2.7

  • Adds GraalVM Reachability Metadata for com.github.docker-java:docker-java:3.4.0

  • Adds GraalVM Reachability Metadata for com.mysql:mysql-connector-j:8.3.0

  • Adds GraalVM Reachability Metadata for io.grpc:grpc-netty:1.65.1

  • Adds GraalVM Reachability Metadata for io.vertx:vertx-core:4.5.1

  • Adds GraalVM Reachability Metadata for io.netty:netty-transport-classes-epoll:4.1.112.Final

  • Adds GraalVM Reachability Metadata for org.apache.groovy:groovy:4.0.22

  • Adds GraalVM Reachability Metadata for org.apache.hadoop:hadoop-common:3.3.6

  • Adds GraalVM Reachability Metadata for org.apiguardian:apiguardian-api:1.1.2

  • Adds GraalVM Reachability Metadata for org.jboss.logging:jboss-logging:3.2.1.Final

  • Adds GraalVM Reachability Metadata for org.jboss.narayana.jta:jta:5.12.7.Final

  • Adds GraalVM Reachability Metadata for org.junit.jupiter:junit-jupiter:5.11.1

  • Adds GraalVM Reachability Metadata for org.testcontainers:junit-jupiter:1.20.2

  • Adds GraalVM Reachability Metadata for com.typesafe:config:1.2.1

  • Adds GraalVM Reachability Metadata for com.google.protobuf:protobuf-java:3.21.12. This sounds like what I missed in Add support for com.google.protobuf:protobuf-java-util:3.21.12 oracle/graalvm-reachability-metadata#167 .

  • Adds GraalVM Reachability Metadata for org.springframework:spring-jcl:5.3.27

  • Adds GraalVM Reachability Metadata for com.clickhouse:clickhouse-jdbc:0.6.3

  • Updates documentation. According to Unable to apply parallel builds to thread-unsafe native-maven-plugin graalvm/native-build-tools#619 , GraalVM Native Build Tools 0.10.3 no longer prevents parallel builds of Maven.

  • As for [GR-49770] Safe Composition of Metadata  oracle/graal#5173, the SCM proposal involving buildArg entered GraalVM CE For JDK22 as an experimental option and became an official option on GraalVM CE For JDK23. The SCM proposal brings more readable Error Logs, which is definitely beneficial to contributors. The Error Logs brought by the SCM proposal are as follows.

-H:+UnlockExperimentalVMOptions -H:ThrowMissingRegistrationErrors= -H:MissingRegistrationReportingMode=Warn
com.oracle.svm.core.jdk.resources.MissingResourceRegistrationError: The program tried to access the resource at path 

   META-INF/services/javax.xml.parsers.SAXParserFactory

 without it being registered as reachable. Add it to the resource metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#resources-and-resource-bundles for help
  [email protected]/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1387)
  [email protected]/javax.xml.parsers.FactoryFinder$1.run(FactoryFinder.java:255)
  [email protected]/java.security.AccessController.executePrivileged(AccessController.java:128)
  [email protected]/java.security.AccessController.doPrivileged(AccessController.java:319)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access method

   ch.qos.logback.core.encoder.Encoder#valueOf(java.lang.String)

 without it being registered for runtime reflection. Add ch.qos.logback.core.encoder.Encoder#valueOf(java.lang.String) to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
  [email protected]/java.lang.Class.getMethod(DynamicHub.java:1070)
  ch.qos.logback.core.joran.util.StringToObjectConverter.getValueOfMethod(StringToObjectConverter.java:101)
  ch.qos.logback.core.joran.util.StringToObjectConverter.followsTheValueOfConvention(StringToObjectConverter.java:110)
  ch.qos.logback.core.joran.util.StringToObjectConverter.canBeBuiltFromSimpleString(StringToObjectConverter.java:39)
  • On ShardingSphere's own GRM side, remove {"excludeClasses": "java.**"}. Because this type of GRM begins to be required by Safe Composition of Metadata. There are obviously too many java.** GRM JSONs that need to be defined.
    {"excludeClasses": "java.**"},
    {"includeClasses": "java.io.BufferedWriter"},
    {"includeClasses": "java.io.Closeable"},
    {"includeClasses": "java.io.Flushable"},
    {"includeClasses": "java.io.InputStream"},
    {"includeClasses": "java.io.OutputStream"},
    {"includeClasses": "java.io.Reader"},
    {"includeClasses": "java.io.Writer"},
    {"includeClasses": "java.lang.Appendable"},
    {"includeClasses": "java.lang.AutoCloseable"},
    {"includeClasses": "java.lang.Boolean"},
    {"includeClasses": "java.lang.ClassLoader"},
    {"includeClasses": "java.lang.CharSequence"},
    {"includeClasses": "java.lang.Comparable"},
    {"includeClasses": "java.lang.Integer"},
    {"includeClasses": "java.lang.Iterable"},
    {"includeClasses": "java.lang.Long"},
    {"includeClasses": "java.lang.Number"},
    {"includeClasses": "java.lang.Process"},
    {"includeClasses": "java.lang.Readable"},
    {"includeClasses": "java.lang.constant.ConstantDesc"},
    {"includeClasses": "java.lang.reflect.AnnotatedElement"},
    {"includeClasses": "java.lang.management.RuntimeMXBean"},
    {"includeClasses": "java.util.AbstractCollection"},
    {"includeClasses": "java.util.AbstractMap"},
    {"includeClasses": "java.util.Collection"},
    {"includeClasses": "java.util.Deque"},
    {"includeClasses": "java.util.Enumeration"},
    {"includeClasses": "java.util.HashSet"},
    {"includeClasses": "java.util.Iterator"},
    {"includeClasses": "java.util.LinkedHashSet"},
    {"includeClasses": "java.util.List"},
    {"includeClasses": "java.util.Map"},
    {"includeClasses": "java.util.NavigableSet"},
    {"includeClasses": "java.util.Properties"},
    {"includeClasses": "java.util.Set"},
    {"includeClasses": "java.util.SortedMap"},
    {"includeClasses": "java.util.SortedSet"},
    {"includeClasses": "java.util.Timer"},
    {"includeClasses": "java.util.concurrent.BlockingQueue"},
    {"includeClasses": "java.util.concurrent.atomic.LongAdder"}
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access

   java.util.Collection.getMethods()

 without it being registered for runtime reflection. Add java.util.Collection.getMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
  [email protected]/java.lang.Class.getMethods(DynamicHub.java:1022)
  org.codehaus.groovy.reflection.stdclasses.CachedSAMClass.getSAMMethod(CachedSAMClass.java:172)
  org.codehaus.groovy.reflection.ClassInfo.isSAM(ClassInfo.java:388)
  org.codehaus.groovy.reflection.ClassInfo.createCachedClass(ClassInfo.java:378)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access

   java.util.List.getMethods()

 without it being registered for runtime reflection. Add java.util.List.getMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
  [email protected]/java.lang.Class.getMethods(DynamicHub.java:1022)
  org.codehaus.groovy.reflection.stdclasses.CachedSAMClass.getSAMMethod(CachedSAMClass.java:172)
  org.codehaus.groovy.reflection.ClassInfo.isSAM(ClassInfo.java:388)
  org.codehaus.groovy.reflection.ClassInfo.createCachedClass(ClassInfo.java:378)

Before committing this PR, I'm sure that I have checked the following options:

  • My code follows the code of conduct of this project.
  • I have self-reviewed the commit code.
  • I have (or in comment I request) added corresponding labels for the pull request.
  • I have passed maven check locally : ./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e.
  • I have made corresponding changes to the documentation.
  • I have added corresponding unit tests for my changes.

Copy link
Member Author

@linghengqian linghengqian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • This PR became quite complicated because the Warning Log even included GraalVM CE's internal missing GRM, which definitely involved multiple potential GraalVM bugs. Before supporting compiling shardingsphere project with GraalVM CE For JDK23, I needed to change -H:ThrowMissingRegistrationErrors= to -H:ThrowMissingRegistrationErrors=org.apache.shardingsphere to avoid a wider range of impacts.
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access

   com.oracle.svm.core.genscavenge.HeapImplMemoryMXBean.getConstructors()

 without it being registered for runtime reflection. Add com.oracle.svm.core.genscavenge.HeapImplMemoryMXBean.getConstructors() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
  [email protected]/java.lang.Class.getConstructors(DynamicHub.java:1028)
  [email protected]/com.sun.jmx.mbeanserver.MBeanIntrospector.findConstructors(MBeanIntrospector.java:461)
  [email protected]/com.sun.jmx.mbeanserver.MBeanIntrospector.getClassMBeanInfo(MBeanIntrospector.java:432)
  [email protected]/com.sun.jmx.mbeanserver.MBeanIntrospector.getMBeanInfo(MBeanIntrospector.java:391)
  • The path scanned by seata client is also suspicious. I don't believe the seata client uses the registry file directly.
com.oracle.svm.core.jdk.resources.MissingResourceRegistrationError: The program tried to access the resource at path 

   registry

 without it being registered as reachable. Add it to the resource metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#resources-and-resource-bundles for help
  [email protected]/java.lang.ClassLoader.getResource(ClassLoader.java:1407)
  org.apache.seata.config.FileConfiguration.getFileFromClasspath(FileConfiguration.java:194)
  org.apache.seata.config.FileConfiguration.getConfigFile(FileConfiguration.java:144)
  org.apache.seata.config.FileConfiguration.<init>(FileConfiguration.java:108)

@strongduanmu strongduanmu merged commit 2b6fcca into apache:master Oct 11, 2024
142 checks passed
@linghengqian linghengqian deleted the scm-re branch October 11, 2024 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants