Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package libraries Java code as Java Modules #10714

Draft
wants to merge 90 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
9c149b9
Must specify --repl to enable debug server
JaroslavTulach Jul 29, 2024
76070ba
Use --repl to enable REPL changelog note
JaroslavTulach Jul 29, 2024
873c418
Towards a ClassPath per Package
JaroslavTulach Jul 30, 2024
3b1bc98
One ClassLoader per each Package
JaroslavTulach Jul 30, 2024
45f62cf
Define ModuleLayer for each Package
JaroslavTulach Jul 30, 2024
8700cc8
Load classes via ModuleLayer
JaroslavTulach Jul 30, 2024
74f8111
Turning std-base and std-aws into Java modules
JaroslavTulach Jul 30, 2024
7e865ce
Turn common-polyglot-core-utils into JPMS module
Akirathan Jul 30, 2024
3a6c335
Add depedencies to org.enso.std.base JPMS module
Akirathan Jul 30, 2024
b23be3d
Turn std-database into JPMS module
Akirathan Jul 30, 2024
f0dd3f7
Add depedencies to org.enso.std.aws JPMS module
Akirathan Jul 30, 2024
c4c1842
Using module-info to register SPI services
JaroslavTulach Jul 31, 2024
149dc7e
Removing another no longer needed dependency on NetBeans Lookup API
JaroslavTulach Jul 31, 2024
48b29c7
Removing add to class path concept
JaroslavTulach Jul 31, 2024
dd5723e
Turn std-table into JPMS module
Akirathan Jul 31, 2024
74fa9e1
Move tests that need internal access to Standard.Base into test/Base_…
JaroslavTulach Jul 31, 2024
1e48979
Resolving merge conficts
JaroslavTulach Jul 31, 2024
bbe754c
No more of Polyglot.Java
JaroslavTulach Aug 1, 2024
1514ad2
Nicer formatting with var
JaroslavTulach Aug 1, 2024
09c6946
Allow Standard.Table to depend on modules from Standard.Base
JaroslavTulach Aug 1, 2024
3148976
language-server/test compiles again
JaroslavTulach Aug 1, 2024
ae64b8a
Host Files values group_builder
JaroslavTulach Aug 5, 2024
c5e41ab
Merging in develop branch
JaroslavTulach Aug 5, 2024
6fdd11c
Merge with most recent develop branch that includes many JPMS
JaroslavTulach Sep 26, 2024
cfdda7b
Using moduleDependencies and internalModuleDependencies for standard …
JaroslavTulach Sep 26, 2024
d50f006
There can be Enso sources without associated package
JaroslavTulach Sep 26, 2024
202b1d8
Merging with develop
JaroslavTulach Oct 1, 2024
b04ce5c
Avoid duplicated modules. Use the one from boot layer.
JaroslavTulach Oct 1, 2024
d94a44d
Load all library Java modules with a single class loader
JaroslavTulach Oct 1, 2024
c71183c
Opening std-table.jar for reflection from Enso code
JaroslavTulach Oct 1, 2024
9bc7a3a
Visualization_Tests run and yield just nine errors
JaroslavTulach Oct 1, 2024
16b82c3
Standard.Test loads classes from Standard.Base
JaroslavTulach Oct 1, 2024
405e88d
Always prefer modules from parent layers
JaroslavTulach Oct 1, 2024
57e64e0
AWS requires access to Standard.Base and Standard.Database JARs
JaroslavTulach Oct 1, 2024
efda30f
Opening up std-table JAR, so Enso can access all Java classes
JaroslavTulach Oct 1, 2024
e9a0456
Less functional boilerplate
JaroslavTulach Oct 2, 2024
c0359e2
Merge remote-tracking branch 'origin/develop' into wip/jtulach/ClassP…
JaroslavTulach Oct 2, 2024
5b14f89
Special Lookup class and registration via a property to allow searchi…
JaroslavTulach Oct 2, 2024
841de5f
Turning Standard.Image into Java module
JaroslavTulach Oct 2, 2024
e6e5c09
Making test/Image_Tests pass OK
JaroslavTulach Oct 2, 2024
3194f6d
Avoid DriverManager. Directly talk to Driver instances.
JaroslavTulach Oct 2, 2024
cbe625f
Register SQLServerConnectionDetailsSPI in std-bits/microsoft/*/module…
JaroslavTulach Oct 3, 2024
c72ab8e
No polyglot symbol for org.enso.base.encoding.NewlineDetector in Tabl…
JaroslavTulach Oct 3, 2024
062b26e
No polyglot symbol for org.enso.table.data.table.Table in Table_Tests
JaroslavTulach Oct 3, 2024
e4cc4b4
No polyglot symbol for org.enso.table.data.table.join.between.SortedL…
JaroslavTulach Oct 3, 2024
9c86f7a
Using for loop
JaroslavTulach Oct 3, 2024
6992103
Make enso-test-java-helpers available to Table_Tests
JaroslavTulach Oct 3, 2024
98eeb64
No need to generate .bindings index to run benchmarks
JaroslavTulach Oct 3, 2024
60b4277
BenchProcessor puts runtime.jar into own layer. Use that one as bootL…
JaroslavTulach Oct 3, 2024
f482b72
test/Benchmarks need access to org.enso.table.data.index.OrderedMulti…
JaroslavTulach Oct 4, 2024
3112158
Introducing AWS wrapper module
JaroslavTulach Oct 7, 2024
e51eb3e
Placeholders for exported packages
JaroslavTulach Oct 7, 2024
a263449
Remove placeholders
Akirathan Oct 7, 2024
d161f9e
Remove aws-wrapper/Compile/moduleDependencies.
Akirathan Oct 7, 2024
c03825c
javaModuleName is the first setting
Akirathan Oct 7, 2024
3ec8ed0
Fix typo in cbor library dependency
Akirathan Oct 7, 2024
41fd4c1
Typo in module name.
Akirathan Oct 7, 2024
4bec7d9
Fix compilation of aws-wrapper
Akirathan Oct 7, 2024
88aaa76
Remove empty settings from std-aws
Akirathan Oct 7, 2024
be66f1b
Fix typo in module name
Akirathan Oct 7, 2024
c97ec77
aws-wrapper-assembly.jar gets into component directory.
Akirathan Oct 7, 2024
4a6e55c
Remove signature from aws-wrapper-assembly jar
Akirathan Oct 7, 2024
011494e
aws-wrapper dependency of std-aws is "provided".
Akirathan Oct 7, 2024
92bc7b4
Ensure aws-wrapper-assembly.jar gets into AWS/polyglot/java directory
Akirathan Oct 7, 2024
06ca5ce
Fix NoSuchElementException in JPMSUtils
Akirathan Oct 7, 2024
34dda82
Revert "aws-wrapper-assembly.jar gets into component directory."
Akirathan Oct 7, 2024
e788644
org.enso.std.aws exports org.enso.aws package
Akirathan Oct 7, 2024
644357d
slf4j is module dependency of aws-wrapper
Akirathan Oct 7, 2024
6c33fef
aws-wrapper depends on org.reactivestreams
Akirathan Oct 7, 2024
4c98a58
aws-wrapper uses its own service.
Akirathan Oct 7, 2024
e81e27c
Add some dependencies to aws-wrapper
Akirathan Oct 7, 2024
686f24d
Add necessary exports
Akirathan Oct 7, 2024
c5b1959
fmt
Akirathan Oct 7, 2024
2bd96d4
FileSystemSPI uses Lookup instead of ServiceLoader
Akirathan Oct 7, 2024
cf605cd
org.enso.std.aws provides DataLinkSPI.
Akirathan Oct 7, 2024
12ebe80
Better error message on duplicated service providers
Akirathan Oct 7, 2024
a31fb9c
Duplicate provider is just a warning
Akirathan Oct 7, 2024
77dc8de
List is easier to == than Array
JaroslavTulach Oct 8, 2024
6388881
runtime-integration-tests are loading classes with empty EnsoClassPath
JaroslavTulach Oct 8, 2024
00b47b4
Java.lookup_class has been removed
JaroslavTulach Oct 8, 2024
4173822
Using List() at the right places
JaroslavTulach Oct 8, 2024
f0c2ae4
std.table module isn't present in NI enso build
JaroslavTulach Oct 8, 2024
f11a26a
Use Lookup to lookup DatabaseConnectionDetailsSPI
JaroslavTulach Oct 8, 2024
b7f6542
An unexpected panic was thrown: (No_Such_Method.Error org.enso.base.n…
JaroslavTulach Oct 8, 2024
baeff36
Initialize ModuleLayer for a library sooner than a class is loaded fr…
JaroslavTulach Oct 8, 2024
a9bc32f
There is no runtime-fat-jar anymore
JaroslavTulach Oct 8, 2024
ec03e10
Add JDK_MODULES to small JDK of engine-runner native image
Akirathan Oct 8, 2024
1d4b172
Catch also ResolutionException
JaroslavTulach Oct 8, 2024
2bf4df1
Always add at least boot layer as parent
JaroslavTulach Oct 9, 2024
e8c220b
Make Tableau_Tests pass
JaroslavTulach Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@
- [Enforce conversion method return type][10468]
- [Renaming launcher executable to ensoup][10535]
- [Space-precedence does not apply to value-level operators][10597]
- [Must specify `--repl` to enable debug server][10709]

[10468]: https:/enso-org/enso/pull/10468
[10535]: https:/enso-org/enso/pull/10535
[10597]: https:/enso-org/enso/pull/10597
[10709]: https:/enso-org/enso/pull/10709

#### Enso IDE

Expand Down
74 changes: 38 additions & 36 deletions engine/runner/src/main/java/org/enso/runner/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
import scala.runtime.BoxedUnit;

/** The main CLI entry point class. */
public final class Main {
public class Main {
private static final String JVM_OPTION = "jvm";
private static final String RUN_OPTION = "run";
private static final String INSPECT_OPTION = "inspect";
Expand Down Expand Up @@ -98,6 +98,8 @@ public final class Main {

private static final org.slf4j.Logger logger = LoggerFactory.getLogger(Main.class);

Main() {}

private static boolean isDevBuild() {
return Info.ensoVersion().matches(".+-SNAPSHOT$");
}
Expand All @@ -106,11 +108,8 @@ private static Option.Builder cliOptionBuilder() {
return Option.builder();
}

/**
* Builds the [[Options]] object representing the CLI syntax.
*
* @return an [[Options]] object representing the CLI syntax
*/
private static final Options CLI_OPTIONS = buildOptions();

private static Options buildOptions() {
var help =
cliOptionBuilder().option("h").longOpt(HELP_OPTION).desc("Displays this message.").build();
Expand Down Expand Up @@ -512,22 +511,22 @@ private static Options buildOptions() {
*
* @param options object representing the CLI syntax
*/
private static void printHelp(Options options) {
new HelpFormatter().printHelp(LanguageInfo.ID, options);
private static void printHelp() {
new HelpFormatter().printHelp(LanguageInfo.ID, CLI_OPTIONS);
}

/** Terminates the process with a failure exit code. */
private static RuntimeException exitFail() {
private RuntimeException exitFail() {
return doExit(1);
}

/** Terminates the process with a success exit code. */
private static RuntimeException exitSuccess() {
private final RuntimeException exitSuccess() {
return doExit(0);
}

/** Shuts down the logging service and terminates the process. */
private static RuntimeException doExit(int exitCode) {
RuntimeException doExit(int exitCode) {
RunnerLogging.tearDown();
System.exit(exitCode);
return null;
Expand Down Expand Up @@ -665,7 +664,7 @@ private void compile(
* @param executionEnvironment name of the execution environment to use during execution or {@code
* null}
*/
private void run(
private void handleRun(
String path,
java.util.List<String> additionalArgs,
String projectPath,
Expand All @@ -675,6 +674,7 @@ private void run(
boolean disablePrivateCheck,
boolean enableAutoParallelism,
boolean enableStaticAnalysis,
boolean enableDebugServer,
boolean inspect,
boolean dump,
String executionEnvironment,
Expand Down Expand Up @@ -708,9 +708,13 @@ private void run(
.options(options);

if (inspect) {
if (enableDebugServer) {
println("Cannot use --inspect and --repl and --run at once");
throw exitFail();
}
options.put("inspect", "");
} else {
// by default running with debug server enabled
}
if (enableDebugServer) {
factory.messageTransport(replTransport());
options.put(DebugServerInfo.ENABLE_OPTION, "true");
}
Expand Down Expand Up @@ -960,7 +964,7 @@ private void displayVersion(boolean useJson) {
}

/** Parses the log level option. */
private static Level parseLogLevel(String levelOption) {
private Level parseLogLevel(String levelOption) {
var name = levelOption.toLowerCase();
var found =
Stream.of(Level.values()).filter(x -> name.equals(x.name().toLowerCase())).findFirst();
Expand All @@ -977,7 +981,7 @@ private static Level parseLogLevel(String levelOption) {
}

/** Parses an URI that specifies the logging service connection. */
private static URI parseUri(String string) {
private URI parseUri(String string) {
try {
return new URI(string);
} catch (URISyntaxException ex) {
Expand All @@ -1001,15 +1005,13 @@ public static void main(String[] args) throws Exception {
/**
* Main entry point for the CLI program.
*
* @param options the command line options
* @param line the provided command line arguments
* @param logLevel the provided log level
* @param logMasking the flag indicating if the log masking is enabled
*/
private void runMain(Options options, CommandLine line, Level logLevel, boolean logMasking)
throws IOException {
final void mainEntry(CommandLine line, Level logLevel, boolean logMasking) throws IOException {
if (line.hasOption(HELP_OPTION)) {
printHelp(options);
printHelp();
throw exitSuccess();
}
if (line.hasOption(VERSION_OPTION)) {
Expand Down Expand Up @@ -1089,7 +1091,7 @@ private void runMain(Options options, CommandLine line, Level logLevel, boolean
}

if (line.hasOption(RUN_OPTION)) {
run(
handleRun(
line.getOptionValue(RUN_OPTION),
Arrays.asList(line.getArgs()),
line.getOptionValue(IN_PROJECT_OPTION),
Expand All @@ -1099,14 +1101,15 @@ private void runMain(Options options, CommandLine line, Level logLevel, boolean
line.hasOption(DISABLE_PRIVATE_CHECK_OPTION),
line.hasOption(AUTO_PARALLELISM_OPTION),
line.hasOption(ENABLE_STATIC_ANALYSIS_OPTION),
line.hasOption(REPL_OPTION),
line.hasOption(INSPECT_OPTION),
line.hasOption(DUMP_GRAPHS_OPTION),
line.getOptionValue(EXECUTION_ENVIRONMENT_OPTION),
scala.Option.apply(line.getOptionValue(WARNINGS_LIMIT))
.map(Integer::parseInt)
.getOrElse(() -> 100));
}
if (line.hasOption(REPL_OPTION)) {
if (line.hasOption(REPL_OPTION) && !line.hasOption(RUN_OPTION)) {
runRepl(
line.getOptionValue(IN_PROJECT_OPTION),
logLevel,
Expand All @@ -1122,7 +1125,7 @@ private void runMain(Options options, CommandLine line, Level logLevel, boolean
preinstallDependencies(line.getOptionValue(IN_PROJECT_OPTION), logLevel);
}
if (line.getOptions().length == 0) {
printHelp(options);
printHelp();
throw exitFail();
}
}
Expand Down Expand Up @@ -1240,16 +1243,15 @@ private static final <T> scala.collection.immutable.List<T> join(
return scala.collection.immutable.$colon$colon$.MODULE$.apply(head, tail);
}

private void println(String msg) {
void println(String msg) {
System.out.println(msg);
}

private void launch(String[] args) throws IOException, InterruptedException, URISyntaxException {
var options = buildOptions();
var line = preprocessArguments(options, args);
var line = preprocessArguments(args);

var logMasking = new boolean[1];
var logLevel = setupLogging(options, line, logMasking);
var logLevel = setupLogging(line, logMasking);

if (line.hasOption(JVM_OPTION)) {
var jvm = line.getOptionValue(JVM_OPTION);
Expand Down Expand Up @@ -1332,36 +1334,36 @@ private void launch(String[] args) throws IOException, InterruptedException, URI
}
}

launch(options, line, logLevel, logMasking[0]);
launch(line, logLevel, logMasking[0]);
}

protected CommandLine preprocessArguments(Options options, String[] args) {
final CommandLine preprocessArguments(String... args) {
var parser = new DefaultParser();
try {
var startParsing = System.currentTimeMillis();
var line = parser.parse(options, args);
var line = parser.parse(CLI_OPTIONS, args);
logger.trace(
"Parsing Language Server arguments took {0}ms",
System.currentTimeMillis() - startParsing);
return line;
} catch (Exception e) {
printHelp(options);
printHelp();
throw exitFail();
}
}

private static Level setupLogging(Options options, CommandLine line, boolean[] logMasking) {
private Level setupLogging(CommandLine line, boolean[] logMasking) {
var logLevel =
scala.Option.apply(line.getOptionValue(LOG_LEVEL))
.map(Main::parseLogLevel)
.map(this::parseLogLevel)
.getOrElse(() -> defaultLogLevel);
var connectionUri = scala.Option.apply(line.getOptionValue(LOGGER_CONNECT)).map(Main::parseUri);
var connectionUri = scala.Option.apply(line.getOptionValue(LOGGER_CONNECT)).map(this::parseUri);
logMasking[0] = !line.hasOption(NO_LOG_MASKING);
RunnerLogging.setup(connectionUri, logLevel, logMasking[0]);
return logLevel;
}

private void launch(Options options, CommandLine line, Level logLevel, boolean logMasking) {
private void launch(CommandLine line, Level logLevel, boolean logMasking) {
if (line.hasOption(LANGUAGE_SERVER_OPTION)) {
try {
var conf = parseProfilingConfig(line);
Expand All @@ -1379,7 +1381,7 @@ private void launch(Options options, CommandLine line, Level logLevel, boolean l
conf,
ExecutionContext.global(),
() -> {
runMain(options, line, logLevel, logMasking);
mainEntry(line, logLevel, logMasking);
return BoxedUnit.UNIT;
});
} catch (IOException ex) {
Expand Down
46 changes: 46 additions & 0 deletions engine/runner/src/test/java/org/enso/runner/EngineMainTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package org.enso.runner;

import static org.junit.Assert.assertEquals;

import java.io.File;
import java.util.ArrayList;
import java.util.List;
import org.junit.Test;
import org.slf4j.event.Level;

public class EngineMainTest {
private final List<String> linesOut = new ArrayList<>();

@Test
public void cannotUseReplAndInspectAtOnce() throws Exception {
try {
var m =
new Main() {
@Override
RuntimeException doExit(int code) {
throw new ExitCode(code);
}

void println(String line) {
linesOut.add(line);
}
};
var file = File.createTempFile("some", ".enso");
file.deleteOnExit();
var line = m.preprocessArguments("--repl", "--inspect", "--run", file.getAbsolutePath());
m.mainEntry(line, Level.INFO, false);
} catch (ExitCode ex) {
assertEquals("Execution fails", 1, ex.exitCode);
assertEquals("One line printed", 1, linesOut.size());
assertEquals("Cannot use --inspect and --repl and --run at once", linesOut.get(0));
}
}

private static final class ExitCode extends RuntimeException {
final int exitCode;

ExitCode(int exitCode) {
this.exitCode = exitCode;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ static AddToClassPathNode build() {
Object doExecute(Object path, @Cached ExpectStringNode expectStringNode) {
var ctx = EnsoContext.get(this);
var file = ctx.getTruffleFile(new File(expectStringNode.execute(path)));
ctx.addToClassPath(file);
var cp = ctx.findClassPath(null);
cp.addToClassPath(file);
return ctx.getBuiltins().nothing();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ static LookupClassNode build() {
@Specialization
@CompilerDirectives.TruffleBoundary
Object doExecute(Object name, @Cached("build()") ExpectStringNode expectStringNode) {
return EnsoContext.get(this).lookupJavaClass(expectStringNode.execute(name));
return EnsoContext.get(this).lookupJavaClass(null, expectStringNode.execute(name));
}

abstract Object execute(Object name);
Expand Down
Loading
Loading