Package libraries Java code as Java Modules

[JaroslavTulach]

Pull Request Description

Implements #7082 by creating a module layer per library. Removes the obsolete add_to_class_path concept as there is no longer a single ClassPath to load from. Each library with polyglot/java JARs gets its own. Need a way to express dependencies between Java APIs among multiple libraries - created requires: flag in package.yaml.

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

• The documentation has been updated, if necessary.
• Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
• All code follows the
  Scala,
  Java,
  TypeScript,
  and
  Rust
  style guides. In case you are using a language not listed above, follow the Rust style guide.
• Unit tests have been written where possible.
• Report decent exception of layer resolution problem

[JaroslavTulach]

Question for @radeusgd - we seem to get the encapsulation with all the consequences. There is dozen of failures:

• Incorrect polyglot import Stream_Utils
• Incorrect polyglot import File_Utils
• CompareException
• Text_Utils
• CaseFoldedString
• some Not_Logged_in.Error
• NoClassDefFoundError Polyglot_Utils
• Autodetect read doesn't seem to work
• Another Polyglot_Utils

I can think of ways to fix the tests, but some failures show intended cross-library usage we don't have any alternative yet.

[radeusgd]

Question for @radeusgd - we seem to get the encapsulation with all the consequences. There is dozen of failures:

• Incorrect polyglot import Stream_Utils
• Incorrect polyglot import File_Utils
• CompareException
• Text_Utils
• CaseFoldedString
• some Not_Logged_in.Error
• NoClassDefFoundError Polyglot_Utils
• Autodetect read doesn't seem to work
• Another Polyglot_Utils

I can think of ways to fix the tests, but some failures show intended cross-library usage we don't have any alternative yet.

I imagine we can come up with some workarounds, although that may not be trivial. I'm worried the most about the read autodetection - does it look like the classpath separation breaks the SPI? How can we fix that?

Alternatively, as suggested in #7082 (title Re-exporting), could we provide a way for a library to opt-in to being able to access modules of another lib? e.g. Standard.Table asking to be able to user Java helpers of Standard.Base, e.g. by adding some entry in its package.yaml? That would still prevent users from accessing the internals (unless they also add such an override) while allowing us to more easily share utilities between our libraries.

[JaroslavTulach]

I can think of ways to fix the tests

My plan was:

enso$ git diff distribution/ test/Base_Tests/src/Data/Text/Utils_Spec.enso
diff --git distribution/lib/Standard/Base/0.0.0-dev/src/Internal/Extra_Imports.enso distribution/lib/Standard/Base/0.0.0-dev/src/Internal/Extra_Imports.enso
index 7771401cf4..c42ee36ec6 100644
--- distribution/lib/Standard/Base/0.0.0-dev/src/Internal/Extra_Imports.enso
+++ distribution/lib/Standard/Base/0.0.0-dev/src/Internal/Extra_Imports.enso
@@ -9,7 +9,7 @@ private
 polyglot java import java.io.PrintStream
 
 # needed by Util_Spec
-polyglot java import org.enso.base.text.CaseFoldedString
+polyglot java import org.enso.base.text.CaseFoldedString as JCaseFoldedString
 polyglot java import org.enso.base.text.CaseFoldedString.Grapheme
 
 # needed by Comparator_Spec:
@@ -29,3 +29,5 @@ polyglot java import java.util.function.Function
 polyglot java import java.lang.Thread
 polyglot java import java.lang.Thread.State
 polyglot java import java.lang.Float
+
+CaseFoldedString = JCaseFoldedString
diff --git test/Base_Tests/src/Data/Text/Utils_Spec.enso test/Base_Tests/src/Data/Text/Utils_Spec.enso
index d0b987bea2..7d8d006ff7 100644
--- test/Base_Tests/src/Data/Text/Utils_Spec.enso
+++ test/Base_Tests/src/Data/Text/Utils_Spec.enso
@@ -1,7 +1,7 @@
 from Standard.Base import all
 
 polyglot java import org.enso.base.Text_Utils
-polyglot java import org.enso.base.text.CaseFoldedString
+import Standard.Base.Internal.Extra_Imports.CaseFoldedString
 polyglot java import com.ibm.icu.text.BreakIterator
 
 from Standard.Test import all

but that requires us to run with --disable-private-check. I can add such option, but then I am getting test failures at

[FAILED] Private constructors: [6/11, 26ms]
  | => enso / runEngineDistribution 111s
    - [FAILED] cannot directly call private constructor [4ms]
        Reason: Expected a Panic Private_Access to be thrown, but the action succeeded and returned [(Cons 42)] (at test/Base_Tests/src/Semantic/Private_Spec.enso:18:13-65).
    - [FAILED] can get reference to private constructor, but cannot call it [2ms]
        Reason: Expected a Panic Private_Access to be thrown, but the action succeeded and returned [(Cons 42)] (at test/Base_Tests/src/Semantic/Private_Spec.enso:27:13-58).
    - [FAILED] cannot get private field [2ms]
        Reason: Expected a Panic Private_Access to be thrown, but the action succeeded and returned [42] (at test/Base_Tests/src/Semantic/Private_Spec.enso:35:13-56).
    - [FAILED] cannot call private constructor from Java [3ms]
        Reason: Expected a Panic Private_Access to be thrown, but the action succeeded and returned [(Cons 42)] (at test/Base_Tests/src/Semantic/Private_Spec.enso:59:13-88).
    - [FAILED] cannot call private constructor from a lambda method [3ms]
        Reason: Expected a Panic Private_Access to be thrown, but the action succeeded and returned [(Cons 42)] (at test/Base_Tests/src/Semantic/Private_Spec.enso:62-64).

[FAILED] Private methods: [2/5, 9ms]

I guess we cannot have both: Run with --disable-private-check and these Private tests.

[JaroslavTulach]

Heuréka. JVM tests are green.

[JaroslavTulach]

This will very likely break Espresso support. We had to remove java.desktop because of Espresso in #9866

[Akirathan]

java.desktop is required by org.apache.poi.poi. The NI failed on Base_Tests because of that.

[JaroslavTulach]

java.desktop is required by org.apache.poi.poi. The NI failed on Base_Tests because of that.

That's scary. Probably another dependency on java.beans - we should change upstream to make this dependency optional.