Export tokens from lib/index.js

[dlqqq]

Description

Exports tokens from lib/index.js to allow labextension configuration.

Motivation

I was not able to configure a custom message footer (#942) by providing the IJaiMessageFooter token in a separate labextension, scaffolded via the latest extension-template.

In my custom extension, I imported the tokens from @jupyter-ai/core/lib/tokens and defined a plugin that provided the IJaiMessageFooter token. I also added this to my package.json file, according to the labextension docs:

    "jupyterlab": {
        "discovery": {
            "server": {
                "managers": [
                    "pip"
                ],
                "base": {
                    "name": "test_extension"
                }
            }
        },
        "extension": true,
        "outputDir": "test_extension/labextension",
        "sharedPackages": {
            "@jupyter-ai/core": {
                "bundled": false,
                "singleton": true
            },
            "@jupyter-ai/core/lib/tokens": {
                "bundled": false,
                "singleton": true
            }
        }
    },

However, this did not add a custom message footer to the chat messages, and I received an error message in the browser console:

Shared module @jupyter-ai/core/lib/tokens doesn't exist in shared scope default

After doing a bit of digging, I noticed that all other labextensions re-export their tokens from src/index.ts, so you can import the tokens from the package directly (instead of a module within that package). I made the code changes in jupyter-ai, built new JavaScript & Python packages via jupyter-releaser and installed these locally.

Then, I updated the import in my extension to:

import { IJaiMessageFooter } from '@jupyter-ai/core';

Finally, I rebuilt my extension. To my surprise, this worked! I could see the new message footer in the chat UI.

It seems that tokens must be imported from a package's main file, lib/index.js. Otherwise, the tokens aren't available as a shared singleton module.

[dlqqq]

It seems that tokens must be imported from a package's main file, lib/index.js. Otherwise, the tokens aren't available as a shared singleton module.

@krassowski Have you encountered this issue before? It doesn't appear to be documented on JupyterLab's extension dev guide, so I'm wondering if this is due to me doing something wrong in the steps I took (documented above).

[krassowski]

Things which should be shared need to be imported from a public API not though lib, yes. Using lib leads to bundling a second copy of the dependency and enums, objects and some types (like including enums) no longer match via identity comparison.

That's the implementation detail of our extension system using webpack module federation leaking out. It's entirely logical when you are familiar with how webpack sharing of dependencies works. Maybe there is a way to tell webpack to merge lib imports with default imports, not sure.

I also explained it a bit here: https://discourse.jupyter.org/t/prebuilt-extension-error-no-provider-for-jupyterlab-coreutils-isettingregistry/28104/3?u=krassowski

[krassowski]

Absolutely fine solution IMO, tokens are almost always pubic API so let's export them :)

[dlqqq]

It's entirely logical when you are familiar with how webpack sharing of dependencies works

That explains my confusion then 😂. Thank you very much for finding this info so quickly!