Use with "esm" and .mjs files does not work

[onlywei]

Hello, I'm unsure how to use this mocking together with mocha, esm and a .mjs file.

I built a reproducible test repo here: https:/onlywei/rewiremock-esm-repro

I've also tried turning on esm's cjs-interop flag and got a different error.
Would like advice.

[theKashey]

It will be interesting to solve.

[theKashey]

Node 10 introduced loader hooks - https://nodejs.org/api/esm.html#esm_loader_hooks
(not usable by now)

[theKashey]

Sorry a "delay".
There is 2 ways how to use it, and one way how not to:

Would not work

• it would not work with .mjs files - Unable to mock .mjs files standard-things/esm#786

Would work

It would work every time you use import to import something. .proxy, you have used in your example is making require which breaks things.

• with .module or .around helpers, which accepts import as a module name

 index = (await rewiremock.module(() => import('./index'), (r) => ({
        './foo': r.withDefault(() => 'fake-foo'),
}))).default;

It's the same as rewiremock.proxy, but for promised resolvers, ie - import.

• with mockery interface

rewiremock(() => import('./foo')).withDefault(() => 'fake-foo');
// or, it does not matter
rewiremock('./foo').withDefault(() => 'fake-foo');

rewiremock.enable();

index = (await import('./index')).default;

[iambumblehead]

@theKashey today I tried updating a package and some dependency-packages from "transpiled" to "native es6". A perfect blend of cjs/mjs incompatibilities prevent my success.

The final problem of getting ava and rewiremock to work with these packages is blocked it seems. AVA doesn't support es modules but can transpile only test files with esm and rewiremock does not support ".mjs" files. When importing rewiremock from an .mjs test file there is an error Error: Rewiremock: there is no "parent module". Is there two HotModuleReplacementPlugins? when importing rewiremock from a regular ".js" test file and using rewiremock.module to import the "mjs" source file, rewiremock does not mock the dependencies imported by that .mjs file.

Unfortunately, I don't have a solution and am just describing my experience.

Setting up packages that nicely provide both commonjs and es modules is complex. The best-working combination I found so far is to identify an ".mjs" file as the "main" file, so that node handles the file correctly when it is imported and doesn't require one to declare type "module" in the package.json (nodejs restricts how "module" type packages can be used and it seems better not to declare that)

I do not remember all of the subtle cjs/esm restrictions and problems to list them here... I wish rewiremock supported .mjs files.

Anyway thank you @theKashey for rewiremock it is a great package :)

[iambumblehead]

@theKashey I'm un-familiar with module-loading details, but if you have a plan for adding esm module support to rewiremock, I might be interested in trying to implement that plan.

The discussion here seems related https://gist.github.com/guybedford/e6687e77d60739f27d1b5449a2872bf4

-- edit
also related:

• Loader Hooks nodejs/modules#351,
• https://nodejs.org/api/esm.html#esm_hooks the hooks sections each have this note: Note: The loaders API is being redesigned. This hook may disappear or its signature may change. Do not rely on the API described below.

[iambumblehead]

I have tried using variations of the 'import' strategy as below

await rewiremock.module( () => import( '../../src/file.js' ), r => ({
  './localfile.js': { abcd: true },
  '@org/package': r.withDefault({ abcd: true })
});

I control the sources at '@org/package' and no matter the changes I make at @org/package or to the rewiremock call, localfile is always mocked succesfully and @org/package is never mocked succesfully. In some combinations, the imported src/file.js script gets an undefined default for the mocked '@org/package' package.

If rewiremock is able to mock the localfile this indicates it may also somehow be able to mock the package...

@theKashey do you have any ideas for me? What can I do so that '@org/package' will be mocked here?

[theKashey]

https://gist.github.com/guybedford/e6687e77d60739f27d1b5449a2872bf4 has an interesting, but not documented moment

  if (url.startsWith('mock:')) {
    return { format: 'dynamic' };
  }

Right now the main problem with "esm mocking" is not the actual "mocking", but clearing the cache later - ESM was designed to be immutable, so dependency level mocking was(?) not possible by design.

[theKashey]

So. Let's clarify one moment - anything which goes not through "old" module system is invisible to rewiremock. That is why mjs is not working, but I am not sure why @org/package is not working for you.

You can try to use the same "name resolution hack"(aka "import") to double check this moment.

await rewiremock.module( () => import( '../../src/file.js' ), r => ({
  './localfile.js': { abcd: true },
  // lets resolve a fill name
  require.resolve('@org/package'): r.withDefault({ abcd: true })
});

or

await rewiremock.around( () => import( '../../src/file.js' ), () => {
  rewiremock('./localfile.js').with({ abcd: true }),
  rewiremock(() => import('@org/package').withDefault({ abcd: true });
});

[iambumblehead]

@theKashey thanks for your assistance. I'll try it out later on at the end of the day :)

[iambumblehead]

@theKashey these lines from my test use the first pattern you gave...

const post = stub().resolves();
const { doHealthPush } = await rewiremock.module( () => import( '../../src/health/health.js' ), r => ({
  '../utils/errors.js': errorsExports,
  [require.resolve( '@platform/template-js-http-request' )]: r.withDefault({ post })
}));

inside "health.js" the package resolves to undefined

import request from '@platform/template-js-http-request';

console.log({ request }); // { request: undefined }

[iambumblehead]

@theKashey I made a small little package to test your setup pattern... it worked. I'll investigate to find what causes the issue in the original package.

edit
I found that the 'health.js' script imports scripts that imported scripts that imported "@platform/template-js-http-request" and when some of those imports are removed, it "resolves" the problem and the mocked definition then appears in the originally targetted file.

I'm trying to make a little sample that replicates the issue by loading the module again elsewhere in a dependency hierarchy outside the target file. but so far am unable to reproduce. In the bigger original package not-importing certain files that import "@platform/template-js-http-request" allows the mocked definition to remain...

using console.log to log imported "@platform/template-js-http-request" definitions like this...

import request from '@platform/template-js-http-request';

console.log({ request });

the shell process shows these definitions...

{ request: { post: [Function: functionStub] } }
{ request: undefined }

if I remove the rewiremock definition for the file, the full un-mocked definition appears in both console.log calls (there is no undefined definition.)

[theKashey]

Sounds like mocking is working, but there is a problem with providing mocked value.
Try

import * as request from '@platform/template-js-http-request';

console.log({ request });

note - output will be different.

[iambumblehead]

@theKashey a little demonstration is here: https:/iambumblehead/rewiremock-discuss

[theKashey]

That's awesome way to tackle the problem. How I will all issues will be as easily reproducible as this one.

[iambumblehead]

@theKashey thank you :) I am glad if I am helpful and thankful for your assistance

[iambumblehead]

@theKashey I wanted to give you a little update...

I tried updating the sources here but things are complicated by the way node's module system works. I made this package, but is slow to process really demanding test loads (fast enough for moderate loads) https:/iambumblehead/esmock

A few anecdotal things I found...

if import is used to load a script, the runtime somehow "knows" when module._load returns a definition that is not the actual module file and it will return undefined, so it is necessary to write mocked definitions directly to the loaded module.

'mjs' files can, in fact, be mocked. Inside module._load, load a blank/dummy.mjs file, write the mock definitions to that and return it.

A problem I was unable to solve is (big problem)... removing an 'es-module' from module._cache causes that modules' definition to revert to the un-mocked definition at runtime inside files that have already imported that definition. The 'solution' at esmock is to use a semaphor to load each mock definition completely before processing the next one.

If I were able to solve that last problem I would maybe try optimizing esmock for general use.

[iambumblehead]

I google-searched this subject again this morning and found this package which uses the new node.js module loader. I haven't tried it yet, but the source code consists of a few amazingly small files

https://www.npmjs.com/package/quibble

[theKashey]

Yep, here are a few details on how it works - https://dev.to/giltayar/mock-all-you-want-supporting-es-modules-in-the-testdouble-js-mocking-library-3gh1

And I don't want to implement the same hacky and smelly solutions for rewiremock.

My opinion is not changing for the last years - mjs/esm are not ready to be used and should be avoided. Especially for tests, which never were considered as an important use case for ESM (immutable cache)

[iambumblehead]

@theKashey thanks for your response. I agree with you and am not a fan of the ESM format :/

[theKashey]

Ideal solution here would be developing a "module cache" polyfill, probably based on quibble approach, to abstract the missing API and let other solutions to integrate to not yet friendly ecosystem with less friction.