Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement overrides for package.json #1134

Closed
tsukinoko-kun opened this issue Aug 23, 2022 · 25 comments
Closed

Implement overrides for package.json #1134

tsukinoko-kun opened this issue Aug 23, 2022 · 25 comments
Labels
atw bun install Something that relates to the npm-compatible client enhancement New feature or request

Comments

@tsukinoko-kun
Copy link

What is the problem this feature will solve?

The dependencies of your project might use old versions of their dependencies. I hat the problem, that lmdb was incompatible with my processor (Apple M2). I had to override the lmdb version to get my project running.

What is the feature you are proposing to solve the problem?

npm has a option called overrides in the package.json. This allows to use another version of a dependency you have no control over.

For me, this was:

"overrides": {
  "lmdb": "2.6.0-alpha6",
  "@lmdb/lmdb-darwin-arm64": "2.6.0-alpha6",
  "@lmdb/lmdb-darwin-x64": "2.6.0-alpha6",
},

What alternatives have you considered?

using the good old npm

@tsukinoko-kun tsukinoko-kun added the enhancement New feature or request label Aug 23, 2022
@ElYaiko
Copy link

ElYaiko commented Oct 25, 2022

I'm having the same issue when using overrides with Bun 0.2.1, I see is not supported yet, a temporary solution as you mentioned was using npm i

@Electroid Electroid added the bun install Something that relates to the npm-compatible client label Nov 3, 2022
@Electroid Electroid changed the title overrides Implement overrides for package.json Nov 3, 2022
@bnussman-akamai
Copy link

This would be a great feature for bun! npm/pnpm calls this overrides. Yarn v1 calls it resolutions.

@god-of-javascript
Copy link

+1
This is a must.

@lewisl9029
Copy link

This was one of the blockers for me when I attempted to migrate an app to bun as well. With a large enough app, packages with broken dependencies are pretty inevitable.

Having some way to deal with these either through patching (requested here) and/or overrides is a life saver. Bun is missing both at the moment unfortunately.

@Jarred-Sumner
Copy link
Collaborator

Jarred-Sumner commented Mar 21, 2023

is only one level deep okay?

like

{
  "overrides": {
    "lodash": "npm:lodash-es@latest"
  }
}

and not supporting

{
  "overrides": {
    "react": {
      "react-dom": "16.0.0"
    }
  }
}

@lewisl9029
Copy link

Can't speak for others in this thread, but 1 level would be enough to unblock my use case. Most of the time, I'm using overrides to force dedupe to a single good version, so don't need the deeper levels.

Would be a great place to start if it makes implementation simpler imo. 🙂

@chrisbbreuer
Copy link

chrisbbreuer commented Jul 26, 2023

One level deep would not unblock our use case, which requires nested overrides as well.

Let me add some context: I am in the process of creating Bun compatibility of a few packages that are lacking compatibility, like unconfig, and, by extension, UnoCSS. Even though it seems UnoCSS is working for some users, unconfig is required in UnoCSS which currently ships with and uses jiti, which fails here.

1 level deep now unblocks us too 👍🏽

@Hebilicious
Copy link

Would love to have this as well.
Supporting the "resolutions" alias could be considered too (used by yarn and pnpm).

@ChiefORZ
Copy link

ChiefORZ commented Sep 9, 2023

would make debugging a hell lot easier. this is a must have feature in the node ecosystem.

cross-referencing #4669

@Tehnix
Copy link

Tehnix commented Sep 9, 2023

Currently also blocking us from being able to utilize bun install, we're running into React type mismatch issues such as the ones reported here vercel/next.js#42292 (comment) for which the solution seems to be resolutions/override.

Our current workaround is to install all dependencies via yarn install (since that's what we used before) and then run as normal with bun after that :)

@ild0tt0re
Copy link

Also for us. overrides and patchedDependencies are blocking.

@juliusmarminge
Copy link

juliusmarminge commented Sep 12, 2023

pnpm patch is so nice to have and definetely a blocking feature before adopting bun in some of my repos

@martinmckenna
Copy link

aw man. was hoping to migrate today until I found this issue 😢

eps1lon added a commit to eps1lon/dom-testing-library that referenced this issue Sep 16, 2023
eps1lon added a commit to eps1lon/dom-testing-library that referenced this issue Sep 16, 2023
@siddharth-golchha
Copy link

Is there a workaround for this at the moment? Trying to replace yarn install (v1) with bun install and it fails because "resolutions" is not supported.

@divmgl
Copy link

divmgl commented Sep 23, 2023

Is there a workaround for this at the moment?

There is no workaround. Similar issues tracked here.

#4642
#5022
#4946
#6085

Unfortunately it's one of the biggest blockers for using Bun as a package manager.

@ricklove
Copy link

ricklove commented Oct 3, 2023

@Jarred-Sumner has work on this already started?

I was going to give it a go, but still working on getting make setup to work (wsl2 on windows - that is a different issue and I'll open a ticket if I can't figure it out).

@divmgl
Copy link

divmgl commented Oct 4, 2023

@ricklove I attempted to work through this and unfortunately the Zig codebase is very difficult to work on. I would hold off on this and let @Jarred-Sumner take a look since he's the most experienced with that part of the system, which is one of the older parts of the codebase (ported from a Go repository) as far as I can tell.

@Jarred-Sumner
Copy link
Collaborator

"overrides" and "resolutions" support will be shipped in Bun v1.0.6, which releases tonight. We will only support one level deep overrides, meaning nested overrides won't work. If that blocks someone, we can open a new issue to track nested overrides specifically

@beorn
Copy link

beorn commented Oct 12, 2023

Are the overrides/resolutions monorepo-wide or per project?

@genesiscz
Copy link

genesiscz commented Oct 12, 2023

I just wasted a few hours trying to resolve a typescript conflict where @mui/* components were requiring @types/react at another version that was incompatible (and installing it to node_modules of the package) and it turns out it is because I had the same package listed in "resolutions" and "dependencies". After I moved it to resolutions it shown me another error

 Type '(propsTextField: TextFieldProps) => JSX.Element' is not assignable to type '(props: OutlinedTextFieldProps | FilledTextFieldProps | StandardTextFieldProps) => ReactElement<any, string | JSXElementConstructor<...>>'.

I tried to move the @mui/material package from dependencies to resolutions and it worked!

I followed this tutorial so I had placed the "@mui/styled-engine": "npm:@mui/styled-engine-sc@latest" on both places, which wasn't working until I deleted it from dependencies and bun update after rm -rf node_modules!

Maybe this helps somebody. Maybe it's expected behavior, but it worked with yarn.

@genesiscz
Copy link

genesiscz commented Oct 13, 2023

On another note, when having

  "resolutions": {
    "@mui/styled-engine": "npm:@mui/styled-engine-sc@^5.14.0"
  },

So basically

  1. resolutions sometimes won't work without moving the item from dependencies to resolutions (packages include node_modules with its own packages which are not respecting the resolutions)
  2. after moving the item, I have to remove the whole node_modules folder (or at least @mui, AND @types folder seems to work)
  3. Without moving @mui/material form dependencies to resolutions, it still installs the node_modules with @types/react that is incompatible to my version stated in resolutions already
  4. bun doesn't remove node_modules folder from the package while running bun update

I know it is rather confusing, so here's my terminal log which shows exactly what happened. Comments are in :

➜  paldock-react git:(fix/non-working) ✗ cat package.json | grep @types/react          
    <redacted types, following lines are in resolutions key>
    "@types/react": "^18.0.28",
    "@types/react-dom": "^18.0.13"
➜  paldock-react git:(fix/non-working) ✗ bun update                                    
[0.20ms] ".env.production"
bun update v1.0.6 (969da088)
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @mui/[email protected]
 + @mui/[email protected]
 + @mui/[email protected]
 + @mui/[email protected] (v6.16.2 available)

 170 packages installed [4.28s]
➜  paldock-react git:(fix/non-working) ✗ ls node_modules/@mui/base/node_modules/@types/      
react --- this should not be here already 
➜  paldock-react git:(fix/non-working) ✗ cat package.json | grep @types/react          
  <redacted types, following lines are in resolutions key - changed to static version>
    "@types/react": "18.2.28",
    "@types/react-dom": "18.2.13"
➜  paldock-react git:(fix/non-working) ✗ bun update
[0.20ms] ".env.production"
bun update v1.0.6 (969da088)

Checked 4985 installs across 2460 packages (no changes) [3.56s]
➜  paldock-react git:(fix/non-working) ✗ ls node_modules/@mui/base/node_modules/@types/
react <this should not be here already>
➜  paldock-react git:(fix/non-working) ✗ rm -rf node_modules/@mui/ node_modules/@types/ <this fixes the problem>
➜  paldock-react git:(fix/non-working) ✗ bun update                                    
[0.17ms] ".env.production"
bun update v1.0.6 (969da088)
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @types/[email protected]
 + @mui/[email protected]
 + @mui/[email protected]
 + @mui/[email protected]
 + @mui/[email protected] (v6.16.2 available)

 168 packages installed [3.07s]
➜  paldock-react git:(fix/non-working) ✗ ls node_modules/@mui/base/node_modules/@types/
ls: node_modules/@mui/base/node_modules/@types/: No such file or directory <WORKS!>

Interestingly enough, when i bun bun.lockb, I see this when I have ^18.0.37 in resolutions:

"@types/react@*", "@types/react@>=16.8", "@types/react@^16.8 || ^17.0 || ^18.0", "@types/react@^16.9.0 || ^17.0.0 || ^18.0.0", "@types/react@^17.0.0 || ^18.0.0":
  version "18.2.22"
  resolved "https://registry.npmjs.org/@types/react/-/react-18.2.22.tgz"
  integrity sha512-60fLTOLqzarLED2O3UQImc/lsNRgG0jE/a1mPW9KjMemY0LMITWEsbS4VvZ4p6rorEHd5YKxxmMKSDK505GHpA==
  dependencies:
    "@types/prop-types" "*"
    "@types/scheduler" "*"
    csstype "^3.0.2"

"@types/react@*", "@types/react@>=16":
  version "18.2.28"
  resolved "https://registry.npmjs.org/@types/react/-/react-18.2.28.tgz"
  integrity sha512-ad4aa/RaaJS3hyGz0BGegdnSRXQBkd1CCYDCdNjBPg90UUpLgo+WlJqb9fMYUxtehmzF3PJaTWqRZjko6BRzBg==
  dependencies:
    "@types/prop-types" "*"
    "@types/scheduler" "*"
    csstype "^3.0.2"

When I change it to ^18.2.23 (^18.2.22 doesn't change anything), I get only this in the lock file:

"@types/react@*", "@types/react@>=16.8", "@types/react@^16.8 || ^17.0 || ^18.0", "@types/react@^16.9.0 || ^17.0.0 || ^18.0.0", "@types/react@^17.0.0 || ^18.0.0":
  version "18.2.22"
  resolved "https://registry.npmjs.org/@types/react/-/react-18.2.22.tgz"
  integrity sha512-60fLTOLqzarLED2O3UQImc/lsNRgG0jE/a1mPW9KjMemY0LMITWEsbS4VvZ4p6rorEHd5YKxxmMKSDK505GHpA==
  dependencies:
    "@types/prop-types" "*"
    "@types/scheduler" "*"
    csstype "^3.0.2"

diff

 ➜  paldock-react git:(a5f18265) ✗ diff yarn3.lock yarn4.lock
3c3
< # bun ./bun.lockb --hash: 96F88981FD1285FE-8f4e5bdc57fdaeae-077C884C79C4FE56-7d7746e92ed9105b
---
> # bun ./bun.lockb --hash: F21E403871B567FC-8c349ecf9bbc232a-81F761BE40AAEACC-bbfcc1cf6ce649bb
5862c5862
< "@types/prop-types@*", "@types/prop-types@^15.7.5":
---
> "@types/prop-types@^15.7.5":
5901,5910c5901
< "@types/react@*", "@types/react@>=16.8", "@types/react@^16.8 || ^17.0 || ^18.0", "@types/react@^16.9.0 || ^17.0.0 || ^18.0.0", "@types/react@^17.0.0 || ^18.0.0":
<   version "18.2.22"
<   resolved "https://registry.npmjs.org/@types/react/-/react-18.2.22.tgz"
<   integrity sha512-60fLTOLqzarLED2O3UQImc/lsNRgG0jE/a1mPW9KjMemY0LMITWEsbS4VvZ4p6rorEHd5YKxxmMKSDK505GHpA==
<   dependencies:
<     "@types/prop-types" "*"
<     "@types/scheduler" "*"
<     csstype "^3.0.2"
< 
< "@types/react@*", "@types/react@>=16":
---
> "@types/react@*", "@types/react@>=16", "@types/react@>=16.8", "@types/react@^16.8 || ^17.0 || ^18.0", "@types/react@^16.9.0 || ^17.0.0 || ^18.0.0", "@types/react@^17.0.0 || ^18.0.0":
6065,6069d6055
<   version "0.16.3"
<   resolved "https://registry.npmjs.org/@types/scheduler/-/scheduler-0.16.3.tgz"
<   integrity sha512-5cJ8CB4yAx7BH1oMvdU0Jh9lrEXyPkar6F9G/ERswkCuvP4KQZfZkSjcMbAICCpQTN4OuZn8tz0HiKv9TGZgrQ==
< 
< "@types/scheduler@*":

and it actually is what changes the behavior (including or not including the node_modules inside @mui folders, while I still have to delete the @mui and @types folders before bun upate). After changing the version back to ^18.2.0 it doesn't change the lock file back after bun update.

Also, Bun installed @mui/styled-engine-sc@latest at version @mui/[email protected], which yarn did not. Is this an expected behavior?

@divmgl
Copy link

divmgl commented Oct 13, 2023

I appreciate the work on this.

Is it unreasonable to expect that the package resolution behavior for Bun matches npm? I feel like npm, yarn and pnpm are all the same in this regard, even if they have different lockfile and storage mechanisms. As in, I can run any of their installations and I have the same results within my project.

@rostero1
Copy link

My org requires Veracode to scan packages. Having nested resolutions has saved me a few times. bun why would also be needed to make the switch, since I need to know why a vulnerable package exists.

@Electroid
Copy link
Contributor

Support for overrides was added in Bun v1.0.6, which you can upgrade to by running bun upgrade.

If you need support for nested overrides, follow #6608 for updates.

@beorn
Copy link

beorn commented Oct 21, 2023

I'm running into some issue using URLs. In yarn this would work:

  "resolutions": {
    "slate": "https:/beorn/slate/raw/custom/packages/slate/package.tgz",
  },

but in Bun I'm getting

# bun install --verbose
Request: GET /beorn/slate/raw/custom/packages/slate/package.tgz
	Connection: keep-alive
	User-Agent: Bun/1.0.7
	Accept: */*
	Host: github.com
	Accept-Encoding: gzip, deflate
Response: < 302 Found
< 	Server: GitHub.com
< 	Date: Sat, 21 Oct 2023 03:56:43 GMT
< 	Content-Type: text/html; charset=utf-8
< 	Vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, Accept-Encoding, Accept, X-Requested-With
< 	Access-Control-Allow-Origin: https://render.githubusercontent.com
< 	Location: https://raw.githubusercontent.com/beorn/slate/custom/packages/slate/package.tgz
 	[...]

Request: GET /beorn/slate/custom/packages/slate/package.tgz
	Connection: keep-alive
	User-Agent: Bun/1.0.7
	Accept: */*
	Host: raw.githubusercontent.com
	Accept-Encoding: gzip, deflate
Response: < 200 OK
< 	Connection: keep-alive
< 	Content-Length: 237470
[...]
< 	X-GitHub-Request-Id: 890A:93AB:5E423D:73F53D:65334ADD
[...]
< 	Source-Age: 285

    [425.94ms] Downloaded slate tarball

[slate] Start extracting .863fac336a9a6640slate
 CHANGELOG.md
 Readme.md
 dist/create-editor.d.ts
 package.json
[...]
[slate] Extracted

error: slate@^0.94.1 failed to resolve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
atw bun install Something that relates to the npm-compatible client enhancement New feature or request
Projects
None yet
Development

No branches or pull requests