Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for stable SIMD in Rust #48556

Closed
3 tasks
alexcrichton opened this issue Feb 26, 2018 · 98 comments
Closed
3 tasks

Tracking issue for stable SIMD in Rust #48556

alexcrichton opened this issue Feb 26, 2018 · 98 comments
Labels
B-RFC-approved Blocker: Approved by a merged RFC but not yet implemented. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@alexcrichton
Copy link
Member

alexcrichton commented Feb 26, 2018

This is a tracking issue for RFC 2325, adding SIMD support to stable Rust. There's a number of components here, including:

The initial implementation of this is being added in #48513 and the next steps would be:


Known issues

@alexcrichton alexcrichton added B-RFC-approved Blocker: Approved by a merged RFC but not yet implemented. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. labels Feb 26, 2018
@scottmcm
Copy link
Member

My one request for the bikeshed (which the current PR already does and may be obvious, but I'll write it down anyway): Please ensure they're not all in the same module as things like undefined_behaviour and [un]likely, so that those rust-defined things don't get lost in the sea of vendor intrinsics.

@cuviper
Copy link
Member

cuviper commented Feb 26, 2018

What will be the story for external LLVM? (lacking MCSubtargetInfo::getFeatureTable())

@alexcrichton
Copy link
Member Author

@scottmcm certainly! I'd imagine that if we ever stabilized Rust-related intrinsics they'd not go into the same module (they probably wouldn't even be platform-specific).

@cuviper currently it's an unresolved question, so if it doesn't get fixed it means that using an external LLVM would basically mean that #[cfg(target_feature = ...)] would always expand to false (or the equivalent thereof)

@hanna-kruppe
Copy link
Contributor

hanna-kruppe commented Feb 26, 2018

I'd imagine that if we ever stabilized Rust-related intrinsics they'd not go into the same module (they probably wouldn't even be platform-specific).

One option raised in the RFC thread (that I personally quite like) was stabilizing std::intrinsics (only the module), keep the stable rust intrinsics in that module (they can already be imported from that location due to a long-standing bug in stability checking) and put these new platform-specific intrinsics in submodules. IIUC this would also satisfy @scottmcm's request.

To be explicit, under that plan the rustdoc page for std::intrinsics would look like this:


Modules

  • x86_64
  • arm
  • ...

Functions

  • copy
  • copy_nonoverlapping
  • drop_in_place
  • ...

@alexcrichton
Copy link
Member Author

Another naming idea I've just had. Right now the feature detection macro is is_target_feature_enabled!, but since it's so target specific it may be more apt to call it is_x86_target_feature_enabled!. This'll make it a pain to call on x86/x86_64 though which could be a bummer.

@nox
Copy link
Contributor

nox commented Mar 5, 2018

Why keep all the leading underscores for the intrinsics? Surely even if we keep the same names as what the vendors chose, we can still remove those signs, right?

@BurntSushi
Copy link
Member

The point is to expose vendor APIs. The vendor APIs have underscores. Therefore, ours do too.

@nox
Copy link
Contributor

nox commented Mar 5, 2018

It is debatable that those underscores are actually part of the name. They only have one because C has no modules and namespacing, AFAICT.

@nox
Copy link
Contributor

nox commented Mar 5, 2018

I would be happy dropping the topic if it was discussed at length already, but I couldn't find any discussion specific to them leading underscores.

@BurntSushi
Copy link
Member

@nox rust-lang/stdarch#212 --- My comment above is basically a summary of that. I probably won't say much else on the topic.

@Centril
Copy link
Contributor

Centril commented Mar 5, 2018

@nox, @BurntSushi Continuing the discussion from there... since it hasn't been mentioned before:

Leading _ for identifiers in rust often means "this is not important" - so just taking the names directly from the vendors may wrongly give this impression.

@alexcrichton
Copy link
Member Author

@nox @Centril the recurring theme of stabilizing SIMD in Rust is "it's not our job to make this nice". Any attempt made to make SIMD different than what the vendors define has ended with uncomfortable questions and intrinsics that end up being left out. To that end the driving force for SIMD intrinsics in Rust is to get anything compiling on stable.

Crates like faster are explicitly targeted at making SIMD usage easy, fast, and ergonomic. The standard library's intrinsics are not intended to be widely used nor use for "intro level" problems. Leveraging the SIMD intrinsics is quite unsafe (due to target feature detection/selection) and can come at a high cost if used incorrectly.

Overall, again, the goal is to not enable ergonomic SIMD in Rust right now, but any SIMD in Rust. Following exactly what the vendors say is the easiest way for us to guarantee that all Rust programs will always have access to vendor intrinsics.

@hanna-kruppe
Copy link
Contributor

I agree that the leading underscores are a C artifact, not a vendor choice (the C standard reserves identifiers of this form, so that's what C compilers use for intrinsics). Removing them is neither "trying to make it nicer/more ergonomic" (it's really only a minor aesthetic difference) nor involves any per-intrinsic judgement calls. It's a dead simple mechanical translation for a difference in language rules, almost as much as __m128 _mm_foo(); is mechanically translated to fn _mm_foo() -> __m128;.

@alexcrichton
Copy link
Member Author

@rkruppe do we have a rock solid guarantee that no vendor will ever in the future add the same name without underscores?

@Centril
Copy link
Contributor

Centril commented Mar 5, 2018

@alexcrichton

@rkruppe do we have a rock solid guarantee that no vendor will ever in the future add the same name without underscores?

Can't speak for CPU vendors, but the probability seems very very low. Why would they add an intrinsic where the difference is only an underscore..? Further, as Rust's influence grows, they might not do this simply because of Rust.

@hanna-kruppe
Copy link
Contributor

A name like mm_foo (no leading underscore at all) is not reserved in the C language, so it can't be used for compiler-supplied extensions without breaking legal C programs. There are a few theoretical possibilities for a vendor to nevertheless create intrinsics without leading underscores:

  • they could expose it only in C++ (with namespacing) -- or, for that matter, another language that isn't C
  • they could break legal C programs (very unlikely, and I'll eat my hat if GCC or Clang developers accept this)
  • A future version of C adds some way of doing namespacing, and people start using it for intrinsics

All extremely unlikely. The first one seems like the only one that doesn't sound like science fiction to me, and if that happens we'd have other problems anyway (such intrinsics may use function overloading and other features Rust doesn't have).

@alexreg
Copy link
Contributor

alexreg commented Mar 5, 2018

It is debatable that those underscores are actually part of the name. They only have one because C has no modules and namespacing, AFAICT.

This. The whole point is that the underscore-leading names were chosen so as to specifically not clash with user-defined functions. Which means they should never be using non-underscore names. It's against well-established C conventions. Hence, we should just rename them to follow Rust conventions, with no real chance there will be any name clash in the future, providing the vendors stay sane and respect C conventions.

@alexcrichton
Copy link
Member Author

@Centril "probability seems very very low" is what I would say as well, but we're talking about stability of functions in the standard library, so "low probability" won't cut it unfortunately.

@rkruppe I definitely agree, yeah, but "extremely unlikely" to me says "follow the vendor spec to the letter and we can figure out ergonomics later".

@alexcrichton
Copy link
Member Author

Another point worth mentioning for staying exactly to the upstream spec is that I believe it actually boosts learnability. You'll have instant familiarity with any SIMD/intrinsic code written in C, of which there's already quite a lot!

If we stray from the beaten path then we'll have to have a section of the documentation which is very clear about defining the mappings between intrinsic names and what we actually expose in Rust.

@pythoneer
Copy link
Contributor

I don't think renaming (no leading underscore or any other alteration) is useful. This is simply not the goal and only introduces pain points. I cannot think of a reason other than "i like that more" to justify that. It only introduces the possibility to naming clashes and "very very unlikely" is not convincing because we can prevent this 100% by not doing it altogether.

I think its the best choice to follow the vendor naming schema as close as possible and i think we should even break compatibility if we ever introduce an error in the "public API" without doing some renaming like _mm_intr_a to _mm_intr_a2 and start diverging the exact naming schema introduced by the vendor.

@nox
Copy link
Contributor

nox commented Mar 5, 2018

@alexcrichton But as @rkruppe said, removing the leading underscore isn't about ergonomics, it's about not porting C defects to Rust blindly.

@nox
Copy link
Contributor

nox commented Mar 5, 2018

Sorry for the double post, but I also want to add that arguing that a vendor may release an unprefixed intrinsic with the same name as a prefixed one is to me as hypothetical as arguing that bool may not be a single byte on some platform we would like to support.

@pythoneer
Copy link
Contributor

@nox but why stop by the _? We could also fully rename the function with ps and pd into f32 and f64 which would be something "more Rust". Its somewhat arbitrary to just remove the leading underscore. And we could argue back and forth what is ergonomics and what isn't but i don't think there is a very good line to distinguish that to a point every body agrees.

@nox
Copy link
Contributor

nox commented Mar 5, 2018

@pythoneer Because the name is what the vendor decided, with a leading underscore because of nondescript limitations of C.

@pythoneer
Copy link
Contributor

@nox and the explicit goal of stdsimd is to expose this (however defect) vendor defined interface.

@alexreg
Copy link
Contributor

alexreg commented Mar 5, 2018

@nox and the explicit goal of stdsimd is to expose this (however defect) vendor defined interface.

Interface, sure, but not necessarily the naming conventions!

@chorman0773
Copy link
Contributor

Can we get the AVX-512 intrinsics stabilized please?

I specifically need _mm_rolv_epi64 to avoid confusing codegen from llvm which caused me 4 hours of headaches trying to figure out, while folding down a shift+shift+or into a proper rotate.

@jplatte
Copy link
Contributor

jplatte commented Oct 4, 2023

Not sure where else to ask: How would I find out about the state of (non-simd) WASM intrinsics? I'm specifically wondering whether the three memory_atomic_ fn's (two of them being required for parking_lot_core's thread parking) could be stabilized soon-ish.

garbear added a commit to garbear/xbmc that referenced this issue Dec 31, 2023
Error was:

  error[E0658]: use of unstable library feature 'stdsimd'
     --> operations.rs:124:24
      |
  124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmut...
      |                        ^^^^^^^^^^
      |
      = note: see issue #48556 <rust-lang/rust#48556> for more information
garbear added a commit to garbear/xbmc that referenced this issue Dec 31, 2023
Error was:

  error[E0658]: use of unstable library feature 'stdsimd'
     --> operations.rs:124:24
      |
  124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmut...
      |                        ^^^^^^^^^^
      |
      = note: see issue #48556 <rust-lang/rust#48556> for more information
github-merge-queue bot pushed a commit to privacy-scaling-explorations/zkevm-circuits that referenced this issue Jan 12, 2024
### Description

Had following built error on my Mac (MacOS 12.6.7, M1 Pro)
```
error[E0658]: use of unstable library feature 'stdsimd'
   --> /Users/kimi/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.8.7/src/operations.rs:124:24
    |
124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
    |                        ^^^^^^^^^^
    |
    = note: see issue #48556 <rust-lang/rust#48556> for more information
    = help: add `#![feature(stdsimd)]` to the crate attributes to enable

...
```

Had following error if only to downgrade `ahash` to `0.8.6`. 

```
error[E0432]: unresolved import `revm_precompile`
 --> bus-mapping/src/precompile.rs:7:5
  |
7 | use revm_precompile::{Precompile, PrecompileError, Precompiles};
  |     ^^^^^^^^^^^^^^^ use of undeclared crate or module `revm_precompile`
  |
```

Fixed by upgrade `revm_precompile` to `2.2.0`.

After upgrade `revm_precompile` to `2.2.0`, CI complained library
`libclang.so` missing. That's bcs `revm_precompile` `2.2.0` having
`c-kzg` feature which needs `libclang`. So in this fix, we also disable
default feature of `revm-precompile` (`default-features=false`).
### Issue Link

N/A

### Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] This change requires a documentation update

### Contents

Pinning a fixed version is a better practice so this PR is not only
update the versions mentioned above but also using `=` to pin the
version.

---------

Co-authored-by: adria0.eth <[email protected]>
fuzzard pushed a commit to fuzzard/xbmc that referenced this issue Jan 21, 2024
cargo-c fails to build on osx aarch64 host with the following failures

  error[E0658]: use of unstable library feature 'stdsimd'
     --> /Users/Shared/xbmc-depends/arm-darwin23.1.0-native/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.8.7/src/operations.rs:124:24
      |
  124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
      |                        ^^^^^^^^^^
      |
      = note: see issue #48556 <rust-lang/rust#48556> for more information

  error[E0658]: use of unstable library feature 'stdsimd'
     --> /Users/Shared/xbmc-depends/arm-darwin23.1.0-native/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.8.7/src/operations.rs:124:35
      |
  124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
      |                                   ^^^^^^^^^
      |
      = note: see issue #48556 <rust-lang/rust#48556> for more information

  error[E0658]: use of unstable library feature 'stdsimd'
     --> /Users/Shared/xbmc-depends/arm-darwin23.1.0-native/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.8.7/src/operations.rs:154:24
      |
  154 |     let res = unsafe { vaesimcq_u8(vaesdq_u8(transmute!(value), transmute!(0u128))) };
      |                        ^^^^^^^^^^^
      |
      = note: see issue #48556 <rust-lang/rust#48556> for more information

  error[E0658]: use of unstable library feature 'stdsimd'
     --> /Users/Shared/xbmc-depends/arm-darwin23.1.0-native/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ahash-0.8.7/src/operations.rs:154:36
      |
  154 |     let res = unsafe { vaesimcq_u8(vaesdq_u8(transmute!(value), transmute!(0u128))) };
      |                                    ^^^^^^^^^
      |
      = note: see issue #48556 <rust-lang/rust#48556> for more information

     Compiling hkdf v0.12.4
  For more information about this error, try `rustc --explain E0658`.
  error: could not compile `ahash` (lib) due to 4 previous errors

Bump to 1.75.0 resolves this.
@Ciel-MC
Copy link

Ciel-MC commented Mar 10, 2024

Some builds have been failing on nightly because of this, and just to check if I'm understanding everything correctly, this has been stabilized on nightly, is that correct? Which is why the feature attribute fails.

@Amanieu
Copy link
Member

Amanieu commented Mar 10, 2024

The stdsimd feature has been split into sub-features: #27731 (comment)

Here is the full set of new tracking issues for what stdsimd was previous tracking:

I'm going to close this issue since it isn't tracking anything anymore.

@Amanieu Amanieu closed this as completed Mar 10, 2024
TomTaehoonKim added a commit to kroma-network/tachyon that referenced this issue Mar 12, 2024
In 88e4660, cargo lock files were updated wrong, causing ci failure with
below error message.

ERROR: /private/var/tmp/_bazel_takim9999/3a7efe9626017559f78ec48c055ca385/external/crate_index__ahash-0.8.8/BUILD.bazel:22:13: Compiling Rust rlib ahash v0.8.8 (14 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__ahash-0.8.8//:ahash) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file bazel-out/darwin_arm64-dbg/bin/external/crate_index__ahash-0.8.8/ahash_build_script.env ... (remaining 33 arguments skipped)
error[E0658]: use of unstable library feature 'stdsimd'
   --> external/crate_index__ahash-0.8.8/src/operations.rs:124:24
    |
124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
    |                        ^^^^^^^^^^
    |
    = note: see issue #48556 <rust-lang/rust#48556> for more information
TomTaehoonKim added a commit to kroma-network/tachyon that referenced this issue Mar 12, 2024
In 88e4660, cargo lock files were updated wrong, causing ci failure with
the error message below.

ERROR: /private/var/tmp/_bazel_takim9999/3a7efe9626017559f78ec48c055ca385/external/crate_index__ahash-0.8.8/BUILD.bazel:22:13: Compiling Rust rlib ahash v0.8.8 (14 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__ahash-0.8.8//:ahash) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file bazel-out/darwin_arm64-dbg/bin/external/crate_index__ahash-0.8.8/ahash_build_script.env ... (remaining 33 arguments skipped)
error[E0658]: use of unstable library feature 'stdsimd'
   --> external/crate_index__ahash-0.8.8/src/operations.rs:124:24
    |
124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
    |                        ^^^^^^^^^^
    |
    = note: see issue #48556 <rust-lang/rust#48556> for more information
dongchangYoo pushed a commit to kroma-network/tachyon that referenced this issue Mar 12, 2024
In 88e4660, cargo lock files were updated wrong, causing ci failure with
the error message below.

ERROR: /private/var/tmp/_bazel_takim9999/3a7efe9626017559f78ec48c055ca385/external/crate_index__ahash-0.8.8/BUILD.bazel:22:13: Compiling Rust rlib ahash v0.8.8 (14 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__ahash-0.8.8//:ahash) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file bazel-out/darwin_arm64-dbg/bin/external/crate_index__ahash-0.8.8/ahash_build_script.env ... (remaining 33 arguments skipped)
error[E0658]: use of unstable library feature 'stdsimd'
   --> external/crate_index__ahash-0.8.8/src/operations.rs:124:24
    |
124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
    |                        ^^^^^^^^^^
    |
    = note: see issue #48556 <rust-lang/rust#48556> for more information
dongchangYoo pushed a commit to kroma-network/tachyon that referenced this issue Mar 12, 2024
In 88e4660, cargo lock files were updated wrong, causing ci failure with
the error message below.

ERROR: /private/var/tmp/_bazel_takim9999/3a7efe9626017559f78ec48c055ca385/external/crate_index__ahash-0.8.8/BUILD.bazel:22:13: Compiling Rust rlib ahash v0.8.8 (14 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__ahash-0.8.8//:ahash) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --env-file bazel-out/darwin_arm64-dbg/bin/external/crate_index__ahash-0.8.8/ahash_build_script.env ... (remaining 33 arguments skipped)
error[E0658]: use of unstable library feature 'stdsimd'
   --> external/crate_index__ahash-0.8.8/src/operations.rs:124:24
    |
124 |     let res = unsafe { vaesmcq_u8(vaeseq_u8(transmute!(value), transmute!(0u128))) };
    |                        ^^^^^^^^^^
    |
    = note: see issue #48556 <rust-lang/rust#48556> for more information
CjS77 added a commit to CjS77/tari_payment_server that referenced this issue May 29, 2024
When providing a Tari address in a Shopify order, we cannot let users provide just
any wallet address there, because this would let folks put other wallet addresses in
there and hope that one day someone makes a payment from that wallet and their order will
then be fulfilled.

This module provides the method and standard format for providing the
wallet address and signature into the memo field f an order that proves
the buyer owns the wallet.

* Updating nightly version

Some updated dependencies were failing on nightly (dalek!)
Updated nightly version to one where stdsimd has been stabilised seems
to fix this:
rust-lang/rust#48556

feat: memo signature utility

Add a utility to generate the order memo signature object.

Run it like this

```
memo_signature <address> <order_id> <secret_key>
```

The result is an object like
```json
{
  "address":"b8971598a865b25b6508d4ba154db228e044f367bd9a1ef50dd4051db42b63143d",
  "order_id":"alice001",
  "signature":"56e39d539f1865742b41993bdc771a2d0c16b35c83c57ca6173f8c1ced34140aeaf32bfdc0629e73f971344e7e45584cbbb778dc98564d0ec5c419e6f9ff5d06"
}
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B-RFC-approved Blocker: Approved by a merged RFC but not yet implemented. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests