• Feature Name: vendor_intrinsics
• Start Date: 2018-02-04
• RFC PR: rust-lang/rfcs#2325
• Rust Issue: rust-lang/rust#48556

Summary

The purpose of this RFC is to provide a framework for SIMD to be used on stable Rust. It proposes stabilizing x86-specific vendor intrinsics, but includes the scaffolding for other platforms as well as a future portable SIMD design

Motivation

Stable Rust today does not typically

The goal of this RFC is to enable using SIMD intrinsics on stable Rust, and in general

Note that this is certainly not the first discussion to broach the topic of SIMD in Rust, but rather this has been an ongoing discussion for quite some time now! For example the simd crate started long ago, we've had rfcs, we've had a lot of discussions on internals, and the stdsimd crate has been implemented.

This RFC draws from much of the historical feedback and design

Guide-level explanation

Let's say you've just heard about this fancy feature called

#![allow(unused)]
fn main() {
pub fn foo(a: &[u8], b: &[u8], c: &mut [u8]) {
    for ((a, b), c) in a.iter().zip(b).zip(c) {
        *c = *a + *b;
    }
}
}

When inspecting the assembly you notice that rustc is making use of the %xmmN registers which you've read is related to SSE on your CPU. You know, however, that your CPU supports up to AVX2 which has bigger registers, so you'd like to get access to them!

Your first solution to this problem is to compile with -C target-feature=+avx2, and after that you see the %ymmN registers being used, yay! Unfortunately though you're publishing this binary

#![allow(unused)]
fn main() {
#[target_feature(enable = "avx2")]
pub unsafe fn foo(a: &[u8], b: &[u8], c: &mut [u8]) {
    for ((a, b), c) in a.iter().zip(b).zip(c) {
        *c = *a + *b;
    }
}
}

And sure enough you see the %ymmN registers getting used in this function! Note, however, that because you've explicitly

#![allow(unused)]
fn main() {
pub fn foo(a: &[u8], b: &[u8], c: &mut [u8]) {
    // Note that this `unsafe` block is safe because we're testing
    // that the `avx2` feature is indeed available on our CPU.
    if is_target_feature_detected!("avx2") {
        unsafe { foo_avx2(a, b, c) }
    } else {
        foo_fallback(a, b, c)
    }
}

#[target_feature(enable = "avx2")]
unsafe fn foo_avx2(a: &[u8], b: &[u8], c: &mut [u8]) {
    foo_fallback(a, b, c) // the function below is inlined here
}

fn foo_fallback(a: &[u8], b: &[u8], c: &mut [u8]) {
    for ((a, b), c) in a.iter().zip(b).zip(c) {
        *c = *a + *b;
    }
}
}

And sure enough once again we see that foo is dispatching at runtime to the appropriate function, and only foo_avx2 is using our %ymmN registers!

Ok great! At this point we've seen how to enable CPU features for functions-at-a-time as well as how they could be used in a larger context

For explicit

For example let's say that we're writing a function that encodes

First up you'll see the dispatch routine like we wrote above:

#![allow(unused)]
fn main() {
fn hex_encode<'a>(src: &[u8], dst: &'a mut [u8]) -> Result<&'a str, usize> {
    let len = src.len().checked_mul(2).unwrap();
    if dst.len() < len {
        return Err(len);
    }

    #[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
    {
        if is_target_feature_detected!("avx2") {
            return unsafe { hex_encode_avx2(src, dst) };
        }
        if is_target_feature_detected!("sse4.1") {
            return unsafe { hex_encode_sse41(src, dst) };
        }
    }

    hex_encode_fallback(src, dst)
}
}

Here we have some routine business about hex encoding in general,

Taking

#![allow(unused)]
fn main() {
let ascii_zero = _mm_set1_epi8(b'0' as i8);
let nines = _mm_set1_epi8(9);
let ascii_a = _mm_set1_epi8((b'a' - 9 - 1) as i8);
let and4bits = _mm_set1_epi8(0xf);
}

As it turns out though, these are all Intel SIMD intrinsics! For example _mm_set1_epi8 is defined

These functions are all imported through std::arch::* at the top of the example (in this case stdsimd::vendor::*). We go on to use a bunch of these intrinsics throughout the hex_encode_sse41 function to actually do the hex encoding.

The example listed

test benches::large_default    ... bench:      73,432 ns/iter (+/- 12,526) = 14279 MB/s
test benches::large_fallback   ... bench:   1,711,030 ns/iter (+/- 286,642) = 612 MB/s
test benches::small_default    ... bench:          30 ns/iter (+/- 18) = 3900 MB/s
test benches::small_fallback   ... bench:         204 ns/iter (+/- 74) = 573 MB/s
test benches::x86::large_avx2  ... bench:      69,742 ns/iter (+/- 9,157) = 15035 MB/s
test benches::x86::large_sse41 ... bench:     108,463 ns/iter (+/- 70,250) = 9667 MB/s
test benches::x86::small_avx2  ... bench:          25 ns/iter (+/- 8) = 4680 MB/s
test benches::x86::small_sse41 ... bench:          25 ns/iter (+/- 14) = 4680 MB/s

Or in other words, our runtime dispatch implementation

With std::arch and is_target_feature_detected! we've now written a program that's 20x faster on supported hardware, yet it also continues to run on older hardware as well! Not bad for a few dozen lines on each function!

Note that this RFC is explicitly

Furthermore LLVM does quite a good job with a portable u32x4 type, for example, in terms

• The intrinsics will not take
  とる
  portable types as arguments.
  引数
  For example u32x4 and __m128i will be different types on x86. The two types, however, will be convertible between one another (either via transmutes or via explicit
  明示的な
  functions). This conversion
  変換
  will have zero run-time
  実行時の(に)
  cost.
• The portable simd types will likely live in a module like std::simd rather than std::arch.

The design

Reference-level explanation

Stable SIMD in Rust ends up requiring a surprising number of both language

The #[target_feature] Attribute

The #[target_feature] attribute was specified

#![allow(unused)]
fn main() {
#[target_feature(enable = "avx2")]
}

The only currently allowed

There's a good number of these features supported by the compiler today. It's expected that when stabilizing other pieces of this RFC the names of the following

• aes
• avx2
• avx
• bmi2
• bmi - to be renamed to bmi1, the name Intel gives it
• fma
• fxsr
• lzcnt
• popcnt
• rdrnd
• rdseed
• sse2
• sse3
• sse4.1
• sse4.2
• sse
• ssse3
• xsave
• xsavec
• xsaveopt
• xsaves

Note that AVX-512 names are missing from this list,

The target_feature value in #[cfg]

In addition

#![allow(unused)]
fn main() {
#[cfg(target_feature = "avx")]
fn foo() {
    // implementation that can use `avx`
}

#[cfg(not(target_feature = "avx"))]
fn foo() {
    // a fallback implementation
}
}

Additionally this is also made available to cfg!:

#![allow(unused)]
fn main() {
if cfg!(target_feature = "avx") {
    println!("this program was compiled with AVX support");
}
}

The #[cfg] attribute and cfg! macro statically resolve

The is_target_feature_detected! Macro

One mode of operation

The crux of this support in libstd is this macro provided

For example on x86 you could write:

#![allow(unused)]
fn main() {
if is_target_feature_detected!("sse4.1") {
    println!("this cpu has sse4.1 features enabled!");
}
}

It would, however, be an error to write this on x86 cpus:

#![allow(unused)]
fn main() {
is_target_feature_detected!("neon"); //~ COMPILE ERROR: neon is an ARM feature, not x86
is_target_feature_detected!("foo"); //~ COMPILE ERROR: unknown target feature for x86
}

The macro is intended to be implemented

• The first time the macro is invoked
  呼び出す
  all the local CPU features will be detected.
• The detected features will then be cached globally (when possible and currently in a bitset) for the rest of the execution
  実行
  of the program.
• Further
  さらなる、それ以上
  invocations
  呼び出し
  of is_target_feature_detected! are expected to be cheap runtime dispatches. (aka load a value and check whether a bit is set)
• Exception:
  例外
  in some cases the result
  結果、戻り値
  of the macro is statically known: for example, is_target_feature_detected!("sse2") when the binary
  ２進数
  is being compiled with "sse42" globally. In these cases, none of the steps above are performed and the macro just expands to true.

The exact method of CPU feature detection various

Note that the implementation

The std::arch Module

This is where the real meat is. A new module will be added

The contents of the arch modules are defined

For example most Intel intrinsics start with _mm_ or _mm256_ for 128 and 256-bit registers. While perhaps unergonomic, we'll be sticking to what Intel says. Note that all intrinsics will also be unsafe, according

Function signatures

The current proposed mapping for x86 intrinsics is:

[0] required to be compile-time constants.

Other than these exceptions

• __m128
• __m128d
• __m128i
• __m256
• __m256d
• __m256i

(note that AVX-512 types will come in the future!)

Infrastructure-wise the contents of std::arch are expected to continue to be defined

Currently today on x86 and ARM platforms the stdsimd crate performs all these checks, but these checks are not yet implemented

It's not expected that the contents of std::arch will remain static

The types in std::arch

It's worth paying close attention to the types in std::arch. Types like __m128i are intended to represent

Instead it will be required that the types defined

Implementation-wise these packed SIMD types are implemented

The Rust ABI will currently be implemented

Again though, note that this section

Intrinsics in std::arch and constant

定数

arguments

引数

There are a number of intrinsics on x86 (and other) platforms that require their arguments

Eventually we will likely have some form

It's hoped that this restriction

Portable packed SIMD

So-called "portable" packed SIMD types are currently implemented

For example u8x16::add may be implemented

It's intended that this RFC neither includes nor rules out the addition

• The portable SIMD types (both packed and scalable) will not be used in intrinsics.
• The per-architecture SIMD types will be distinct
  区別された/独立した
  types from the portable SIMD types.

Or, in other words, it's intended that portable SIMD types are entirely decoupled from intrinsics. If they both end up being implemented

Not stabilizing MMX in this RFC

This RFC proposed notably omitting

• MMX intrinsics aren't always desirable
• LLVM codegen errors happen with debuginfo enabled and MMX
• LLVM codegen errors with MMX types and i586

Due to these issues having an unclear conclusion as well as a seeming lack of desire to stabilize MMX intrinsics, the __m64 and all related intrinsics will not be stabilized via this RFC.

Drawbacks

This RFC represents

Due to the enormity of what's being added

Rationale and alternatives

代わりのもの、選択肢

Over the years quite a few iterations

Portable types in architecture interfaces

It was initially attempted in the stdsimd crate that we would use the portable types on all of the intrinsics. For example instead of:

#![allow(unused)]
fn main() {
pub unsafe fn _mm_set1_epi8(val: i8) -> __m128i;
}

we would instead define

#![allow(unused)]
fn main() {
pub unsafe fn _mm_set1_epi8(val: i8) -> i8x16;
}

The latter definition

The downside of this approach, however, is that Intel isn't telling us what to do. While that may sound simple, this RFC is proposing an addition

Furthermore not all intrinsics from Intel actually have an interpretation

Consequently

There is interest by both current stdsimd maintainers and users to expose a "better-typed" SIMD API in crates.io that builds on top of the intrinsics proposed for stabilization here.

Stabilizing SIMD implementation

実装

details

Another alternative

This avenue was decided against, however, for a few reasons:

• Such raw
  生の
  interfaces may change over time as they simply represent
  表現する
  LLVM as a current point in time rather than what LLVM wants to do in the future.
• Alternate implementations
  実装
  of rustc or alternate rustc backends like Cranelift may not expose the same sort of functionality that LLVM provides,
  与える
  or implementing
  実装する
  the interfaces may be much more difficult in alternate backends than in LLVM's.

As a result,

Unresolved questions

There's a number of unresolved questions around stabilizing SIMD today which don't pose serious blockers and may also wish to be considered

Relying on unexported LLVM APIs

The static

Unfortunately though LLVM does not actually export this information for us to consume (as far as we know). As a result

It appears

Packed SIMD types in extern functions are not sound

The packed SIMD types have particular care paid to them with respect to their ABI in Rust and how they're passed between functions, notably to ensure

A consequence

What if we're wrong?

Despite the CI infrastructure of the stdsimd crate it seems inevitable that we'll get an intrinsic wrong at some point. What do we do in a situation like that? This situation is somewhat analagous to the libc crate but there you can fix the problem downstream (just have a corrected type/definition) for vendor intrinsics it's not so easy.

Currently it seems that our only recourse would be to add a 2 suffix to the function name or otherwise