KokkosBlas: Add GEMV that wraps BLAS (and cuBLAS)

[mhoemmen]

Blocks: trilinos/Trilinos#1169

KokkosBlas::gemv currently exists, but it does not currently call the BLAS library or cuBLAS where appropriate.

There is a subtlety in whether Tpetra uses this GEMV as a matrix-vector product, or as the dot product of each column of a MultiVector with a single Vector. The difference is that matrix-vector products do all computations with Scalar values (actually Tpetra::MultiVector::impl_scalar_type), while the intermediate sums in a dot product have type Tpetra::MultiVector::dot_type. The two types are usually the same, except for Scalar types that come from Stokhos. We have to decide whether we want GEMV to support both cases, or just one of them.

In practice, BLAS implementations only support types for which impl_scalar_type == dot_type. Thus, this is really about the interface that we present to users.

[crtrott]

OK we need to fix this with respect to BLAS and CuBLAS. The seconds thing is another issue.

[crtrott]

So what does this need to do for Stokhos?

[mhoemmen]

@crtrott When you say "the second thing," do you mean the issue of whether impl_scalar_type == dot_type? That's something specific to Tpetra; I wouldn't expect kokkos-kernels to have to deal with that. Tpetra would prefer to defer to kokkos-kernels in the common case where impl_scalar_type == dot_type, though. Thanks!

So what does this need to do for Stokhos?

For some Stokhos Scalar types in Tpetra, impl_scalar_type != dot_type. Stokhos uses double (or whatever the underlying scalar type is) as the dot_type, since it is the result of the dot product of two Stokhos Scalar types.