integration.md

Framework/Library Integration

• NumPy
• SciPy
• pandas
• PyTorch
• TensorFlow
• MXNet
• DLPack

NumPy

as_numpy

Wraps the specified DenseTensor as a NumPy array. This is a zero-copy operation; at the end both the tensor and the NumPy array will share the same data.

mlio.integ.numpy.as_numpy(tensor : DenseTensor)

• tensor: A DenseTensor instance.

to_numpy

Copies the specified DenseTensor as a NumPy array.

mlio.integ.numpy.to_numpy(tensor : DenseTensor)

• tensor: A DenseTensor instance.

as_tensor

Wraps the specified NumPy array as a DenseTensor. This is a zero-copy operation; at the end both the tensor and the NumPy array will share the same data.

mlio.integ.numpy.as_tensor(arr : ndarray)

• arr: A NumPy array.

to_tensor

Copies the specified NumPy array as a DenseTensor.

mlio.integ.numpy.to_tensor(arr : ndarray)

• arr: A NumPy array.

SciPy

to_coo_matrix

Copies the specified CooTensor as a SciPy coo_matrix. Note that coo_matrix supports only up to two dimensions. If your CooTensor instance has more than two dimensions, the function will raise an error.

mlio.integ.scipy.to_coo_matrix(tensor : CooTensor)

• tensor: A CooTensor instance.

to_tensor

Copies the specified SciPy coo_matrix as a CooTensor.

mlio.integ.scipy.to_tensor(mtx : coo_matrix)

• mtx: A coo_matrix instance.

pandas

to_pandas

Copies the specified Example to a pandas DataFrame.

mlio.integ.pandas.to_pandas(exm : Example)

• exm: The Example instance.

PyTorch

as_torch

Wraps the specified DenseTensor as a Torch tensor. This is a zero-copy operation; at the end both tensors will share the same data.

mlio.integ.torch.as_torch(tensor : DenseTensor)

• tensor: A DenseTensor instance.

TensorFlow

to_tf

Copies the specified Tensor to a dense or sparse TensorFlow tensor.

mlio.integ.tensorflow.to_tf(tensor : Tensor)

• tensor: A Tensor instance.

make_tf_dataset

Constructs a TensorFlow dataset from the specified DataReader instance.

mlio.integ.tensorflow.make_tf_dataset(data_reader : DataReader,
                                      features : Sequence[str],
                                      dtypes : Sequece[tf.DType])

• data_reader: The DataReader instance to wrap.
• features: The list of feature names.
• dtypes: The data types of features.

MXNet

as_mxnet

Wraps the specified DenseTensor as an MXNet array. This is a zero-copy operation; at the end both the tensor and the MXNet array will share the same data.

mlio.integ.mxnet.as_mxnet(tensor : DenseTensor)

• tensor: A DenseTensor instance.

DLPack

as_dlpack

Wraps the specified DenseTensor as a DLPack tensor returned in a Python capsule. This is a zero-copy operation; at the end both tensors will share the same data.

mlio.integ.dlpack.as_dlpack(tensor : DenseTensor, version : int = 0x10)

• tensor: A DenseTensor instance.
• version: The DLPack specification version that the tensor should be compatible with.