Documentation

Computes the bitwise NOT of the given input tensor. The input tensor must be of integral or Boolean types. For bool tensors, it computes the logical NOT.

>>> dtype &&& shape $ bitwiseNot (ones :: CPUTensor 'D.Bool [3,3])
(Bool,[3,3])

Computes the element-wise logical NOT of the given input tensor. If not specified, the output tensor will have the bool dtype. If the input tensor is not a bool tensor, zeros are treated as False and non-zeros are treated as True.

sumAll

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Int) $ sumAll (ones :: CPUTensor 'D.Bool '[2, 3])
(Int64,([],6))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Int) $ sumAll (ones :: CPUTensor 'D.UInt8 '[2, 3])
(Int64,([],6))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Int) $ sumAll (ones :: CPUTensor 'D.Int8 '[2, 3])
(Int64,([],6))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Int) $ sumAll (ones :: CPUTensor 'D.Int16 '[2, 3])
(Int64,([],6))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Int) $ sumAll (ones :: CPUTensor 'D.Int32 '[2, 3])
(Int64,([],6))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Int) $ sumAll (ones :: CPUTensor 'D.Int64 '[2, 3])
(Int64,([],6))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Float) $ sumAll (ones :: CPUTensor 'D.Float '[2, 3])
(Float,([],6.0))
>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: Double) $ sumAll (ones :: CPUTensor 'D.Double '[2, 3])
(Double,([],6.0))

sumDim

>>> dtype &&& shape $ sumDim @0 (ones :: CPUTensor 'D.Float '[3,4,5])
(Float,[4,5])
>>> sumDim @1 (ones :: CPUTensor 'D.Float '[2,4])
Tensor Float [2] [ 4.0000   ,  4.0000   ]

abs

>>> dtype &&& shape $ abs (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

ceil

>>> dtype &&& shape $ ceil (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

floor

>>> dtype &&& shape $ floor (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

min

>>> dtype &&& shape $ min (ones :: CPUTensor 'D.Float '[2,2])
(Float,[])

max

>>> dtype &&& shape $ max (ones :: CPUTensor 'D.Float '[2,2])
(Float,[])

Computes the mean while carrying out a full reduction of all tensor dimensions.

>>> meanAll (ones :: CPUTensor 'D.Float '[])
Tensor Float []  1.0000
>>> meanAll (zeros :: CPUTensor 'D.Float '[2,2])
Tensor Float []  0.0000

Computes the mean while carrying out a full reduction of all tensor dimensions. This version is not restricted and can return NaN.

>>> unsafeMeanAll (ones :: CPUTensor 'D.Float '[])
Tensor Float []  1.0000
>>> unsafeMeanAll (ones :: CPUTensor 'D.Float '[0])
Tensor Float [] NaN
>>> unsafeMeanAll (zeros :: CPUTensor 'D.Float '[2,2])
Tensor Float []  0.0000

Computes the mean and reduces the tensor over the specified dimension.

>>> t = ones :: CPUTensor 'D.Float '[3,4,5]
>>> dtype &&& shape $ meanDim @0 t
(Float,[4,5])
>>> dtype &&& shape $ meanDim @1 t
(Float,[3,5])
>>> dtype &&& shape $ meanDim @2 t
(Float,[3,4])

Computes the mean and reduces the tensor over the specified dimension.

>>> import Torch.Typed.Factories
>>> import Data.Default.Class
>>> t = def :: NamedTensor '( D.CPU, 0) 'D.Float '[Vector 3, Vector 4, Vector 5]
>>> dtype &&& shape $ meanNamedDim @(Vector 4) t
(Float,[3,5])

Computes the mean and optionally reduces the tensor over the specified dimension.

See https://pytorch.org/docs/stable/torch.html#torch.mean for more information.

>>> t = fromJust [[5, 1], [3, 2], [4, 1], [2, 7]] :: CPUTensor 'D.Float '[4, 2]
>>> mean @0 @KeepDim t
Tensor Float [1,2] [[ 3.5000   ,  2.7500   ]]

Computes the median while carrying out a full reduction of all tensor dimensions.

>>> dtype &&& shape $ medianAll (ones :: CPUTensor 'D.Float '[2,2])
(Float,[])

Computes the median and reduces the tensor over the specified dimension.

>>> t = ones :: CPUTensor 'D.Float '[3,4,5]
>>> dtype &&& shape $ fst $ medianDim @0 t
(Float,[4,5])
>>> dtype &&& shape $ fst $ medianDim @1 t
(Float,[3,5])
>>> dtype &&& shape $ fst $ medianDim @2 t
(Float,[3,4])

Computes the median and optionally reduces the tensor over the specified dimension.

See https://pytorch.org/docs/stable/torch.html#torch.median for more information.

>>> t = fromJust [[5, 1], [3, 2], [4, 1], [2, 7]] :: CPUTensor 'D.Float '[4, 2]

• - libtorch 1.7.0
• - (Tensor Float [1,2] [[ 3.0000 , 1.0000 ]],Tensor Int64 [1,2] [[ 1, 0]])
• - libtorch 1.8.0 >>> median 0 KeepDim t (Tensor Float [1,2] [[ 3.0000 , 1.0000 ]],Tensor Int64 [1,2] [[ 1, 2]])

Returns a tuple '(modes, indices)' where modes is the mode value of each row of the input tensor in the given dimension dim, i.e. a value which appears most often in that row, and indices is the index location of each mode value found.

See https://pytorch.org/docs/stable/torch.html#torch.mode for more information.

>>> t = fromJust [[0, 5], [0, 2], [3, 5]] :: CPUTensor 'D.Int64 '[3, 2]

>>> (modes :: CPUTensor 'D.Int64 '[2], indices :: CPUTensor 'D.Int64 '[2]) = mode @0 @DropDim t
>>> (dtype modes, shape modes, D.asValue (toDynamic modes) :: [Int])
(Int64,[2],[0,5])
>>> (dtype indices, shape indices, D.asValue (toDynamic indices) :: [Int])
(Int64,[2],[1,2])

>>> t = fromJust [[0, 0], [0, 1], [3, 3]] :: CPUTensor 'D.Float '[3, 2]

>>> (modes :: CPUTensor 'D.Float '[3,1], indices :: CPUTensor 'D.Int64 '[3,1]) = mode @1 @KeepDim t
>>> (dtype modes, shape modes, D.asValue (toDynamic modes) :: [[Float]])
(Float,[3,1],[[0.0],[0.0],[3.0]])
>>> (dtype indices, shape indices, D.asValue (toDynamic indices) :: [[Int]])
(Int64,[3,1],[[1],[0],[1]])

addScalar TODO: what dtypes is this defined for? TODO: what scalar types is this defined for?

>>> dtype &&& shape $ addScalar 1 (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

subScalar TODO: what dtypes is this defined for? TODO: what scalar types is this defined for?

>>> dtype &&& shape $ subScalar 1 (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

mulScalar TODO: what dtypes is this defined for? TODO: what scalar types is this defined for?

>>> dtype &&& shape $ mulScalar 2 (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

divScalar TODO: what dtypes is this defined for? TODO: what scalar types is this defined for?

>>> dtype &&& shape $ divScalar 2 (ones :: CPUTensor 'D.Float '[2,2])
(Float,[2,2])

powScalar TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> dtype &&& shape $ powScalar 2 (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

erf

>>> dtype &&& shape $ erf (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

exp

>>> dtype &&& shape $ exp (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

log1p

>>> dtype &&& shape $ log1p (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

log2 >>> dtype &&& shape $ log2 (ones :: CPUTensor 'D.Float '[3,2]) (Float,[3,2])

log10

>>> dtype &&& shape $ log10 (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

pow this operation supports broadcasting TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> dtype &&& shape $ pow (2 :: CPUTensor 'D.Float '[]) (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

relu activation function

>>> dtype &&& shape $ relu (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

selu

>>> dtype &&& shape $ selu (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

mish mish is a smooth activation function, see https://arxiv.org/abs/1908.08681 for details.

>>> dtype &&& shape &&& (\t -> D.asValue (toDynamic t) :: [[Float]]) $ mish (ones :: CPUTensor 'D.Float '[3,2])
(Float,([3,2],[[0.86509836,0.86509836],[0.86509836,0.86509836],[0.86509836,0.86509836]]))

sigmoid

>>> dtype &&& shape $ sigmoid (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

sin

>>> dtype &&& shape $ sin (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

sinh

>>> dtype &&& shape $ sinh (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

cos

>>> dtype &&& shape $ cos (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

sqrt

tanh

ConditionalReduction

>>> :kind! ConditionalReduction '[3,2] ReduceNone
ConditionalReduction '[3,2] ReduceNone :: [Natural]
= '[3, 2]
>>> :kind! ConditionalReduction '[3,2] ReduceMean
ConditionalReduction '[3,2] ReduceMean :: [Natural]
= '[]

binary cross entropy

>>> t = ones :: CPUTensor 'D.Float '[2,2]
>>> dtype &&& shape $ binaryCrossEntropy @ReduceNone t t t
(Float,[2,2])
>>> dtype &&& shape $ binaryCrossEntropy @ReduceMean t t t
(Float,[])
>>> dtype &&& shape $ binaryCrossEntropy @ReduceSum t t t
(Float,[])

mseLoss

>>> t = ones :: CPUTensor 'D.Float '[2,2]
>>> dtype &&& shape $ mseLoss @ReduceNone t t
(Float,[2,2])
>>> dtype &&& shape $ mseLoss @ReduceMean t t
(Float,[])
>>> dtype &&& shape $ mseLoss @ReduceSum t t
(Float,[])

softmax

>>> t = ones :: CPUTensor 'D.Float '[2,2]
>>> dtype &&& shape $ softmax @0 t
(Float,[2,2])
>>> dtype &&& shape $ softmax @1 t
(Float,[2,2])

logSoftmax

>>> t = ones :: CPUTensor 'D.Float '[2,2]
>>> dtype &&& shape $ logSoftmax @0 t
(Float,[2,2])
>>> dtype &&& shape $ logSoftmax @1 t
(Float,[2,2])

inverse TODO: if rank < n for any tensors in the batch, then this will not work. we can't decide this statically, but we should prevent runtime errors. therefore, return Maybe?

>>> t <- randn :: IO (CPUTensor 'D.Float '[3,2,2])
>>> dtype &&& shape $ inverse t
(Float,[3,2,2])
>>> t <- randn :: IO (CPUTensor 'D.Float '[2,2])
>>> dtype &&& shape $ inverse t
(Float,[2,2])

symeig Warning: torch.symeig is deprecated in favor of torch.linalg.eigh and will be removed in a future PyTorch release. The default behavior has changed from using the upper triangular portion of the matrix by default to using the lower triangular portion. L, _ = torch.symeig(A, upper=upper) should be replaced with L = torch.linalg.eigvalsh(A, UPLO=U if upper else L) and L, V = torch.symeig(A, eigenvectors=True) should be replaced with L, V = torch.linalg.eigh(A, UPLO=U if upper else L) (function operator())

>>> t <- rand :: IO (CPUTensor 'D.Float '[3,2,2])
>>> (eigenVals,eigenVecs) = symeig Upper t
>>> dtype &&& shape $ eigenVals -- Skip warning
...
>>> dtype &&& shape $ eigenVals
(Float,[3,2])
>>> :t eigenVals
eigenVals :: Tensor '( 'D.CPU, 0) 'D.Float '[3, 2]
>>> dtype &&& shape $ eigenVecs
(Float,[3,2,2])
>>> :t eigenVecs
eigenVecs :: Tensor '( 'D.CPU, 0) 'D.Float '[3, 2, 2]
>>> (eigenVals,eigenVecs) = symeig Lower t
>>> dtype &&& shape $ eigenVals
(Float,[3,2])
>>> dtype &&& shape $ eigenVecs
(Float,[3,2,2])

symeigvalues

>>> t <- rand :: IO (CPUTensor 'D.Float '[3,2,2])
>>> eigenVals = symeigvalues Upper t
>>> dtype &&& shape $ eigenVals
(Float,[3,2])
>>> :t eigenVals
eigenVals :: Tensor '( 'D.CPU, 0) 'D.Float '[3, 2]

eig Warning: torch.eig is deprecated in favor of torch.linalg.eig and will be removed in a future PyTorch release. torch.linalg.eig returns complex tensors of dtype cfloat or cdouble rather than real tensors mimicking complex tensors. L, _ = torch.eig(A) should be replaced with L_complex = torch.linalg.eigvals(A) and L, V = torch.eig(A, eigenvectors=True) should be replaced with L_complex, V_complex = torch.linalg.eig(A) (function operator())

>>> t <- rand :: IO (CPUTensor 'D.Float '[3,3])
>>> (eigenVals,eigenVecs) = eig @EnableEigenVectors t
>>> dtype &&& shape $ eigenVals -- Skip warning
...
>>> dtype &&& shape $ eigenVals
(Float,[3,2])
>>> :t eigenVals
eigenVals :: Tensor '( 'D.CPU, 0) 'D.Float '[3, 2]
>>> dtype &&& shape $ eigenVecs
(Float,[3,3])
>>> :t eigenVecs
eigenVecs :: Tensor '( 'D.CPU, 0) 'D.Float '[3, 3]
>>> (eigenVals,eigenVecs) = eig @DisableEigenVectors t
>>> dtype &&& shape $ eigenVals
(Float,[3,2])
>>> dtype &&& shape $ eigenVecs
(Float,[0])
>>> :t eigenVecs
eigenVecs :: Tensor '( 'D.CPU, 0) 'D.Float '[0]

Singular Value Decomposition TODO: When compute_uv is False, backward cannot be performed since u and v from the forward pass are required for the backward operation. There is no way to encode in the types at this point in time. Thus, only True is supported currently.

This function returns a tuple `(u, s, v)` which is the singular value decomposition of a input real matrix or batches of real matrices input such that `input = U×diag(S)×V^T`.

>>> a <- randn :: IO (CPUTensor 'D.Float '[3, 5])
>>> (u, s, v) = svd @'ThinSVD a
>>> dtype &&& shape $ u
(Float,[3,3])
>>> dtype &&& shape $ s
(Float,[3])
>>> dtype &&& shape $ v
(Float,[5,3])
>>> (u, s, v) = svd @'FullSVD a
>>> dtype &&& shape $ u
(Float,[3,3])
>>> dtype &&& shape $ s
(Float,[3])
>>> dtype &&& shape $ v
(Float,[5,5])
>>> a <- randn :: IO (CPUTensor 'D.Float '[5, 3])
>>> (u, s, v) = svd @'ThinSVD a
>>> dtype &&& shape $ u
(Float,[5,3])
>>> dtype &&& shape $ s
(Float,[3])
>>> dtype &&& shape $ v
(Float,[3,3])
>>> (u, s, v) = svd @'FullSVD a
>>> dtype &&& shape $ u
(Float,[5,5])
>>> dtype &&& shape $ s
(Float,[3])
>>> dtype &&& shape $ v
(Float,[3,3])

cholesky TODO: cholesky can throw if the input is not positive-definite. Computes the Cholesky decomposition of a symmetric positive-definite matrix. The operation supports batching.

Warning: torch.cholesky is deprecated in favor of torch.linalg.cholesky and will be removed in a future PyTorch release. L = torch.cholesky(A) should be replaced with L = torch.linalg.cholesky(A) and U = torch.cholesky(A, upper=True) should be replaced with U = torch.linalg.cholesky(A.transpose(-2, -1).conj()).transpose(-2, -1).conj() (function operator())

>>> t <- rand :: IO (CPUTensor 'D.Float '[2,2])
>>> u = cholesky Upper (t `matmul` transpose2D t) -- Skip warning
...
>>> dtype &&& shape $ u
(Float,[2,2])
>>> :t u
u :: Tensor '( 'D.CPU, 0) 'D.Float '[2, 2]

choleskyInverse Computes the inverse of a symmetric positive-definite matrix using its Cholesky factor, returned, e.g., by cholesky. Unlike cholesky, this operation does not support batching. The inverse is computed using the LAPACK routine `?potri`.

>>> t <- rand :: IO (CPUTensor 'D.Float '[2,2])
>>> tri = Upper
>>> u = cholesky tri (t `matmul` transpose2D t)
>>> dtype &&& shape $ choleskyInverse tri u
(Float,[2,2])

choleskySolve Solves the system of linear equations represented by `a c = b` using the Cholesky factor matrix u of a (returned, e.g., by cholesky), where a is a positive semidefinite matrix. The operation supports batching.

>>> t <- rand :: IO (CPUTensor 'D.Float '[3,3])
>>> a = t `matmul` transpose2D t
>>> b <- rand :: IO (CPUTensor 'D.Float '[3,2])
>>> tri = Upper
>>> u = cholesky tri a
>>> dtype &&& shape $ choleskySolve tri b u
(Float,[3,2])

solve Solves the system of linear equations represented by `a c = b` and also returns the LU decomposition of a. a has to be a positive semidefinite matrix. The operation supports batching.

Warning: torch.solve is deprecated in favor of torch.linalg.solveand will be removed in a future PyTorch release. torch.linalg.solve has its arguments reversed and does not return the LU factorization. To get the LU factorization see torch.lu, which can be used with torch.lu_solve or torch.lu_unpack. X = torch.solve(B, A).solution should be replaced with X = torch.linalg.solve(A, B) (function operator())

>>> t <- rand :: IO (CPUTensor 'D.Float '[10,10])
>>> a = t `matmul` transpose2D t
>>> b <- rand :: IO (CPUTensor 'D.Float '[10,3])
>>> (c,lu) = solve b a
>>> dtype &&& shape $ c -- Skip warning
...
>>> dtype &&& shape $ c
(Float,[10,3])
>>> dtype &&& shape $ lu
(Float,[10,10])
>>> :t c
c :: Tensor '( 'D.CPU, 0) 'D.Float '[10, 3]
>>> :t lu
lu :: Tensor '( 'D.CPU, 0) 'D.Float '[10, 10]

geqrf TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? geqrf computes a QR decomposition of the given input matrix, but without constructing Q and R as explicit separate matrices. Rather, this function directly calls the underlying LAPACK function `?geqrf` which produces a tuple `(a, tau)` of intermediate results as defined in the LAPACK documentation for `?geqrf`.

You can use orgqr on `(a, tau)` to compute the real orthogonal matrix Q, but in general you may just want to use qr instead.

See the LAPACK documentation for `?geqrf` for further details, https://software.intel.com/en-us/node/521004.

>>> (a, tau) = geqrf (ones :: CPUTensor 'D.Float '[3,4])
>>> dtype &&& shape $ a
(Float,[3,4])
>>> dtype &&& shape $ tau
(Float,[3])
>>> (a, tau) = geqrf (ones :: CPUTensor 'D.Float '[4,3])
>>> dtype &&& shape $ a
(Float,[4,3])
>>> dtype &&& shape $ tau
(Float,[3])

orgqr TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? Computes the orthogonal matrix Q of a QR factorization from the `(a, tau)` tuple returned by geqrf.

This directly calls the underlying LAPACK function `?orgqr`. See the LAPACK documentation for `?orgqr` for further details, https://software.intel.com/en-us/mkl-developer-reference-c-orgqr.

When libtorch-1.7, this function behavior is changed. First dimention should be greater than second dimention.

>>> dtype &&& shape $ orgqr (ones :: CPUTensor 'D.Float '[4,3]) (ones :: CPUTensor 'D.Float '[3])
(Float,[4,3])

sign works for all dtypes

>>> dtype &&& shape $ sign (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

Transpose

>>> :kind! Transpose '[3,2] 0 1
Transpose '[3,2] 0 1 :: [Natural]
= '[2, 3]
>>> :kind! Transpose '[3,2,1] 1 2
Transpose '[3,2,1] 1 2 :: [Natural]
= '[3, 1, 2]

transpose See "........depspytorchatensrcATennativeTensorShape.cpp".

>>> dtype &&& shape $ transpose @0 @1 (ones :: CPUTensor 'D.Float '[3,2])
(Float,[2,3])
>>> dtype &&& shape $ transpose @0 @1 (ones :: CPUTensor 'D.Float '[3,2,1])
(Float,[2,3,1])
>>> dtype &&& shape $ transpose @1 @2 (ones :: CPUTensor 'D.Float '[3,2,1])
(Float,[3,1,2])

transpose2d, special case for a 2D tensor

>>> dtype &&& shape $ transpose2D (ones :: CPUTensor 'D.Float '[3,2])
(Float,[2,3])

diag

>>> dtype &&& shape $ diag @'Upper @0 (ones :: CPUTensor 'D.Float '[3,2])
(Float,[2])
>>> dtype &&& shape $ diag @'Upper @1 (ones :: CPUTensor 'D.Float '[3,2])
(Float,[1])
>>> dtype &&& shape $ diag @'Lower @1 (ones :: CPUTensor 'D.Float '[3,2])
(Float,[2])

all See https://pytorch.org/docs/stable/tensors.html#torch.BoolTensor.all.

>>> t = all (fromJust [False, False] :: CPUTensor 'D.Bool '[2])
>>> toInt t == 1
False

>>> t = all (fromJust [False, True] :: CPUTensor 'D.Bool '[2])
>>> toInt t == 1
False

>>> t = all (fromJust [True, True] :: CPUTensor 'D.Bool '[2])
>>> toInt t == 1
True

any See https://pytorch.org/docs/stable/tensors.html#torch.BoolTensor.any.

>>> t = any (fromJust [False, False] :: CPUTensor 'D.Bool '[2])
>>> toInt t == 1
False

>>> t = any (fromJust [False, True] :: CPUTensor 'D.Bool '[2])
>>> toInt t == 1
True

>>> t = any (fromJust [True, True] :: CPUTensor 'D.Bool '[2])
>>> toInt t == 1
True

allDim See https://pytorch.org/docs/stable/tensors.html#torch.BoolTensor.all.

>>> t = fromJust [[True, True], [True, False], [True, True], [True, True]] :: CPUTensor 'D.Bool '[4, 2]

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Bool]) $ allDim @1 @DropDim t
(Bool,([4],[True,False,True,True]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Bool]]) $ allDim @1 @KeepDim t
(Bool,([4,1],[[True],[False],[True],[True]]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Bool]) $ allDim @0 @DropDim t
(Bool,([2],[True,False]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Bool]]) $ allDim @0 @KeepDim t
(Bool,([1,2],[[True,False]]))

anyDim See https://pytorch.org/docs/stable/tensors.html#torch.BoolTensor.any.

>>> t = fromJust [[True, True], [True, False], [True, True], [True, True]] :: CPUTensor 'D.Bool '[4, 2]

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Bool]) $ anyDim @1 @DropDim t
(Bool,([4],[True,True,True,True]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Bool]]) $ anyDim @1 @KeepDim t
(Bool,([4,1],[[True],[True],[True],[True]]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Bool]) $ anyDim @0 @DropDim t
(Bool,([2],[True,True]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Bool]]) $ anyDim @0 @KeepDim t
(Bool,([1,2],[[True,True]]))

dropout TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? TODO: get rid of IO by exposing the RNG state TODO: can we use D.Scalar for the dropout probability?

>>> t = ones :: CPUTensor 'D.Float '[3,2]
>>> t' <- dropout 0.5 False t
>>> dtype &&& shape $ t'
(Float,[3,2])
>>> t'' <- dropout 0.5 False t
>>> t ==. t''
Tensor Bool [3,2] [[ 1,  1],
                   [ 1,  1],
                   [ 1,  1]]
>>> t''' <- dropout 0.0 True t
>>> t ==. t'''
Tensor Bool [3,2] [[ 1,  1],
                   [ 1,  1],
                   [ 1,  1]]
>>> t'''' <- dropout 1.0 True t
>>> t''''
Tensor Float [3,2] [[ 0.0000,  0.0000],
                    [ 0.0000,  0.0000],
                    [ 0.0000,  0.0000]]

featureDropout TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? TODO: why not IO? TODO: can we use D.Scalar for the dropout probability?

>>> c = featureDropout 0.1 True (ones :: CPUTensor 'D.Float '[2,2])
>>> dtype &&& shape $ c
(Float,[2,2])

alphaDropout TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? TODO: why not IO? TODO: can we use D.Scalar for the dropout probability?

>>> c = alphaDropout 0.1 True (ones :: CPUTensor 'D.Float '[2,2])
>>> dtype &&& shape $ c
(Float,[2,2])

featureAlphaDropout TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? TODO: why not IO? TODO: can we use D.Scalar for the dropout probability?

>>> c = featureAlphaDropout 0.1 True (ones :: CPUTensor 'D.Float '[2,2])
>>> dtype &&& shape $ c
(Float,[2,2])

acos

>>> dtype &&& shape $ acos (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

avgPool1d TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> t = avgPool1d @1 @1 @0 (ones :: CPUTensor 'D.Float '[1,3,4])
>>> shape t
[1,3,4]
>>> :t t
t :: Tensor '( 'D.CPU, 0) 'D.Float '[1, 3, 4]

adaptiveAvgPool1d TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> t = adaptiveAvgPool1d @8 (ones :: CPUTensor 'D.Float '[1,3,16])
>>> shape t
[1,3,8]
>>> :t t
t :: Tensor '( 'D.CPU, 0) 'D.Float '[1, 3, 8]

adaptiveMaxPool1d TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> tt = adaptiveMaxPool1d @8 (ones :: CPUTensor 'D.Float '[1,3,16])
>>> shape . fst $ tt
[1,3,8]
>>> :t tt
tt
  :: (Tensor '( 'D.CPU, 0) 'D.Float '[1, 3, 8],
      Tensor '( 'D.CPU, 0) 'D.Int64 '[1, 3, 8])

addmv TODO: probably only defined for floating point tensors, or maybe numeric type is lifted? TODO: can we use D.Scalar for beta and alpha?

>>> t = addmv 1 1 (ones :: CPUTensor 'D.Float '[3,2]) (zeros :: CPUTensor 'D.Float '[2]) (ones :: CPUTensor 'D.Float '[])
>>> dtype &&& shape $ t
(Float,[3])
>>> :t t
t :: Tensor '( 'D.CPU, 0) 'D.Float '[3]

allclose

>>> allclose 0.1 0.1 True (ones :: CPUTensor 'D.Float '[3,3]) (ones :: CPUTensor 'D.Float '[3,3])
True

argmax See https://pytorch.org/docs/stable/torch.html#torch.argmax.

>>> t = fromJust [[0, 1], [-1, 2], [0, 1], [0, -2]] :: CPUTensor 'D.Float '[4, 2]

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Int]) $ argmax @1 @DropDim t
(Int64,([4],[1,1,1,0]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Int]]) $ argmax @1 @KeepDim t
(Int64,([4,1],[[1],[1],[1],[0]]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Int]) $ argmax @0 @DropDim t
(Int64,([2],[0,1]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Int]]) $ argmax @0 @KeepDim t
(Int64,([1,2],[[0,1]]))

argmin See https://pytorch.org/docs/stable/torch.html#torch.argmin.

>>> t = fromJust [[0, 1], [-1, 2], [0, 1], [0, -2]] :: CPUTensor 'D.Float '[4, 2]

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Int]) $ argmin @1 @DropDim t
(Int64,([4],[0,0,0,1]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Int]]) $ argmin @1 @KeepDim t
(Int64,([4,1],[[0],[0],[0],[1]]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [Int]) $ argmin @0 @DropDim t
(Int64,([2],[1,3]))

>>> dtype &&& shape &&& (\t' -> D.asValue (toDynamic t') :: [[Int]]) $ argmin @0 @KeepDim t
(Int64,([1,2],[[1,3]]))

asin

>>> dtype &&& shape $ asin (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

atan

>>> dtype &&& shape $ atan (ones :: CPUTensor 'D.Float '[3,2])
(Float,[3,2])

baddbmm TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> t = baddbmm 1 1 (ones :: CPUTensor 'D.Float '[5,3,2]) (zeros :: CPUTensor 'D.Float '[5,2,4]) (ones :: CPUTensor 'D.Float '[])
>>> dtype &&& shape $ t
(Float,[5,3,4])
>>> :t t
t :: Tensor '( 'D.CPU, 0) 'D.Float '[5, 3, 4]

batched matrix multiplication TODO: probably only defined for floating point tensors, or maybe numeric type is lifted?

>>> dtype &&& shape $ bmm (ones :: CPUTensor 'D.Float '[5,3,2]) (zeros :: CPUTensor 'D.Float '[5,2,4])
(Float,[5,3,4])

BroadcastTensorsImpl

>>> type Ty = BroadcastTensorsImpl '[] 'Nothing
>>> :kind! Ty
Ty :: Maybe ([Natural], D.DType, (D.DeviceType, Natural))
= 'Nothing
>>> type Ty = BroadcastTensorsImpl '[Tensor '( 'D.CPU, 0) 'D.Float '[1, 3], Tensor '( 'D.CPU, 0) 'D.Float '[2, 1]] 'Nothing
>>> :kind! Ty
Ty :: Maybe ([Natural], D.DType, (D.DeviceType, Natural))
= 'Just '( '[2, 3], 'D.Float, '( 'D.CPU, 0))
>>> type Ty = BroadcastTensorsImpl '[Tensor '( 'D.CPU, 0) 'D.Float '[1, 3], Tensor '( 'D.CPU, 0) 'D.Float '[2, 1], Tensor '( 'D.CPU, 0) 'D.Float '[5, 1, 1]] 'Nothing
>>> :kind! Ty
Ty :: Maybe ([Natural], D.DType, (D.DeviceType, Natural))
= 'Just '( '[5, 2, 3], 'D.Float, '( 'D.CPU, 0))
>>> type Ty = BroadcastTensorsImpl '[Tensor '( 'D.CPU, 0) 'D.Float '[1, 3], Tensor '( 'D.CPU, 0) 'D.Float '[2, 1], Tensor '( 'D.CPU, 0) 'D.Float '[1, 5, 1]] 'Nothing
>>> :kind! Ty
Ty :: Maybe ([Natural], D.DType, (D.DeviceType, Natural))
= 'Nothing

broadcast tensors TODO: broadcastTensors returns garbage data and is hence broken See https://pytorch.org/docs/stable/_modules/torch/functional.html#broadcast_tensors.

>>> x = ones :: CPUTensor 'D.Float '[1, 3]
>>> y = ones :: CPUTensor 'D.Float '[2, 1]
>>> z = ones :: CPUTensor 'D.Float '[5, 1, 1]

• - >>> x' :. y' :. z' :. HNil = broadcastTensors (x :. y :. z :. HNil)
• - >>> :type x'
• - x' :: Tensor '( 'D.CPU, 0) 'D.Float '[5, 2, 3]
• - >>> dtype &&& shape &&& (t -> D.asValue (toDynamic t) :: [[[Float]]]) $ x'
• - >>> :type y'
• - y' :: Tensor '( 'D.CPU, 0) 'D.Float '[5, 2, 3]
• - >>> dtype &&& shape &&& (t -> D.asValue (toDynamic t) :: [[[Float]]]) $ y'
• - >>> :type z'
• - z' :: Tensor '( 'D.CPU, 0) 'D.Float '[5, 2, 3]
• - >>> dtype &&& shape &&& (t -> D.asValue (toDynamic t) :: [[[Float]]]) $ z'

Cat

>>> type Ty = Cat 0 '[Tensor '( 'D.CPU, 0) 'D.Float '[1]]
>>> :kind! Ty
Ty :: ([Natural], D.DType, (D.DeviceType, Natural))
= '( '[1], 'D.Float, '( 'D.CPU, 0))
>>> type Ty = Cat 0 '[Tensor '( 'D.CPU, 0) 'D.Float '[1], Tensor '( 'D.CPU, 0) 'D.Float '[2]]
>>> :kind! Ty
Ty :: ([Natural], D.DType, (D.DeviceType, Natural))
= '( '[3], 'D.Float, '( 'D.CPU, 0))
>>> type Ty = Cat 0 '[Tensor '( 'D.CPU, 0) 'D.Float '[1, 3], Tensor '( 'D.CPU, 0) 'D.Float '[2, 3]]
>>> :kind! Ty
Ty :: ([Natural], D.DType, (D.DeviceType, Natural))
= '( '[3, 3], 'D.Float, '( 'D.CPU, 0))
>>> type Ty = Cat 1 '[Tensor '( 'D.CPU, 0) 'D.Float '[3, 1], Tensor '( 'D.CPU, 0) 'D.Float '[3, 2]]
>>> :kind! Ty
Ty :: ([Natural], D.DType, (D.DeviceType, Natural))
= '( '[3, 3], 'D.Float, '( 'D.CPU, 0))
>>> type Ty = Cat 1 '[Tensor '( 'D.CPU, 0) 'D.Float '[2, 5, 4, 2], Tensor '( 'D.CPU, 0) 'D.Float '[2, 1, 4, 2], Tensor '( 'D.CPU, 0) 'D.Float '[2, 3, 4, 2], Tensor '( 'D.CPU, 0) 'D.Float '[2, 1, 4, 2]]
>>> :kind! Ty
Ty :: ([Natural], D.DType, (D.DeviceType, Natural))
= '( '[2, 10, 4, 2], 'D.Float, '( 'D.CPU, 0))

cat

>>> t = ones :: CPUTensor 'D.Float '[2,2]
>>> t' = cat @0 (t :. HNil)
>>> :type t'
t' :: Tensor '( 'D.CPU, 0) 'D.Float '[2, 2]
>>> dtype &&& shape &&& (\t'' -> D.asValue (toDynamic t'') :: [[Float]]) $ t'
(Float,([2,2],[[1.0,1.0],[1.0,1.0]]))
>>> t' = cat @1 (t :. HNil)
>>> :type t'
t' :: Tensor '( 'D.CPU, 0) 'D.Float '[2, 2]
>>> dtype &&& shape &&& (\t'' -> D.asValue (toDynamic t'') :: [[Float]]) $ t'
(Float,([2,2],[[1.0,1.0],[1.0,1.0]]))
>>> t' = cat @0 (t :. t :. t :. HNil)
>>> :type t'
t' :: Tensor '( 'D.CPU, 0) 'D.Float '[6, 2]
>>> dtype &&& shape &&& (\t'' -> D.asValue (toDynamic t'') :: [[Float]]) $ t'
(Float,([6,2],[[1.0,1.0],[1.0,1.0],[1.0,1.0],[1.0,1.0],[1.0,1.0],[1.0,1.0]]))
>>> t' = cat @1 (t :. t :. t :. HNil)
>>> :type t'
t' :: Tensor '( 'D.CPU, 0) 'D.Float '[2, 6]
>>> dtype &&& shape &&& (\t'' -> D.asValue (toDynamic t'') :: [[Float]]) $ t'
(Float,([2,6],[[1.0,1.0,1.0,1.0,1.0,1.0],[1.0,1.0,1.0,1.0,1.0,1.0]]))

chunk

• - >>> :type chunk 3 1 (ones :: CPUTensor 'D.Float '[2, 2])
• - chunk 3 1 (ones :: CPUTensor 'D.Float '[2, 2])
• - :: HList
• - '[Tensor '( 'D.CPU, 0) 'D.Float '[2, 1],
• - Tensor '( 'D.CPU, 0) 'D.Float '[2, 1]] >>> t0 :. t1 :. HNil = chunk 3 1 (ones :: CPUTensor 'D.Float '[2, 2]) >>> dtype &&& shape $ t0 (Float,[2,1]) >>> dtype &&& shape $ t1 (Float,[2,1])
• - >>> :type chunk 3 1 (ones :: CPUTensor 'D.Float '[1, 0, 3])
• - chunk 3 1 (ones :: CPUTensor 'D.Float '[1, 0, 3])
• - :: HList
• - '[Tensor '( 'D.CPU, 0) 'D.Float '[1, 0, 3],
• - Tensor '( 'D.CPU, 0) 'D.Float '[1, 0, 3],
• - Tensor '( 'D.CPU, 0) 'D.Float '[1, 0, 3]] >>> t0 :. t1 :. t2 :. HNil = chunk 3 1 (ones :: CPUTensor 'D.Float '[1, 0, 3]) >>> dtype &&& shape $ t0 (Float,[1,0,3]) >>> dtype &&& shape $ t1 (Float,[1,0,3]) >>> dtype &&& shape $ t2 (Float,[1,0,3])
• - >>> :type chunk 6 0 (ones :: CPUTensor 'D.Float '[19, 4])
• - chunk 6 0 (ones :: CPUTensor 'D.Float '[19, 4])
• - :: HList
• - '[Tensor '( 'D.CPU, 0) 'D.Float '[4, 4],
• - Tensor '( 'D.CPU, 0) 'D.Float '[4, 4],
• - Tensor '( 'D.CPU, 0) 'D.Float '[4, 4],
• - Tensor '( 'D.CPU, 0) 'D.Float '[4, 4],
• - Tensor '( 'D.CPU, 0) 'D.Float '[3, 4]] >>> t0 :. t1 :. t2 :. t3 :. t4 :. HNil = chunk 6 0 (ones :: CPUTensor 'D.Float '[19, 4]) >>> dtype &&& shape $ t0 (Float,[4,4]) >>> dtype &&& shape $ t1 (Float,[4,4]) >>> dtype &&& shape $ t2 (Float,[4,4]) >>> dtype &&& shape $ t3 (Float,[4,4]) >>> dtype &&& shape $ t4 (Float,[3,4])