import Control.Monad.State
import Torch
import Torch.Internal.Managed.Type.Context (manual_seed_L)

Randomness

Create a randomly initialized matrix:

w <- randIO' [2, 2]

w => 
Tensor Float [2,2] [[ 9.5247e-2,  0.7988   ],
                    [ 0.2953   ,  0.2042   ]]

Note that random initialization without specifying the random number generator (RNG) state as above is necessarily impure, because we expect a different result each time randIO' is called with the same arguments.

w' <- randIO' [2, 2]

w' => 
Tensor Float [2,2] [[ 0.3321   ,  0.1659   ],
                    [ 0.4619   ,  0.8728   ]]

To make a computation deterministic, the RNG can be explicitly seeded:

manual_seed_L 42
w1 <- randIO' [2, 2]

manual_seed_L 42
w2 <- randIO' [2, 2]

w1 == w2 => 
True

Pure vs. Impure

Hasktorch also includes pure variants of the random initialization functions that accept the RNG state as an argument and return the updated RNG state with the result. For example, the rand' function in Torch.Random:

rand' :: [Int] -> Generator -> (Tensor, Generator)

where Generator is the type representing the RNG state.

By convention, random initialization functions with the IO suffix thread the RNG state implicitly through the IO context, while the analogous functions without the IO suffix expect a Generator argument and return a new generator along with the result.

To use the "pure" style, we first need to initialize a Generator, and then explicitly thread it through our computation:

rng0 <- mkGenerator (Device CPU 0) 31415

let (x1, rng1) = rand' [2, 2] rng0
    (x2, rng2) = rand' [2, 2] rng1

(x1, x2) => 
(Tensor Float [2,2] [[ 0.3296   ,  0.2113   ],
                    [ 0.8446   ,  0.9235   ]],Tensor Float [2,2] [[ 0.5002   ,  0.4071   ],
                    [ 0.5099   ,  0.4392   ]])

The benefit of this approach is the clear separation between deterministic and nondeterministic parts of the computation, in this example manifested in the explicit threading of the Generator value.

Note that we can use the State monad to make this threading implicit while still making use of the type system to enforce the boundary between deterministic and nondeterministic code:

let randomPair = do x1 <- state $ rand' [2, 2]
                    x2 <- state $ rand' [2, 2]
                    pure (x1, x2)

We can then pass in a Generator to get a pair of random tensors like this:

runState randomPair rng0 => 
((Tensor Float [2,2] [[ 0.3296   ,  0.2113   ],
                    [ 0.8446   ,  0.9235   ]],Tensor Float [2,2] [[ 0.5002   ,  0.4071   ],
                    [ 0.5099   ,  0.4392   ]]),UnsafeGenerator {unGenerator = _})