Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
The PyTorch doc Reproducibility has very detailed instructions on how to make your model training reproducible. Basically, you need the following code.
torch.manual_seed(args.seed)
np.random.seed(args.seed)
random.seed(args.seed)
Notice that torch.manual_seed set seeds of RNGs on all devices (both CPU and GPUs).
There is no need to make additional calls of torch.cuda.manual_seed
or torch.manual_seed_all.
References
Difference between torch.manual_seed and torch.cuda.manual_seed