Ben Chuanlong Du's Blog

It is never too late to learn.

Hands on Tensor in PyTorch

In [1]:
import torch
import numpy as np
In [5]:
x = torch.tensor(
    [
        [1.0, 2, 3, 4, 5],
        [6.0, 7, 8, 9, 10],
    ]
)
x
Out[5]:
tensor([[ 1.,  2.,  3.,  4.,  5.],
        [ 6.,  7.,  8.,  9., 10.]])

torch.Tensor vs torch.tensor vs torch.as_tensor

  1. torch.Tensor always returns torch.FloatTensor. torch.tensor infers the data type and allows users to specify the data type. It is suggested that you use torch.tensor instead of torch.Tensor.

  2. torch.tensor always copies data while torch.as_tensor avoids copying data if possible. One such an example is when you convert a numpy array to a Tensor. However, notice that both torch.tensor and torch.as_tensor copies data if a list is feeded to them.

  3. In most situations, you should use torch.tensor. Never use torch.Tensor. Be cautious if you use torch.as_tensor.

In [16]:
a1 = np.array([1, 2, 3])
t1 = torch.Tensor(arr)
t1
Out[16]:
tensor([1., 2., 3.])
In [17]:
a1[0] = 1000
t1
Out[17]:
tensor([1., 2., 3.])
In [18]:
a2 = np.array([1, 2, 3])
t2 = torch.Tensor(arr)
t2
Out[18]:
tensor([1., 2., 3.])
In [19]:
a2[0] = 1000
t2
Out[19]:
tensor([1., 2., 3.])
In [20]:
a3 = np.array([1, 2, 3])
t3 = torch.as_tensor(a3)
t3
Out[20]:
tensor([1, 2, 3])
In [21]:
a3[0] = 1000
t3
Out[21]:
tensor([1000,    2,    3])

torch.rand

In [4]:
torch.rand(10)
Out[4]:
tensor([0.7872, 0.1416, 0.2451, 0.7463, 0.3549, 0.6301, 0.3934, 0.4746, 0.0036,
        0.0931])

torch.stack

Concatenates sequence of tensors along a new dimension. All tensors need to be of the same size.

In [2]:
data = [torch.tensor([1, 2, 3]), torch.tensor([4, 5, 6])]
In [3]:
torch.stack(data)
Out[3]:
tensor([[1, 2, 3],
        [4, 5, 6]])

torch.stack takes a list/tuple of tensors. It does NOT work on a generator/iterator of tensors.

In [6]:
torch.stack(t for t in data)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-4a0a55595f8d> in <module>
----> 1 torch.stack(t for t in data)

TypeError: stack(): argument 'tensors' (position 1) must be tuple of Tensors, not generator

Tensor.detach

Returns a new Tensor, detached from the current graph.

The result will never require gradient.

Note: Returned Tensor shares the same storage with the original one. In-place modifications on either of them will be seen, and may trigger errors in correctness checks. IMPORTANT NOTE: Previously, in-place size / stride / storage changes (such as resize_ / resizeas / set / transpose) to the returned tensor also update the original tensor. Now, these in-place changes will not update the original tensor anymore, and will instead trigger an error. For sparse tensors: In-place indices / values changes (such as zero / copy / add_) to the returned tensor will not update the original tensor anymore, and will instead trigger an error.

In [3]:
?x.detach

Docstring:
Returns a new Tensor, detached from the current graph.

The result will never require gradient.

.. note::

  Returned Tensor shares the same storage with the original one.
  In-place modifications on either of them will be seen, and may trigger
  errors in correctness checks.
  IMPORTANT NOTE: Previously, in-place size / stride / storage changes
  (such as `resize_` / `resize_as_` / `set_` / `transpose_`) to the returned tensor
  also update the original tensor. Now, these in-place changes will not update the
  original tensor anymore, and will instead trigger an error.
  For sparse tensors:
  In-place indices / values changes (such as `zero_` / `copy_` / `add_`) to the
  returned tensor will not update the original tensor anymore, and will instead
  trigger an error.
Type:      builtin_function_or_method

Tensor.mean

In [6]:
x.mean()
Out[6]:
tensor(5.5000)

Tensor.item

In [7]:
x.item()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-3396a1b2b617> in <module>
----> 1 x.item()

ValueError: only one element tensors can be converted to Python scalars
In [11]:
y = torch.tensor([2])
y
Out[11]:
tensor([2])
In [13]:
y.item()
Out[13]:
2
In [15]:
y = torch.tensor(2)
y
Out[15]:
tensor(2)
In [16]:
y.item()
Out[16]:
2

Tensor.backward

Generally speaking, you only need to call this method on the loss tensor.

Move a Tensor to a Specific Device in PyTorch

Please refer to Move a Tensor to a Specific Device in PyTorch for more details.

Convert a Tensor to a Numpy Array in PyTorch

Please refer to Convert a Tensor to a Numpy Array in PyTorch for more details.

Tensor Transformations in TorchVision

Please refer to Tensor Transformations in TorchVision for more details.

Resize a Tensor in PyTorch

Please refer to Resize a Tensor in PyTorch for more details.

Rerferences

torch.Tensor

torch.as_tensor

torch.tensor

torch.as_tensor fails to create named tensors

In [ ]:

Comments