Tips on Dataset in PyTorch
If your data can be fit into the CPU memory, it is a good practice to save your data into one pickle file (or other format that you know how to deserialize). This comes with several advantages. First, it is easier and faster to read from a single big file rather than many small files. Second, it avoids the possible system error of openning too many files (even though avoiding lazying data loading is another way to fix the issue). Some example datasets (e.g., MNIST) have separate training and testing files (i.e., 2 pickle files), so that research work based on it can be easily reproduced. I personally suggest that you keep only 1 file containing all data when implementing your own Dataset class. You can always use the function
torch.utils.data.random_split
Hands on the Python submodule os.path
Comments¶
- It is suggested that you use use
pathlibpackage instead ofos.path.
Hands on the Python module glob
Comment¶
pathlib.Path.glob is preferred to the
globmodule!!The list returned is not sorted.
Maven Dependencies
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Properties
<properties>
<scala.version>2.11.11</scala.version>
</properties>
Dependencies
Unit Test
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
specs
<dependency>
<groupId …Tips on Deno
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Add the following into the file .vscode/settings.json under the root directory of your project.
{
"deno.enable": true,
}
References
https://mp.weixin.qq.com/s …