checkmate¶

checkmate breaks the GPU memory wall by enabling researchers to train large state-of-the-art models that do not fit in GPU memory. Checkmate applies optimal tensor rematerialization (as detailed in the paper) to trade off space and time.

Tips and Traps¶

1. Adpative pooling an generate fixed-sized output, which enables a model to handle varying sized input. The models in torchvision are such examples.

AdaptiveMaxPool2d¶

References¶

Torchvision ResNet Input Size

set_grad_enable¶