Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Delta Table

convert to delta [db_name.]table_name [partitioned by ...] [vacuum [retain number hours]]

vaccum

describe history db_name.table_name

can select from historical snapshot can also rollback to a historical snapshot rollback …

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Pytorch appears to be crashing due to OOM prematurely?

How to deal with excessive memory usages of PyTorch?

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

https://github.com/linkedin/datahub

https://engineering.linkedin.com/blog/2019/data-hub DataHub: A generalized metadata search & discovery tool

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips

1. Training a model requires significantly more CPU/GPU memories than running inference using the model.

2. torch.cuda.empty_cache() doesn't help if memory is not enough

3. It is suggested that you …

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Data Modeling with RDF(S)

Learn OWL and RDFS

RDF: Resource Description Framework OWL: Web Ontology Language

Products

neo4j

RDFox

Unique features of RDFox include the very powerful rule language (roughly …

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

1. torchvision pretrained models

2. Model Zoo