Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Tips and Traps¶
By defaut,
csv::Readerrequires headers.When the csv crate is used together with the serde crate for deserialization, CSV files to be parsed have to be strictly well formatted. For example, the headers in CSV files have to match the defintion in the serde struct. Otherwise, the code will panic with an error of "missing fields".
Spark Issue: _Pickle.Picklingerror: Args[0] from __Newobj__ Args Has the Wrong Class
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Please refer to Spark Issue: Task Not Serializable for a similar serialization issue in Spark/Scala.
Symptom
Cause
For example, if you have the following import
from nltk.corpus import stopwords …Hands on the json Module in Python
Tips and Traps¶
It is suggested that you avoid using JSON for serializing and deserializing data. Please refer to Shotcomes of JSON for detailed discussions on this. TOML and YAML are better text-based alternatives to JSON. If serialization and deserialization is done in Python only, pickle
Save and Load PyTorch Models
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
-
PyTorch uses pickle to serialize and deserialize objects.
-
The PyTorch convention is to use the file extension
.ptor.pthfor saving model (or its parameters) and use the file extension …
Representation of Machine Learning Models
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Open Neural Network Exchange (ONNX)
https://github.com/onnx
https://github.com/onnx/onnxmltools
Predictive Model Markup Language (PMML)
https://github.com/jpmml/sklearn2pmml
PMML4S is a PMML (Predictive Model Markup …