Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Object Detection Using Deep Learning
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Concepts
Image Classification
Image Localization
Image Classification: Predict the type or class of an object in an image. Input: An image with a single object, such as a photograph. Output: A …
The Case Statement and the when Function in Spark
Tips and Traps¶
Watch out for NaNs ..., behave might not what you expect ...
None can be used for otherwise and yield null in DataFrame.
Column alias and postional columns can be used in group by in Spark SQL!!!
Notice the function when behaves like if-else.
Arithmetic Functions and Operations in Spark
Collection Functions in Spark
Tips and Traps¶
If you use PySpark instead of Spark/Scala, pandas udf is a great alternative to all those (complicated) collections functions discussed here. Leveraging pandas udf, each partition of a Spark DataFrame can be converted to a pandas DataFrame without copying the underlying data, you can then do transforms on pandas DataFrames which will be converted back to partitons of a Spark DataFrame.