Ben Chuanlong Du's Blog

It is never too late to learn.

Machine Learning Libraries, Computing Frames and Programming Languages

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. GPU is more accisible for average individual people. GPU is still the main tool for deep learning right now.

  2. Python Distributed Computing Frameworks (Ray, Modin, etc.) servers as a mid solution …

Spark Issue: Max Number of Executor Failures Reached

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptom

21/06/01 15:03:28 INFO ApplicationMaster: Final app status: FAILED, exitCode: 11, (reason: Max number of executor failures (6) reached)

Possible Causes

The option spark.yarn.max.executor …

Write Documentation for Python Packages Using Sphinx

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Installation

You can install Sphinx and necessary extensions using the following command.

pip3 install sphinx sphinx-autodoc-typehints

Or simply

xinstall sphinx -ic

Since the above commands installs Sphinx to the user's local …

Messsage Queue Implementations

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Kafka seems to be the most popular message queue currently. Redis and memcached are more suitaable for situations where you need cache in memory.

Protocols

AMQP is the Advanced Message Queuing …

Show Image in Terminal

** Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

viu is a simple terminal image viewer written in Rust. It works perfectly in kitty . In terminals which does not support inline image display protocols, viu will show the image in …