Ben Chuanlong Du's Blog

It is never too late to learn.

Tips on Microk8S

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement! 1. Microk8s works on Linux only.

References

MicroK8s quick start guide

https://microk8s.io/

https://microk8s.io/#get-started

https://github.com/ubuntu/microk8s

https://microk8s.io/docs/

Tips on OpenFaaS

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tutorials

https://blog.alexellis.io/quickstart-openfaas-cli/

https://www.youtube.com/watch?time_continue=6&v=LQa8-JvIeWA&feature=emb_logo

Deploy OpenFaaS with Kubernetes

Deploy OpenFaaS from static YAML, via helm, or via new …

Job Scheduling and Management Using Apache Airflow

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Installation (MySQL)

  1. Install Apache AirFlow.

    wajig install \
        python3-dev python3-pip \
        mysql-server libmysqlclient-dev
    sudo AIRFLOW_GPL_UNIDECODE=yes pip3 install apache-airflow[mysql]
    
  2. Add the following content into your my.cnf …

Machine Learning Libraries, Computing Frames and Programming Languages

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. GPU is more accisible for average individual people. GPU is still the main tool for deep learning right now.

  2. Python Distributed Computing Frameworks (Ray, Modin, etc.) servers as a mid solution …

Spark Issue: Max Number of Executor Failures Reached

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptom

21/06/01 15:03:28 INFO ApplicationMaster: Final app status: FAILED, exitCode: 11, (reason: Max number of executor failures (6) reached)

Possible Causes

The option spark.yarn.max.executor …