Ben Chuanlong Du's Blog

It is never too late to learn.

Python Packaging Tools

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Python packaging:

https://python-packaging.readthedocs.io/en/latest/minimal.html

  1. It is recommended to use PyBuilder to manager Python projects.

pyinstaller

setuptools

pip

pip can install a package from a GitHub …

Parameterized SQL

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. Parameters are not supported in a DDL statement.

Use Hive in Zeppelin

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Configuration

It seems that you have set a queue using the following code even if you have already set a queue in properties.

%hive-interpreter

set mapred.job.queue.name = your_queue …

Set Up External Dependencies in Zeppelin

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

http://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/index.html

  1. %spark.dep seems troublesome. What if I set interpreter dependency directly? And can I do it for a single notebook? How …