Ben Chuanlong Du's Blog

It is never too late to learn.

Rust and Spark

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

The simplest and best way is to leverage pandas_udf in PySpark. In the pandas UDF, you can call subprocess.run to run any shell command and capture its output.

from pathlib …

JavaScript Alternatives to Shell

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

https://github.com/shelljs/shelljs

https://tutorialedge.net/javascript/nodejs/executing-shell-scripts-with-nodejs/

Check Whether a Shell Command Exists

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Python

shutil.which

Shell

which

Reference

  • https://stackoverflow.com/questions/11210104/check-if-a-program-exists-from-a-python-script/34177358

  • https://stackoverflow.com/questions/592620/how-to-check-if-a-program-exists-from-a-bash-script

"." Is Not Recognized as An Internal or External Command

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

I encountered the following error message when running a Windows command !./osqueryi.exe .tables in the IPython shell.

In[15] >>> !./osqueryi.exe .tables
'.' is not recognized as an internal or external …