Ben Chuanlong Du's Blog

It is never too late to learn.

Rust and Spark

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

The simplest and best way is to leverage pandas_udf in PySpark. In the pandas UDF, you can call subprocess.run to run any shell command and capture its output.

from pathlib …

Zellij Is the Best Terminal Multiplexer

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

zellij options --disable-mouse-mode

https://github.com/zellij-org/zellij

Persistent Sessions

A detached session becomes a persistent session.

ctrol + o: d

You can re-attach a session using

zellij attach session_name

Make a Rust Project Both a Library and a Binary

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

.
├── Cargo.toml
└── src
    ├── main.rs
    │   
    └── lib.rs
[package]
name = "package_name"
version = "0.0.1"
authors = ["me <me@gmail.com>"]

[lib]
name = "lib_name"
path = "src/lib.rs"

[[bin]]
name = "binary_name"
path = "src …