A data pipeline commonly has three phases: extract, transform, and load (ETL). The Unix philosophy of “do one thing well” can apply here by making each program handle only one ETL phase; for example, a program that only extracts data. This strategy works well whether orchestration is done with a shell script or with a Python-based de facto tool such as Apache Airflow. In contrast to mixing ETL within a single program, Airflow and shell scripts using GNU Parallel can handle retries, concurrency, logging, etc. Data pipeline engineering is not a rare case. Still, other kinds of applications, such as GUI, TUI, and web apps, are rarely built on shell scripts; however, shell scripts may work behind the scenes for many apps. Moreover, “do one thing well” still applies, but the unit may be a function, module or service instead.

wsl --install installs Ubuntu by default. If I had to use WSL, I’d try this first too.

The hard part for me was dynamic scope, which I found it was explained nicely in The Common Lisp Condition System: Beyond Exception Handling with Control Flow Mechanisms. Since then I have begun to feel that dynamic scope in Bash is a feature, not limitation.