tag

#data-engineering

2 posts

WSLDeep Dive October 4, 2026 7 min read

Apache Beam in WSL: DirectRunner Semantics and Local Pipeline Tests

Use Apache Beam's DirectRunner in WSL to test pipeline logic, serialization, ordering assumptions, and assertions before validating a distributed runner.

#wsl#apache-beam#data-engineering

WSLDeep Dive October 4, 2026 7 min read

Apache Spark in WSL: Local Mode, Data Paths, and Test Boundaries

Develop Spark jobs in WSL local mode, keep JVM and filesystem boundaries clear, and test transformations without mistaking a laptop for a cluster.

#wsl#apache-spark#data-engineering