Tag: DataFrames rss


06 March 2020 / / Programming
When working on Spark jobs (in Scala), we often sequentially write the code in a single class, giving more attention to the transformations we do and forgetting how our code is structured or even if it's tested. Today I'll be talking about how I personally like to structure and design my Spark jobs, such as they are highly maintainable and testable.