How to configure Spark / Glue to avoid creation of empty $_folder_$ after Glue job successful execution
I have a simple glue etl job which is triggered by Glue workflow. It drop duplicates data from a crawler table and writes back the result into a S3 bucket. The job is completed successfully . However the empty folders that spark generates "$folder$" remain...