Total directory contents size increased after spark partition over gzip files in spark sql
I have 10files in my directory with gzip compression and total size of the directory contents is 800mb. When I repartitioned in spark SQL from 10 files to 5 files, I assumed that the total directory size will still be 800MB and then the file count will become 5. But to my surprise, the file size content went to 1.7GB (File count is 5, which is expected).
Any insights please?