Is your feature request related to a problem or challenge?
I would like to add a Spark-compatible Parquet reader to the datafusion-spark crate, based on functionally that already exists in Comet.
The Parquet reader should support per-Spark-version behavior. For example, there were significant differences between Spark 3.x and 4.x with regards to type widening and handling of TimestampNTZ.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
Is your feature request related to a problem or challenge?
I would like to add a Spark-compatible Parquet reader to the
datafusion-sparkcrate, based on functionally that already exists in Comet.The Parquet reader should support per-Spark-version behavior. For example, there were significant differences between Spark 3.x and 4.x with regards to type widening and handling of
TimestampNTZ.Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response