[SPARK-53655][SQL][TESTS] Fix the intention of 'read parquet footers in parallel' test

[yaooqinn]

What changes were proposed in this pull request?

Fix the intention of 'read parquet footers in parallel' test

Why are the changes needed?

The 'read parquet footers in parallel' test creates a json file as an invalid input for the parquet footer reader, skipping the hidden files like normal reads

Does this PR introduce any user-facing change?

no

How was this patch tested?

modified test

Was this patch authored or co-authored using generative AI tooling?

no

[dongjoon-hyun]

Thank you, @yaooqinn .

1. Please fix the compilation failure due to the unused import statement.

[error] /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormatSuite.scala:22:30: Unused import
[error] Applicable -Wconf / @nowarn filters for this fatal warning: msg=<part of the message>, cat=unused-imports, site=org.apache.spark.sql.execution.datasources.parquet
[error] import org.apache.hadoop.fs.{FileSystem, Path}
[error]        

2. We need a JIRA issue for this test improvement because this is a logic change, not an indentation.

- val fileStatuses =
-   Seq(fs.listStatus(path1), fs.listStatus(path2), fs.listStatus(path3)).flatten
+ val fileStatuses = HadoopFSUtils.listFiles(
+   new Path(basePath),
+   hadoopConf,
+   (path: Path) => path.getName != "_SUCCESS").flatMap(_._2)

[yaooqinn]

Thank you @dongjoon-hyun for the suggested changes

[dongjoon-hyun]

+1, LGTM. Thank you, @yaooqinn .