Table topic fails to commit partition metadata to Iceberg, resulting in null partition values

[Gezi-lzq]

Bug Description

When a table topic is configured with a partitioning strategy (e.g., automq.table.topic.partition.by=[hour(timestamp)]), the partition metadata is not correctly committed to the Iceberg manifests. This results in queries showing only null values for partition columns, even though the data files are correctly written to partitioned paths in S3.

Root Cause

The root cause is that the CommitResponse sent from the Worker to the TableCoordinator is constructed with an empty partitionType (Types.StructType.of()). This prevents the DataFile objects from being serialized with the correct partition values, leading to null partition information in the manifest file.

Proposed Solution

The fix is to ensure the correct partitionType is provided when constructing the CommitResponse. The PartitioningWriter has the necessary partition spec and schema, and this information must be propagated to the commit response.