[Feature][Connector-V2] Support sink connector for AWS DSQL #9726

[cloud456]

Purpose of this pull request

At the Re:invent conference in 2024,AWS announced Amazon Aurora DSQL, a new serverless, distributed SQL database. Up to now, there is no tool that can synchronize to DSQL well.

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• If you are contributing the connector code, please check that the following files are updated:
  1. Update plugin-mapping.properties and add new connector information in it
  2. Update the pom file of seatunnel-dist
  3. Add ci label in label-scope-conf
  4. Add e2e testcase in seatunnel-e2e
  5. Update connector plugin_config

[Hisoka-X]

I found dsql support jdbc. Why not add new catalog and dialect in jdbc connector? Please refer https://github.com/apache/seatunnel/tree/dev/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog and https://github.com/apache/seatunnel/tree/dev/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect

[cloud456]

I found dsql support jdbc. Why not add new catalog and dialect in jdbc connector? Please refer https://github.com/apache/seatunnel/tree/dev/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog and https://github.com/apache/seatunnel/tree/dev/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect

The reason for not using JDBC is that the connection password for DSQL requires generating a temporary token to serve as the password. This token is only valid for 15 minutes by default, therefore requiring token renewal and dataSource reset within that 15-minute timeframe. Implementing this mechanism within JDBC would, in my opinion, be an intrusive behavior and not standard practice. Please refer https://docs.aws.amazon.com/aurora-dsql/latest/userguide/SECTION_authentication-token.html

[Hisoka-X]

JDBC connector has many features available out of the box and is easier to maintain.

requires generating a temporary token to serve as the password.

So I think we can add new method named refreshPassword in JdbcConnectionProvider to adapt this situation.

[cloud456]

Glad to hear it.Do you have any schedule for this? Just to mention, it's not only about implementing refreshPassword in JdbcConnectionProvider, but also redesigning JdbcConnectionConfig. DSQL doesn't require a username and password; instead, it creates a password using ak/sk or profile combined with region and cluster endpoint. Therefore, its parameter requirements for JdbcConnectionConfig are different, potentially changing the original parameter requirements, such as making username and password mandatory in the original design

[Hisoka-X]

Glad to hear it.Do you have any schedule for this? Just to mention, it's not only about implementing refreshPassword in JdbcConnectionProvider, but also redesigning JdbcConnectionConfig. DSQL doesn't require a username and password; instead, it creates a password using ak/sk or profile combined with region and cluster endpoint. Therefore, its parameter requirements for JdbcConnectionConfig are different, potentially changing the original parameter requirements, such as making username and password mandatory in the original design

Yes, this indeed different. How about implements jdbc sink and overwrite key method to let this works?

[cloud456]

Initially, I did consider using JDBC to implement the DSQL sink functionality. However, due to DSQL's unique characteristics that differ from other databases, implementing it based on JDBC might alter existing logic and affect other databases. This would require investing significant time in testing and verification to ensure correct logic without impacting other database functionalities. Currently, I don't have much time to invest in this approach, though perhaps it could be done in the future. And this DSQL sink connector I've submitted is implemented based on real user migration needs, and I hope it can be merged. Thank you!

[Hisoka-X]

Can you provide a screenshot of it running normally?

[Hisoka-X]

We don't need README. We need docs in https://github.com/apache/seatunnel/tree/dev/docs/en/connector-v2/sink

[Hisoka-X]

Also please check these TODO