Support MongoDB CDC connector

[wuchong]

Debezium supports to capture change data from MongoDB. It should be possible to support MongoDB CDC connector based on the DebeziumSourceFunction.

[dongdongking008]

I'm afraid it is not enough:

https://debezium.io/documentation/reference/connectors/mongodb.html#mongodb-update-events

In MongoDB’s oplog, update events do not contain the before or after states of the changed document. Consequently, it is not possible for a Debezium connector to provide this information. However, a Debezium connector provides a document’s starting state in create and read events. Downstream consumers of the stream can reconstruct document state by keeping the latest state for each document and comparing the state in a new event with the saved state. Debezium connector’s are not able to keep this state.

So, we also need Change Streams.

After database snapshot phase, use Change Streams to capture streaming change data from MongoDB.

[FrancisChan-dev]

when flink-cdc-connect can support mongodb?

[chethanuk]

use Change Streams to capture streaming change data from MongoDB

There is already an Offical Change Streams Kafka Connector, But format it's Mongo's CSE Document Format

[Jiabao-Sun]

WIP