-
Notifications
You must be signed in to change notification settings - Fork 2.1k
[oceanbase] fix regex for change events #1369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Please help review this pr when you are available. @leonardBang @GOODBOY008 @Jiabao-Sun The ci failure is not related to my modification. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @whhe for the excellent work to push this feature forward.
From my point of view, I think we can do some optimizations mentioned in the comments, especially in the scenario of a single database or a single table.
In addition, using not fully qualified table regex does not seem to be able to partially filter the tables with same name but in different databases. Shall we use the fully qualified tableList
like MySQL CDC connector?
CC @leonardBang
...ctor-oceanbase-cdc/src/main/java/com/ververica/cdc/connectors/oceanbase/OceanBaseSource.java
Outdated
Show resolved
Hide resolved
Thanks for your advice. I used regex here at the beginning and so I keep it in this pr. And it make sense to use fully qualified table list, the monitor list can be more accurate in this way. I would change it asap. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @whhe. I left a few comments.
...ctor-oceanbase-cdc/src/main/java/com/ververica/cdc/connectors/oceanbase/OceanBaseSource.java
Show resolved
Hide resolved
.../src/main/java/com/ververica/cdc/connectors/oceanbase/table/OceanBaseTableSourceFactory.java
Show resolved
Hide resolved
Signed-off-by: He Wang <[email protected]>
Signed-off-by: He Wang <[email protected]>
@Jiabao-Sun Docs updated, please take a look when you are available. |
Thanks @whhe for the update. |
.../src/main/java/com/ververica/cdc/connectors/oceanbase/table/OceanBaseTableSourceFactory.java
Show resolved
Hide resolved
@leonardBang Comments addressed. PTAL. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @whhe for the update, LGTM. Btw, the logic in OceanBaseRichSourceFunction
is too long to read, we can split it in the future.
…tables (apache#1369) This closes apache#1369.
fnmatch
is used to match pattern of change events, whileregex
is used for snapshot reading.In this pr, I make the log client to fetch all the data at first and do filter with regex in the process part.