Skip to content

Commit edc376a

Browse files
whhewhhe
committed
add jdbc options and support oracle mode
1 parent c2a6456 commit edc376a

File tree

13 files changed

+467
-40
lines changed

13 files changed

+467
-40
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ This README is meant as a brief walkthrough on the core features of CDC Connecto
1111
|------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
1212
| [mongodb-cdc](docs/content/connectors/mongodb-cdc.md) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.1 |
1313
| [mysql-cdc](docs/content/connectors/mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.27 |
14-
| [oceanbase-cdc](/docs/content/connectors/oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase) (MySQL mode): 2.x, 3.x, 4.x | JDBC Driver: 5.1.4x |
14+
| [oceanbase-cdc](/docs/content/connectors/oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
1515
| [oracle-cdc](docs/content/connectors/oracle-cdc.md) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19 | Oracle Driver: 19.3.0.0 |
1616
| [postgres-cdc](docs/content/connectors/postgres-cdc.md) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12 | JDBC Driver: 42.2.27 |
1717
| [sqlserver-cdc](docs/content/connectors/sqlserver-cdc.md) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 7.2.2.jre8 |

‎docs/content/about.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The CDC Connectors for Apache Flink<sup>®</sup> integrate Debezium as the engin
1111
|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
1212
| [mongodb-cdc](connectors/mongodb-cdc.md) | <li> [MongoDB](https://www.mongodb.com): 3.6, 4.x, 5.0 | MongoDB Driver: 4.3.1 |
1313
| [mysql-cdc](connectors/mysql-cdc.md) | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | JDBC Driver: 8.0.27 |
14-
| [oceanbase-cdc](connectors/oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase) (MySQL mode): 2.x, 3.x, 4.x | JDBC Driver: 5.1.4x |
14+
| [oceanbase-cdc](connectors/oceanbase-cdc.md) | <li> [OceanBase CE](https://open.oceanbase.com): 3.1.x, 4.x <li> [OceanBase EE](https://www.oceanbase.com/product/oceanbase): 2.x, 3.x, 4.x | OceanBase Driver: 2.4.x |
1515
| [oracle-cdc](connectors/oracle-cdc.md) | <li> [Oracle](https://www.oracle.com/index.html): 11, 12, 19 | Oracle Driver: 19.3.0.0 |
1616
| [postgres-cdc](connectors/postgres-cdc.md) | <li> [PostgreSQL](https://www.postgresql.org): 9.6, 10, 11, 12 | JDBC Driver: 42.2.12 |
1717
| [sqlserver-cdc](connectors/sqlserver-cdc.md) | <li> [Sqlserver](https://www.microsoft.com/sql-server): 2012, 2014, 2016, 2017, 2019 | JDBC Driver: 7.2.2.jre8 |

‎docs/content/connectors/oceanbase-cdc(ZH).md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,16 @@ OceanBase CDC 连接器允许从 OceanBase 读取快照数据和增量数据。
1515
</dependency>
1616
```
1717

18+
如果您是要连接企业版的 OceanBase，您可能需要使用 OceanBase 官方的 JDBC 驱动，这时需要引入如下依赖。
19+
20+
```xml
21+
<dependency>
22+
<groupId>com.oceanbase</groupId>
23+
<artifactId>oceanbase-client</artifactId>
24+
<version>2.4.2</version>
25+
</dependency>
26+
```
27+
1828
## 下载 SQL 客户端 JAR 包
1929

2030
```下载链接仅在已发布版本可用，请在文档网站左下角选择浏览已发布的版本。```
@@ -23,6 +33,8 @@ OceanBase CDC 连接器允许从 OceanBase 读取快照数据和增量数据。
2333

2434
**注意:** flink-sql-connector-oceanbase-cdc-XXX-SNAPSHOT 版本是开发分支`release-XXX`对应的快照版本，快照版本用户需要下载源代码并编译相应的 jar。用户应使用已经发布的版本，例如 [flink-sql-connector-oceanbase-cdc-2.3.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oceanbase-cdc) 当前已发布的所有版本都可以在 Maven 中央仓库获取。
2535

36+
对于 JDBC 驱动，上述的 cdc jar 文件中已经包含了我们推荐的 MySQL 驱动版本 5.1.47。由于开源许可证的原因，我们不能在上述 cdc jar 文件中包含 OceanBase 的官方 JDBC 驱动，如果您需要使用它，可以从[这里](https://repo1.maven.org/maven2/com/oceanbase/oceanbase-client/2.4.2/oceanbase-client-2.4.2.jar)下载，然后放到 `<FLINK_HOME>/lib/` 目录下，同时需要将配置项 `jdbc.driver` 设为 `com.oceanbase.jdbc.Driver`。
37+
2638
### 配置 OceanBase 数据库和 oblogproxy 服务
2739

2840
1. 按照 [文档](https://github.com/oceanbase/oceanbase#quick-start) 配置 OceanBase 集群。
@@ -248,6 +260,20 @@ OceanBase CDC 连接器包括用于 SQL 和 DataStream API 的选项，如下表
248260
<td>String</td>
249261
<td>日志代理中 `libobcdc` 的工作模式 , 可以是 `storage` 或 `memory`。</td>
250262
</tr>
263+
<tr>
264+
<td>jdbc.driver</td>
265+
<td>否</td>
266+
<td style="word-wrap: break-word;">com.mysql.jdbc.Driver</td>
267+
<td>String</td>
268+
<td>全量读取时使用的 jdbc 驱动类名。</td>
269+
</tr>
270+
<tr>
271+
<td>jdbc.properties.*</td>
272+
<td>否</td>
273+
<td style="word-wrap: break-word;">无</td>
274+
<td>String</td>
275+
<td>传递自定义 JDBC URL 属性的选项。用户可以传递自定义属性，如 'jdbc.properties.useSSL' = 'false'。</td>
276+
</tr>
251277
</tbody>
252278
</table>
253279
</div>
@@ -396,6 +422,7 @@ public class OceanBaseSourceExample {
396422
.tableName("^test_table$")
397423
.hostname("127.0.0.1")
398424
.port(2881)
425+
.jdbcDriver("com.mysql.jdbc.Driver")
399426
.logProxyHost("127.0.0.1")
400427
.logProxyPort(2983)
401428
.serverTimeZone(serverTimeZone)
@@ -597,3 +624,100 @@ public class OceanBaseSourceExample {
597624
</tbody>
598625
</table>
599626
</div>
627+
628+
### Oracle 模式
629+
630+
<div class="wy-table-responsive">
631+
<table class="colwidths-auto docutils">
632+
<thead>
633+
<tr>
634+
<th class="text-left">OceanBase type</th>
635+
<th class="text-left">Flink SQL type</th>
636+
<th class="text-left">NOTE</th>
637+
</tr>
638+
</thead>
639+
<tbody>
640+
<tr>
641+
<td>NUMBER(1)</td>
642+
<td>BOOLEAN</td>
643+
<td></td>
644+
</tr>
645+
<tr>
646+
<td>NUMBER(p, s <= 0), p - s < 3 </td>
647+
<td>TINYINT</td>
648+
<td></td>
649+
</tr>
650+
<tr>
651+
<td>NUMBER(p, s <= 0), p - s < 5 </td>
652+
<td>SMALLINT</td>
653+
<td></td>
654+
</tr>
655+
<tr>
656+
<td>NUMBER(p, s <= 0), p - s < 10 </td>
657+
<td>INT</td>
658+
<td></td>
659+
</tr>
660+
<tr>
661+
<td>NUMBER(p, s <= 0), p - s < 19 </td>
662+
<td>BIGINT</td>
663+
<td></td>
664+
</tr>
665+
<tr>
666+
<td>NUMBER(p, s <= 0), 19 <=p - s <=38</td>
667+
<td>DECIMAL(p - s, 0)</td>
668+
<td></td>
669+
</tr>
670+
<tr>
671+
<td>NUMBER(p, s > 0)</td>
672+
<td>DECIMAL(p, s)</td>
673+
</tr>
674+
<tr>
675+
<td>NUMBER(p, s <= 0), p - s> 38 </td>
676+
<td>STRING</td>
677+
<td></td>
678+
</tr>
679+
<tr>
680+
<td>
681+
FLOAT<br>
682+
BINARY_FLOAT
683+
</td>
684+
<td>FLOAT</td>
685+
<td></td>
686+
</tr>
687+
<tr>
688+
<td>BINARY_DOUBLE</td>
689+
<td>DOUBLE</td>
690+
<td></td>
691+
</tr>
692+
<tr>
693+
<td>
694+
DATE<br>
695+
TIMESTAMP [(p)]
696+
</td>
697+
<td>TIMESTAMP [(p)]</td>
698+
<td></td>
699+
</tr>
700+
<tr>
701+
<td>
702+
CHAR(n)<br>
703+
NCHAR(n)<br>
704+
VARCHAR(n)<br>
705+
VARCHAR2(n)<br>
706+
NVARCHAR2(n)<br>
707+
CLOB<br>
708+
</td>
709+
<td>STRING</td>
710+
<td></td>
711+
</tr>
712+
<tr>
713+
<td>
714+
RAW<br>
715+
BLOB<br>
716+
ROWID
717+
</td>
718+
<td>BYTES</td>
719+
<td></td>
720+
</tr>
721+
</tbody>
722+
</table>
723+
</div>

‎docs/content/connectors/oceanbase-cdc.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,16 @@ In order to set up the OceanBase CDC connector, the following table provides dep
1616
</dependency>
1717
```
1818

19+
If you want to use OceanBase JDBC driver to connect to the enterprise edition database, you should also include the following dependency in your class path.
20+
21+
```xml
22+
<dependency>
23+
<groupId>com.oceanbase</groupId>
24+
<artifactId>oceanbase-client</artifactId>
25+
<version>2.4.2</version>
26+
</dependency>
27+
```
28+
1929
### SQL Client JAR
2030

2131
```Download link is available only for stable releases.```
@@ -24,6 +34,8 @@ Download [flink-sql-connector-oceanbase-cdc-2.4-SNAPSHOT.jar](https://repo1.mave
2434

2535
**Note:** flink-sql-connector-oceanbase-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users need to download the source code and compile the corresponding jar. Users should use the released version, such as [flink-sql-connector-oceanbase-cdc-2.2.1.jar](https://mvnrepository.com/artifact/com.ververica/flink-sql-connector-oceanbase-cdc), the released version will be available in the Maven central warehouse.
2636

37+
For JDBC driver, the cdc jar above already contains MySQL JDBC driver 5.1.47, which is our recommended version. Due to the license issue, we can not include the OceanBase JDBC driver in the cdc jar. If you need to use it, you can download it from [here](https://repo1.maven.org/maven2/com/oceanbase/oceanbase-client/2.4.2/oceanbase-client-2.4.2.jar) and put it under `<FLINK_HOME>/lib/`, you also need to set the start option `jdbc.driver` to `com.oceanbase.jdbc.Driver`.
38+
2739
Setup OceanBase and LogProxy Server
2840
----------------------
2941

@@ -251,6 +263,20 @@ The OceanBase CDC Connector contains some options for both sql and stream api as
251263
<td>String</td>
252264
<td>Working mode of `obcdc` in LogProxy, can be `storage` or `memory`.</td>
253265
</tr>
266+
<tr>
267+
<td>jdbc.driver</td>
268+
<td>optional</td>
269+
<td style="word-wrap: break-word;">com.mysql.jdbc.Driver</td>
270+
<td>String</td>
271+
<td>JDBC driver class for snapshot reading.</td>
272+
</tr>
273+
<tr>
274+
<td>jdbc.properties.*</td>
275+
<td>optional</td>
276+
<td style="word-wrap: break-word;">(none)</td>
277+
<td>String</td>
278+
<td>Option to pass custom JDBC URL properties. User can pass custom properties like 'jdbc.properties.useSSL' = 'false'.</td>
279+
</tr>
254280
</tbody>
255281
</table>
256282
</div>
@@ -404,6 +430,7 @@ public class OceanBaseSourceExample {
404430
.tableName("^test_table$")
405431
.hostname("127.0.0.1")
406432
.port(2881)
433+
.jdbcDriver("com.mysql.jdbc.Driver")
407434
.logProxyHost("127.0.0.1")
408435
.logProxyPort(2983)
409436
.serverTimeZone(serverTimeZone)
@@ -598,3 +625,100 @@ Data Type Mapping
598625
</tbody>
599626
</table>
600627
</div>
628+
629+
### Oracle Mode
630+
631+
<div class="wy-table-responsive">
632+
<table class="colwidths-auto docutils">
633+
<thead>
634+
<tr>
635+
<th class="text-left">OceanBase type</th>
636+
<th class="text-left">Flink SQL type</th>
637+
<th class="text-left">NOTE</th>
638+
</tr>
639+
</thead>
640+
<tbody>
641+
<tr>
642+
<td>NUMBER(1)</td>
643+
<td>BOOLEAN</td>
644+
<td></td>
645+
</tr>
646+
<tr>
647+
<td>NUMBER(p, s <= 0), p - s < 3 </td>
648+
<td>TINYINT</td>
649+
<td></td>
650+
</tr>
651+
<tr>
652+
<td>NUMBER(p, s <= 0), p - s < 5 </td>
653+
<td>SMALLINT</td>
654+
<td></td>
655+
</tr>
656+
<tr>
657+
<td>NUMBER(p, s <= 0), p - s < 10 </td>
658+
<td>INT</td>
659+
<td></td>
660+
</tr>
661+
<tr>
662+
<td>NUMBER(p, s <= 0), p - s < 19 </td>
663+
<td>BIGINT</td>
664+
<td></td>
665+
</tr>
666+
<tr>
667+
<td>NUMBER(p, s <= 0), 19 <=p - s <=38</td>
668+
<td>DECIMAL(p - s, 0)</td>
669+
<td></td>
670+
</tr>
671+
<tr>
672+
<td>NUMBER(p, s > 0)</td>
673+
<td>DECIMAL(p, s)</td>
674+
</tr>
675+
<tr>
676+
<td>NUMBER(p, s <= 0), p - s> 38 </td>
677+
<td>STRING</td>
678+
<td></td>
679+
</tr>
680+
<tr>
681+
<td>
682+
FLOAT<br>
683+
BINARY_FLOAT
684+
</td>
685+
<td>FLOAT</td>
686+
<td></td>
687+
</tr>
688+
<tr>
689+
<td>BINARY_DOUBLE</td>
690+
<td>DOUBLE</td>
691+
<td></td>
692+
</tr>
693+
<tr>
694+
<td>
695+
DATE<br>
696+
TIMESTAMP [(p)]
697+
</td>
698+
<td>TIMESTAMP [(p)]</td>
699+
<td></td>
700+
</tr>
701+
<tr>
702+
<td>
703+
CHAR(n)<br>
704+
NCHAR(n)<br>
705+
VARCHAR(n)<br>
706+
VARCHAR2(n)<br>
707+
NVARCHAR2(n)<br>
708+
CLOB<br>
709+
</td>
710+
<td>STRING</td>
711+
<td></td>
712+
</tr>
713+
<tr>
714+
<td>
715+
RAW<br>
716+
BLOB<br>
717+
ROWID
718+
</td>
719+
<td>BYTES</td>
720+
<td></td>
721+
</tr>
722+
</tbody>
723+
</table>
724+
</div>

‎flink-connector-mysql-cdc/src/main/java/com/ververica/cdc/connectors/mysql/table/JdbcUrlUtils.java renamed to ‎flink-connector-debezium/src/main/java/com/ververica/cdc/debezium/utils/JdbcUrlUtils.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
* limitations under the License.
1515
*/
1616

17-
package com.ververica.cdc.connectors.mysql.table;
17+
package com.ververica.cdc.debezium.utils;
1818

1919
import java.util.Map;
2020
import java.util.Properties;

‎flink-connector-mysql-cdc/src/main/java/com/ververica/cdc/connectors/mysql/table/MySqlTableSourceFactory.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
import com.ververica.cdc.connectors.mysql.source.offset.BinlogOffset;
3030
import com.ververica.cdc.connectors.mysql.source.offset.BinlogOffsetBuilder;
3131
import com.ververica.cdc.debezium.table.DebeziumOptions;
32+
import com.ververica.cdc.debezium.utils.JdbcUrlUtils;
3233
import org.slf4j.Logger;
3334
import org.slf4j.LoggerFactory;
3435

0 commit comments

Comments
 (0)