Replication Factor in Compute Storage Decoupled Mode

[Nj-kol]

When defining tables, you usually specify a replication factor. Now, this certainly makes sense in a Compute Storage Coupled style, where, if the replication factor is 3, there will be three copies in different BE nodes. However, when it comes to Compute Storage Decoupled mode, where data is stored in S3, what is the relevance of the replication factor?

Is replication factor even necessary while declaring a table?

[httpshirley]

Hi there, thanks for your interest in the Apache Doris Compute-Storage Decoupled Mode. In this mode, the replication factor of 3 is only for the data in the Doris internal table. The number of external data copies is determined by the external data source parameters.

[Nj-kol]

Hi there, thanks for your interest in the Apache Doris Compute-Storage Decoupled Mode. In this mode, the replication factor of 3 is only for the data in the Doris internal table. The number of external data copies is determined by the external data source parameters.

But @httpshirley, even for the internal Doris table, in the compute storage decoupled mode the data ends up being stored on S3, not on the BE nodes' disk. Then what is the relevance of replication factor ( say 3 ), because data is only stored once on S3

[ixzc]

hi @Nj-kol for Compute-Storage Decoupled Mode, you needn't to set replica number, because the high availability is guaranteed by the features of s3 or hdfs itself, not by multiple copies of doris.

[Nj-kol]

@ixzc Thanks for confirming. That is also what I figured. But I think this should be clearly articulated on the Doris document as it serves as a point of confusion for newcomers to the project. Because doris exists in two modes now, storage attached and detached wherever possible, this distinction can be shown

[ixzc]

thanks for your advice. we will consider this