Skip to content

Conversation

Chiro11
Copy link
Contributor

@Chiro11 Chiro11 commented Jun 10, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR makes the per-database barrier_interval_ms and checkpoint_frequency settings work.

Following up on PR #22052 which set up the API and storage, this change modifies the GlobalBarrierWorker (especially in PeriodicBarriers) in meta service to schedule and inject barriers according to each database's specific configuration. Databases without custom settings will fall back to the global SystemParams.

To verify the new scheduling logic, focused unit tests have been added to the PeriodicBarriers component.

Related Issues

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • I have checked the Release Timeline and Currently Supported Versions to determine which release branches I need to cherry-pick this PR into.

Documentation

  • My PR needs documentation updates.
Release note

Users can now configure barrier_interval_ms and checkpoint_frequency for each database individually, providing more granular control over latency and resource consumption within a single RisingWave cluster.

Previously, these settings were global, forcing all databases to share the same configuration. This change is particularly impactful for users running diverse workloads. If no specific values are set for a database, it will automatically use the global settings defined in SystemParams, ensuring backward compatibility and a seamless experience for existing users.

You can configure these settings using the following SQL commands:

On creation:

CREATE DATABASE <db_name> WITH barrier_interval_ms = <value> checkpoint_frequency = <value>;

For existing databases:

ALTER DATABASE <db_name> SET barrier_interval_ms = <value>;
ALTER DATABASE <db_name> SET checkpoint_frequency = <value>;

To revert to the global default:

ALTER DATABASE <db_name> SET barrier_interval_ms = DEFAULT;
ALTER DATABASE <db_name> SET checkpoint_frequency = DEFAULT;

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for per-database barrier intervals and checkpoint frequencies by extending DDL, catalog controller, and the barrier scheduler.

  • DDL controller now notifies the global barrier manager on database creation and alteration.
  • Catalog controller methods return the internal database model for barrier configuration.
  • The barrier scheduler/store is refactored to maintain per-database timer streams and state updates.

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/meta/src/rpc/ddl_controller.rs Notify GlobalBarrierManager on create/alter database ops
src/meta/src/controller/catalog/create_op.rs Change create_database to return the database model
src/meta/src/controller/catalog/alter_op.rs Change alter_database_param to return the database model
src/meta/src/barrier/worker.rs Bootstrap and recovery use per-database PeriodicBarriers
src/meta/src/barrier/schedule.rs Refactor scheduling to use IntervalStream per database
src/meta/src/barrier/mod.rs Add UpdateDatabaseBarrier request variant
src/meta/src/barrier/manager.rs Expose update_database_barrier API
src/meta/src/barrier/context/recovery.rs Include database_infos in recovery snapshot
src/common/src/catalog/mod.rs Derive Clone/Debug for AlterDatabaseParam
Comments suppressed due to low confidence (2)

src/meta/src/barrier/schedule.rs:357

  • Per-database scheduling logic is introduced here; consider adding unit tests to cover PeriodicBarriers::new and next_barrier to validate interval streams and checkpoint logic for multiple databases.
pub(super) fn new(

src/meta/src/barrier/manager.rs:90

  • Public API update_database_barrier is added; please add a Rustdoc comment describing its purpose, parameters, and failure modes to aid future maintainers.
pub fn update_database_barrier(

@Chiro11 Chiro11 requested review from Li0k and wenym1 June 11, 2025 09:45
@Chiro11 Chiro11 force-pushed the chiro/barrier_per_db_loop branch from 7f4dba8 to 2ca8d2a Compare June 12, 2025 16:58
@Chiro11 Chiro11 requested a review from wenym1 June 16, 2025 05:28
Copy link
Contributor

@wenym1 wenym1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to handle the unresolved comments.

@wenym1 wenym1 added the user-facing-changes Contains changes that are visible to users label Jun 16, 2025
@wenym1
Copy link
Contributor

wenym1 commented Jun 16, 2025

Please also update the release note in the Documentation section of the PR description, since this PR is a user facing change.

@Chiro11 Chiro11 requested a review from wenym1 June 16, 2025 11:39
Copy link
Contributor

@wenym1 wenym1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

The added unit tests look flaky. May need some way to make it more stable.

@Chiro11 Chiro11 force-pushed the chiro/barrier_per_db_loop branch from affa33b to 200d132 Compare June 16, 2025 13:22
@Chiro11 Chiro11 enabled auto-merge June 16, 2025 13:23
@Chiro11 Chiro11 force-pushed the chiro/barrier_per_db_loop branch from 19091bc to 3911042 Compare June 16, 2025 15:39
@Chiro11 Chiro11 added this pull request to the merge queue Jun 16, 2025
Merged via the queue into main with commit b4a6931 Jun 16, 2025
31 of 32 checks passed
@Chiro11 Chiro11 deleted the chiro/barrier_per_db_loop branch June 16, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: allow different configurations for different database/resource group
2 participants