Skip to content

[Feature] compact_database: detect misconfigured including_tables in combined mode #10453

Description

@p-eye

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

including_tables / excluding_tables of compact_database must match the whole databaseName.tableName.
A short-name pattern such as table_* matches no table, and only divided mode reports it (2.0.0 and master):

  • divided: fails with no tables to be compacted ...
  • combined, batch: finishes successfully without compacting anything
  • combined, streaming: keeps running without compacting anything, with no log above DEBUG

The Flink docs example used this form with mode => 'combined' (see #10452 ).

Solution

  1. combined, batch: fail at build time when no table matches, as divided does. Is there a reason to keep finishing successfully in this case?
  2. combined, streaming: log a WARN from the source when no table matches, mentioning the databaseName.tableName format. Keep running, because a streming job may start before its tables are created.
  3. divided: mention the databaseName.tableName format in the error message.

Anything else?

makes a combined batch job with no matching table fail instead of finishing.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions