[Colin-Cheng]

This change adds MultiZoneConfig support to DataprocCreateClusterOperator (via cluster_config), allowing users to configure multi-zonal clusters across multiple zones within a region during Dataproc cluster creation. This functionality is enabled by bumping the google-cloud-dataproc package to >=5.32.0.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Gemini CLI following the guidelines

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[github-actions]

uv.lock on main just moved via #75689 ("Allow FastAPI 0.137+ by requiring cadwyn 8.1"), commit b70a439 and this PR currently conflicts.

Quickest fix:

git fetch upstream main && git rebase upstream/main
rm uv.lock && uv lock
git add uv.lock && git rebase --continue
git push --force-with-lease

Automated nudge — ignore if you're not ready to rebase. This comment is updated in place on future uv.lock bumps.

[olegkachur-e]

If we pass config just as dict will it fail? Is there strong reason for additional transformation code?

[Colin-Cheng]

Removed the custom MultiZoneConfig dataclass and __init__ conversion from dataproc.py (and the corresponding unit test), keeping the google-cloud-dataproc>=5.32.0 version bump and adding documentation with a multi_zone_config example in providers/google/docs/operators/cloud/dataproc.rst.

[olegkachur-e]

Thank you for the contribution! A few suggestions:

• Version bump is enough: with google-cloud-dataproc>=5.32.0, DataprocCreateClusterOperator already passes cluster_config to the API as is, so {"gce_cluster_config": {"multi_zone_config": {"target_shape": "ANY"}}} (or a native dataproc_v1.MultiZoneConfig) works without operator changes. The MultiZoneConfig dataclass and the conversion in __init__ can be removed.
• Docs: a short feature description and config example in docs/operators/cloud/dataproc.rst, with a link to the official Dataproc docs, would be very welcome.
• System test result: could you please share the result of a system test run with the cluster config modified to use multi_zone_config (e.g. example_dataproc_cluster_deferrable.py)?
• Rebase: please rebase on the latest main and regenerate uv.lock to resolve the conflict.

Thanks!

[Colin-Cheng]

Thanks for the review @olegkachur-e! I have updated the PR to address your comments:

1. Removed MultiZoneConfig dataclass & operator changes: Reverted the custom MultiZoneConfig class and DataprocCreateClusterOperator.__init__ changes in providers/google/src/airflow/providers/google/cloud/operators/dataproc.py (as well as the unit test in test_dataproc.py), relying directly on google-cloud-dataproc>=5.32.0 passing cluster_config through to the API.

2. Documentation: Added a Multi-zone clusters section with a CLUSTER_CONFIG (multi_zone_config) example and links to the official Dataproc MultiZoneConfig and GceClusterConfig documentation in providers/google/docs/operators/cloud/dataproc.rst.

3. System Test (example_dataproc_cluster_deferrable.py with multi_zone_config):
   Ran providers/google/tests/system/google/cloud/dataproc/example_dataproc_cluster_deferrable.py with CLUSTER_CONFIG updated to use multi_zone_config:

   CLUSTER_CONFIG = {
       "gce_cluster_config": {
           "multi_zone_config": {
               "target_shape": "ANY",
           },
       },
       "master_config": {
           "num_instances": 1,
           "machine_type_uri": "n1-standard-4",
           "disk_config": {"boot_disk_type": "pd-standard", "boot_disk_size_gb": 32},
       },
       "worker_config": {
           "num_instances": 2,
           "machine_type_uri": "n1-standard-4",
           "disk_config": {"boot_disk_type": "pd-standard", "boot_disk_size_gb": 32},
       },
       "secondary_worker_config": {
           "num_instances": 1,
           "machine_type_uri": "n1-standard-4",
           "disk_config": {
               "boot_disk_type": "pd-standard",
               "boot_disk_size_gb": 32,
           },
           "is_preemptible": True,
           "preemptibility": "PREEMPTIBLE",
       },
   }

   Test execution result (1 passed):

   providers/google/tests/system/google/cloud/dataproc/example_dataproc_cluster_deferrable.py
   [2026-10-21T19:40:37.211610Z] {dagrun.py:1402} INFO - Marking run <DagRun dataproc_cluster_def @ 2026-01-01 00:00:00+00:00: manual__2026-10-21T19:36:15.093321+00:00, state:running, queued_at: None. run_type: manual> successful
   [2026-10-21T19:40:37.212101Z] {dagrun.py:1453} INFO - DagRun Finished: dag_id=dataproc_cluster_def, logical_date=2026-01-01 00:00:00+00:00, run_id=manual__2026-10-21T19:36:15.093321+00:00, run_start_date=2026-10-21 19:36:15.206363+00:00, run_end_date=2026-10-21 19:40:37.211635+00:00, run_duration=262.005272, state=success, external_trigger=True, run_type=manual, data_interval_start=2026-01-01 00:00:00+00:00, data_interval_end=2026-01-01 00:00:00+00:00,
   ...
   ================== 1 passed, 2 warnings in 677.89s (0:11:17) ===================
   

   All tasks in dataproc_cluster_def (create_cluster -> update_cluster -> delete_cluster) succeeded.

[olegkachur-e]

LGTM 👍
Thank you!

[MaksYermak]

Hello @Colin-Cheng , thank you for the PR!
LGTM, @shahar1 what do you think about this PR?