Skip to content

Add Quickstart: Run PySpark on Apache Iceberg Tables with Snowpark Connect for Spark - #3734

Merged
sfc-gh-amadan merged 3 commits into
masterfrom
quickstart/scos-iceberg-churn
Nov 20, 2026
Merged

sfc-gh-amadan merged 3 commits into
masterfrom
quickstart/scos-iceberg-churn

Conversation

@sfc-gh-vduraisamy

Copy link
Copy Markdown
Contributor

Summary

Adds a Quickstart that runs an existing PySpark churn pipeline (fictional StreamCo) on Snowflake with Snowpark Connect for Spark, writing Snowflake-managed Apache Iceberg tables. It covers DataFrameWriterV2, day partitions and overwritePartitions(), Iceberg tags, incremental reads and the changelog, time travel, Write-Audit-Publish on branches (preview) and lineage via metadata tables.

Guide

  • Title: Run PySpark on Apache Iceberg Tables with Snowpark Connect for Spark
  • Slug: pyspark-iceberg-churn-snowpark-connect
  • Path: site/sfguides/src/pyspark-iceberg-churn-snowpark-connect/

Test plan

  • Executed every code block in order on a clean account.
  • Verified output messages match reader expectations.
  • Confirmed the role grants block runs cleanly as ACCOUNTADMIN.
  • Confirmed DROP ROLE IF EXISTS SCOS_CHURN_ROLE cleans up in the Clean Up step.

context-ticket: N/A

@sfc-gh-vduraisamy
sfc-gh-vduraisamy requested a review from a team as a code owner November 18, 2026 21:04
@github-actions

Copy link
Copy Markdown
Contributor

Helpful Information

Non-Image Files in Assets Folder

Non-image files found in /assets folders will **NOT** be uploaded to snowflake.com. If you are referencing them in your guide, you should link to them directly (e.g. [using permanent GitHub links](htt…

  • site/sfguides/src/pyspark-iceberg-churn-snowpark-connect/assets/scos-iceberg-churn-demo.ipynb

These are informational messages and will not block your PR from being merged.

@github-actions

Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: pyspark-iceberg-churn-snowpark-connect
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/pyspark-iceberg-churn-snowpark-connect-4d55f342af6b66d1b148791116e3409ee4631ecd

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@github-actions

Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: pyspark-iceberg-churn-snowpark-connect
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/pyspark-iceberg-churn-snowpark-connect-45d9df84a28c5ff91cffc52a2689280c2e644c53

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

@sfc-gh-amadan sfc-gh-amadan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

@sfc-gh-amadan
sfc-gh-amadan merged commit 8ac7142 into master Nov 20, 2026
3 of 4 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

🔗 Staging Preview Ready

Quickstart: pyspark-iceberg-churn-snowpark-connect
Language: en

👉 Preview URL: https://publish-p57963-e462098.adobeaemcloud.com/en/developers/guides/pyspark-iceberg-churn-snowpark-connect-d11c40463f92df6877aba5770d9bd1fe6469fce0

Note: Images are uploaded asynchronously. If images are missing, wait a few minutes and refresh.


Generated by GitHub Actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants