ci: bump actions/checkout from 4 to 7 - #3
Open
dependabot[bot] wants to merge 8 commits into
Open
Conversation
Add the standard files a credible open-source project is expected to have, several of which were missing: - LICENSE: full Apache-2.0 text (pyproject already declared Apache-2.0 but no LICENSE file existed, so GitHub reported no license) - CONTRIBUTING.md: dev setup (uv), test and lint commands, PR guidelines - CODE_OF_CONDUCT.md: Contributor Covenant 2.1 - SECURITY.md: private vulnerability reporting policy - CHANGELOG.md: Keep a Changelog format, 0.1.0 plus Unreleased - .github issue forms (bug, feature), chooser config, and PR template - .github/dependabot.yml: weekly pip, github-actions, and docker updates - README badges: CI, license, Python version, ruff No source or test changes; 42 tests still pass and ruff is clean.
Add an opt-in 'flink' Docker Compose profile that runs a small Apache Flink cluster (JobManager + TaskManager) against the SAME Postgres catalog and MinIO warehouse Floe uses, plus a one-shot Flink SQL job that reads a Floe-authored silver table and writes a gold rollup back into the catalog. This demonstrates that Floe's outputs are open, engine-agnostic Iceberg tables and previews the roadmap's Flink-based compute. Floe's own refresh engine remains DuckDB. Because 'docker compose up' must stay unchanged, the Flink services are gated behind a Compose profile and only start with --profile flink. Also make the compute engine setting honest: config.compute.engine now validates its value and rejects unimplemented engines (such as flink) at load time with a clear message, instead of silently falling back to DuckDB. Details: - docker/flink/Dockerfile: stock Flink 2.1 + iceberg-flink-runtime, iceberg-aws-bundle (S3FileIO for MinIO), and the Postgres JDBC driver - docker/flink/run-demo.sh + demo.sql: register Floe's Iceberg catalog and run an engine-agnostic gold rollup - .gitattributes: force LF on shell/Docker assets so the container shebang is not broken by a Windows CRLF checkout - config validation covered by 3 new tests; full suite at 45 passing Note: the Flink profile is validated against the Iceberg/Flink docs and pinned to known-good versions, but has not been run end to end here (no Docker in this environment). The interop to verify on first run is PyIceberg SqlCatalog against Flink JdbcCatalog over the shared Postgres.
… profile Iceberg core references org.apache.hadoop.conf.Configuration when it initializes any catalog, even when all IO goes through S3FileIO, so the experimental Flink profile failed at CREATE CATALOG with a ClassNotFoundException. Bundle flink-shaded-hadoop-2-uber into the image to provide that class. The one-shot SQL container also needs sql-client.execution.result-mode=TABLEAU to render the trailing SELECT in non-interactive mode; set it in demo.sql. With both fixes the profile runs end to end: Flink reads a Floe-authored silver table and writes a gold rollup back into the shared Iceberg catalog.
Add an opt-in large-dataset mode to the quickstart seeder: when FLOE_SEED_DELIVERIES is set, generate that many foreign-key-consistent synthetic delivery rows (with proportional dimensions, safety events, and defects) in DuckDB instead of the small curated dataset. The default path is unchanged. Expose the FLOE_SEED_* knobs through docker-compose.yml and document them in .env.example so the containerized stack can be driven at any scale. Add a benchmarks/ directory with a portable native harness and a README that records reproducible native and containerized numbers for a full DAG materialization.
Add an opt-in `ui` Compose profile with Dozzle, a lightweight read-only web dashboard for watching every container in the project in real time (live logs and CPU / memory stats, grouped by Compose service). It only mounts the Docker socket read-only and is useful when running the engine headless, without Docker Desktop's built-in Compose view. Document all of the live views in the README under "Dashboards and observability": Floe's own `floe watch` terminal dashboard, the MinIO console, the Flink Web UI, and the new container UI, with the URL and how to open each.
Document how to inspect a table's schema, snapshot history, and data files using the `pyiceberg` CLI (already a Floe dependency) pointed at the Compose stack's catalog, plus DuckDB and Trino for SQL previews, per-snapshot row counts, and time travel. No Floe-specific code is required: every option reads the same Iceberg metadata, so the engine stays small.
Bring up a browser-based SQL workbench over the exact Iceberg tables Floe
manages, so they can be queried like a database (ad-hoc SQL, row previews,
snapshot history, time travel) instead of reading raw Parquet blobs in the
MinIO console. Both pieces are off-the-shelf open source and read the SAME
Postgres catalog and MinIO warehouse the rest of the stack uses, with no
Floe-specific code:
- trino: Iceberg connector over the shared JDBC catalog and S3/MinIO. Secrets
come from .env via Trino's built-in ${ENV:...} substitution, and the catalog
name matches Floe's project so it resolves the very same rows PyIceberg wrote.
- metabase: web SQL editor and charts, using its bundled Starburst (Trino)
driver; its app DB persists to a volume so first-run setup survives restarts.
Enable with `docker compose --profile sql up -d` (Metabase on :3000, Trino on
:8080). Documented in the README's "Inspecting a table" and dashboards sections.
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Author
LabelsThe following labels could not be found: Please fix the above issues or remove invalid values from |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps actions/checkout from 4 to 7.
Release notes
Sourced from actions/checkout's releases.
... (truncated)
Changelog
Sourced from actions/checkout's changelog.
... (truncated)
Commits
9c091bbupdate error wording (#2467)1044a6dgetting ready for checkout v7 release (#2464)f028218Bump the minor-npm-dependencies group across 1 directory with 3 updates (#2462)d914b26upgrade module to esm and update dependencies (#2463)537c7efBump@actions/coreand@actions/tool-cacheand Remove uuid (#2459)130a169Bump js-yaml from 4.1.0 to 4.2.0 (#2461)7d09575Bump flatted from 3.3.1 to 3.4.2 (#2460)0f9f3aaBump actions/publish-immutable-action (#2458)f9e715ablock checking out fork pr for pull_request_target and workflow_run (#2454)df4cb1cUpdate changelog for v6.0.3 (#2446)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)