[SPARK-59033][INFRA][DOCS] Coverage badges at branch root + fix udfs page rendering

.github/workflows/coverage.yml

@@ -173,14 +173,6 @@ jobs:
           print(f"Python coverage: {coverage_pct:.1f}% ({color})")
           PYTHON_EOF
 
-      - name: Ensure badges directory exists
-        run: mkdir -p badges
-
-      - name: Copy badges to output directory
-        run: |
-          cp coverage-rust-badge.json badges/coverage-rust.json
-          cp coverage-python-badge.json badges/coverage-python.json
-
       - name: Check if badges branch exists
         id: check-badges-branch
         run: |
@@ -209,13 +201,12 @@ jobs:
             git rm -rf .
           fi
 
-          # Copy new badges into the branch
-          mkdir -p badges
-          cp coverage-rust-badge.json badges/coverage-rust.json
-          cp coverage-python-badge.json badges/coverage-python.json
+          # Copy new badges into the branch root (not in a subdirectory)
+          cp coverage-rust-badge.json coverage-rust.json
+          cp coverage-python-badge.json coverage-python.json
 
           # Commit and push
-          git add badges/
+          git add coverage-rust.json coverage-python.json
           git commit -m "ci: update coverage badges" || echo "No changes to commit"
           git push -f origin badges
 
@@ -225,6 +216,6 @@ jobs:
         with:
           name: coverage-badges
           path: |
-            badges/coverage-rust.json
-            badges/coverage-python.json
+            coverage-rust-badge.json
+            coverage-python-badge.json
           retention-days: 5

README.md

@@ -20,9 +20,10 @@
 # Spark Connect Rust Client
 
 A fast, native **Rust** client for **Apache Spark Connect** - and a drop-in
-`pyspark` replacement. It builds `spark.connect` protobuf plans, manages the gRPC
-channel, and decodes Arrow results in Rust, speaking the same protocol and
-returning the same results as the reference client.
+`pyspark` replacement with **100% public-API parity with PySpark 4.2.0**. It
+builds `spark.connect` protobuf plans, manages the gRPC channel, and decodes
+Arrow results in Rust, speaking the same protocol and returning the same results
+as the reference client.
 
 [![PyPI](https://img.shields.io/pypi/v/pyspark-client-rust?color=c2410c&label=pyspark-client-rust)](https://pypi.org/project/pyspark-client-rust/)
 ![Spark](https://img.shields.io/badge/Apache%20Spark-4.2.0%2B-c2410c)
@@ -84,45 +85,22 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
 See the [documentation](https://apache.github.io/spark-connect-rust/) for the full
 API, running a Spark Connect server, and more.
 
-## API Parity & Drop-in Replacement
-
-The Python drop-in (`pyspark-client-rust`) achieves **100% public-API parity with
-PySpark 4.2.0**. The Rust client (crate `apache-spark-connect`) has the same
-parity and supports the complete DataFrame, SQL, Streaming, Catalog, and Type
-system. The two internal APIs intentionally absent (`Column.to_plan` and
-`SparkSession.client`) are implementation details and not part of the public
-interface.
-
 ## User-Defined Functions (UDFs)
 
-Rust UDFs are compiled to WebAssembly, shipped to the executors, and can be used
-via the DataFrame API or registered by name for SQL. The runner is cloudpickled
-by value, so executors need only the `wasmtime` Python package:
+Write UDFs as plain Rust functions and call them directly — `#[spark_wasm_udf]`
+compiles them to WebAssembly and ships them to the executors:
 
 ```rust
-use spark_connect::functions::col;
-use spark_connect::wasm_udf::{udf, AbiType};
-
-fn main() -> Result<(), Box<dyn std::error::Error>> {
-    let spark = SparkSession::builder().remote("sc://localhost:15002").get_or_create()?;
-
-    // Load a prebuilt module: fn shout(s: String) -> String
-    let wasm = std::fs::read("shout.wasm")?;
-    let shout = udf("shout", wasm, "shout", vec![AbiType::Str], AbiType::Str);
-
-    // DataFrame API:
-    let df = spark.range(0, 3)?.select(vec![shout.call(vec![col("id").cast_str("string")])?])?;
-    df.show(20)?;
-
-    // ...or register by name and call from SQL (mirrors spark.udf.register):
-    spark.udf().register("shout", &shout)?;
-    spark.sql("SELECT shout(name) FROM people")?.show(20)?;
-    Ok(())
+#[spark_wasm_udf]
+mod udfs {
+    pub fn add_one(x: i64) -> i64 { x + 1 }
 }
+
+spark.range(5)?.select(vec![udf::add_one(col("id"))?]).show(20)?;
 ```
 
 See the [WASM UDF guide](https://apache.github.io/spark-connect-rust/udfs/) for
-building modules with the `spark_wasm_udf` macro and more examples.
+the full setup, SQL registration, supported types, and more.
 
 ## Contributing
 

docs/udfs.md

@@ -14,11 +14,11 @@ toolchain server-side.
 
 Annotate a module of functions with `#[spark_wasm_udf]`. For each function the
 macro infers the Spark signature from the Rust types, exports it to WASM, and
-generates a constructor under `udf::` - a direct call `udf::<name>(col0, col1,
-...)` (one column per argument, **arity checked at compile time**) that returns the
+generates a constructor under `udf::` - a direct call `udf::<name>(col0, col1, ...)`
+(one column per argument, **arity checked at compile time**) that returns the
 result `Column`. A one-line `build.rs` compiles the module.
 
-```rust,ignore
+```{.rust .no-run}
 // src/main.rs
 use spark_connect_macros::spark_wasm_udf;
 
@@ -52,7 +52,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
 }
 ```
 
-```rust,ignore
+```{.rust .no-run}
 // build.rs
 fn main() {
     spark_connect_build::embed_wasm_udf("src/main.rs");
@@ -77,7 +77,6 @@ Run it:
 
 ```bash
 rustup target add wasm32-unknown-unknown            # once
-export SPARK_CONNECT_WASM_PACKER_PATH=$PWD/python    # so the client finds the packer
 cargo run
 ```
 
@@ -103,8 +102,7 @@ Arguments and results cross the WASM boundary with a length-prefixed binary ABI
 | `Vec<T>`    | `ArrayType` (of `T`) |
 | `Option<T>` | nullable `T`         |
 
-These nest arbitrarily - e.g. `Vec<Option<String>>` -> `ArrayType(StringType,
-nullable)`.
+These nest arbitrarily - e.g. `Vec<Option<String>>` -> `ArrayType(StringType, nullable)`.
 
 ## How it works
 
@@ -143,7 +141,7 @@ Nothing here is needed unless the `wasm-udf` feature is enabled:
 For non-deterministic UDFs or custom packer configuration, the macro also
 generates a builder `udf::<name>_udf()`:
 
-```rust,ignore
+```{.rust .no-run}
 udf::add_one_udf()
     .as_nondeterministic()
     .call(vec![col("id")])?;
@@ -152,7 +150,7 @@ udf::add_one_udf()
 If you'd rather load a prebuilt module and spell out the types yourself, the
 lower-level factory mirrors `pyspark.sql.functions.udf`:
 
-```rust,ignore
+```{.rust .no-run}
 use spark_connect::wasm_udf::{udf, AbiType};
 
 let wasm = std::fs::read("shout.wasm")?;

scripts/check_doc_snippets.py

@@ -27,8 +27,13 @@
 ``rustc`` against the already-built ``apache-spark-connect`` rlib. Only compilation
 (type-checking) is performed - snippets are never executed, so no server is needed.
 
-A block that must be skipped (pseudo-code, or a deliberate compile-error example)
-can be fenced as ```rust,ignore``` and it is not checked.
+A block that must be skipped (pseudo-code, a deliberate compile-error example, or
+a snippet that needs a feature/macro the snippet harness doesn't build - e.g. the
+`wasm-udf` `#[spark_wasm_udf]` examples) should be fenced as ```{.rust .no-run}```.
+That brace form still renders as a Rust code block in mkdocs/superfences (unlike
+```rust,ignore```, whose comma makes superfences drop the fence entirely and print
+the backticks literally), and it is not matched by the checker below, so it is not
+compiled.
 
 Usage:
     cargo build -p apache-spark-connect
@@ -48,8 +53,8 @@
 RLIB = REPO / "target" / "debug" / "libspark_connect.rlib"
 DEPS = REPO / "target" / "debug" / "deps"
 
-# ```rust``` (checked) vs ```rust,ignore``` / ```rust,no_run``` (skipped). We only
-# check plain ```rust``` fences.
+# ```rust``` (checked) vs ```{.rust .no-run}``` (rendered but skipped). We only check
+# plain ```rust``` fences; any info string with a comma/space/brace is left alone.
 _BLOCK = re.compile(r"^```rust\s*$\n(.*?)^```\s*$", re.DOTALL | re.MULTILINE)
 
 # Harness: bind the identifiers the fragments assume from earlier blocks. `use`