Repository navigation
fix: prepare earcut payload inside the queued worker task - #230
Conversation
_earcutPolygonData() and _earcutMultiPolygonData() interleaved and cloned the geometry with preparePostMessage(..., true) before queueing the task, then waited on pool.completed(). Every layer waiting for a worker therefore held its own copy of the geometry, and each layer waited for, and could be rejected by, unrelated tasks in a shared pool. Do the interleaving and cloning inside the queued task, and await that task directly. Copies are now limited by the number of active pool tasks, and each layer resolves with its own triangulation. Closes geoarrow#225
Use a real threads Pool with an in-process worker that clones, rehydrates, and triangulates the payload like the earcut worker. For Polygon and MultiPolygon data with interleaved and separated coordinates, check that: - while the only worker is busy, neither getInterleavedPolygon nor preparePostMessage has run - once the worker is released, each runs as expected and the layer gets the correct triangulation - a layer resolves with its own result while another pool task is running Pools are drained after each test.
|
This overlaps with #232, which cancels queued triangulation tasks when a layer is "finalized". As mentioned in that PR body, they both complement eachother. There is a plan for how to handle if #230 merges first, and this would probably be my recommended order. If PR #232 is already merged, rebase #230 on top of main and keep geometry preparation inside the callback, but pass that callback through Don't trust git conflict markers, check the surrounding code as well. Git can retain a direct pool.queue() call outside the conflict markers. If maintainers think both issues these PRs address should be fixed, and this merge-order makes reviewing difficult, I'm happy to combine the PRs into one. When i started out, i thought there would be a cleaner separation between them, but implementations were slightly more entangled than i thought. |
Closes #225
Background
_earcutPolygonData()and_earcutMultiPolygonData()cloned the geometry withpreparePostMessage(..., true)(and interleaved separated coordinates) before callingpool.queue(), then waited onpool.completed().When many record batches create polygon layers faster than earcut can keep up, for example with progressive rendering:
completed()rejects on any task failure, one layer's error could fail another layer's triangulationChange
Interleaving and
preparePostMessagenow run inside the queued task, and each layer awaits the task returned bypool.queue(). Copies are limited by the number of active pool tasks, which is the pool size at the default concurrency of one. Each layer resolves or fails with its own triangulation only.metricstiming is unchanged.Tests
Added
tests/solid-polygon-layer-earcut.test.ts. It uses a realthreadsPool with an in-process worker that clones, rehydrates, and triangulates the payload the same way the earcut worker does. For Polygon and MultiPolygon data, with both interleaved and separated coordinates:getInterleavedPolygonnorpreparePostMessagehas runpreparePostMessageruns once,getInterleavedPolygonruns only for separated coordinates, and the result matches main-thread earcutAll eight tests from 8558c54 fail on
main(the last one by timing out) and pass after applying de1478a.pnpm typecheck,pnpm test, andbiome ci .pass.