Skip to content

fix(data_collection): avoid redundant JPEG re-encode when extracting RGB frames - #175

Open
liuquangao wants to merge 1 commit into
AgibotTech:mainfrom
liuquangao:fix/rgb-passthrough
Open

fix(data_collection): avoid redundant JPEG re-encode when extracting RGB frames#175
liuquangao wants to merge 1 commit into
AgibotTech:mainfrom
liuquangao:fix/rgb-passthrough

Conversation

@liuquangao

Copy link
Copy Markdown
Contributor

Problem

The RGB camera stream is published via image_transport republish raw compressed, so every sensor_msgs/CompressedImage in the bag already carries JPEG bytes. When extracting, the code nonetheless ran message_to_cvimage() (cv2.imdecode) and then cv2.imwrite("*.jpg") on each frame — decoding and re-encoding every
frame. This is pure overhead and adds a second lossy JPEG pass on top of the one image_transport already performed.

Change

Add an rgb_write_mode option (default passthrough) that writes the raw CompressedImage bytes straight into the existing camera/<frame>/*.jpg layout, so both downstream consumers (this extractor's ffmpeg step and agibot_to_lerobot) keep working unchanged. The legacy decode + re-encode path remains available via rgb_write_mode: "opencv".

Only the CompressedImage path is touched; the raw Image, depth and semantic paths are unchanged.

Benchmark

Depth-enabled episode, 578 frames, 5 RGB + 3 depth cameras, extractor run inside the data_collection container (medians consistent across repeats).

Case RGB handling MP4 write_frames total CPU (user+sys) Output
opencv decode + re-encode off 39.95 s 49.11 s 168.5 s 2.065 GiB
passthrough (this PR) off 16.89 s 26.23 s 76.6 s 2.060 GiB
  • write_frames: 39.95 s → 16.89 s (2.36× faster, −58%), ~23 s saved.
  • total extraction (video off): 49.11 s → 26.23 s (1.87× faster, −47%).
  • CPU time: 168.5 s → 76.6 s.
  • Output size essentially unchanged (2.065 → 2.060 GiB), confirming no quality regression from removing the second lossy JPEG pass.
  • On the full default pipeline (MP4 on) end-to-end drops 68.8 s → 45.7 s (1.51×).

The absolute RGB saving (~23 s) is independent of depth; with depth off the same change cuts write_frames from 25.2 s to 1.0 s. Here depth PNG writing (~16 s) is a shared floor in both modes, which is why the ratio is smaller but still large.

…RGB frames

The RGB camera stream is published via `image_transport republish raw
compressed`, so each `sensor_msgs/CompressedImage` in the bag already carries
JPEG bytes. The extractor nonetheless ran `message_to_cvimage()` (cv2.imdecode)
and then `cv2.imwrite("*.jpg")`, decoding and re-encoding every frame. That is
pure overhead and adds a second lossy JPEG pass.

Add an `rgb_write_mode` option (default "passthrough") that writes the raw
CompressedImage bytes straight to the existing `camera/<frame>/*.jpg` layout,
so both downstream consumers (this extractor's ffmpeg step and
agibot_to_lerobot) keep working unchanged. The legacy decode+re-encode path is
still available via "opencv" for A/B comparison.

Only the CompressedImage path is changed; raw Image, depth and semantic paths
are untouched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant