fix(data_collection): avoid redundant JPEG re-encode when extracting RGB frames - #175
Open
liuquangao wants to merge 1 commit into
Open
fix(data_collection): avoid redundant JPEG re-encode when extracting RGB frames#175liuquangao wants to merge 1 commit into
liuquangao wants to merge 1 commit into
Conversation
…RGB frames
The RGB camera stream is published via `image_transport republish raw
compressed`, so each `sensor_msgs/CompressedImage` in the bag already carries
JPEG bytes. The extractor nonetheless ran `message_to_cvimage()` (cv2.imdecode)
and then `cv2.imwrite("*.jpg")`, decoding and re-encoding every frame. That is
pure overhead and adds a second lossy JPEG pass.
Add an `rgb_write_mode` option (default "passthrough") that writes the raw
CompressedImage bytes straight to the existing `camera/<frame>/*.jpg` layout,
so both downstream consumers (this extractor's ffmpeg step and
agibot_to_lerobot) keep working unchanged. The legacy decode+re-encode path is
still available via "opencv" for A/B comparison.
Only the CompressedImage path is changed; raw Image, depth and semantic paths
are untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The RGB camera stream is published via
image_transport republish raw compressed, so everysensor_msgs/CompressedImagein the bag already carries JPEG bytes. When extracting, the code nonetheless ranmessage_to_cvimage()(cv2.imdecode) and thencv2.imwrite("*.jpg")on each frame — decoding and re-encoding everyframe. This is pure overhead and adds a second lossy JPEG pass on top of the one image_transport already performed.
Change
Add an
rgb_write_modeoption (defaultpassthrough) that writes the raw CompressedImage bytes straight into the existingcamera/<frame>/*.jpglayout, so both downstream consumers (this extractor's ffmpeg step andagibot_to_lerobot) keep working unchanged. The legacy decode + re-encode path remains available viargb_write_mode: "opencv".Only the CompressedImage path is touched; the raw Image, depth and semantic paths are unchanged.
Benchmark
Depth-enabled episode, 578 frames, 5 RGB + 3 depth cameras, extractor run inside the data_collection container (medians consistent across repeats).
write_frames: 39.95 s → 16.89 s (2.36× faster, −58%), ~23 s saved.The absolute RGB saving (~23 s) is independent of depth; with depth off the same change cuts
write_framesfrom 25.2 s to 1.0 s. Here depth PNG writing (~16 s) is a shared floor in both modes, which is why the ratio is smaller but still large.