Skip to content

fix(linux/vulkan): preserve host aspect ratio in encoder output#5130

Merged
ReenigneArcher merged 3 commits into
LizardByte:masterfrom
hebo6:master
May 27, 2026
Merged

fix(linux/vulkan): preserve host aspect ratio in encoder output#5130
ReenigneArcher merged 3 commits into
LizardByte:masterfrom
hebo6:master

Conversation

@hebo6
Copy link
Copy Markdown
Contributor

@hebo6 hebo6 commented May 16, 2026

Description

Fixes a behavior inconsistency between the VAAPI and Vulkan encoders on Linux: when the encoder target resolution differs in aspect from the captured source, the VAAPI path letterboxes the source into the encoded frame while the Vulkan path stretches it to fill.

Screenshot

Issues Fixed or Closed

Roadmap Issues

Type of Change

  • feat: New feature (non-breaking change which adds functionality)
  • fix: Bug fix (non-breaking change which fixes an issue)
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Other changes that don't modify src or test files
  • revert: Reverts a previous commit
  • BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Code has been commented, particularly in hard-to-understand areas
  • Code docstring/documentation-blocks for new or existing methods/components have been added or updated
  • Unit tests have been added or updated for any new or modified functionality

AI Usage

  • None: No AI tools were used in creating this PR
  • Light: AI provided minor assistance (formatting, simple suggestions)
  • Moderate: AI helped with code generation or debugging specific parts
  • Heavy: AI generated most or all of the code changes

@hebo6 hebo6 changed the title vulkan: preserve host aspect ratio in encoder output fix(linux/vulkan): preserve host aspect ratio in encoder output May 16, 2026
@ReenigneArcher ReenigneArcher added the ai PR has signs of heavy ai usage (either indicated by user or assumed) label May 16, 2026
@ReenigneArcher

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor

@neatnoise neatnoise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the original author of the Vulkan encoder path — this looks correct and clean. Single dispatch, no extra passes or sync, negligible overhead.

A few notes:

  1. rgb2yuv.comp L43-47: The black value computation assumes range_y/range_uv are (1,0). Worth a comment noting this assumption so it doesn't break silently if those values change in the future.

  2. vulkan_encode.cpp L115-116: Push constant struct layout changed (fields inserted in the middle) — fine since both C++ and GLSL are updated together, but worth being aware of for any future out-of-tree shader variants.

  3. vulkan_encode.cpp L290-293: The & ~1 rounding on eff_w/eff_h and offsets could theoretically leave a 1-pixel gap if frame dimensions are odd. Might want to assert or clamp that dst_offset + eff_size <= frame_size.

@ReenigneArcher ReenigneArcher added this to the vulkan milestone May 20, 2026
@hebo6
Copy link
Copy Markdown
Contributor Author

hebo6 commented May 25, 2026

I clarified the black-fill comment to describe that it uses the normal RGB=(0,0,0) conversion formula, and clamped the aspect-fit size after even alignment so dst_offset + dst_size stays within the full frame while preserving UV-plane alignment.

The push constant layout is updated on both the C++ and GLSL sides in this PR, so I think that part is okay as-is.

@ReenigneArcher ReenigneArcher requested a review from neatnoise May 26, 2026 02:57
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Bundle Report

Bundle size has no change ✅

@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.79%. Comparing base (87182c9) to head (4fde9aa).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/platform/linux/vulkan_encode.cpp 0.00% 17 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #5130     +/-   ##
=========================================
  Coverage   17.79%   17.79%             
=========================================
  Files         111      111             
  Lines       24143    24154     +11     
  Branches    10687    10688      +1     
=========================================
+ Hits         4296     4299      +3     
- Misses      15364    16532   +1168     
+ Partials     4483     3323   -1160     
Flag Coverage Δ
Archlinux 11.18% <0.00%> (-0.01%) ⬇️
FreeBSD-aarch64 ?
FreeBSD-amd64 13.30% <0.00%> (-0.03%) ⬇️
Homebrew-ubuntu-22.04 13.52% <0.00%> (-0.02%) ⬇️
Linux-AppImage 12.10% <0.00%> (-0.01%) ⬇️
Windows-AMD64 14.85% <ø> (ø)
Windows-ARM64 13.19% <ø> (+<0.01%) ⬆️
macOS-arm64 18.86% <ø> (ø)
macOS-x86_64 18.34% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/platform/linux/vulkan_encode.cpp 0.32% <0.00%> (-0.01%) ⬇️

... and 54 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87182c9...4fde9aa. Read the comment docs.

Copy link
Copy Markdown
Contributor

@neatnoise neatnoise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine for me

Copy link
Copy Markdown
Member

@ReenigneArcher ReenigneArcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one nitpic

Comment thread src_assets/linux/assets/shaders/vulkan/rgb2yuv.comp Outdated
@sonarqubecloud
Copy link
Copy Markdown

@ReenigneArcher ReenigneArcher merged commit 3c7952b into LizardByte:master May 27, 2026
73 of 74 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai PR has signs of heavy ai usage (either indicated by user or assumed)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants