Skip to content

Add avifCalloc() and harden allocation call sites#3210

Open
metsw24-max wants to merge 2 commits into
AOMediaCodec:mainfrom
metsw24-max:allocator-hardening-calloc-sweep
Open

Add avifCalloc() and harden allocation call sites#3210
metsw24-max wants to merge 2 commits into
AOMediaCodec:mainfrom
metsw24-max:allocator-hardening-calloc-sweep

Conversation

@metsw24-max
Copy link
Copy Markdown
Contributor

Introduce avifCalloc() as a centralized overflow-checked, zero-initializing allocation helper and convert existing open-coded allocation patterns to use it.

This change replaces repeated patterns of:

p = avifAlloc(...);
memset(p, 0, ...);

with a single reusable helper that performs allocation arithmetic validation and zero-initialization in one place.

Changes

  • Add new public allocation helper:

    • avifCalloc(size_t count, size_t size)
  • Centralize:

    • multiplication overflow validation
    • zero-initialization behavior
    • allocation failure handling
  • Convert existing allocation sites across multiple files from:

    • avifAlloc(...) + memset(...)
    • avifAlloc(count * sizeof(...))

    to:

    • avifCalloc(...)

Security / Hardening Impact

This is a defensive hardening change and does not modify codec, parsing, or image-processing logic.

The patch reduces reliance on open-coded allocation arithmetic and removes repeated allocation+memset patterns throughout the codebase.

Several converted call sites previously performed count * sizeof(T) calculations directly before allocation. These allocations are now validated centrally through avifCalloc(), preventing unchecked size multiplication from propagating to allocation routines.

Comment thread include/avif/avif.h Outdated
// allocation failure, if either argument is 0, or if count * size overflows size_t.
// Using avifCalloc() instead of avifAlloc() + memset() ensures the multiplication
// is overflow-checked at a single, audited site.
AVIF_API void * avifCalloc(size_t count, size_t size);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move the declaration of avifCalloc() to include/avif/internal.h and remove AVIF_API.

The avifAlloc() and avifFree() functions were exported in the public interface unintentionally. These functions are for internal use by libavif.

Comment thread src/mem.c Outdated

void * avifCalloc(size_t count, size_t size)
{
// Match the contract documented in avif.h: reject zero or overflow rather than
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Please see my comment at line 24 below.) Remove "or overflow" from this comment.

Comment thread src/mem.c Outdated
if (count == 0 || size == 0) {
return NULL;
}
if (count > SIZE_MAX / size) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just call calloc(count, size) here.

If you agree, also remove #include <stdint.h> and #include <string.h>.

Comment thread include/avif/avif.h Outdated
// Allocates count * size bytes and zero-initializes them. Returns NULL on memory
// allocation failure, if either argument is 0, or if count * size overflows size_t.
// Using avifCalloc() instead of avifAlloc() + memset() ensures the multiplication
// is overflow-checked at a single, audited site.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Please change this comment block to the following:

// Allocates count * size bytes and zero-initializes them. Returns NULL on memory
// allocation failure, including the case when count * size overflows size_t.

Comment thread src/codec_svt.c Outdated
goto cleanup;
}
uvPlanes = avifAlloc(uvSize);
uvPlanes = avifCalloc(1, uvSize);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this should be

        uvPlanes = avifCalloc(uvSize, 1);

where 1 could be changed to sizeof(uint8_t).

Comment thread src/gainmap.c Outdated
const int maxNumBuckets = 10000;
const int numBuckets = AVIF_MIN((int)ceilf((max - min) / bucketSize), maxNumBuckets);
int * histogram = avifAlloc(sizeof(int) * numBuckets);
int * histogram = avifCalloc((size_t)numBuckets, sizeof(int));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The size_t cast shouldn't be necessary. Did you get a compiler warning without the size_t cast?

Addressed:
- Move avifCalloc() declaration from the public avif.h to internal.h
  and drop AVIF_API; avifAlloc()/avifFree() were exported in the public
  interface unintentionally and the new helper is for internal use only.
- Update the declaration comment to describe behavior (incl. the
  overflow case) rather than the rationale for the helper.
- Simplify the body to a single call to libc calloc(), which already
  handles count*size overflow per C11. Drop the now-unused <stdint.h>
  and <string.h> includes.
- codec_svt.c: pass uvSize as the element count, sizeof(uint8_t) as
  the element size, matching the count-then-size convention.
- gainmap.c: drop the redundant (size_t) cast on numBuckets.
@metsw24-max
Copy link
Copy Markdown
Contributor Author

metsw24-max commented May 20, 2026

Addressed:

  • Move avifCalloc() declaration from the public avif.h to internal.h and drop AVIF_API; avifAlloc()/avifFree() were exported in the public interface unintentionally and the new helper is for internal use only.
  • Update the declaration comment to describe behavior (incl. the overflow case) rather than the rationale for the helper.
  • Simplify the body to a single call to libc calloc(), which already handles count*size overflow per C11. Drop the now-unused <stdint.h> and <string.h> includes.
  • codec_svt.c: pass uvSize as the element count, sizeof(uint8_t) as the element size, matching the count-then-size convention.
  • gainmap.c: drop the redundant (size_t) cast on numBuckets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants