Clang PGO profiles can be overwritten when PIDs are reused #160380

Bug report

CPython's Clang PGO profiles are named code-%p.profclangr. If a PID is reused during training, a later profile can overwrite an earlier one before llvm-profdata merge runs.

I have not observed lost profiles in a real-world build. This is a defense-in-depth proposal.

In python-build-standalone, we use code-%128m.profclangr instead. LLVM merges profiles into a bounded file pool with locking. I'd like to use the same pattern in CPython; see astral-sh/python-build-standalone#1034.

The clanker wrote a reproduction: The overwrite behavior can be reproduced with Clang/LLVM 18 on Linux using the program below. It enters main five times under one PID, writing a profile before each exec. The %p profile records a main function count of 1; %128m records 5.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int __llvm_profile_write_file(void);
int main(int argc, char **argv) {
    int remaining = atoi(argv[1]);
    if (remaining > 0) {
        char next[32];
        snprintf(next, sizeof(next), "%d", remaining - 1);
        if (__llvm_profile_write_file() != 0) return 2;
        unsetenv("__LLVM_PROFILE_RT_INIT_ONCE");
        execl(argv[0], argv[0], next, (char *)0);
        return 3;
    }
    return 0;
}

clang-18 -fprofile-instr-generate profile.c -o profile
mkdir profiles-pid profiles-merge
LLVM_PROFILE_FILE='profiles-pid/code-%p.profraw' ./profile 4
LLVM_PROFILE_FILE='profiles-merge/code-%128m.profraw' ./profile 4
llvm-profdata-18 merge profiles-pid/*.profraw -o pid.profdata
llvm-profdata-18 merge profiles-merge/*.profraw -o merged.profdata
llvm-profdata-18 show --counts --all-functions pid.profdata
llvm-profdata-18 show --counts --all-functions merged.profdata

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux