Skip to content

Commit d828335

Browse files
donislawdevclaude
andcommitted
format/png: the first compressed format, exact to the byte
PNG is where the output size stops following from the content. What lands on disk is whatever deflate decides, so the exact size comes from a padding chunk that makes up the difference. The padding channel was measured before it was built, because the architecture document had recorded it as an open question that had to be measured rather than assumed. Five independent decoders - Pillow, FFmpeg, Tcl/Tk, the Windows Imaging Component and exiftool - all read an image with an extra chunk between IDAT and IEND, and all return identical pixels. The chunk is private rather than tEXt. Both work everywhere, but exiftool reports "Text/EXIF chunk(s) found after PNG IDAT" for tEXt, and a warning in the tool a tester checks with is what makes them think the generator is broken. That is the same cost already recorded for 7Z. A private chunk draws no comment from any of the five. Everything before the closing chunk streams out while being counted, so nothing is buffered whatever the picture size. Planning encodes once to learn the exact size, which is what lets an unreachable size be refused before any file exists - measured at about 176 ms for a 3840x2160 gradient and a couple of milliseconds at the default 640x480. PNG has sizes it cannot reach and that is a property of the format, not a defect. The smallest is 73 bytes, the next reachable is 83, and 74 to 82 cannot be hit because the smallest possible chunk costs 12 bytes on its own. Measured by scanning every size from 0 to 400. Asking for one of them gets an error naming both neighbours. That finding reshaped the size guard into the rule it actually protects: exact or an error, never a different size in silence. It now also walks sizes below the declared minimum, so the code that refuses is exercised rather than assumed. Mutation then found two holes in the guards themselves. The seed test only ran with the label on, and the label carries the seed in its text, so a generator could have ignored the seed everywhere else and still passed. And nothing ever asked for a size below the minimum. Both closed. The label is drawn with a 3 by 5 bitmap font written here rather than taken from a font file - no dependency, no licence to check, identical pixels everywhere. At three pixels wide M, N and W stay similar, which is recorded rather than pretended away. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent ef2e752 commit d828335

8 files changed

Lines changed: 738 additions & 21 deletions

File tree

‎CHANGELOG-DEV.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,45 @@ configuration, architectural decisions. Anything a user would notice goes to
66

77
## Unreleased
88

9+
### 2026-08-01 - the first compressed format
10+
11+
- `png` is the first generator where the output size does not follow from the
12+
content. Everything before the closing chunk streams straight out while
13+
being counted, then a padding chunk makes up the difference. Nothing is
14+
buffered, whatever the picture size.
15+
- **The padding channel was measured before it was built.** Five independent
16+
decoders - Pillow, FFmpeg, Tcl/Tk, the Windows Imaging Component and
17+
exiftool - all read an image with an extra chunk between IDAT and IEND and
18+
return identical pixels. This had been recorded as an open question that
19+
had to be measured rather than assumed, and it was.
20+
- **A private chunk rather than tEXt.** Both work everywhere, but exiftool
21+
reports "Text/EXIF chunk(s) found after PNG IDAT" for tEXt. A warning in
22+
the checking tool is what makes a tester think the generator is broken,
23+
which is the same cost already recorded for 7Z.
24+
- **PNG has unreachable sizes and that is a property of the format.** The
25+
smallest is 73 bytes, the next reachable is 83, and 74 to 82 cannot be hit
26+
because the smallest possible chunk costs 12 bytes. Measured by scanning
27+
every size from 0 to 400.
28+
- The exact size guard was restated as the rule it actually protects: exact
29+
or an error, never a different size in silence. It now also walks sizes
30+
below the declared minimum, so the refusal path is exercised rather than
31+
assumed.
32+
- **Mutation found two holes in the guards themselves.** The seed test only
33+
ran with the label on, and the label carries the seed in its text - so a
34+
generator could ignore the seed everywhere else and still pass. And no test
35+
ever asked for a size below the minimum, so the code that refuses could
36+
have been deleted unnoticed. Both closed.
37+
- `imagelabel` draws a 3 by 5 bitmap font written here rather than taken from
38+
a font file. No dependency, no licence to check, and identical pixels on
39+
every machine. Known limit: at three pixels wide M, N and W stay similar.
40+
- `--set key=value` is repeatable and maps one to one onto the properties
41+
block of a recipe.
42+
- Measured while building this: a gradient compresses to 0.1% of its raw
43+
size, noise to 75%. A 3840x2160 gradient encodes in about 176 ms, and
44+
writing it to disk instead of counting it costs nothing measurable - so the
45+
temp file option would have bought time it does not buy and cost disk space
46+
it does not have.
47+
948
### 2026-08-01 - review after the first format
1049

1150
A read through everything written so far, before starting the second format.

‎CHANGELOG.md‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,15 @@ because it turns other people's test suites red.
1212

1313
### Added
1414

15-
- `tfg generate` produces text files of an exact size. Ask for 10 485 761
16-
bytes and you get exactly that, from 0 bytes upward.
15+
- `tfg generate` produces **text and PNG files** of an exact size. Ask for
16+
10 485 761 bytes and you get exactly that.
17+
- PNG carries the label burned into the picture, and its size is made exact
18+
by a private chunk that decoders ignore. Checked against Pillow, FFmpeg,
19+
Tcl/Tk, the Windows Imaging Component and exiftool - all five read the
20+
image, the pixels are unchanged and none of them reports a warning.
21+
- `--set` sets a format property and can be repeated:
22+
`--set width=1920 --set height=1080`. Leave it out and the picture size is
23+
chosen to suit the file size you asked for.
1724
- `tfg formats` lists what this build supports, with the fidelity level, the
1825
determinism level, the minimum size and the padding channel of each format.
1926
- Every run writes a `manifest.json` next to the files - path, size, SHA-256,
@@ -57,5 +64,10 @@ because it turns other people's test suites red.
5764
never a silent difference from what you ordered.
5865
- **The last word can be clipped.** Hitting the exact size wins over ending
5966
on a word boundary.
67+
- **A few PNG sizes cannot be reached.** The smallest PNG is 73 bytes and the
68+
next reachable size is 83, because the chunk that makes up any difference
69+
costs 12 bytes on its own. Anything from 83 bytes upward works. Asking for
70+
a size in between gets an error naming both neighbours - never a file of a
71+
different size.
6072
- Text files are UTF-8 with LF line endings. Other encodings and line endings
6173
are not implemented yet.

‎internal/cli/cli.go‎

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,13 @@ func generate(ctx context.Context, args []string, out, errOut io.Writer) int {
9797
asJSON = fs.Bool("json", false, "write the manifest to standard output")
9898
)
9999

100+
// One repeatable flag rather than a separate flag per format property.
101+
// Twenty five formats with a dozen properties each would give a surface
102+
// nobody reads in --help, and this maps one to one onto the properties
103+
// block of a recipe, so both surfaces speak the same words.
104+
props := propertyFlag{}
105+
fs.Var(&props, "set", "format property, repeatable: --set width=1920 --set height=1080")
106+
100107
fs.Usage = func() {
101108
fmt.Fprint(errOut, "tfg generate - produce files.\n\nFlags:\n")
102109
fs.PrintDefaults()
@@ -133,13 +140,14 @@ func generate(ctx context.Context, args []string, out, errOut io.Writer) int {
133140
}
134141

135142
target := engine.Target{
136-
ID: *id,
137-
Format: *formatID,
138-
Count: *count,
139-
Bytes: bytesWanted,
140-
NameTmpl: *name,
141-
Label: !*clean,
142-
Expected: *expected,
143+
ID: *id,
144+
Format: *formatID,
145+
Count: *count,
146+
Bytes: bytesWanted,
147+
NameTmpl: *name,
148+
Label: !*clean,
149+
Expected: *expected,
150+
Properties: props,
143151
}
144152
opt := engine.Options{
145153
OutDir: *outDir,
@@ -263,6 +271,25 @@ func classify(err error) int {
263271
return ExitRuntime
264272
}
265273

274+
// propertyFlag collects repeated --set key=value pairs.
275+
type propertyFlag map[string]string
276+
277+
func (p propertyFlag) String() string { return "" }
278+
279+
func (p propertyFlag) Set(v string) error {
280+
key, value, found := strings.Cut(v, "=")
281+
if !found || key == "" {
282+
return fmt.Errorf("expected key=value, got %q", v)
283+
}
284+
if _, exists := p[key]; exists {
285+
// Setting the same property twice is a mistake worth naming. One of
286+
// the two values would be lost, and nobody would know which.
287+
return fmt.Errorf("%s is set more than once", key)
288+
}
289+
p[key] = value
290+
return nil
291+
}
292+
266293
func args2(args []string) []string {
267294
out := []string{"generate"}
268295
return append(out, args...)

‎internal/format/all/all.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,6 @@ package all
88

99
import (
1010
// Each format registers itself when its package is loaded.
11+
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/png"
1112
_ "github.com/donislawdev/TestingFilesGenerator/internal/format/txt"
1213
)

‎internal/format/imagelabel/font.go‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
package imagelabel
2+
3+
// A 3 by 5 pixel font, drawn here rather than taken from a font file.
4+
//
5+
// Two reasons. A font file is a dependency with a licence to check, and this
6+
// project answers that question often enough already. And a bitmap font is
7+
// exact - the same label renders to the same pixels on every machine and
8+
// every version, which the byte stability promise needs.
9+
//
10+
// Letters are drawn in upper case only. At three pixels wide, lower case is
11+
// not legible, and the label is metadata rather than content.
12+
//
13+
// Each glyph is five rows of three columns. A '#' is a lit pixel.
14+
15+
const (
16+
glyphWidth = 3
17+
glyphHeight = 5
18+
)
19+
20+
var glyphs = map[rune]string{
21+
'A': "###" + "#.#" + "###" + "#.#" + "#.#",
22+
'B': "##." + "#.#" + "##." + "#.#" + "##.",
23+
'C': "###" + "#.." + "#.." + "#.." + "###",
24+
'D': "##." + "#.#" + "#.#" + "#.#" + "##.",
25+
'E': "###" + "#.." + "##." + "#.." + "###",
26+
'F': "###" + "#.." + "##." + "#.." + "#..",
27+
'G': "###" + "#.." + "#.#" + "#.#" + "###",
28+
'H': "#.#" + "#.#" + "###" + "#.#" + "#.#",
29+
'I': "###" + ".#." + ".#." + ".#." + "###",
30+
'J': "..#" + "..#" + "..#" + "#.#" + "###",
31+
'K': "#.#" + "#.#" + "##." + "#.#" + "#.#",
32+
'L': "#.." + "#.." + "#.." + "#.." + "###",
33+
'M': "#.#" + "###" + "###" + "#.#" + "#.#",
34+
'N': "#.#" + "###" + "###" + "###" + "#.#",
35+
'O': "###" + "#.#" + "#.#" + "#.#" + "###",
36+
'P': "###" + "#.#" + "###" + "#.." + "#..",
37+
'Q': "###" + "#.#" + "#.#" + "###" + "..#",
38+
'R': "###" + "#.#" + "##." + "#.#" + "#.#",
39+
'S': "###" + "#.." + "###" + "..#" + "###",
40+
'T': "###" + ".#." + ".#." + ".#." + ".#.",
41+
'U': "#.#" + "#.#" + "#.#" + "#.#" + "###",
42+
'V': "#.#" + "#.#" + "#.#" + "#.#" + ".#.",
43+
'W': "#.#" + "#.#" + "###" + "###" + "#.#",
44+
'X': "#.#" + "#.#" + ".#." + "#.#" + "#.#",
45+
'Y': "#.#" + "#.#" + ".#." + ".#." + ".#.",
46+
'Z': "###" + "..#" + ".#." + "#.." + "###",
47+
48+
'0': "###" + "#.#" + "#.#" + "#.#" + "###",
49+
'1': ".#." + "##." + ".#." + ".#." + "###",
50+
'2': "###" + "..#" + "###" + "#.." + "###",
51+
'3': "###" + "..#" + "###" + "..#" + "###",
52+
'4': "#.#" + "#.#" + "###" + "..#" + "..#",
53+
'5': "###" + "#.." + "###" + "..#" + "###",
54+
'6': "###" + "#.." + "###" + "#.#" + "###",
55+
'7': "###" + "..#" + "..#" + "..#" + "..#",
56+
'8': "###" + "#.#" + "###" + "#.#" + "###",
57+
'9': "###" + "#.#" + "###" + "..#" + "###",
58+
59+
' ': "..." + "..." + "..." + "..." + "...",
60+
'-': "..." + "..." + "###" + "..." + "...",
61+
'.': "..." + "..." + "..." + "..." + ".#.",
62+
':': "..." + ".#." + "..." + ".#." + "...",
63+
'_': "..." + "..." + "..." + "..." + "###",
64+
'/': "..#" + "..#" + ".#." + "#.." + "#..",
65+
'+': "..." + ".#." + "###" + ".#." + "...",
66+
'#': "#.#" + "###" + "#.#" + "###" + "#.#",
67+
}
68+
69+
// unknown stands in for any character with no glyph, so a label never
70+
// silently loses part of itself.
71+
const unknown = "###" + "#.#" + "#.#" + "#.#" + "###"
72+
73+
func glyph(r rune) string {
74+
if r >= 'a' && r <= 'z' {
75+
r -= 'a' - 'A'
76+
}
77+
if g, ok := glyphs[r]; ok {
78+
return g
79+
}
80+
return unknown
81+
}
Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
// Package imagelabel draws the self describing label into pixels.
2+
//
3+
// Seven image formats burn the same label, so they share one rasteriser
4+
// rather than carrying seven copies of it.
5+
package imagelabel
6+
7+
import (
8+
"image"
9+
"image/color"
10+
"image/draw"
11+
)
12+
13+
// Padding around the text inside its band, in scaled pixels.
14+
const pad = 2
15+
16+
// Draw burns text into the top left of img and returns the height of the band
17+
// it used.
18+
//
19+
// The band gets a solid background so the text stays readable whatever the
20+
// picture underneath looks like. Without that, a label over a bright gradient
21+
// is invisible in exactly the screenshot somebody attaches to a bug report.
22+
//
23+
// It draws what fits and stops. A picture too small for the label is not a
24+
// reason to fail - the caller decides whether to ask for a label at all, and
25+
// says so in the manifest when it is left out.
26+
func Draw(img draw.Image, text string) int {
27+
b := img.Bounds()
28+
if b.Dx() <= 0 || b.Dy() <= 0 || text == "" {
29+
return 0
30+
}
31+
32+
scale := scaleFor(b.Dx(), len(text))
33+
if scale == 0 {
34+
return 0
35+
}
36+
37+
bandH := glyphHeight*scale + 2*pad
38+
if bandH > b.Dy() {
39+
bandH = b.Dy()
40+
}
41+
42+
band := image.Rect(b.Min.X, b.Min.Y, b.Max.X, b.Min.Y+bandH)
43+
draw.Draw(img, band, image.NewUniform(color.RGBA{R: 16, G: 16, B: 16, A: 255}), image.Point{}, draw.Src)
44+
45+
ink := color.RGBA{R: 240, G: 240, B: 240, A: 255}
46+
x := b.Min.X + pad
47+
for _, r := range text {
48+
if x+glyphWidth*scale > b.Max.X {
49+
break
50+
}
51+
drawGlyph(img, glyph(r), x, b.Min.Y+pad, scale, ink)
52+
x += (glyphWidth + 1) * scale
53+
}
54+
return bandH
55+
}
56+
57+
// scaleFor picks the largest whole scale at which the whole label fits across
58+
// the image. Whole numbers only, so a glyph is never resampled and the result
59+
// is identical on every machine.
60+
func scaleFor(width, chars int) int {
61+
if chars <= 0 {
62+
return 0
63+
}
64+
needed := chars*(glyphWidth+1) + 2*pad
65+
for s := 4; s >= 1; s-- {
66+
if needed*s <= width {
67+
return s
68+
}
69+
}
70+
return 0
71+
}
72+
73+
func drawGlyph(img draw.Image, g string, x, y, scale int, ink color.Color) {
74+
for row := 0; row < glyphHeight; row++ {
75+
for col := 0; col < glyphWidth; col++ {
76+
if g[row*glyphWidth+col] != '#' {
77+
continue
78+
}
79+
for dy := 0; dy < scale; dy++ {
80+
for dx := 0; dx < scale; dx++ {
81+
img.Set(x+col*scale+dx, y+row*scale+dy, ink)
82+
}
83+
}
84+
}
85+
}
86+
}
87+
88+
// Fits says whether a label would be drawn at all at this width. The caller
89+
// needs to know before planning, so that a picture too small to carry one is
90+
// reported rather than quietly unlabelled.
91+
func Fits(width, chars int) bool {
92+
return scaleFor(width, chars) > 0
93+
}

0 commit comments

Comments
 (0)