SESHAT

Named after the ancient Egyptian goddess of writing, wisdom, and knowledge.

Two product pipelines around Gardiner-sign Egyptian hieroglyphs, both built to generalize to any symbol inventory / ancient script:

1. Generation (pipelines/generation/) — produce human-looking handwriting samples for any canonical symbol (procedural engine, zero training; or One-DM latent-diffusion style mimicry, fine-tuned).
2. Matching (pipelines/matching/) — recognize a drawn/handwritten symbol against a canonical-glyph inventory (dictionary-app backend; open-set, metric-learning encoder + nearest-prototype index).

Start here: pipelines/README.md for the map and both runbooks.

Results

Full HPC (A100) training, adversarial review and stress testing, and repro commands are in each pipeline's README: pipelines/matching/README.md, pipelines/generation/README.md.

Matching (production recipe: resnet34@160, 100 epochs + missing-stroke augmentation):

• Held-out handwriting retrieval top-1 0.935 / top-5 0.982 (n=7344, group-disjoint)
• Unseen-writer (a person not in the training data) probe top-1 0.840 / top-5 0.984 (n=256); adding stick-figure abstraction augmentation (skeletal renders of all 769 glyphs as extra training data) lifts this to 0.973 at −1.5 pt held-out — the candidate next model, pending confirmation on user-collected drawings.
• Corruption robustness (n=800/case): clean 0.938, scan-cell frame 0.911, shaky-hand wobble 0.866, 28%-block occlusion 0.811, severed/missing strokes 0.718; blur and low-resolution capture remain the sharpest failure modes (~0.33–0.40), and half-finished drawings score 0.11 (a partial-stroke augmentation triples that to 0.35 but costs ~2 pt everywhere else — shipped as an off-by-default flag, --p-partial).

Real held-out handwriting queries (never seen in training), run through the production model. Column 1: the drawn query. Columns 2–6: its top-5 canonical matches by cosine similarity — correct hit outlined green, always ranked first here.

Generation — two complementary engines, not two attempts at the same thing:

• Procedural (skeletonize canonical → re-stroke with wobble/pressure, zero training, CPU, any symbol): deterministic, always on-class, the reliable default.
• One-DM (latent diffusion, style-conditioned, fine-tuned on real handwriting): recognizability of its output is top-1 0.567 / top-5 0.600 on held-out signs which is more indicative of flaws in the generator than in the matcher. A synthetic-data feedback loop (One-DM output → matcher training) was built with anti-collapse guardrails and tested but currently not used due to the flawed-generator (see pipelines/matching/README.md for more details).

Four signs, both engines, same canonical source. Procedural output is deterministic and always on-class every time; One-DM output is real production-model inference, selected for legibility from a wider, lower-quality sample.

Repository layout

Path	Contents
app/	Hieroglyph Dictionary — installable PWA (Android/iOS phone + tablet): draw a sign, get the top-5 canonical matches, tap through to its meaning; all inference on-device (see app/README.md)
pipelines/	The two product pipelines (generation, matching) + showcase/ (README figures above)
One-DM/	Vendored + extended One-DM diffusion handwriting generator (the learned generation engine)
hiero_data/	Datasets: the Hand-drawn Hieroglyph Dataset (handwriting) and the archaeohack-starterpack (canonical glyphs, Gardiner↔Unicode mapping, font, single-writer probe set)
misc/	Shared procedural scripts, portable uv-based environments, project notes (PROJECT_NOTES.md, RESETUP.md, REJECTED_SOFTWARE.md)
slurm/	SLURM job scripts for HPC training/evaluation (reproduction record for the results above)

Setup

bash misc/resetup.sh

Rebuilds all three environments (main scripts, One-DM, matching) relative to wherever the repo lives. See misc/RESETUP.md for details and GPU notes.

What's not in this repo

Large, regenerable artifacts are excluded via .gitignore rather than committed: raw datasets (hiero_data/Hand-drawn Hieroglyph Dataset/, One-DM/data/hiero*/), model weights (One-DM/model_zoo/, One-DM/Saved/, *.pt/*.pth checkpoints, app/data/model.onnx), SLURM job logs (slurm/logs/), and the .venv/tool caches that misc/resetup.sh rebuilds. Small evidence artifacts that make results reproducible/auditable without the raw data — matcher eval/stress JSONs and prototype indexes under pipelines/matching/runs/, and the prepped-dataset metadata under One-DM/data/ — are kept.

Citations and acknowledgements

This project builds on:

One-DM (the diffusion-based generation engine, One-DM/):

@inproceedings{one-dm2024,
  title={One-Shot Diffusion Mimicker for Handwritten Text Generation},
  author={Dai, Gang and Zhang, Yifan and Ke, Quhui and Guo, Qiangya and Huang, Shuangping},
  booktitle={European Conference on Computer Vision},
  year={2024}
}

github.com/dailenson/One-DM · arXiv:2409.04004

Hand-drawn Hieroglyph Dataset (the handwriting corpus, hiero_data/Hand-drawn Hieroglyph Dataset/):

@inproceedings{aneesh2024hieroglyph,
  title={Exploring Hieroglyph Recognition: A Deep Learning Approach},
  author={Aneesh, N. A. and Somasundaram, Anush and Ameen, Azhar and Garimella, Govind Sreekar and Jayashree, R.},
  booktitle={2024 2nd International Conference on Computer, Communication and Control (IC4)},
  year={2024},
  doi={10.1109/IC457434.2024.10486368}
}

IEEE Xplore

archaeohack-starterpack (canonical Gardiner glyphs, Unicode mapping, font): github.com/ArchaeoHack/archaeohack-starterpack

In addition, this project originates from and builds upon work from the ArchaeoHack 2025 hackathon, and has been assisted by Claude Fable 5, Opus 4.8 and Sonnet 5.

Contact

App files are distributed only upon request, for now.

Dion HO: dh3065@columbia.edu