- Reading path through the examples
- Running Cosmolike projects (Basic instructions)
- Baryonic feedback on EXAMPLE_EVALUATE1 and EXAMPLE_EVALUATE3
- Running ML emulators
- Running Hybrid Cosmolike-ML emulators
- Running Fisher
- Unit tests
- Computing covariances
- Exploring notebooks
- Appendix: Which accuracy settings are available?
Start with one evaluation of EXAMPLE_EVALUATE1.yaml
(basic instructions): it checks the
installation that every later example uses.
flowchart TB
A["EXAMPLE_EVALUATE1.yaml: first evaluation"] --> B["EXAMPLE_EVALUATE1.ipynb: cosmic shear and Fisher"]
B --> C["EXAMPLE_EVALUATE2.ipynb: lensing and clustering"]
C --> D["EXAMPLE_EVALUATE3.ipynb: bfmt feedback"]
C --> E["EXAMPLE_EVALUATE_COVARIANCE.ipynb: covariance terms"]
E --> F["compute_covariance.py: production covariance"]
A --> G["EXAMPLE_MCMC1.yaml and EXAMPLE_EMUL_*: chains and minima"]
G --> H["EXAMPLE_EMUL_PROFILE*.py: profiles"]
G --> I["scripts/EXAMPLE_PLOT_*.py: figures"]
H --> I
A --> J["EXAMPLE_EMUL2_*: hybrid examples"]
- The three data-vector notebooks share one fiducial point.
EXAMPLE_EVALUATE1.ipynbintroduces the wrapper calls on cosmic shear,EXAMPLE_EVALUATE2.ipynbadds the lens probes of the 3×2pt data vector, andEXAMPLE_EVALUATE3.ipynbapplies thebfmtfeedback models to that vector. The last one needsbfmtand its emulators installed (baryonic feedback). - The covariance notebook and the production CLI need the optional covariance build (computing covariances).
- The CosmoLike chain and the emulator samplers write chains and minima
under
chains/. The profile scripts read a minimum (--minfile) and, optionally, a chain covariance (--cov); the plotting scripts read the chains, minima and profiles (ML emulators). - The hybrid examples (hybrid emulators) use neither the data-vector emulators nor those chains.
From Cocoa/Readme instructions:
Note
setup_cocoa.sh and compile_cocoa.sh install and compile the cosmolike projects selected in set_installation_options.sh. A commented IGNORE_COSMOLIKE_*_CODE key enables a project; an active key skips it. The defaults enable LSST Y1, DES × Planck and Roman real.
[Adapted from Cocoa/set_installation_options.sh shell script]
#export IGNORE_COSMOLIKE_LSST_Y1_CODE=1
export IGNORE_COSMOLIKE_DES_Y3_CODE=1
#export IGNORE_COSMOLIKE_DESXPLANCK_CODE=1
export IGNORE_COSMOLIKE_ROMAN_FOURIER_CODE=1
#export IGNORE_COSMOLIKE_ROMAN_REAL_CODE=1
export IGNORE_COSMOLIKE_ROMAN_KL_CODE=1
export IGNORE_COSMOLIKE_DES_CLUSTER_CODE=1
# WARNING: des_y6 is not production ready.
export IGNORE_COSMOLIKE_DES_Y6_CODE=1
(...)
export LSST_Y1_URL="https://github.com/CosmoLike/cocoa_lsst_y1.git"
export LSST_Y1_NAME="lsst_y1"
export LSST_Y1_GIT_TAG="v5.05"
Each released project is pinned to a tag. To select another revision, set only one of its GIT_COMMIT, GIT_BRANCH or GIT_TAG keys (here LSST_Y1_GIT_COMMIT, LSST_Y1_GIT_BRANCH or LSST_Y1_GIT_TAG): a commit takes precedence over a branch, and a branch over a tag. With none set, Cocoa clones the repository's default branch.
Note
Rerunning setup_cocoa.sh (or ./installation_scripts/setup_cosmolike_projects.sh) keeps an existing projects/lsst_y1 folder, so uncommitted work survives. To replace the folder with a fresh clone, add the key below to set_installation_options.sh before rerunning; it deletes the folder first.
export OVERWRITE_EXISTING_COSMOLIKE_CODE=1 # dangerous (possible loss of uncommitted work)
Note
If users want to recompile cosmolike, there is no need to rerun the Cocoa general scripts. Instead, run the following three commands:
source start_cocoa.sh
and
source ./installation_scripts/setup_cosmolike_projects.sh
and
source ./installation_scripts/compile_all_projects.sh
or (in case users just want to compile the lsst-y1 project)
source ./projects/lsst_y1/scripts/compile_lsst_y1.sh
Tip
Assuming Cocoa is installed on a local (not remote!) machine, type the command below after step 2️⃣ to run Jupyter Notebooks.
jupyter notebook --no-browser --port=8888
The terminal will then show a message similar to the following template:
(...)
[... NotebookApp] Jupyter Notebook 6.1.1 is running at:
[... NotebookApp] http://f0a13949f6b5:8888/?token=XXX
[... NotebookApp] or http://127.0.0.1:8888/?token=XXX
[... NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Now go to the local internet browser and type http://127.0.0.1:8888/?token=XXX, where XXX is the previously saved token displayed on the line
[... NotebookApp] or http://127.0.0.1:8888/?token=XXX
The project lsst-y1 contains jupyter notebook examples located at projects/lsst_y1.
Note
The example notebooks load their shared support functions from
Cocoa/external_modules/code/cosmolike_core/cosmolike_notebook_utils/:
the CAMB run packaged for cosmolike, the data-vector plots, and the
Fisher-forecast helpers, called through the cnu namespace. What is
specific to this project lives in
interface/cosmolike_lsst_y1_notebook_wrappers.py, imported as nw:
the fiducial point and the probe, bfmt
wrappers around the compiled-interface calls.
To run the example
Step 1️⃣: activate the Cocoa Conda environment, and the private Python environment
conda activate cocoa
and
source start_cocoa.sh
Step 2️⃣: Select the number of OpenMP cores (the commands below set it to 8).
-
Linux
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=close; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1 -
macOS (arm)
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=disabled; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
Step 3️⃣: The folder projects/lsst_y1 contains examples. So, run the cobaya-run on the first example following the commands below.
Warning
(Linux only) In some HPC nodes, numa can cause you problems. If that is the case,
replace numa with slot
-
One model evaluation:
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ cobaya-run ./projects/lsst_y1/EXAMPLE_EVALUATE1.yaml -f -
macOS (arm)
mpirun -n 1 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EVALUATE1.yaml -f
-
-
MCMC (Metropolis-Hastings Algorithm):
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ cobaya-run ./projects/lsst_y1/EXAMPLE_MCMC1.yaml -f -
macOS (arm)
mpirun -n 4 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_MCMC1.yaml -f
-
EXAMPLE_EVALUATE1.yaml can apply an external baryonic feedback suppression to the
matter power spectrum via the bfmt theory block (SP(k), BCEmu, Flamingo, BACCOemu,
or BCemu2025). By default, the example runs without feedback. The notebook
EXAMPLE_EVALUATE3.ipynb compares these models on the 3×2pt data vector
(below).
Step 1️⃣: ensure the lines below are commented out in set_installation_options.sh
before running setup_cocoa.sh and compile_cocoa.sh. By default, these lines should
be commented out, but it is worth checking.
[Adapted from Cocoa/set_installation_options.sh shell script]
#export IGNORE_PYSPK_CODE=1 # SP(k)
#export IGNORE_BCEMU_CODE=1 # BCEmu and BCemu2025
#export IGNORE_FBRE_CODE=1 # FlamingoBaryonResponseEmulator
#export IGNORE_BACCOEMU_CODE=1 # BACCOemu
#export IGNORE_BFMT_CODE=1 # Baryon Feedback Theory Block
Step 2️⃣: in EXAMPLE_EVALUATE1.yaml, uncomment the bfmt theory block and select
the model:
theory:
bfmt:
baryon_model: 2 # 1 = SP(k), 2 = BCEmu, 3 = FlamingoEmulator, 4 = BACCOemu, 5 = BCemu2025
Step 3️⃣: set external_baryon_suppression: True on the lsst_y1.cosmic_shear
likelihood block.
Step 4️⃣: uncomment the selected model's parameters in the params block and in
the sampler: evaluate: override block (the example carries a commented block for each
model).
Note
BACCOemu (baryon_model: 4) rejects this example's fiducial omegab: 0.04, the edge
of its baryon-density training box. Move omegab inside that box before selecting it;
the feedback tests evaluate BACCOemu at omegab: 0.049.
Tip
For the sampled parameters of each model, their validity ranges, and the bfmt
options, see Cocoa/external_modules/code/baryon_suppression/README.md.
EXAMPLE_EVALUATE3.ipynb drives the same bfmt block for the
three SP(k) relations, BCEmu, Flamingo and BCemu2025 at the example's fiducial point;
BACCOemu is left out for the reason above. The notebook builds its own Cobaya models, so it needs the installation keys of
Step one but none of the YAML edits. For each model it applies the suppression
Its section 6 prints the
Cocoa contains a few transformer- and CNN-based neural network emulators capable of simulating the CMB, cosmolike outputs, matter power spectrum, and distances. The scripts below exemplify their API. To run them, keep the following lines commented out in set_installation_options.sh before running setup_cocoa.sh and compile_cocoa.sh; they are commented out by default. The examples marked Planck CMB (l < 396) + SN + BAO + LSST-Y1 combine LSST Y1 cosmic shear with Planck 2018, DES Y5 supernovae and DESI DR2 BAO.
[Adapted from Cocoa/set_installation_options.sh shell script]
# keep the # symbol (i.e., leave these environmental keys unset in `set_installation_options.sh`)
#export IGNORE_EMULTRF_CODE=1 # SaraivanovZhongZhu (SZZ) transformer/CNN-based emulators
#export IGNORE_EMULTRF_DATA=1 # CMB, distance and power networks (Planck + SN + BAO and hybrid examples)
#export IGNORE_PLANCK_LIKELIHOOD_CODE=1 # Planck + SN + BAO examples
#export IGNORE_PLANCK_CMB_DATA=1 # Planck + SN + BAO examples
#export IGNORE_SN_DATA=1 # DES Y5 supernovae
#export IGNORE_BAO_DATA=1 # DESI DR2 BAO
#export IGNORE_NAUTILUS_SAMPLER_CODE=1 # EXAMPLE_EMUL_NAUTILUS[1-2].py
#export IGNORE_POLYCHORD_SAMPLER_CODE=1 # EXAMPLE_EMUL_POLY[1-2].yaml
#export IGNORE_GETDIST_CODE=1 # Nautilus scripts and scripts/EXAMPLE_PLOT_COMPARE_CHAINS_EMUL*.py
Tip
If one of these keys was active when setup_cocoa.sh ran, see the Cocoa
FAQ: How can users compile a single external module (not involving Cosmolike)?
Now, users must follow all the steps below.
Step 1️⃣: Activate the private Python environment by sourcing the script start_cocoa.sh
source start_cocoa.sh
Step 2️⃣: Ensure OpenMP is OFF.
export OMP_NUM_THREADS=1
Step 3️⃣ Run cobaya-run on the first emulator example following the commands below.
-
One model evaluation:
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by slot \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_EVALUATE1.yaml -f -
macOS (arm)
mpirun -n 1 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_EVALUATE1.yaml -f
-
-
MCMC (Metropolis-Hastings Algorithm):
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by slot \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_MCMC1.yaml -r -
macOS (arm)
mpirun -n 4 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_MCMC1.yaml -r
or (Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ )-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by slot \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_MCMC2.yaml -r -
macOS (arm)
mpirun -n 4 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_MCMC2.yaml -r
-
Note
The examples below may require a large number of MPI workers. Before running them, it may be necessary to increase
the limit of threads that can be created (at UofA HPC type ulimit -u 1000000), otherwise users
may encounter the error libgomp: Thread creation failed
-
PolyChord:
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_POLY1.yaml -r -
macOS (arm)
mpirun -n 12 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_POLY1.yaml -r
or (Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ )-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_POLY2.yaml -r -
macOS (arm)
mpirun -n 12 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL_POLY2.yaml -r
-
For multi-node runs, see MPI across nodes.
Note
Note on core oversubscription: an MPI process that is waiting still burns 100% of its
core, checking for messages in a loop. With more processes than cores, this stalls the
processes doing real work. Open MPI usually detects this and makes waiting processes give
up the CPU, but its detection can be fooled. Adding --mca mpi_yield_when_idle 1 forces
that behavior; it is harmless otherwise.
The Nautilus, Minimizer, Profile, and Emcee scripts below contain an internally defined yaml_string that specifies priors,
likelihoods, and the theory code, all following Cobaya Conventions.
-
Nautilus:
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python -m mpi4py.futures ./projects/lsst_y1/EXAMPLE_EMUL_NAUTILUS1.py \ --root ./projects/lsst_y1/ --outroot "EXAMPLE_EMUL_NAUTILUS1" \ --maxfeval 750000 --nlive 2048 --neff 15000 \ --flive 0.01 --nnetworks 5 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python -m mpi4py.futures ./projects/lsst_y1/EXAMPLE_EMUL_NAUTILUS1.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_NAUTILUS1" \ --maxfeval 750000 --nlive 2048 --neff 15000 \ --flive 0.01 --nnetworks 5
The Colab example Test Nautilus illustrates how stable Nautilus results are as a function of
nliveor (Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ )-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python -m mpi4py.futures ./projects/lsst_y1/EXAMPLE_EMUL_NAUTILUS2.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_NAUTILUS2" \ --maxfeval 850000 --nlive 3072 --neff 15000 \ --flive 0.01 --nnetworks 5 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python -m mpi4py.futures ./projects/lsst_y1/EXAMPLE_EMUL_NAUTILUS2.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_NAUTILUS2" \ --maxfeval 850000 --nlive 3072 --neff 15000 \ --flive 0.01 --nnetworks 5
What if the user runs a
Nautiluschain with--maxfevalinsufficient for producingneffsamples?Nautilussaves the chain checkpoint atchains/outroot_checkpoint.hdf5. -
-
Emcee:
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 51 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python ./projects/lsst_y1/EXAMPLE_EMUL_EMCEE1.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_EMCEE1" \ --maxfeval 1000000 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL_EMCEE1.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_EMCEE1" \ --maxfeval 1000000
or (Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ )-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 114 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python ./projects/lsst_y1/EXAMPLE_EMUL_EMCEE2.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_EMCEE2" \ --maxfeval 2000000 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL_EMCEE2.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_EMCEE2" \ --maxfeval 2000000
The number of steps per MPI worker is
$n_{\rm sw} = {\rm maxfeval}/n_{\rm w}$ , with the number of walkers being$n_{\rm w}={\rm max}(3n_{\rm params},n_{\rm MPI})$ .For proper convergence, each walker should traverse 50 times the autocorrelation length (
$\tau$ ), which is provided in the header of the output chain file. A reasonable rule of thumb is to assume$\tau > 200$ and therefore set${\rm maxfeval} > 10,000 \times n_{\rm w}$ . Finally, the script sets burn-in (per walker) at$5 \times \tau$ .With these numbers, users may ask when
Emceeis preferable toMetropolis-Hastings? Here are a few numbers from thePlanck CMB (l < 396) + SN + BAO + LSST-Y1test case.-
MHachieves convergence with$n_{\rm sw} \sim 150,000$ (number of steps per walker), but only requires four walkers. -
Emceehas$\tau \sim 300$ , so it requires$n_{\rm sw} \sim 15,000$ when running with$n_{\rm w}=114$ .
Conclusion:
Emceerequires$\sim 3$ more evaluations in this case, but the number of evaluations per MPI worker (assuming one MPI worker per walker) is reduced by$\sim 10$ . Therefore,Emceeseems well-suited for chains where the evaluation of a single cosmology is time-consuming (and there is no slow/fast decomposition).What if the user runs an
Emceechain with--maxfevalinsufficient for convergence?Emceesaves the chain checkpoint atchains/outroot.h5. -
-
Sampler Comparison
The scripts that generated the plots below are provided at
scripts/EXAMPLE_PLOT_COMPARE_CHAINS_EMUL[2].py.The Google Colab notebooks Example Sampler Comparison (LSST-Y1 only) and Example Sampler Comparison (LSST+Others) can also reconstruct a similar version of these figures.
Another Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ : -
Global Minimizer:
The minimizer reimplements
Procoli, developed by Karwal et al (arXiv:2401.14225).-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 51 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python ./projects/lsst_y1/EXAMPLE_EMUL_MINIMIZE1.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_MIN1" \ --nstw 350 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL_MINIMIZE1.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_MIN1" \ --nstw 350
or (Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ )-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 114 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python ./projects/lsst_y1/EXAMPLE_EMUL_MINIMIZE2.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_MIN2" \ --nstw 750 -
macOS (arm)
mpirun -n 12 \ --oversubscribe python ./projects/lsst_y1/EXAMPLE_EMUL_MINIMIZE2.py \ --root ./projects/lsst_y1/ \ --outroot "EXAMPLE_EMUL_MIN2" \ --nstw 750
The number of steps per Emcee walker per temperature is
$n_{\rm stw}$ , and the number of walkers is$n_{\rm w}={\rm max}(3n_{\rm params},n_{\rm MPI})$ . The minimum number of total evaluations is$3n_{\rm params} \times n_{\rm T} \times n_{\rm stw}$ , which can be distributed among$n_{\rm MPI} = 3n_{\rm params}$ MPI processes for faster results.The scripts that generated the plots below are provided at
scripts/EXAMPLE_PLOT_MIN_COMPARE_CONV[2].pyIn the tests of these examples,
$n_{\rm stw} \sim 200$ worked reasonably well up to$n_{\rm param} \sim \mathcal{O}(10)$ . The case below, with$n_{\rm param} = 38$ , illustrates the need for performing convergence tests on a case-by-case basis.In this example, the total number of evaluations for a reliable minimum is approximately
$319,200$ ($n_{\rm stw} \sim 700$ ), distributed among$n_{\rm MPI} = 114$ processes for faster results. With the use of emulators, such minima can be computed with$\mathcal{O}(1)$ MPI workers. -
-
Profile:
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 51 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python ./projects/lsst_y1/EXAMPLE_EMUL_PROFILE1.py \ --root ./projects/lsst_y1/ --cov 'chains/EXAMPLE_EMUL_MCMC1.covmat' \ --outroot "EXAMPLE_EMUL_PROFILE1" \ --factor 3 --nstw 350 --numpts 10 \ --profile 1 \ --minfile="./projects/lsst_y1/chains/EXAMPLE_EMUL_MIN1.txt" -
macOS (arm)
mpirun -n 51 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL_PROFILE1.py \ --root ./projects/lsst_y1/ \ --cov 'chains/EXAMPLE_EMUL_MCMC1.covmat' \ --outroot "EXAMPLE_EMUL_PROFILE1" \ --factor 3 --nstw 350 --numpts 10 --profile 1 \ --minfile="./projects/lsst_y1/chains/EXAMPLE_EMUL_MIN1.txt"
or (Example with
Planck CMB (l < 396) + SN + BAO + LSST-Y1-$n_{\rm param} = 38$ )-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 114 --oversubscribe \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x OMP_DYNAMIC \ -x ROOTDIR -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES \ -x CLIK_PLUGIN -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH \ -x CLIK_DATA --mca mpi_yield_when_idle 1 --rank-by slot --map-by slot \ --mca pml ob1 --mca btl vader,tcp,self --bind-to core:overload-allowed \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 --report-bindings \ python ./projects/lsst_y1/EXAMPLE_EMUL_PROFILE2.py \ --root ./projects/lsst_y1/ \ --cov 'chains/EXAMPLE_EMUL_MCMC2.covmat' \ --outroot "EXAMPLE_EMUL_PROFILE2" \ --factor 3 --nstw 750 --numpts 10 --profile 1 \ --minfile="./projects/lsst_y1/chains/EXAMPLE_EMUL_MIN2.txt" -
macOS (arm)
mpirun -n 114 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL_PROFILE2.py \ --root ./projects/lsst_y1/ \ --cov 'chains/EXAMPLE_EMUL_MCMC2.covmat' \ --outroot "EXAMPLE_EMUL_PROFILE2" \ --factor 3 --nstw 750 --numpts 10 --profile 1 \ --minfile="./projects/lsst_y1/chains/EXAMPLE_EMUL_MIN2.txt"
The argument
factorspecifies the start and end of the parameter being profiled:start value ~ minimum value - factor*np.sqrt(np.diag(cov)) end value ~ minimum value + factor*np.sqrt(np.diag(cov))Use
${\rm factor} \sim 3$ for parameters that are well constrained by the data when a covariance matrix is provided. Ifcovis not supplied, the code estimates one internally from the prior. If a parameter is poorly constrained orcovis not given, use${\rm factor} \ll 1$ .The script of the plot below is provided at
projects/lsst_y1/scripts/EXAMPLE_PLOT_PROFILE1[2].pyProfile 1:
LSST-Y1 Cosmic Shear onlyThe Google Colab Profile Likelihood (LSST-Y1 only) can be used to reconstruct a similar figure.
Profile 2:
Planck CMB (l < 396) + SN + BAO + LSST-Y1 Cosmic Shear -
Note
These hybrid examples remain experimental. The checks below verify the workflow; assess emulator accuracy and posterior convergence for your analysis.
The EXAMPLE_EMUL2 examples emulate the background expansion and matter
power spectra. CosmoLike still computes the survey projections, bias and
intrinsic-alignment contributions. Changing n(z) or nuisance parameters does
not require retraining a survey data-vector network.
The shared theory networks live in external_modules/data/emultrf. Install
them through the main Cocoa emulator recipe.
These networks assume mnu = 0.06 eV; do not sample neutrino mass. Their
cold-matter power approximation is not a calibrated massive-neutrino halo
model. Check their training range before widening cosmological priors.
We assume Cocoa and this project are installed, the Cocoa Conda environment
is active, the shell is Bash, and the current folder is cocoa/Cocoa/.
Step 1️⃣: activate Cocoa.
source start_cocoa.shStep 2️⃣: Select the number of OpenMP cores (below, we set it to 4).
-
Linux
export OMP_NUM_THREADS=4; export OMP_PROC_BIND=close; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1 -
macOS (arm)
export OMP_NUM_THREADS=4; export OMP_PROC_BIND=disabled; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
Step 3️⃣: remove GPU access on Linux; these examples use the CPU.
export CUDA_VISIBLE_DEVICES=""Step 4️⃣: evaluate the first hybrid example.
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 1 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_EVALUATE1.yaml --force -
macOS (arm)
mpirun -n 1 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_EVALUATE1.yaml --force
The YAML selects the CPU for the distance emulator. Keep BLAS at one thread
per MPI rank (OPENBLAS_NUM_THREADS=1, MKL_NUM_THREADS=1); on macOS also
use VECLIB_MAXIMUM_THREADS=1. The Python sampler entry points set these
BLAS limits before importing numerical libraries.
| Example | Configuration 1 | Configuration 2 |
|---|---|---|
| Fixed evaluation | EXAMPLE_EMUL2_EVALUATE1.yaml | EXAMPLE_EMUL2_EVALUATE2.yaml |
| Cobaya MCMC | EXAMPLE_EMUL2_MCMC1.yaml | EXAMPLE_EMUL2_MCMC3.yaml |
| Annealed minimization | EXAMPLE_EMUL2_MINIMIZE1.py | EXAMPLE_EMUL2_MINIMIZE2.py |
| Parameter profile | EXAMPLE_EMUL2_PROFILE1.py | EXAMPLE_EMUL2_PROFILE2.py |
| Nautilus sampling | EXAMPLE_EMUL2_NAUTILUS1.py | EXAMPLE_EMUL2_NAUTILUS2.py |
Configuration 1 uses lsst_y1.cosmic_shear, NLA, and lsst_y1_M1_GGL0.05.dataset.
Configuration 2 uses lsst_y1.combo_3x2pt, NLA, and lsst_y1_M1_GGL0.05.dataset.
EXAMPLE_EMUL2_MCMC2.yaml adds Planck 2018 CMB
(l < 396 plik TTTEEE, low-ℓ TT and EE), DESI DR2 BAO and DES Y5 supernovae
to configuration 1: the Planck CMB (l < 396) + SN + BAO + LSST-Y1
combination of the emulator examples.
The minimization, profile and Nautilus scripts read the corresponding
EXAMPLE_EMUL2_EVALUATE1.yaml or 2.yaml; --input selects another evaluate
YAML. They require cocoa_hybrid_sampling.py from the matching shared core
revision. They do not maintain separate embedded cosmologies. --check evaluates
the specified fiducial and prints the sampled parameter order without sampling.
Use a new --outroot for each run; these scripts refuse to overwrite results.
With Cocoa activated in Bash from cocoa/Cocoa/, run the first MCMC
example. Use
EXAMPLE_EMUL2_MCMC3.yaml for configuration 2 and EXAMPLE_EMUL2_MCMC2.yaml
for the Planck + BAO + SN combination. Check chain convergence before
interpreting posterior constraints.
Step 1️⃣: Select the number of OpenMP cores (below, we set it to 8).
-
Linux
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=close; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1 -
macOS (arm)
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=disabled; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
Step 2️⃣: remove GPU access on Linux; these examples use the CPU.
export CUDA_VISIBLE_DEVICES=""Step 3️⃣: start Cobaya's hybrid MCMC.
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 4 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_MCMC1.yaml -
macOS (arm)
mpirun -n 4 --oversubscribe \ cobaya-run ./projects/lsst_y1/EXAMPLE_EMUL2_MCMC1.yaml
We assume Cocoa and this project are installed, the Cocoa Conda environment
is active, the shell is Bash, and the current folder is cocoa/Cocoa/.
Step 1️⃣: Select the number of OpenMP cores (below, we set it to 8).
-
Linux
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=close; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1 -
macOS (arm)
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=disabled; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
Step 2️⃣: remove GPU access on Linux; these examples use the CPU.
export CUDA_VISIBLE_DEVICES=""Step 3️⃣: check the hybrid setup before a long run.
python ./projects/lsst_y1/EXAMPLE_EMUL2_MINIMIZE1.py --checkStep 4️⃣: search for a minimum.
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 21 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \ -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \ -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \ -x CLIK_PLUGIN -x CUDA_VISIBLE_DEVICES --mca mpi_yield_when_idle 1 \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ python ./projects/lsst_y1/EXAMPLE_EMUL2_MINIMIZE1.py --nstw 200 --outroot hybrid_min1 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL2_MINIMIZE1.py --nstw 200 --outroot hybrid_min1
Step 5️⃣: profile the first sampled parameter using that saved minimum.
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 21 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \ -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \ -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \ -x CLIK_PLUGIN -x CUDA_VISIBLE_DEVICES --mca mpi_yield_when_idle 1 \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ python ./projects/lsst_y1/EXAMPLE_EMUL2_PROFILE1.py --profile 0 --nstw 200 --numpts 11 --factor 1 --minfile ./projects/lsst_y1/chains/hybrid_min1.json --outroot hybrid_profile1 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL2_PROFILE1.py --profile 0 --nstw 200 --numpts 11 --factor 1 --minfile ./projects/lsst_y1/chains/hybrid_min1.json --outroot hybrid_profile1
Step 6️⃣: run Nautilus as an independent sampling example.
-
Linux
"${CONDA_PREFIX}"/bin/mpirun -n 90 --oversubscribe \ --mca pml ob1 --mca btl vader,tcp,self \ -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX -x ROOTDIR \ -x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \ -x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CLIK_PATH -x CLIK_DATA \ -x CLIK_PLUGIN -x CUDA_VISIBLE_DEVICES --mca mpi_yield_when_idle 1 \ --mca btl_tcp_if_exclude lo,docker0,virbr0,ib0 \ --bind-to core:overload-allowed --report-bindings \ --rank-by slot --map-by numa:pe=${OMP_NUM_THREADS} \ python ./projects/lsst_y1/EXAMPLE_EMUL2_NAUTILUS1.py --nlive 1000 --neff 10000 --maxfeval 100000 --outroot hybrid_nautilus1 -
macOS (arm)
mpirun -n 12 --oversubscribe \ python ./projects/lsst_y1/EXAMPLE_EMUL2_NAUTILUS1.py --nlive 1000 --neff 10000 --maxfeval 100000 --outroot hybrid_nautilus1
The annealed Emcee search follows the DES × Planck template. Its objective
is −2 log posterior, including nuisance and cosmological priors; the
profile is therefore a penalized profile, not a pure likelihood profile.
--nstw sets steps per walker per temperature. More steps and independent
starts are needed to assess whether a minimum is reliable.
--profile accepts a sampled-parameter name or its printed zero-based index.
--factor gives the half-width in proposal standard deviations, clipped to
the prior bounds. --cov accepts a covariance whose header lists the sampled
parameters in order; without it, the prior covariance sets the proposal.
The minimum JSON must come from the same evaluate YAML and parameter order.
Older plain-text minimum files are not accepted. Set any additional priors
in the input YAML; these scripts do not insert hidden cosmological priors.
Nautilus writes weighted GetDist-compatible rows and a JSON convergence
record. Reaching --maxfeval is not convergence. If the budget ends before
any posterior samples are retained, only the checkpoint and a JSON record
with converged: false are saved. Its prior transform uses
Cobaya's one-dimensional prior distributions; external prior factors enter
once as additional log weight. Evidence with unnormalized external priors
has that normalization limitation. These examples do not certify emulator
accuracy or posterior convergence.
Details on the matter power spectrum emulator designs will be presented in the emulator_code repository.
Neural networks generalize the syren-new Eq. 6 of arXiv:2410.14623 formula for the linear power spectrum (w0waCDM with a fixed neutrino mass of
Note
Users can decide not to correct the syren-new formula for the linear power spectrum (flag in the yaml). The caveats of the syren-new approximation have not been studied extensively; it appears sufficient for w0waCDM forecasts when combined with the Euclid Emulator to compute the nonlinear boost.
For back-of-the-envelope LCDM calculations (e.g., to test cosmolike features), users can also choose not to correct the syren-Halofit formula for the LCDM nonlinear boost (see figure below). In this case, the overhead on top of cosmolike computations is minimum, at the order of
Note
Running on more than one node.
- Network interface. TCP must use an interface routable between compute
nodes. A common exclusion list is
--mca btl_tcp_if_exclude lo,docker0,virbr0,ib0; adapt it to the cluster. Keepib0if routable IP-over-InfiniBand is the intended network. These examples exchange parameter vectors and scalar scores, so communication volume is small; actual scaling still depends on the machine. - Environment. Remote ranks need the same Cocoa paths and libraries:
ROOTDIR,PATH,LD_LIBRARY_PATH,PYTHONPATH,CONDA_PREFIX, OpenMP and BLAS settings, andCLIK_PATH/CLIK_DATA/CLIK_PLUGINwhen used. Slurm normally exports the submitting environment (--export=ALL). Explicit-xoptions also forward these variables with SSH launchers. Activate Cocoa before launching; build/download flags do not replace runtime paths. All nodes must see the same files at the same paths. - Slurm geometry. Keep
ntasks-per-node × cpus-per-taskwithin the allocated physical cores per node. SetOMP_NUM_THREADStoSLURM_CPUS_PER_TASKand use--map-by numa:pe=${OMP_NUM_THREADS}. The minimization, profile and Nautilus pool reserves one MPI rank as coordinator; the remaining ranks evaluate the model.
Within a Slurm allocation, first activate Cocoa in Bash on the launch node. The following steps assume Open MPI 4 and shared installation/data paths. Omit optional CLIK exports if those variables are not set.
Step 1️⃣: match OpenMP threads to the scheduler allocation.
export OMP_NUM_THREADS="${SLURM_CPUS_PER_TASK}"; export OMP_PROC_BIND=close; \
export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \
export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
Step 2️⃣: launch the hybrid minimizer across the allocated ranks.
"${CONDA_PREFIX}"/bin/mpirun -n "${SLURM_NTASKS}" \
--mca pml ob1 --mca btl vader,tcp,self \
--mca btl_tcp_if_exclude lo,docker0,virbr0 \
--map-by numa:pe=${OMP_NUM_THREADS} --bind-to core --report-bindings \
-x ROOTDIR -x PATH -x LD_LIBRARY_PATH -x PYTHONPATH -x CONDA_PREFIX \
-x OMP_NUM_THREADS -x OMP_PROC_BIND -x OMP_PLACES -x OMP_DYNAMIC \
-x OPENBLAS_NUM_THREADS -x MKL_NUM_THREADS -x CUDA_VISIBLE_DEVICES \
python ./projects/lsst_y1/EXAMPLE_EMUL2_MINIMIZE1.py --nstw 200 --outroot hybrid_multinodeFor a Planck likelihood add -x CLIK_PATH -x CLIK_DATA -x CLIK_PLUGIN when
those variables are defined. Follow the cluster's MPI module and Slurm
launch policy; do not oversubscribe a production allocation. Outside Slurm,
supply the hosts and slots with the cluster's --hostfile or --host recipe.
Section 18 of the Jupyter notebook projects/lsst_y1/EXAMPLE_EVALUATE1.ipynb computes cosmic-shear Fisher matrices for 17 parameters (five cosmological, two NLA, five source photo-z shifts and five shear calibrations) with five-point-stencil derivatives. It tests how they depend on AccuracyBoost and on the step size, and repeats the AccuracyBoost test with DerivativeKit derivatives. Gaussian priors on the photo-z shifts and shear calibrations enter the Fisher matrix; hard (flat) priors only clip the GetDist contours.
The tests/ folder compares likelihood predictions against stored reference
values, checks for race conditions from OpenMP threading, and measures the
numerical error of the default accuracy settings. It also checks the
accuracy of the hybrid emulated pipelines.
tests/README.md describes the checks, how to run them,
the input data snapshots, and how to refresh them.
Armadillo was chosen to make a convenient Python API for notebook exploration. This C++ library provides vectors, matrices and three-dimensional arrays called cubes. A small interface layer connects them to NumPy through pybind11, with CARMA handling array conversion. The notebooks expose intermediate quantities; production calculations use the CLI interfaces.
We assume Cocoa and this project are installed, the Cocoa Conda environment
is active, the shell is Bash, and the current folder is cocoa/Cocoa/.
Compile the project first; the covariance notebook also needs the optional covariance build described below.
Step 1️⃣: activate Cocoa.
source start_cocoa.shStep 2️⃣: Select the number of OpenMP cores (below, we set it to 8).
-
Linux
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=close; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1 -
macOS (arm)
export OMP_NUM_THREADS=8; export OMP_PROC_BIND=disabled; \ export OMP_PLACES=cores; export OMP_DYNAMIC=FALSE; \ export OPENBLAS_NUM_THREADS=1; export MKL_NUM_THREADS=1
Step 3️⃣: start Jupyter.
jupyter notebook --no-browser --port=8888Step 4️⃣: open the printed URL and choose a notebook below.
Step 5️⃣: select Kernel → Restart Kernel and Run All Cells.
| Notebook | Contents |
|---|---|
| EXAMPLE_EVALUATE1.ipynb | Cosmic shear: baryon_sims), CAMB and CosmoLike accuracy, |
| EXAMPLE_EVALUATE2.ipynb | Galaxy–galaxy lensing ( |
| EXAMPLE_EVALUATE3.ipynb | The bfmt feedback models (three SP(k) relations, BCEmu, Flamingo, BCemu2025) applied to the 3×2pt data vector, their bfmt installed (baryonic feedback). |
| EXAMPLE_EVALUATE_COVARIANCE.ipynb | G, SSC, cNG, total, separate 1h–4h matter trispectra and matrix diagnostics; needs the covariance build. |
Each notebook opens with its model-and-data table and a "Before running" note on requirements and run time; read them before running all cells. Choose the Python kernel from the activated Cocoa environment and restart it after recompiling. The covariance guide explains the forecast files, figures and refinement workflow.
The covariance tools compute Gaussian (G), super-sample (SSC), connected non-Gaussian (cNG) and total matrices before scale cuts. covariance/README.md explains compilation, the production CLI, exploratory notebooks, accuracy settings and output files.
Data-vector options belong to the selected likelihood block. Covariance
options belong to the evaluate YAML's covariance block. They use separate
names and settings; changing one does not refine the other.
| Data-vector setting | What it changes |
|---|---|
accuracyboost |
Overall interpolation-table resolution. |
integration_accuracy |
Quadrature resolution; levels 5/6/7 allow 2048/4096/8192 points per Limber panel. Refine independently of interpolation. |
internal_accuracyboost |
C-FAST-PT convolution grid. |
nonlimber_accuracyboost |
Non-Limber distance sampling. |
pk_z_refinement |
Nested redshift refinement of matter-power inputs. |
lmax |
Real-space angular-transform cutoff, where a real-space transform is used. |
kmax_boltzmann |
Requested Boltzmann power range; coordinate it with the theory settings. |
adopt_limber_gg and adopt_limber_gs choose a projection approximation.
photoz_interpolation_type chooses how n(z) is interpolated, while
photoz_zmid_convention describes the input coordinates. These are modeling
or input-convention choices, not interchangeable accuracy boosts.
photoz_histogram_akima: 0 retains the original midpoint n(z) reconstruction; 1 selects histogram preparation followed by Akima for source and lens samples. photoz_histogram_akima_factor chooses 3 (default), 5 or 7 preparation cells per original uniform bin. See the reconstruction comparisons and area plots. The selected first factor never changes with an accuracy boost.
The default data-vector power grid has 1,500 wavenumbers at boost 1.
Covariance alone uses power_accuracyboost: 8 to prepare 11,993 nodes by
natural cubic interpolation before C linear lookup. Its accuracy_boost
refines tables and cutoffs; its integration_accuracy independently selects
quadrature levels 0–4. See the complete covariance accuracy table.
CAMB's theory.camb.extra_args.AccuracyBoost controls CAMB, not CosmoLike.
Check interpolation, quadrature, input-power sampling and transform cutoffs
separately at fixed cosmology and measurement bins. Narrow n(z) overlaps
particularly require a quadrature check; increasing accuracyboost alone
is not that check. The data-vector test guide
and covariance test guide state what each set of tests
actually verifies. A passing regression or a larger boost is not a general
claim of survey or Fisher convergence.








