qed

A web based visualization tool for large rasters

Getting started

If you are reading this from within qed, you should be able to skip to the launching instructions at the end of this section.

Similarly, if you are lucky enough to work on a machine that is managed professionally, qed may already be installed. Please contact the system administrators for instructions about how to access it, and skip to the end of this section.

If you are comfortable with jupyter notebooks, the current release tarball contains a notebook that can walk you through the installation procedure with minimal tweaking. It is located in etc/mamba/qed.ipynb. Please report any difficulties you encounter.

The same tarball has instructions on how to build a docker instance with a qed installation. You will find support for many recent ubuntu distributions in etc/docker/dev.

If none of these options work for you, read on to the next section for a walk through of how to build qed from source.

Dependencies

The easiest way to install the qed dependencies is using micromamba and conda-forge. The following configuration file will install everything qed needs in an environment named qed. Feel free to add these packages to an existing environment, rather than building one from scratch, although it might be easier at first if you avoid any potential conflicts with your existing environment.

# -*- yaml -*-

name: qed

channels:
  - conda-forge

dependencies:
  - binutils
  - python
  - graphene
  - pybind11
  - pyyaml
  # for access to S3 buckets
  - awscli
  - boto3
  # for access to earthdata
  - earthaccess
  # optional: building from source
  # tools
  - git
  - gcc
  - gxx
  - make
  - nodejs
  # external libraries
  - hdf5

# end of file

On macOS, you will want to use clang instead of gcc. Make sure you activate this environment before moving on to build qed so that you can access the conda-forge provided compilers:

~> micromamba activate qed

It is also possible to use other package managers to install these dependencies. Both pyre and qed have been tested on both ubuntu and macOS using their native environments, as well as macports, homebrew, and spack. The automatic dependency discovery described below also supports macports and the debian/ubuntu package manager, via pkgdb: macports and pkgdb: dpkg respectively. Please keep in mind that most enterprise systems provide compilers and packages that are too old to build this code, so you will have to lean on something else to satisfy the dependencies. Further, you may have to adjust your PATH, LD_LIBRARY_PATH, PYTHONPATH, and perhaps other environment variables that control access to binaries, shared objects, and python packages on your machine.

Cloning the repositories

We will need a place to clone the necessary source repositories. For the sake of concreteness, let's pick ~/dv as the source directory.

~> mkdir ~/dv
~> cd ~/dv

GitHub allows access through ssh or https, with slightly different syntax. If you already have your ssh key installed on your GitHub account, you can clone the three repositories using

~/dv> git clone git@github.com:aivazis/mm
~/dv> git clone git@github.com:pyre/pyre
~/dv> git clone git@github.com:aivazis/qed

Alternatively, you can use the https protocol for anonymous access:

~> mkdir ~/dv
~> cd ~/dv
~/dv> git clone https://github.com/aivazis/mm
~/dv> git clone https://github.com/pyre/pyre
~/dv> git clone https://github.com/aivazis/qed

You can probably get away with installing pyre from conda-forge, but qed and pyre are currently evolving together rather quickly. Being tied to a slower release cycle may delay access to the latest features. Generally speaking, the HEAD of all three repositories is a safe place to pull from, as they are thoroughly tested.

Setting up the build system

The build system needs to know two things: where to deposit the build products, and where to find the headers and libraries of the qed dependencies. When you work in a conda environment, mm can figure out both on its own: --mode=conda installs the build products directly into the active environment, and --pkgdb=conda discovers the dependencies by reading the environment's package database. Record these choices in a configuration file so every invocation of mm picks them up:

~/dv> cd ~
~> mkdir -p .config/pyre
~> mkdir -p .config/mm

The configuration for the build system lives in ~/.config/pyre/mm.yaml:

# -*- yaml -*-

# mm configuration
mm:

  # install the build products into the active conda environment
  mode: conda
  # resolve external dependencies by reading the conda package database
  pkgdb: conda

  # targets
  target: "opt, shared"

  # compilers
  compilers: "gcc, python/python3"

# end of file

On macOS, replace gcc with clang. Incidentally, the directory ~/.config/pyre is the home for configuration files for all pyre applications, including qed, so we will be adding more files here later on.

The one dependency conda cannot discover is pyre itself, since we are about to build it from source rather than install it as a conda package. Let mm know where it will end up by placing the following in ~/.config/mm/config.mm:

# -*- Makefile -*-

# external dependencies
# pyre is built from source and installed into the conda environment
pyre.dir := ${CONDA_PREFIX}
# qed uses the pyre hdf5 support, so add {pyre-h5} to the default set of libraries
pyre.libraries := pyre-h5 pyre journal

# end of file

Building

The next step is to build pyre and qed. We will invoke mm a few times, so you may find it convenient to create an alias for it.

~/dv> alias mm='python3 ${HOME}/dv/mm/mm'

You might want to make this more permanent by also adding it to your shell startup file, e.g. your ~/.bash_profile.

Let's verify that everything is ok so far. Let's go to the pyre source directory and ask mm to show details about the build. The first invocation also generates the package database, so mm will report the dependencies it discovered in your environment before printing the directory layout. The output should look similar to the following:

~/dv> cd pyre
~/dv/pyre> mm builder.info
mm (mm.pkgdb):
 -- building conda package database
 --   environment: qed
 --   prefix: /Users/mga/.local/envs/qed
 --   db: /Users/mga/dv/pyre/builds/qed/opt-shared-darwin-arm64/pkg-conda.db
mm (mm.pkgdb):
 -- found 3 of 32 supported packages
 --   hdf5: 2.1.0
 --   pybind11: 3.0.3
 --   python: 3.14

    mm 5.3.0
    Michael Aïvázis <michael.aivazis@para-sim.com>
    copyright 1998-2026 all rights reserved

builder directory layout:
  staging layout:
           tmp = /Users/mga/dv/pyre/builds/qed/opt-shared-darwin-arm64/
  install layout:
        prefix = /Users/mga/.local/envs/qed/
           bin = /Users/mga/.local/envs/qed/bin/
           doc = /Users/mga/.local/envs/qed/doc/
           inc = /Users/mga/.local/envs/qed/include/
           lib = /Users/mga/.local/envs/qed/lib/
         share = /Users/mga/.local/envs/qed/share/
           pyc = /Users/mga/.local/envs/qed/lib/python3.14/site-packages/
~/dv/pyre>

The exact paths and package versions reflect your environment, of course. Make sure the install prefix points to your conda environment, and that hdf5, pybind11, and python show up in the list of discovered packages.

You may see mm download a pyre archive from github to bootstrap the process. This is normal, as mm is itself a pyre application.

If anything goes wrong at this stage that cannot be resolved by retracing your steps looking for typos, please file an issue at the qed repository, and attach a log file or a screenshot to help diagnose the problem.

If everything looks ok, let's build and install pyre. By default, mm launches a parallel build using all the cores on your machine; use --slots=<n> if you'd like to restrict the load:

~/dv/pyre> mm

If all goes well, you will have a functional pyre in the site-packages directory of your python installation. Let's verify:

~/dv/pyre> python3
>>> import pyre
>>> pyre.__file__

Both statements should succeed, and the latter should print out the pyre installation location.

Similarly for qed:

~/dv/pyre> cd ../qed
~/dv/qed> mm builder.info
~/dv/qed> mm

Again, if anything goes wrong, please reach out, and include as much information about the failure as you can gather.

Launching

In conda mode, mm deposits qed in the bin directory of your environment, which is already on your path, so there are no additional steps to make it accessible. If you chose a different installation strategy, you may have to tweak your environment variables.

By default, the application operates as a command line tool:

~/dv/qed> qed

  qed 0.9.5 revision 847415b9
  copyright (c) 1998-2026 all rights reserved

  The main action dispatcher

  authors:
      michael a.g. aïvázis <michael.aivazis@para-sim.com>

  commands:
         s3: access to S3 buckets
    inspect: a collection of introspection utilities
      about: information about this application
     config: configuration information
       info: helpful information
      debug: debugging information

  options:
      --archives: the list of registered data archives [list]
      --datasets: the list of datasets to display [list]
         --views: the initial list of views [list]
        --reader: the reader to use when opening datasets [str]
          --cell: the format string that specifies the type of the dataset payload [qed.datatypes]
        --origin: the smallest possible index [tuple]
         --shape: the shape of the dataset, in (lines x samples) [tuple]
         --lines: set the vertical dimension of the dataset [int]
       --samples: set the horizontal dimension of the dataset [int]
         --uint8: set the cell type to an unsigned 8-bit integer [bool]
        --uint16: set the cell type to an unsigned 16-bit integer [bool]
        --uint32: set the cell type to an unsigned 32-bit integer [bool]
        --uint64: set the cell type to an unsigned 64-bit integer [bool]
          --int8: set the cell type to an 8-bit integer [bool]
         --int16: set the cell type to a 16-bit integer [bool]
         --int32: set the cell type to a 32-bit integer [bool]
         --int64: set the cell type to a 64-bit integer [bool]
       --float32: set the cell type to a 32-bit float [bool]
       --float64: set the cell type to a 64-bit float [bool]
     --complex64: set the cell type to a 64-bit complex [bool]
    --complex128: set the cell type to a 32-bit complex [bool]
       --logfile: file that captures all journal output [path]
         --shell: my hosting strategy [pyre.shells]
         --DEBUG: debugging mode [bool]

Launching it in the browser requires a couple of command line arguments

~/dv/qed> qed --shell=web --shell.auto=yes
qed: web server on '0.0.0.0':51122

This shows that the qed web server has started and is listening for connections at a specific port. The option --shell.auto=yes instructs qed to start a browser automatically and point it to this port. You can make this behavior the default by creating a configuration file. Settings that affect all invocations of qed should be placed in a configuration file at ~/.config/pyre/qed.yaml

# -*- yaml -*-

# launch the web server
shell: web
# connect a new browser window/tab to the qed server automatically
shell.auto: yes

# end of file

Settings from a qed.yaml in the current working directory override these global defaults, so you have complete control over how the application behaves by creating local configuration files next to your data. More details about application configuration, both from the command line and from within configuration files can be found in the section on configuring the app.