Conversation
deb9ba7 to
b6099f3
Compare
| interface Props { | ||
| url: string; | ||
| filepath: string; | ||
| domain: string; |
There was a problem hiding this comment.
Using the HSDS lingo is less confusing I think. The domain is still exposed as filepath through the DataContext, though.
| domain: string; | ||
| resetKeys?: unknown[]; | ||
| fetcher: Fetcher; | ||
| fetcher?: Fetcher; |
There was a problem hiding this comment.
The API works with a default basic fetcher, without authentication. This allows testing with the internal HSDS support server instead of Bosquet.
|
|
||
| const response = await fetch(`${url}?${queryParams.toString()}`, { | ||
| ...fetchOpts, | ||
| ...moreFetchOpts, |
There was a problem hiding this comment.
This lets me pass other options like method: 'POST' and headers with each request.
| } from './utils'; | ||
|
|
||
| export class HsdsApi extends DataProviderApi { | ||
| private readonly entities = new Map<string, HsdsEntity<ProvidedEntity>>(); |
There was a problem hiding this comment.
GET / with the h5path query param allows fetching the metadata of an entity at a given path. We don't need to walk the entire tree from the root any more, so we don't need to cache the entities along the way either.
| path: '/', | ||
| name: this.filepath, | ||
| }); | ||
| if (response.class === 'group') { |
There was a problem hiding this comment.
This block handles groups specifically as we need to fetch the metadata of the children. GET /?h5path= with include_links=true gives us the links of the group but doesn't resolve them.
| this.fetchAttributeWithValue(collection, id, attr.name), | ||
| ); | ||
|
|
||
| const attrsWithValues = await Promise.all(attrsPromises); |
There was a problem hiding this comment.
We used to make one value request per attribute!
b6099f3 to
b627d54
Compare
b627d54 to
bb584b1
Compare
This branch has not been deployed
1 participant
I'm rewriting
hsds-api.tsfrom the ground up using the latest API endpoints documented at https://github.com/HDFGroup/hsds/blob/v1.0.1/openapi.ymlBatching
The screenshots below show the number of requests going from 64 to only 2 requests when opening the
sample-hsds.h5sample file:This is thanks to the following endpoints and features:
GET /withh5path,include_linksandinclude_attrsquery params — this combines fetching the domain, then the root group, then the attributes of the group, and then the links of the group into a single requestPOST/withparent_idandinclude_attrsquery params, and{ h5paths : string[] }` request body — this allows us to fetch the metadata of all the children of a group in a single requestGET /{collection}/{id}/attributeswithIncludeData— this allows getting the values of all the attributes of an entity in a single requestTyped arrays
GET /datasets/{id}/valuesupports theAccept: application/octet-streamheader to request a binary response instead of JSON. So we replicate thegetValuelogic fromh5grove-api.tsto return typed arrays when possible.The screenshots below show that we now get true
InfintyandNaNvalues instead of strings, and typed arrays for opaque datasets:Opening as draft, as I'd like to keep testing with other HDF5 files, notably soft/external links, NeXux attributes, binary images, etc.