GeoZarr Extension

[christophenoel]

Dear Community,

In the context of an European Space Agency (ESA) GSTP (General Support Technology Programme) we prepared a first draft for a geospatial extension to Zarr (GeoZarr).

This specification is an early draft (v0.1) and a prototype implementation of GeoZarr-based (and Cloud based) geospatial data store has been demonstrated to the Agency in the frame of the project. We are also developing a map viewer (based on OpenLayers) to demonstrate multiscaling and integration of the store in an Earth Observation catalogue.

If you have any interest in Space domain, we would be very interested to get your feedback and ideas to consolidate this extension. We mostly based the specification on what xarray already supports.

Here is the repository: https://github.com/christophenoel/geozarr-spec/blob/main/geozarr-spec.md

Kind regards,

Christophe Noël, Spacebel.

[joshmoore]

First off, big 👍 for this, @christophenoel.

an alternative channel to share this information

Probably https://github.com/zarr-developers/zarr-specs would be best for the moment. Happy to transfer if there are no objections. (I'll point out though that as with #50 it may be that portions are intended for the core spec while portions are more an extension or a community convention)

This specification is an early draft (v0.1) ...

It looks like there's also some overlap with upcoming proposals for NGFF (the proposal that developed out of #50) like use of UDUNITS (ome/ngff#57 (comment)) which is re-assuringly complementary.

We are also developing ...

Links welcome.

If you have any interest in Space domain, we would be very interested to get your feedback and ideas to consolidate this extension. We mostly based the specification on what xarray already supports.

👍 A push to better unify xarray and zarr is just getting underway so this is really great timing. Thanks again!

cc: @jhamman @aurghs etc.

[rabernat]

This is a great effort @christophenoel!

I am leading the Zarr OGC community standard effort. The google document linked below is a "preamble" which will soon be voted on by OGC - https://docs.google.com/document/d/199yucgyJbU3tTZcO1lEbCEWHIvkUXoR1/edit?usp=sharing&ouid=102969561847050549657&rtpof=true&sd=true. It mentions geospatial data in several ways. I would love to have your comments.

It would be great to align efforts. I will try to review your spec over the next week.

[jhamman]

+1 to transferring this issue to zarr-specs

And +1 to hearing more about the map viewer work (perhaps that could use its own issue).

[christophenoel]

Thanks a lot for your friendly encouragement.

Some feedback:

• Multiscale: I used draft 50 ideas as a baseline. It was migrated to image.sc but misses the geospatial aspects. From my point of view, resolution property is relevant for geospatial data to ease the computation of the scale according to zoom level (instead of computing the value from the coordinates variables)
• Demonstrators: the dissemination phase is not started yet but presentations are planned with the Agency and related organization such as the OGC. I will keep you inform if any public meeting is scheduled during the next months. In particular, we have worked with ScanWorld company who developed real use cases examples consuming the (serverless) cloud-native data store. I will share the link if they become available publicly.

[christophenoel]

I am leading the Zarr OGC community standard effort. The google document linked below is a "preamble" which will soon be voted on by OGC - https://docs.google.com/document/d/199yucgyJbU3tTZcO1lEbCEWHIvkUXoR1/edit?usp=sharing&ouid=102969561847050549657&rtpof=true&sd=true. It mentions geospatial data in several ways. I would love to have your comments.

Very interesting. I have read very carefully the outcome from the COG/Zarr report from OGC Testbed 17. I keep in touch with you.

[jakirkham]

Have gone ahead and transferred this issue to zarr-specs since it seems like we are ok with that

[joshmoore]

@christophenoel, I've just added #125 which will be ramping up over the course of the next week or two. Hopefully this would help us to refactor any of the shared issues from the multiscale conventions into a shared document.

[christophenoel]

It would be great to align efforts. I will try to review your spec over the next week.

Hi Ryan, it tried to reach you by email unsuccessfully. I will be working on this by mid february.

[rabernat]

Hi Christophe, Sorry for my slow replies. I am eager to work with you on this, just bad at email! 🙃 A good first step would be for you to review the OGC document I sent you and check that it aligns with what you are hoping to do for GeoZarr. Are you planning to put GeoZarr through the OGC process?

…

-Ryan

On Tue, Jan 25, 2022 at 2:37 AM Christophe Noel (Spacebel) < ***@***.***> wrote: It would be great to align efforts. I will try to review your spec over the next week. Hi Ryan, it tried to reach you by email unsuccessfully. I will be working on this by mid february. — Reply to this email directly, view it on GitHub <#124 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJEKJS5VF7MYYVLX6PAVDTUXZHLFANCNFSM5MIEICJA> . You are receiving this because you commented.Message ID: ***@***.***>

[christophenoel]

Dear all,

I'm very happy to share with you a first bunch of resources (presentations, demos) presenting the project, and showing the assets of GeoZarr.

We finally refined the multiscales support and recently introduced visual portrayals and symbology (see OGC WMS SLD) directly in the data to facilitate the display on map viewers.

I hope this helps !

Christophe.

---

Specification Document - Change Log

Demonstration Videos (Youtube channel):

• Project Presentation (at WGISS-53) GeoZarr Data Store - Context of the ESA GSTP project
• Project Presentation (at DAP) Hyperspectral Data Store and Access Project
• Demo: GeoZarr Visual Portrayals and OpenLayers extension
• Demo: GeoZarr Fast Time Series Plotting
• Demo: GeoZarr Compute and plot NDWI index at runtime
• Demo: GeoZarr Catalogue Integration
• Demo: GeoZarr Serverless Visualisation and Pixel-Based Access
• Comparison: GeoZarr vs COG Performances

OpenLayers extension prototype: https://github.com/spacebel/geozarr-openlayers