Feature/update deployment #61

KarlG-nbis · 2025-09-25T12:50:53Z

Update the images and configuration for the sda-pipeline service to the latest

Merge the quickstart.md and README.md into one README.md to contain the relevant information to get started.

…upport starter-kit-htsget

…pdate to latest sda stack, use ghcr.io repo for sda download image, update config example to the now expected c4gh formating

jbygdell

We should also update the Terraform examples so they match the latest version.

jbygdell · 2025-09-25T12:56:09Z

config/TLS-example/docker-compose.yml

  auth:
    container_name: auth
-    image: ghcr.io/neicnordic/sensitive-data-archive:v0.2.23-auth
+    image: ghcr.io/neicnordic/sensitive-data-archive:v3.0.14-auth


Suggested change

image: ghcr.io/neicnordic/sensitive-data-archive:v3.0.14-auth

image: ghcr.io/neicnordic/sensitive-data-archive:v3.0.14

Auth doesn't have a separate image anymore.

Updated to use ghcr.io/neicnordic/sensitive-data-archive:v3.0.14 with command: sda-auth, is there any env vars that we need to populate as well?

README.md

jbygdell · 2025-09-25T12:59:35Z

README.md

+| database     | PostgreSQL database, [SDA-DB](https://github.com/neicnordic/sensitive-data-archive/tree/main/postgresql).                                                                                                            |
+| storage      | S3 object store, demo uses Minio S3.                                                                                                                                                                                 |
+| auth         | OpenID Connect relaying party and authentication service, [SDA-auth](https://github.com/neicnordic/sensitive-data-archive/tree/main/sda-auth).                                                                       |
+| s3inbox      | Proxy inbox to the S3 backend store, [SDA-S3Inbox](https://github.com/neicnordic/sensitive-data-archive/tree/main/sda).                                                                                              |


Suggested change

| s3inbox | Proxy inbox to the S3 backend store, [SDA-S3Inbox](https://github.com/neicnordic/sensitive-data-archive/tree/main/sda). |

| s3inbox | Proxy inbox to the S3 backend store, [SDA-S3Inbox](https://github.com/neicnordic/sensitive-data-archive/tree/main/sda/cmd/s3inbox/s3inbox.md). |

Maybe this should be included in the SDA below.

Hmm yeah makes sense, given its part of the cmd/sda, but its not mentioned in the cmd/sda/sda.md atm, and would make sense to include auth there as well if its been moved there

zeidlitz

Looks good to me. Added some small suggestions for consistency.

README.md

nanjiangshu · 2025-09-25T13:23:20Z

README.md

+
+## Authentication for users with LS-AAI (mock or alive)
+
+To interact with SDA services, users need to provide [JSON Web Token](https://jwt.io/) (JWT) authorization. Ultimately, tokens can be fetched by [LS-AAI](https://lifescience-ri.eu/ls-login/) upon user login to an OpenID Connect (OIDC) relaying party (RP) service that is [registered with LS-AAI](https://spreg-legacy.aai.elixir-czech.org/). An example of such an RP service is the [sda-auth](https://github.com/neicnordic/sda-auth), which is included in the present stack.


I think this link https://spreg-legacy.aai.elixir-czech.org/ is outdated. Maybe https://services.aai.lifescience-ri.eu/spreg/ is the correct one. Is that correct @jbygdell ?

Also the link to sda-auth should be changed to https://github.com/neicnordic/sensitive-data-archive/tree/main/sda/cmd/auth

Yes, all mentions of elixir-czech should be converted to the new ones from lifescience-ri.eu.
That means the URLs needs to be verified since they are not just a hostname change.

nanjiangshu · 2025-09-25T13:29:29Z

README.md

+
+### The sda-cli tool
+
+Instead of the tools above, users are **encouraged** to use [`sda-cli`](https://github.com/NBISweden/sda-cli), which is a tool specifically developed to perform all common SDA user-related tasks in a convenient and unified manner. It is recommended to use precompiled executables for `sda-cli` which can be found at [https://github.com/NBISweden/sda-cli/releases](https://github.com/NBISweden/sda-cli/releases)


We probably need to specify a version of sda-cli to use.

README.md

nanjiangshu · 2025-09-25T13:38:40Z

README.md

-From the root of the `starter-kit-storage-and-interfaces` folder and run:
+### Downloading data
+
+Users can directly download data from the SDA via `sda-download`, for more details see the service's [api reference](https://github.com/neicnordic/sda-download/blob/main/docs/API.md). In short, given a [valid JW token](#sda-auth), `$token`,  a user can download the file with file ID, `$fileID` by issuing the following command:


Suggested change

Users can directly download data from the SDA via `sda-download`, for more details see the service's [api reference](https://github.com/neicnordic/sda-download/blob/main/docs/API.md). In short, given a [valid JW token](#sda-auth), `$token`, a user can download the file with file ID, `$fileID` by issuing the following command:

Users can directly download data from the SDA via `sda-download`, for more details see the service's [api reference](https://github.com/neicnordic/sensitive-data-archive/blob/main/sda-download/api/api.md). In short, given a [valid JW token](#sda-auth), `$token`, a user can download the file with file ID, `$fileID` by issuing the following command:

Also, the documentation for sda-download is outdated. I created this issue neicnordic/sensitive-data-archive#1996 to address it.

Updated link, and for now the text to mention we only support delivering files encrypted with a key provided by the client

nanjiangshu · 2025-09-25T13:42:59Z

README.md

-### Download a specific file
+#### Download a specific encrypted file

 The `sda-download` service offers multiple methods for downloading files through the API, with options for both encrypted and unencrypted results. Below, you will find an example illustrating each of these methods.


We need to discuss if downloading unencrypted data should be supported.

…n image

…nds examples, remove unecessary header

KarlG-nbis added 5 commits September 25, 2025 11:16

feat: merge quickstart.md and README.md to one document

56bfc47

feat: add info noting that current version of sda-pipeline does not s…

5e35b14

…upport starter-kit-htsget

feat: update images, python:3.10-slim -> python:3.10-slim-bookworm, u…

d4d5a2e

…pdate to latest sda stack, use ghcr.io repo for sda download image, update config example to the now expected c4gh formating

feat: update images and config for the TLS example

2628b1e

feat: add refernce to TLS-example/README.md from README.md

0c6d105

jbygdell reviewed Sep 25, 2025

View reviewed changes

zeidlitz reviewed Sep 25, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

README.md Show resolved Hide resolved

nanjiangshu reviewed Sep 25, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

nanjiangshu reviewed Sep 25, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

nanjiangshu reviewed Sep 25, 2025

View reviewed changes

feat: fix auth service image in TlS-example as it no longer in its ow…

a0b3e3e

…n image

KarlG-nbis force-pushed the feature/update_deployment branch 2 times, most recently from fa63db9 to a12a41b Compare September 26, 2025 07:05

feat: update README.md, fix links to outdated docs, fix sda-cli comma…

4e41124

…nds examples, remove unecessary header

KarlG-nbis force-pushed the feature/update_deployment branch from a12a41b to 4e41124 Compare September 26, 2025 07:08

	image: ghcr.io/neicnordic/sensitive-data-archive:v3.0.14-auth
	image: ghcr.io/neicnordic/sensitive-data-archive:v3.0.14

	\| s3inbox \| Proxy inbox to the S3 backend store, [SDA-S3Inbox](https://github.com/neicnordic/sensitive-data-archive/tree/main/sda). \|
	\| s3inbox \| Proxy inbox to the S3 backend store, [SDA-S3Inbox](https://github.com/neicnordic/sensitive-data-archive/tree/main/sda/cmd/s3inbox/s3inbox.md). \|


		## Authentication for users with LS-AAI (mock or alive)

		To interact with SDA services, users need to provide [JSON Web Token](https://jwt.io/) (JWT) authorization. Ultimately, tokens can be fetched by [LS-AAI](https://lifescience-ri.eu/ls-login/) upon user login to an OpenID Connect (OIDC) relaying party (RP) service that is [registered with LS-AAI](https://spreg-legacy.aai.elixir-czech.org/). An example of such an RP service is the [sda-auth](https://github.com/neicnordic/sda-auth), which is included in the present stack.


		### The sda-cli tool

		Instead of the tools above, users are encouraged to use [`sda-cli`](https://github.com/NBISweden/sda-cli), which is a tool specifically developed to perform all common SDA user-related tasks in a convenient and unified manner. It is recommended to use precompiled executables for `sda-cli` which can be found at [https://github.com/NBISweden/sda-cli/releases](https://github.com/NBISweden/sda-cli/releases)

	Users can directly download data from the SDA via `sda-download`, for more details see the service's [api reference](https://github.com/neicnordic/sda-download/blob/main/docs/API.md). In short, given a [valid JW token](#sda-auth), `$token`, a user can download the file with file ID, `$fileID` by issuing the following command:
	Users can directly download data from the SDA via `sda-download`, for more details see the service's [api reference](https://github.com/neicnordic/sensitive-data-archive/blob/main/sda-download/api/api.md). In short, given a [valid JW token](#sda-auth), `$token`, a user can download the file with file ID, `$fileID` by issuing the following command:

Feature/update deployment #61

Are you sure you want to change the base?

Feature/update deployment #61

Uh oh!

Conversation

KarlG-nbis commented Sep 25, 2025

Uh oh!

jbygdell left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KarlG-nbis Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeidlitz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nanjiangshu Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

KarlG-nbis Sep 26, 2025 •

edited

Loading

nanjiangshu Sep 25, 2025 •

edited

Loading