Skip to content

Sync fork 11#22

Merged
kkochel merged 88 commits intomainfrom
sync-fork-11
Jan 31, 2026
Merged

Sync fork 11#22
kkochel merged 88 commits intomainfrom
sync-fork-11

Conversation

@kkochel
Copy link
Collaborator

@kkochel kkochel commented Jan 31, 2026

Related issue(s) and PR(s)

This PR closes [issue number].

Description

How to test

neicnordic and others added 30 commits January 16, 2026 11:08
…ration-tests

Hotfix/chart integration tests
Bumps the all-modules group in /sda with 1 update: [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus).


Updates `github.com/sirupsen/logrus` from 1.9.3 to 1.9.4
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](sirupsen/logrus@v1.9.3...v1.9.4)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-version: 1.9.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-modules
...

Signed-off-by: dependabot[bot] <support@github.com>
…les/sda/all-modules-a2c59b68ad

build(deps): bump github.com/sirupsen/logrus from 1.9.3 to 1.9.4 in /sda in the all-modules group
Bumps the all-modules group in /sda-download with 1 update: [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus).


Updates `github.com/sirupsen/logrus` from 1.9.3 to 1.9.4
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](sirupsen/logrus@v1.9.3...v1.9.4)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-version: 1.9.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-modules
...

Signed-off-by: dependabot[bot] <support@github.com>
The same column is used for both POSIX and S3.
…ports multiple storage backends, seperate reading / writing to seperate interfaces, etc, still pending unit tests, documentation, possibly missing functionality, etc
…writers with information of object count and size of a location, and add logic to switch to free location, also add some unit test for the s3 writer
…as posix writer, but only with path as only required config for reader
… that a location is configured for posix when reading / deleting
…seekable reader from s3 reader to seperate package
…were removed in migration 21 step, update new 22 migration step to create register_file func as expected
…ng of storage v2 config from config package as its done in storage/v2 packages
…ileWithLocation instead of modifying existing to perserve backwards compatability for now
KarlG-nbis and others added 29 commits January 23, 2026 12:58
…oker cache TTL, update storage/v2/READMD.md to mention the location broker config
…file has been added to a dataset instead of stable id being set to known when a file has been removed from the submission location
…mapper, finalize, ingest, sync, and verify, such that they can query GetSizeAndObjectCountOfLocation
…calls to ensure number of calls match expected
… skip currently active as we know it wont be active in that scenario
… permission grants doesn't change the DB schema so it should not increase the dbschema_version value, and just keep new needed grants in 04_grants.sql
…epth of sub directories, etc, and fix linting issues
… as its signals that it will not walk into that directory/file
…rs, and update WriteFile to ensure files parent directories exists, add locking between WriteFile and RemoveFile such that we are not deleting / writing files at the same time, add unit test for writing / removing files in sub directories
…nst concurrent map read / writes, and update to use DB for directory volumes as well
…meing to avoid writing partial files, ensure we dont override existing files with partial files, create a 'tmp' folder in location for temporary files, add unit tests to check for such scenarios
…ackends-wip

Feature/multiple storage backends
…les/sda-download/all-modules-a2c59b68ad

build(deps): bump github.com/sirupsen/logrus from 1.9.3 to 1.9.4 in /sda-download in the all-modules group
Bumps the all-modules group in /sda-download with 1 update: [github.com/neicnordic/crypt4gh](https://github.com/neicnordic/crypt4gh).


Updates `github.com/neicnordic/crypt4gh` from 1.14.0 to 1.14.1
- [Release notes](https://github.com/neicnordic/crypt4gh/releases)
- [Commits](neicnordic/crypt4gh@v1.14.0...v1.14.1)

---
updated-dependencies:
- dependency-name: github.com/neicnordic/crypt4gh
  dependency-version: 1.14.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-modules
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps the all-modules group in /sda with 3 updates: [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2), [github.com/go-viper/mapstructure/v2](https://github.com/go-viper/mapstructure) and [github.com/neicnordic/crypt4gh](https://github.com/neicnordic/crypt4gh).


Updates `github.com/aws/aws-sdk-go-v2/feature/s3/manager` from 1.20.19 to 1.21.0
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/changelog-template.json)
- [Commits](aws/aws-sdk-go-v2@feature/s3/manager/v1.20.19...v1.21.0)

Updates `github.com/go-viper/mapstructure/v2` from 2.4.0 to 2.5.0
- [Release notes](https://github.com/go-viper/mapstructure/releases)
- [Changelog](https://github.com/go-viper/mapstructure/blob/main/CHANGELOG.md)
- [Commits](go-viper/mapstructure@v2.4.0...v2.5.0)

Updates `github.com/neicnordic/crypt4gh` from 1.14.0 to 1.14.1
- [Release notes](https://github.com/neicnordic/crypt4gh/releases)
- [Commits](neicnordic/crypt4gh@v1.14.0...v1.14.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-version: 1.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all-modules
- dependency-name: github.com/go-viper/mapstructure/v2
  dependency-version: 2.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all-modules
- dependency-name: github.com/neicnordic/crypt4gh
  dependency-version: 1.14.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-modules
...

Signed-off-by: dependabot[bot] <support@github.com>
…les/sda/all-modules-cf3c80e30c

build(deps): bump the all-modules group in /sda with 3 updates
…les/sda-download/all-modules-778b039c27

build(deps): bump github.com/neicnordic/crypt4gh from 1.14.0 to 1.14.1 in /sda-download in the all-modules group
# Conflicts:
#	charts/sda-db/Chart.yaml
#	charts/sda-mq/Chart.yaml
#	charts/sda-svc/Chart.yaml
@kkochel kkochel merged commit 485872a into main Jan 31, 2026
41 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants