Skip to content

Add a config key to allow transfer to a single raw directory#803

Open
stephen-riggs wants to merge 2 commits into
mainfrom
single-data-dir
Open

Add a config key to allow transfer to a single raw directory#803
stephen-riggs wants to merge 2 commits into
mainfrom
single-data-dir

Conversation

@stephen-riggs
Copy link
Copy Markdown
Contributor

One possible option for how to resolve the b24 data transfer where we want everything to go to the same raw folder, but using different rsyncers and analysers.

This adds a config key to override the suggested path logic and return raw each time. It also turns the limited transfer behaviour off in for all rsyncers made by this method so the files are analysed.

@stephen-riggs stephen-riggs requested review from d-j-hatton and tieneupin and removed request for tieneupin May 19, 2026 10:28
@codecov
Copy link
Copy Markdown

codecov Bot commented May 19, 2026

Codecov Report

❌ Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.30%. Comparing base (82b4177) to head (6117832).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #803      +/-   ##
==========================================
- Coverage   52.32%   52.30%   -0.02%     
==========================================
  Files          96       96              
  Lines       10322    10327       +5     
  Branches     1357     1360       +3     
==========================================
+ Hits         5401     5402       +1     
- Misses       4642     4645       +3     
- Partials      279      280       +1     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

log.debug("MultigridDirWatcher thread stop completed")

def _handle_metadata(self, directory: Path, extra_directory: str):
def _handle_metadata(self, directory: Path, extra_directory: str, limited=True):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think _handle_subdir might be a suitable replacement name for _handle_metadata, as it seems to be notifying the notifying the subscribed components to create an additional Analyser for the sub-directory stated in extra_directory.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly, as discussed the name is unchanged but a comment has been added

Copy link
Copy Markdown
Contributor

@tieneupin tieneupin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code seems fine!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants