fix: use FileLock to avoid downloading models concurrently into shared storage

[marcelklehr]

Summary by CodeRabbit

• Bug Fixes
  • Improved model download reliability by preventing concurrent downloads of the same model, ensuring safe and consistent operations.

[Copilot]

Pull request overview

This PR aims to prevent concurrent downloads of machine learning models into shared storage by adding file locking mechanisms. The implementation wraps model download operations with FileLock to ensure only one process downloads a model at a time, avoiding conflicts in multi-instance deployments.

Changes:

• Added lockfile dependency to project dependencies
• Wrapped file-based model download logic with FileLock context manager
• Wrapped HuggingFace snapshot download with FileLock context manager

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
pyproject.toml	Adds lockfile>=0.12.2 dependency to support file locking
nc_py_api/ex_app/integration_fastapi.py	Imports FileLock and wraps model download operations with file locks

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

❌ Patch coverage is 2.38095% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.53%. Comparing base (8e8cf61) to head (c672479).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
nc_py_api/ex_app/integration_fastapi.py	2.38%	41 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #395      +/-   ##
==========================================
- Coverage   94.65%   94.53%   -0.13%     
==========================================
  Files          45       45              
  Lines        5392     5400       +8     
==========================================
+ Hits         5104     5105       +1     
- Misses        288      295       +7     

Files with missing lines	Coverage Δ
nc_py_api/ex_app/integration_fastapi.py	22.59% <2.38%> (-0.48%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[oleksandr-nc]

SoftFileLock can lead to broken deploys if the process crashes during downloading.
FileLock on other hand auto-releases when the process dies.

[marcelklehr]

I had concerns about whether the file system in the shared storage would support locking, that's why I chose softlocks.

Also, it has this:

To mitigate stale locks, the lock file contains the PID and hostname of the holding process. On contention, if the holder is on the same host and its PID no longer exists, the stale lock is broken automatically.

But not sure if that helps us with k8s.

[oleksandr-nc]

Alternative version just as idea(I did not test it in practice):

#396

[coderabbitai]

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Adds file-based locking to model download operations in the FastAPI integration module using the filelock library. The __fetch_model_as_file and display() methods now acquire per-file/per-model locks before executing downloads and etag validation, preventing concurrent access to the same target resource.

Changes

Cohort / File(s)	Summary
Model Download Concurrency Protection nc_py_api/ex_app/integration_fastapi.py	Introduces SoftFileLock usage in two locations: wraps __fetch_model_as_file operations with per-file locking, and wraps display() model downloads with per-model locking using sanitized model names. Adds import for SoftFileLock from filelock module.
Dependency Management pyproject.toml	Adds runtime dependency on filelock>=3.24.2,<4 to support file-based locking mechanisms.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Locks now guard our downloads fair,
No concurrent conflicts bare,
With SoftFileLock in place so true,
Models sync without a queue! 🔒✨

✨ Finishing Touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/k8s

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}