- Resolve and authorize the canonical destination from the request payload
- Reject upload capabilities that cross virtual storage mount boundaries
- Remove the unrelated File-Path middleware authorization check
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
* feat(multipart): add chunk reassembly window
- Reassemble concurrently uploaded chunks into a sequential stream through a ring file, bounding disk usage to slots*chunkSize per session
- Park writers up to a deadline when their slot is busy instead of rejecting instantly, so flow control does not surface as connection errors in browsers
- Record a per-chunk CRC32 table for re-fill verification and keep it readable after close
- Propagate cancellation to blocked readers via CloseWithError so drivers treat aborts like canceled requests
- Cover ordering, backpressure, idempotent resends and close/abort wake-ups with race-enabled tests
* feat(multipart): add pipelined upload session manager
- Start the driver upload at session init over a sequential stream backed by the window, so client-to-server and server-to-storage transfers run concurrently
- Attach client-provided hashes to the stream so drivers can attempt rapid upload before any chunk arrives, and absorb chunks racing pipeline completion idempotently
- Keep only metadata and chunk CRCs after a failed attempt: re-sending chunk 0 re-fills a fresh window, and content changes between attempts are rejected
- Resume receiving sessions only when client hashes prove the same file; failed_retriable sessions resume unconditionally
- Reclaim sessions with a sliding-TTL GC and sweep orphaned ring files at startup
- Cover the state machine with race-enabled tests over a stubbed storage layer
* feat(setting): add multipart upload settings
- Add multipart_enabled and multipart_chunk_size (MB) as public traffic settings
- Validate the chunk size on save (integer within 1-90) via the setting item hook
* feat(server): add multipart upload API
- Add /api/fs/multipart init/chunk/complete/status/abort endpoints with headers aligned with /fs/put
- Gate init behind the FsUp permission checks and reuse the client upload rate limiter for chunk uploads
- Drain the request body before answering chunk requests on every path, so browsers do not see early responses as network errors
- Start the multipart session GC when the router is initialized to reclaim ring files orphaned by a previous run
* refactor(multipart): remove unused overwrite session field
- The overwrite flag was stored on the session but never read: the handler
performs the pre-check and op.Put owns the overwrite semantics
* feat(setting): allow any positive multipart chunk size
- Validate the setting as a positive integer only; self-hosted admins decide
the ceiling themselves instead of an arbitrary 90MB cap
- Keep clamping the client-suggested X-Chunk-Size to the admin value: the
server buffers a window of 8 chunks per session, so an unbounded client
suggestion would translate directly into server-side disk usage
* refactor(server): simplify multipart chunk size clamp
- Fold the two-step clamp into one branch: the ceiling is already floored,
so a client suggestion just lowers the size with a 1MB floor
* feat(driver): add Cloudflare Image Bed support
* fix: restore accidentally deleted file and name
* refactor: rename driver to cloudflare_imgbed and fix module structure
- Rename driver identifier and directory to 'cloudflare_imgbed' for consistency.
- Remove invalid 'replace' directive in go.mod.
- Restore accidentally modified/deleted files in public/dist.
- Update driver registration in drivers/all.go.
Co-authored-by: Copilot <copilot@github.com>
* fix: use base.NewRestyClient() and use e.g
Co-authored-by: Copilot <copilot@github.com>
* fix:go fmt
* feat(driver/cloudflare-imgbed): enhance cloudflare_imgbed API integration with improved error handling and pagination
* refactor
* feat(cloudflare_imgbed): implement upload functionality and optimize performance
- Added support for standard multipart form upload with zero-copy streaming.
- Implemented HuggingFace LFS direct upload for large files (>20MB).
- Integrated with OpenList global rate limiter and progress tracking.
- Optimized memory usage using io.MultiReader for request body construction.
- Added configurable upload threads for chunked HF uploads.
- Support auto mkdir dir when in upload
* refactor: simplify path handling logic
* refactor(cloudflare_imgbed): streamline API endpoint constants and improve initialization logic
* refactor(cloudflare_imgbed): clean up upload logic and remove unused functions
* docs: update help descriptions to English in cloudflare_imgbed
* feat(cloudflare_imgbed): add virtual directory support
* refactor(weak_cache): iImprove weak pointer cleanup handling
Store a cleanup handle alongside weak pointers and stop previous cleanups when overwriting entries to avoid stale removals and leaked cleanup handlers.
Ensure Delete signals success and stops associated cleanup. Add Clear to stop all active cleanups. Use entry identity in the cleanup callback to avoid removing newly inserted values.
* fix bug
* Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
* Adds uploadFolder param and streams base64 sample
Adds an uploadFolder query parameter to forward the destination path to the backend. Replaces the fixed-size sample read with a streaming base64 encoder to avoid truncation and reduce allocations
Co-authored-by: Copilot <copilot@github.com>
* refactor: add context parameter to doRequest and related calls
* fix: update List method to check args.Refresh before virtual directory mask
* refactor: optimize List method and improve error handling in doRequest
* feat: add public URL support and multi-channel chunked upload
- Support multi-channel chunk size configurations (e.g., Telegram, CFR2, S3, Discord).
- Fix 401 unauthorized error when merging chunks in HuggingFace channel.
---------
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Add end-to-end offline download support for torrent files, magnet links, and ed2k links, including torrent parse and generate APIs, CAS-based rapid upload for Cloud189, and protocol-aware tool routing with transfer flow improvements.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: Suyunjing <suyunmeng@oplist.org.cn>
* refactor(HybridCache)!: extract hybrid_cache package and rename cache_threshold to auto_memory_limit
* split HybridCache and stores into internal/hybrid_cache package
* introduce BackingStore abstraction with BufferStore and FileStore
* rename CacheThreshold to AutoMemoryLimit in config/env/bootstrap
* update stream/request integration and related tests
* rename singleFileCache and MultiFileCache to singleFileStore and MultiFileStore
* refactor(HybridCache): improve memory management strategies in NewHybridCache
* rename
* alias hybrid_cache to hcache
* omments
- Switch default SQLite path to github.com/glebarez/sqlite to reduce CGO dependency pressure.
- Introduce a unified openSQLite entry in bootstrap and split driver selection by build tags.
- Add sqlite_cgo_compat fallback for linux mips, mips64, loong64 and mipsle to keep legacy target builds working.
- Update build.sh musl build flow to apply compatibility tag for mips-family targets.
- Update beta_release workflow to pass compatibility tag cleanly and avoid conflicting flag composition.
* refactor(permission): rename permission check functions for clarity
- User.CanWrite() → User.CanCreateFilesOrFolders()
- common.CanWrite() → common.CanWriteContentBypassUserPerms()
- common.IsApply() → common.MetaCoversPath()
Improves code readability by making function names more descriptive.
The new MetaCoversPath name clearly indicates it checks if a meta rule
covers a specific path. It better conveys that it's a query function
rather than an action, and the applyToSubFolder parameter is more
explicit than applySub.
Also adds comprehensive test coverage:
- 10 tests for MetaCoversPath core logic
- 6 tests for CanWriteContent
UserPerms
- 7 tests for getReadme
- 5 tests for getHeader
- 6 tests for isEncrypt
- 9 tests for whetherHide
Total: 43 test scenarios covering all path matching and permission
inheritance logic. Tests verify both normal behavior and bug fixes
for Readme/Header information leakage and write permission bypass.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(permission): implement fine-grained user permissions for read/write operations
Add per-user read and write permission controls at the meta level to enable
more granular access control beyond the existing permission flags.
Key changes:
- Add ReadUsers/WriteUsers fields to Meta model with sub-directory inheritance flags
- Implement CanRead and CanWrite permission check functions in server/common
- Filter file list results based on user read permissions
- Add permission checks across all file operations (FTP, HTTP handlers, WebDAV)
- Simplify error handling pattern for MetaNotFound errors throughout codebase
This allows administrators to restrict specific users from accessing or modifying
certain paths, providing finer control over file system permissions.
Note: Batch and recursive operations (FsMove, FsCopy, FsRemove, FsRecursiveMove,
FsBatchRename, FsRegexRename) currently check parent directory permissions only.
Individual item permission checks are not performed for performance reasons.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* test(permission): add comprehensive tests for CanRead, CanWrite, and combined permission checks
Add TestCanRead, TestCanWrite, TestCanAccessWithReadPermissions, and
TestWritePermissionCombinations to validate the three-layer permission
system including nil user/meta, sub-path inheritance, user whitelists,
and root-level restrictions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(webdav): use safe type assertion for MetaPassKey to prevent panic
Bearer-token and OPTIONS auth paths do not set MetaPassKey in context,
causing a panic when handlers perform a forced type assertion on nil.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(permission): treat nil user as system context in CanRead/CanWrite
Previously, CanRead/CanWrite returned false for nil user, causing
filterReadableObjs to return an empty list when fs.List is called from
internal contexts without a user (e.g. context.Background()). A nil user
represents an internal/system call and should bypass per-user restrictions,
consistent with how whetherHide already handles nil user.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(fsmanage): prevent path traversal in FsRemove
The previous check only skipped names that resolved to "/", but did not
prevent traversal to sibling directories (e.g. "../secret"), which could
bypass the CanWrite permission check that is only applied to req.Dir.
Replace with a post-join prefix check to ensure each resolved path stays
within reqPath.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(webdav): align MetaPassKey behavior with FTP auth logic
For guest users, the WebDAV password input serves as the meta folder
password (consistent with FTP anonymous/guest handling). For authenticated
users, MetaPassKey is set to empty string since their login password is
not the meta folder password.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(permission): require write auth for fs list refresh
* refactor(permission): use MetaCoversPath in CanRead/CanWrite for consistency
Replace inline `(Sub || meta.Path == path)` logic with MetaCoversPath,
consistent with CanWriteContentBypassUserPerms. Also fix a copy-paste
error in the CanWrite comment (read → write).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
* fix(FileTransferTask): skip copying if destination directory does not exist
* pass only object not found error
---------
Co-authored-by: cyk <dr_arc@163.com>
Co-authored-by: KirCute <951206789@qq.com>
* Remove the `OnlyProxy` restriction and obtain the redirected link to support 302
* Add `driver.Config` `PreferProxy` to recommend user to enable the proxy by default
---------
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* feat(drivers): support getting disk usage of some drivers
* feat(drivers/degoo): implement GetDetails
* fix(fs/storage-details): fill used space rather than free space
* fix mega
* fix bsize type
Nostalgia