- Track lastActivity on each multipart upload, updated on create and
every part upload, so idle uploads can be detected
- Add a background reaper per backend instance that removes uploads
inactive for longer than the TTL (default 24h) and cleans their temp
directories
- Add a startup sweep that removes leftover s3-multipart-* directories
older than the TTL, recovering part files from a previous crash
- Add a configurable s3.multipart_ttl (env S3_MULTIPART_TTL) duration,
parsed via time.ParseDuration with a 24h default
- Add tests for TTL-based reaping and stale-directory cleanup
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
- Implement gofakes3 MultipartBackend (Create/UploadPart/Complete/Abort)
on s3Backend so multipart parts stream to local temp files instead of
being buffered in memory
- Track in-progress uploads via a new uploads sync.Map on s3Backend
- Refactor the PutObject body into a reusable putStream helper shared with
the multipart Complete path
- Validate part ordering, etags and existence, reject short reads, and
return S3-style multipart etags
- Add end-to-end multipart tests against a real Local driver
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
The 115 and pikpak multipart uploaders checked for cancellation with a
`case <-ctx.Done()` inside a select, and a break there only leaves the
select, not the enclosing `for retry := 0; retry < 3; retry++` loop.
Cancelling an upload therefore ran all three attempts for every
remaining chunk, each allocating a chunk-sized buffer and reading it
off disk before firing a request that could not succeed, and reported
a transport error instead of the cancellation.
Move the check ahead of the select and use utils.IsCanceled, matching
the pattern the other drivers already use. Assigning ctx.Err() to err
is load-bearing: without it a cancelled chunk takes the success branch,
counts toward progress, and appends a zero-value UploadPart.
Found with staticcheck (SA4011).
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
* feat(strm): add local save permission mode
- add private and shared permission modes for local STRM files
- repair shared-mode directory and file permissions during generation
- add permission handling tests
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* fix(strm): respect deployment umask for local directories
- remove application-level permission modes and chmod operations
- create local STRM directories with umask-controlled permissions
- preserve existing permissions and test the behavior
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
---------
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* feat(ilanzou): modernize console API session and copy
Use an isolated cookie-backed API session and preserve raw appToken syntax required by current iLanzou endpoints.
Resolve CDN download responses more robustly, align upload metadata with the console protocol, and delegate copies to OpenList background tasks instead of blocking requests.
* fix(ilanzou): escape appToken query values
Escape opaque appToken values while preserving the literal colon required by iLanzou endpoints. Ignore CDN HEAD response lengths unless the response status is successful.
* fix(ilanzou): address driver review feedback
Use the ilanzou package name consistently, document upload result polling, and bound CDN size probes. Keep the appToken and CDN challenge handling covered by focused tests and comments.
* fix(ilanzou): use context timeout for HEAD probe
* feat(strm): add file size filtering for STRM generation
- add ParseSize helper function in pkg/utils to parse human-readable byte sizes
- add minFileSize setting field to STRM driver Addition struct
- filter out files below minFileSize threshold during STRM generation
* refactor(strm): use MB as default unit for minFileSize instead of ParseSize
Replace string-based ParseSize auto-conversion with a simple int64 number
field defaulting to MB, consistent with other drivers (Google Drive,
Teldrive). Remove the now-unused ParseSize function and its tests.
* feat(multipart): add chunk reassembly window
- Reassemble concurrently uploaded chunks into a sequential stream through a ring file, bounding disk usage to slots*chunkSize per session
- Park writers up to a deadline when their slot is busy instead of rejecting instantly, so flow control does not surface as connection errors in browsers
- Record a per-chunk CRC32 table for re-fill verification and keep it readable after close
- Propagate cancellation to blocked readers via CloseWithError so drivers treat aborts like canceled requests
- Cover ordering, backpressure, idempotent resends and close/abort wake-ups with race-enabled tests
* feat(multipart): add pipelined upload session manager
- Start the driver upload at session init over a sequential stream backed by the window, so client-to-server and server-to-storage transfers run concurrently
- Attach client-provided hashes to the stream so drivers can attempt rapid upload before any chunk arrives, and absorb chunks racing pipeline completion idempotently
- Keep only metadata and chunk CRCs after a failed attempt: re-sending chunk 0 re-fills a fresh window, and content changes between attempts are rejected
- Resume receiving sessions only when client hashes prove the same file; failed_retriable sessions resume unconditionally
- Reclaim sessions with a sliding-TTL GC and sweep orphaned ring files at startup
- Cover the state machine with race-enabled tests over a stubbed storage layer
* feat(setting): add multipart upload settings
- Add multipart_enabled and multipart_chunk_size (MB) as public traffic settings
- Validate the chunk size on save (integer within 1-90) via the setting item hook
* feat(server): add multipart upload API
- Add /api/fs/multipart init/chunk/complete/status/abort endpoints with headers aligned with /fs/put
- Gate init behind the FsUp permission checks and reuse the client upload rate limiter for chunk uploads
- Drain the request body before answering chunk requests on every path, so browsers do not see early responses as network errors
- Start the multipart session GC when the router is initialized to reclaim ring files orphaned by a previous run
* refactor(multipart): remove unused overwrite session field
- The overwrite flag was stored on the session but never read: the handler
performs the pre-check and op.Put owns the overwrite semantics
* feat(setting): allow any positive multipart chunk size
- Validate the setting as a positive integer only; self-hosted admins decide
the ceiling themselves instead of an arbitrary 90MB cap
- Keep clamping the client-suggested X-Chunk-Size to the admin value: the
server buffers a window of 8 chunks per session, so an unbounded client
suggestion would translate directly into server-side disk usage
* refactor(server): simplify multipart chunk size clamp
- Fold the two-step clamp into one branch: the ceiling is already floored,
so a client suggestion just lowers the size with a 1MB floor
The Login endpoint may return an acw_sc__v2 validation page when accessed,
which previously caused login failures. This change adds the same retry
logic and cookie handling already used in request() to automatically solve
the challenge, ensuring login success even when the anti-bot mechanism is
triggered.
MailCookiesstring`json:"mail_cookies" required:"true" type:"text" help:"Cookies from mail.139.com used for login authentication."`
Authorizationstring`json:"authorization" type:"text" help:"Authorization can be used alone. If empty, use mail_cookies alone for fast login, or mail_cookies + username + password for full login fallback."`
Usernamestring`json:"username" help:"Required only when using password login fallback with mail_cookies."`
Passwordstring`json:"password" secret:"true" help:"Required only when using password login fallback with mail_cookies."`
SmsCodestring`json:"sms_code" secret:"true" help:"Fill this only after OpenList reports that a 139 Mail SMS verification code was sent, then save the storage again."`
MailCookiesstring`json:"mail_cookies" type:"text" help:"Cookies from mail.10086.cn. Used for fast login only when Authorization is empty; otherwise retained as device context for password login fallback."`
LinkIDstring`json:"link_id" type:"text" help:"Multiple shares are separated by commas or new lines. Use link_id#password for password-protected shares."`
returnfmt.Errorf("please provide api_key+user_id or username+password")
ifstrings.TrimSpace(d.Username)==""{
returnfmt.Errorf("please provide api_key+user_id or username(+password)")
}
iferr:=d.login(ctx);err!=nil{
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
MadDogOwner