- Refresh OSS credentials and retry the rejected multipart operation
- Preserve upload IDs, completed parts, callbacks, and outer part retries
- Rewind parts before resending and pass the upload context to OSS requests
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* fix(alist_v3): set child paths so nested directories resolve
- Set `Path` on every object returned by `List`, matching the OpenList
driver. `op.Get` hands a child object straight back to `List`, so a
child without a path made the driver request `""` from the upstream
server, which answered with its own root: every directory below the
mount point served the same listing back, endlessly.
- Add tests covering the child paths and a three-level descent.
Co-authored-by: Claude <81847+claude@users.noreply.github.com>
* test(alist_v3): trim the child-path test to a single case
Collapse the two tests into one two-level descent, drop the recorder
type and the helper funcs, and inline the driver setup. 173 -> 65 lines.
Co-authored-by: Claude <81847+claude@users.noreply.github.com>
---------
Co-authored-by: Fighting <3899648+Elity@users.noreply.github.com>
Co-authored-by: Claude <81847+claude@users.noreply.github.com>
test(s3): encode multipart fixture paths
- Serialize the Local storage addition instead of interpolating filesystem paths into JSON.
- Keep multipart fixture behavior portable across Windows and Linux.
Co-authored-by: nostalume <nostalucent@gmail.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
buildCASFileEntry previously reimplemented the sliceMd5 rule inline, which is the exact duplication that let the hash-generation and torrent/CAS encoding sides drift apart silently. Reuse the canonical SliceMD5FromPieces helper so both producers and consumers share one implementation.
Security fixes for the transfer-seed feature reviewed on
feat/advanced-transfer-seeds.
SSRF via redirect (torrent.go):
- Source validation only pinned the first hop while http.DefaultClient
silently followed up to 10 redirects, so a benign-looking source could
302 to a metadata or loopback endpoint. Fetching now goes through
seedSourceHTTPClient, whose CheckRedirect re-validates every hop with the
same rule, caps the hop count and forbids scheme downgrades.
- Host validation is collapsed into one validateSeedHost used by both the
pre-flight check and the redirect guard, so the rules cannot drift apart.
- Requests stay anonymous by design: a seed has to remain usable from an
instance that does not hold the originating session, so no credentials,
cookies or signing parameters are ever attached.
Seed content fetching (torrent.go):
- Propagate the request context instead of context.Background(), so
cancellation actually stops the download.
- Stream the body through an io.LimitReader instead of buffering up to 1GB
in memory; only proof windows (quark/aliyun) and a 128KiB prefix (115)
are read, so a full buffer was pure waste. Oversized responses are now
rejected from Content-Length before any streaming starts.
Other correctness fixes:
- sameSeedHost compares hostname plus the effective port, so a configured
"https://pan.example.com" and an embedded "...:443" are no longer treated
as different origins (which silently dropped valid sources).
- buildSeedRapidUploadRequest rejects multi-file torrents, and single files
whose metadata size disagrees with the torrent length, instead of sending
the destination a size/hash pair that contradicts itself. Both call sites
now handle the nil result instead of dereferencing it.
- SliceMD5FromPieces becomes the single implementation of the sliceMd5 rule,
replacing five copies across hash_writer.go, torrent.go, generate.go,
189/torrent.go and 189pc/torrent.go. Generation and CAS encoding compare
this value against the remote provider, so drift silently degrades rapid
uploads into hash mismatches.
- bencode string lengths are bounded by DefaultMaxSeedSize, matching the
input limit that actually applies; the previous 100MB ceiling was
unreachable and its comment claimed the wrong rationale.
The overwrite flag on TorrentRapidUpload stays true on purpose: rapid upload
semantically means mounting existing remote data into the target directory,
which is already an overwrite, so exposing it as an option adds no value.
Tests:
- pkg/torrent/seed_security_test.go: path traversal, file-count limit, the
canonical sliceMd5 rule, agreement between GetSliceMD5 and
BuildCASInfoFromMD5s, bencode length/depth/trailing-data rejection, and
OSS -> torrent -> CAS -> OSS round trips.
- server/handles/torrent_seed_test.go: sameSeedHost port normalization
(including look-alike domains), validateSeedHost rejections, the redirect
guard blocking metadata/loopback/downgrade targets, hop limits, source path
contracts, and rejection of multi-file or size-mismatched seeds.
go build ./... passes; go test ./pkg/torrent/... and
go test ./server/handles/... pass.
The previous commit (624fdd24) introduced the authoritative
driver.SeedRapidUploader interface, but left every driver's seed_rapid.go
on the older, incompatible API, so the branch did not compile at all.
Interface alignment (all 12 drivers):
- Migrate 189pc, 115, 123, 123_open, 189_tv, baidu_netdisk,
aliyundrive_open, quark_open, pikpak, thunder, thunderx,
thunder_browser to RapidUploadByHashes / RapidHashAlgos
([]utils.HashType, no longer []*utils.HashType) / RapidHashNeedsPieces
- Add shared driver.SeedHashStream as a complete model.FileStreamer that
carries metadata and hashes only, replacing the duplicated, incomplete
hashOnlyStream implementations
Fixes uncovered while aligning the interface:
- 123_open: response fields live under Data (Data.Reuse / Data.FileID)
- quark_open: pre.Data.FID -> pre.Data.Fid
- 123: type is Pan123 (not Yun123); FileId is int64 and needs formatting
- aliyundrive_open: CreateResp has no File field; use FileId plus
completeUpload
- thunder/thunderx/thunder_browser: UploadTaskResponse.File is a Files
value type, return &resp.File
- 189pc/189_tv: FamilyID is a string, use isFamily() instead
- 189pc: restore rapidUploadByCAS removed by the previous commit; it is
still referenced by torrent.go. Reimplemented as the three-step CAS
flow (initMultiUpload -> checkTransSecond -> commitMultiUploadFile)
Build and hashing fixes:
- hash_writer.go: HashType exposes NewFunc; GCID.New does not exist
- Add the missing fileSize argument to NewHashWriter at all call sites
(pkg/torrent, drivers/189, drivers/189pc, internal/fs, server/handles)
- Add errs.ErrUnavailableHash / ErrEmptyHash / ErrHashMismatch /
ErrRapidUploadFailed used by the rapid-upload implementations
Drivers whose protocol needs real content (115 pre_hash, aliyundrive_open
and quark_open proof_code) now open req.Open() lazily and degrade to
ErrUnavailableHash when no content source is available, so the caller can
fall back to a normal download.
Note: go vet warnings for non-constant format strings in 189pc/utils.go
are pre-existing and intentionally left untouched.
- add an opt-in Local driver setting for PDF thumbnails
- render PDF first pages with macOS Quick Look
- preserve unsupported-platform behavior and cover the renderer with tests
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Co-authored-by: ILoveScratch <ilovescratch@foxmail.com>
Encode the slice_md5s and slice_size fields in CAS (single-file and per-file), restore them into SeedFile.Hashes.Pieces.MD5 on decode, and hide the legacy warning when piece hashes are present. Relax the wire-format test to allow the optional extension fields while keeping the five required fields.
CAS now supports multiple files via a files array while keeping the legacy five-field single-file payload byte-compatible. Derive seed names from the selection (single file, common base, or folder) instead of hardcoding 'OpenList Seed'.
CAS is a single-file container, so multi-file generation now emits one .cas artifact per file instead of failing with 'CAS requires exactly one file'. Capabilities no longer gate cas on single-file selection.
Export NormalizeSeedFormats and EncodeGeneratedSeed so fsup.go can reuse them after the generation logic moved into internal/fs. Drop the now-unused slices import.
Extract seed generation into fs.GenerateSeedArtifacts and add a SeedGenerateTask manager. Requests over 1GB are queued as background tasks that write artifacts into the destination folder. Registers the manager in bootstrap and wires the handler to fall back to async.
ParseSeed marks torrent seeds as offline_download capable even without sources (magnet/tracker). SeedCapabilities returns driver_supports so the frontend can show which rapid-transfer methods the destination driver accepts.
Capabilities now report streamable/direct_source_available/share_available and the configured tracker list. Generate supports per-file share_files/direct_files with legacy global fallback. Add seed_default_trackers setting.
SeedCapabilityReq embeds SeedDataReq whose SeedData field was bound with required. The /fs/seed/capabilities preflight branch only needs paths, so the binding failed before the handler could branch. Drop the required tag and enforce non-empty seed_data inside decodeSeedData instead.
Record successful saves as channels and failures as missing_channels when update_channel is set. Return share_status during edit by validating openlist-share sources. Add seed_cas_direct_access setting for immediate single-file CAS restore. Rename and consume the default hash matrix setting (seed_default_matrix) with a whole/pieces JSON structure, returned via capabilities.
ci(github): enforce AI disclosures and lock invalid issues
- Require exactly one AI disclosure option and a model name when AI is used
- Comment, close, lock, and label issues invalid for missing or malformed AI disclosures or missing model names
- Lock issues closed for unchanged placeholder titles
- Lock issues closed for selecting the unread checklist option
- Exclude AI disclosure options from unchecked task detection
- Skip automatic checks for announcement titles
- Update bilingual closure replies to mention locking and request a new issue
- Split issue replies and PR title checks into separate workflows without changing PR title rules
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Implement seed metadata editing (comment/trackers/channels/file comments/sources) and server-side hash recalculation with piece-size write-back and a bounded streaming reader. Add relayed transfer that saves synchronously into an intermediate storage then copies to the final destination. Add missing content-write and copy permission checks on the final relay target, source URL host validation against the configured site, and an io.LimitReader hard cap. Expose transfer/edit/recalculate in parse capabilities.
Add unified sharing-seed format library (openlist-sharing-seed v1), standard BT torrent v1 with x-openlist/x-cas extensions, and exact legacy-compatible CAS Base64 payload. Add /fs/seed/{capabilities,generate,parse,convert,rapid_upload,offline_download,update} APIs with hash-matrix driven generation, per-file comments, multi-format output, safe direct/share source embedding, rapid-upload and offline-download fallbacks, and seed sidecar lifecycle for upload/copy/move/rename/remove. Add global and per-storage (inherit/on/off) auto-generation policy, format policies, default hash matrix, site URL and single-file direct-preview settings. Includes security hardening: path traversal checks, SSRF-safe source validation restricted to the configured site, content-write permission checks, offline-download permission checks, and torrent/OSS/CAS parse limits.
* fix(drivers/139): improve mail login credential renewal
* fix(drivers/139): guard mail login client initialization
Fall back to base.NewRestyClient() when base.RestyClient has not been initialized, while preserving cloned global-client behavior and the login/SMS retry and redirect policies.
- Decode JSON time strings before normalizing Unicode spaces in both 189 drivers
- Exercise escaped spaces and existing date formats through JSON unmarshalling
- Cover invalid JSON input and XML time parsing
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
- Resolve and authorize the canonical destination from the request payload
- Reject upload capabilities that cross virtual storage mount boundaries
- Remove the unrelated File-Path middleware authorization check
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
* fix(cmd/start): use absolute executable path for child process
* fix(cmd/start): detect force-bin-dir flag variants
---------
Co-authored-by: Zoe Lee <zoelee@gmail.com>
multipartUploadToOSS passed the UpdateProgress callback into
stream.NewStreamSectionReader, which ignores the up argument, so the
callback was never invoked while uploading parts. As a result,
cross-storage copy tasks targeting GuangYaPan stayed at 0% and showed no
progress bar or speed in the task list.
Fix by tracking the uploaded byte count and invoking up after every
part is uploaded, keeping consistent with other drivers' upload flow.
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
The 115 and pikpak multipart uploaders checked for cancellation with a
`case <-ctx.Done()` inside a select, and a break there only leaves the
select, not the enclosing `for retry := 0; retry < 3; retry++` loop.
Cancelling an upload therefore ran all three attempts for every
remaining chunk, each allocating a chunk-sized buffer and reading it
off disk before firing a request that could not succeed, and reported
a transport error instead of the cancellation.
Move the check ahead of the select and use utils.IsCanceled, matching
the pattern the other drivers already use. Assigning ctx.Err() to err
is load-bearing: without it a cancelled chunk takes the success branch,
counts toward progress, and appends a zero-value UploadPart.
Found with staticcheck (SA4011).
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
* feat(strm): add local save permission mode
- add private and shared permission modes for local STRM files
- repair shared-mode directory and file permissions during generation
- add permission handling tests
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* fix(strm): respect deployment umask for local directories
- remove application-level permission modes and chmod operations
- create local STRM directories with umask-controlled permissions
- preserve existing permissions and test the behavior
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
---------
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* feat(ilanzou): modernize console API session and copy
Use an isolated cookie-backed API session and preserve raw appToken syntax required by current iLanzou endpoints.
Resolve CDN download responses more robustly, align upload metadata with the console protocol, and delegate copies to OpenList background tasks instead of blocking requests.
* fix(ilanzou): escape appToken query values
Escape opaque appToken values while preserving the literal colon required by iLanzou endpoints. Ignore CDN HEAD response lengths unless the response status is successful.
* fix(ilanzou): address driver review feedback
Use the ilanzou package name consistently, document upload result polling, and bound CDN size probes. Keep the appToken and CDN challenge handling covered by focused tests and comments.
* fix(ilanzou): use context timeout for HEAD probe
* feat(strm): add file size filtering for STRM generation
- add ParseSize helper function in pkg/utils to parse human-readable byte sizes
- add minFileSize setting field to STRM driver Addition struct
- filter out files below minFileSize threshold during STRM generation
* refactor(strm): use MB as default unit for minFileSize instead of ParseSize
Replace string-based ParseSize auto-conversion with a simple int64 number
field defaulting to MB, consistent with other drivers (Google Drive,
Teldrive). Remove the now-unused ParseSize function and its tests.
* feat(multipart): add chunk reassembly window
- Reassemble concurrently uploaded chunks into a sequential stream through a ring file, bounding disk usage to slots*chunkSize per session
- Park writers up to a deadline when their slot is busy instead of rejecting instantly, so flow control does not surface as connection errors in browsers
- Record a per-chunk CRC32 table for re-fill verification and keep it readable after close
- Propagate cancellation to blocked readers via CloseWithError so drivers treat aborts like canceled requests
- Cover ordering, backpressure, idempotent resends and close/abort wake-ups with race-enabled tests
* feat(multipart): add pipelined upload session manager
- Start the driver upload at session init over a sequential stream backed by the window, so client-to-server and server-to-storage transfers run concurrently
- Attach client-provided hashes to the stream so drivers can attempt rapid upload before any chunk arrives, and absorb chunks racing pipeline completion idempotently
- Keep only metadata and chunk CRCs after a failed attempt: re-sending chunk 0 re-fills a fresh window, and content changes between attempts are rejected
- Resume receiving sessions only when client hashes prove the same file; failed_retriable sessions resume unconditionally
- Reclaim sessions with a sliding-TTL GC and sweep orphaned ring files at startup
- Cover the state machine with race-enabled tests over a stubbed storage layer
* feat(setting): add multipart upload settings
- Add multipart_enabled and multipart_chunk_size (MB) as public traffic settings
- Validate the chunk size on save (integer within 1-90) via the setting item hook
* feat(server): add multipart upload API
- Add /api/fs/multipart init/chunk/complete/status/abort endpoints with headers aligned with /fs/put
- Gate init behind the FsUp permission checks and reuse the client upload rate limiter for chunk uploads
- Drain the request body before answering chunk requests on every path, so browsers do not see early responses as network errors
- Start the multipart session GC when the router is initialized to reclaim ring files orphaned by a previous run
* refactor(multipart): remove unused overwrite session field
- The overwrite flag was stored on the session but never read: the handler
performs the pre-check and op.Put owns the overwrite semantics
* feat(setting): allow any positive multipart chunk size
- Validate the setting as a positive integer only; self-hosted admins decide
the ceiling themselves instead of an arbitrary 90MB cap
- Keep clamping the client-suggested X-Chunk-Size to the admin value: the
server buffers a window of 8 chunks per session, so an unbounded client
suggestion would translate directly into server-side disk usage
* refactor(server): simplify multipart chunk size clamp
- Fold the two-step clamp into one branch: the ceiling is already floored,
so a client suggestion just lowers the size with a 1MB floor
The Login endpoint may return an acw_sc__v2 validation page when accessed,
which previously caused login failures. This change adds the same retry
logic and cookie handling already used in request() to automatically solve
the challenge, ensuring login success even when the anti-bot mechanism is
triggered.
* fix(drivers/139): update path handling for family
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* perf(drivers/139): add retry go for family upload
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* feat(drivers/139): add FamilyCloudHost and GroupCloudHost handling
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* perf(drivers/139): add retry go for personalnew upload
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): use new batchpartinfos for remaining parts
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(driver/139): do not use path in id
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(driver/139): refactor RootPath handling
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(driver/139): remove root path stripping in Init method
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* chore(drivers/139): reduce upload retry attempts to 3
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): add rate limiting for MetaPersonalNew upload
- Wrap stream with LimitedUploadStream before creating StreamSectionReader
- Aligns with the same pattern used in the MetaPersonal/MetaGroup/MetaFamily path
Co-authored-by: GitHub Copilot <copilot@github.com>
* fix(drivers/139): fix section reader leak and seek check in retry block
- Check rd.Seek() return value and free the section reader on error
- Call ss.FreeSectionReader(rd) on all error paths within retry.Do closure
- Prevents buffer.Block leak when retrying failed upload chunks
Co-authored-by: GitHub Copilot <copilot@github.com>
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): move GetSectionReader outside retry loop
- Move ss.GetSectionReader() before retry.Do() so the sequential
stream section reader is only called once per chunk
- Remove redundant ss.FreeSectionReader() calls from inside the
retry closure since the reader is now owned by the outer scope
- Fixes 'stream not cached' errors when retrying failed chunk
uploads during cross-storage WebDAV COPY operations
Co-authored-by: GitHub Copilot <copilot@github.com>
* feat(drivers/139): add UseOldStreamUpload option
- Add UseOldStreamUpload option
- Implement newRequest, newPost
- Support rapid upload for PersonalNew and Group/Family
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): add more param for group/family put
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): correct group params for new put
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): update personalPost to use getPersonalCloudHost
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* chore(drivers/139): correct typo
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): pass full partInfos slice for batched upload
- Pass the global partInfos slice instead of the batch subset to
uploadPersonalParts, so that PartNumber-1 indexing does not go
out of bounds when a file has more than 100 parts.
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* fix(drivers/139): avoid double-counting progress on upload retries
- Save p.Done before each part and reset it inside the retry function
so that bytes from failed attempts are not counted toward progress.
- Each part is counted exactly once, on the successful attempt.
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* fix(drivers/139): gate route-host checks by cloud type
- Only validate GroupCloudHost and FamilyCloudHost for MetaGroup
and MetaFamily storage types, so that personal-only accounts do
not fail initialization when the route policy omits group/family.
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
* fix(drivers/139): add ProviderRoot back for groupgetfiles
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): set path to 0 when group old upload
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* fix(drivers/139): improve error handling for family root path retrieval
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
* refactor(drivers/139): update condition checks for CloudHost initialization
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
---------
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: GitHub Copilot <copilot@github.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Recover temporarily missing or incomplete 115 Open path metadata from parent
and ancestor directory listings. Use 115-sdk-go v0.2.6 to normalize missing
object errors, while preserving valid zero-byte files and ensuring recursive
parent resolution progresses correctly.
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-22 12:59:41 +08:00
207 changed files with 17613 additions and 2096 deletions
⚠️ Please modify the title to better describe your issue or request, and remove the example prompt. This issue will be automatically closed and locked. If you wish to proceed, please create a new issue.
`;
} else if (confirmNotRead || !validAiDisclosure || missingAiModel) {
⚠️ Your issue does not comply with the submission rules. Please read the guidelines before submitting again. This issue will be automatically closed and locked. If you wish to proceed, please confirm that you have reviewed the rules before creating a new issue.
`;
} else if (/- \[ \] (?!我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody.replace(aiSection[0], ''))) {
comment = `感谢您联系OpenList。我们会尽快回复您。
Thanks for contacting OpenList. We will reply to you as soon as possible.
comment += "⚠️ Please modify the title to better describe your issue or request, and remove the example prompt. This issue will be automatically closed. If you wish to proceed, please create a new issue.\n";
comment += "⚠️ Your issue does not comply with the submission rules. Please read the guidelines before submitting again. This issue will be automatically closed. If you wish to proceed, please confirm that you have reviewed the rules before reopening or creating a new issue.\n";
await github.rest.issues.createComment({
...context.repo,
issue_number: context.issue.number,
body: comment
});
await github.rest.issues.update({
...context.repo,
issue_number: context.issue.number,
state: 'closed',
state_reason: 'not_planned',
labels: ['invalid']
});
return;
}
if (confirmHasRead) {
comment = "感谢您联系OpenList。我们会尽快回复您。\n";
comment += "Thanks for contacting OpenList. We will reply to you as soon as possible.\n\n";
comment += "⚠️ The PR title must start with `feat(): `, `docs(): `, `fix(): `, `style(): `, or `refactor(): `, `chore(): `. For example: `feat(component): add new feature`.\n\n";
⚠️ The PR title must start with \`feat(): \`, \`docs(): \`, \`fix(): \`, \`style(): \`, or \`refactor(): \`, \`chore(): \`. For example: \`feat(component): add new feature\`.
如果跨多个组件，请使用主要组件作为前缀，并在标题中枚举、描述中说明。
If it spans multiple components, use the main component as the prefix and enumerate in the title, describe in the body.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Pikachu Ren