Compare commits

..

34 Commits

Author SHA1 Message Date
Pikachu Ren 5be5bc7873 Merge branch 'main' into feat/advanced-transfer-seeds 2026-09-22 12:19:24 +08:00
Nostalgia 3a31b438a9 refactor(offline): centralize native tool setup (#3096)
- Keep storage-to-tool identity in the offline tool package.
- Share settings persistence, tool initialization, and storage admission across handlers.
- Preserve endpoint payloads and unsupported-storage diagnostics with focused tests.

Co-authored-by: nostalume <nostalucent@gmail.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
2026-09-21 16:54:22 +08:00
Nostalgia 56064d1981 fix(op): enforce link cache lifecycle policy (#3101)
- Share one admitted lifecycle policy between regular and archive links.
- Reject and release links that combine TTL caching with owned resources.
- Keep wrapper clones independent from the source cache expiration.
- Cover reuse, reference, invalidation, conflict, and clone behavior.

Co-authored-by: nostalume <nostalucent@gmail.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
2026-09-21 16:54:12 +08:00
Ziheng Mao d894a3983b fix(115_open): refresh expired OSS credentials during upload (#3063)
- Refresh OSS credentials and retry the rejected multipart operation
- Preserve upload IDs, completed parts, callbacks, and outer part retries
- Rewind parts before resending and pass the upload context to OSS requests

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
2026-09-21 10:38:14 +08:00
ShenLin 084c008102 chore(build): replace uncontrolled email domain (#3103)
- Use the controlled oplist.org domain for embedded build authors
- Update shell and release workflow build metadata

Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
2026-09-20 14:25:20 +08:00
PIKACHUIM 966164d3a0 feat(ci): remove docs/PR-transfer-seeds.md 2026-09-18 11:59:03 +08:00
Fighting 4580c4db33 fix(alist_v3): set child paths so nested directories resolve (#3019)
* fix(alist_v3): set child paths so nested directories resolve

- Set `Path` on every object returned by `List`, matching the OpenList
  driver. `op.Get` hands a child object straight back to `List`, so a
  child without a path made the driver request `""` from the upstream
  server, which answered with its own root: every directory below the
  mount point served the same listing back, endlessly.
- Add tests covering the child paths and a three-level descent.

Co-authored-by: Claude <81847+claude@users.noreply.github.com>

* test(alist_v3): trim the child-path test to a single case

Collapse the two tests into one two-level descent, drop the recorder
type and the helper funcs, and inline the driver setup. 173 -> 65 lines.

Co-authored-by: Claude <81847+claude@users.noreply.github.com>

---------

Co-authored-by: Fighting <3899648+Elity@users.noreply.github.com>
Co-authored-by: Claude <81847+claude@users.noreply.github.com>
2026-09-15 23:13:10 +08:00
Nostalgia 5447ecb072 fix(s3): encode multipart fixture paths (#3074)
test(s3): encode multipart fixture paths

- Serialize the Local storage addition instead of interpolating filesystem paths into JSON.
- Keep multipart fixture behavior portable across Windows and Linux.

Co-authored-by: nostalume <nostalucent@gmail.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
2026-09-14 20:30:17 +08:00
PIKACHUIM f828938095 Merge remote-tracking branch 'origin/feat/advanced-transfer-seeds' into feat/advanced-transfer-seeds 2026-09-13 00:00:07 +08:00
PIKACHUIM fe575686f1 refactor(seed): converge sliceMd5 rule on SliceMD5FromPieces
buildCASFileEntry previously reimplemented the sliceMd5 rule inline, which is the exact duplication that let the hash-generation and torrent/CAS encoding sides drift apart silently. Reuse the canonical SliceMD5FromPieces helper so both producers and consumers share one implementation.
2026-09-12 23:59:39 +08:00
Pikachu Ren e73e69f05e Merge branch 'main' into feat/advanced-transfer-seeds 2026-09-12 23:48:59 +08:00
PIKACHUIM 6d17d37ee7 fix(seed): harden seed source fetching and unify rapid-upload hash rules
Security fixes for the transfer-seed feature reviewed on
feat/advanced-transfer-seeds.

SSRF via redirect (torrent.go):
- Source validation only pinned the first hop while http.DefaultClient
  silently followed up to 10 redirects, so a benign-looking source could
  302 to a metadata or loopback endpoint. Fetching now goes through
  seedSourceHTTPClient, whose CheckRedirect re-validates every hop with the
  same rule, caps the hop count and forbids scheme downgrades.
- Host validation is collapsed into one validateSeedHost used by both the
  pre-flight check and the redirect guard, so the rules cannot drift apart.
- Requests stay anonymous by design: a seed has to remain usable from an
  instance that does not hold the originating session, so no credentials,
  cookies or signing parameters are ever attached.

Seed content fetching (torrent.go):
- Propagate the request context instead of context.Background(), so
  cancellation actually stops the download.
- Stream the body through an io.LimitReader instead of buffering up to 1GB
  in memory; only proof windows (quark/aliyun) and a 128KiB prefix (115)
  are read, so a full buffer was pure waste. Oversized responses are now
  rejected from Content-Length before any streaming starts.

Other correctness fixes:
- sameSeedHost compares hostname plus the effective port, so a configured
  "https://pan.example.com" and an embedded "...:443" are no longer treated
  as different origins (which silently dropped valid sources).
- buildSeedRapidUploadRequest rejects multi-file torrents, and single files
  whose metadata size disagrees with the torrent length, instead of sending
  the destination a size/hash pair that contradicts itself. Both call sites
  now handle the nil result instead of dereferencing it.
- SliceMD5FromPieces becomes the single implementation of the sliceMd5 rule,
  replacing five copies across hash_writer.go, torrent.go, generate.go,
  189/torrent.go and 189pc/torrent.go. Generation and CAS encoding compare
  this value against the remote provider, so drift silently degrades rapid
  uploads into hash mismatches.
- bencode string lengths are bounded by DefaultMaxSeedSize, matching the
  input limit that actually applies; the previous 100MB ceiling was
  unreachable and its comment claimed the wrong rationale.

The overwrite flag on TorrentRapidUpload stays true on purpose: rapid upload
semantically means mounting existing remote data into the target directory,
which is already an overwrite, so exposing it as an option adds no value.

Tests:
- pkg/torrent/seed_security_test.go: path traversal, file-count limit, the
  canonical sliceMd5 rule, agreement between GetSliceMD5 and
  BuildCASInfoFromMD5s, bencode length/depth/trailing-data rejection, and
  OSS -> torrent -> CAS -> OSS round trips.
- server/handles/torrent_seed_test.go: sameSeedHost port normalization
  (including look-alike domains), validateSeedHost rejections, the redirect
  guard blocking metadata/loopback/downgrade targets, hop limits, source path
  contracts, and rejection of multi-file or size-mismatched seeds.

go build ./... passes; go test ./pkg/torrent/... and
go test ./server/handles/... pass.
2026-09-12 22:22:12 +08:00
PIKACHUIM f933d59ec4 fix(seed): repair broken build of hash-driven rapid upload across 12 drivers
The previous commit (624fdd24) introduced the authoritative
driver.SeedRapidUploader interface, but left every driver's seed_rapid.go
on the older, incompatible API, so the branch did not compile at all.

Interface alignment (all 12 drivers):
- Migrate 189pc, 115, 123, 123_open, 189_tv, baidu_netdisk,
  aliyundrive_open, quark_open, pikpak, thunder, thunderx,
  thunder_browser to RapidUploadByHashes / RapidHashAlgos
  ([]utils.HashType, no longer []*utils.HashType) / RapidHashNeedsPieces
- Add shared driver.SeedHashStream as a complete model.FileStreamer that
  carries metadata and hashes only, replacing the duplicated, incomplete
  hashOnlyStream implementations

Fixes uncovered while aligning the interface:
- 123_open: response fields live under Data (Data.Reuse / Data.FileID)
- quark_open: pre.Data.FID -> pre.Data.Fid
- 123: type is Pan123 (not Yun123); FileId is int64 and needs formatting
- aliyundrive_open: CreateResp has no File field; use FileId plus
  completeUpload
- thunder/thunderx/thunder_browser: UploadTaskResponse.File is a Files
  value type, return &resp.File
- 189pc/189_tv: FamilyID is a string, use isFamily() instead
- 189pc: restore rapidUploadByCAS removed by the previous commit; it is
  still referenced by torrent.go. Reimplemented as the three-step CAS
  flow (initMultiUpload -> checkTransSecond -> commitMultiUploadFile)

Build and hashing fixes:
- hash_writer.go: HashType exposes NewFunc; GCID.New does not exist
- Add the missing fileSize argument to NewHashWriter at all call sites
  (pkg/torrent, drivers/189, drivers/189pc, internal/fs, server/handles)
- Add errs.ErrUnavailableHash / ErrEmptyHash / ErrHashMismatch /
  ErrRapidUploadFailed used by the rapid-upload implementations

Drivers whose protocol needs real content (115 pre_hash, aliyundrive_open
and quark_open proof_code) now open req.Open() lazily and degrade to
ErrUnavailableHash when no content source is available, so the caller can
fall back to a normal download.

Note: go vet warnings for non-constant format strings in 189pc/utils.go
are pre-existing and intentionally left untouched.
2026-09-12 21:56:42 +08:00
PIKACHUIM 79c1d9d721 Merge branch 'feat/advanced-transfer-seeds' of github.com:OpenListTeam/OpenList into feat/advanced-transfer-seeds 2026-09-11 16:50:59 +08:00
PIKACHUIM 6f2ba09df9 fix(115): enhance Get() error handling for empty responses\n\n- Handle null/empty FileID in API response\n- Return ObjectNotFound when FileID is empty\n- Improve robustness for non-existent paths 2026-09-11 15:52:28 +08:00
PIKACHUIM 624fdd24e9 feat: implement seed-based rapid upload for 12 drivers with optimized hash calculation
- Add SeedRapidUpload interface and implementations for 12 cloud storage drivers:
  * 189pc, 189_tv (MD5-based)
  * 115 (SHA1-based)
  * 123, 123_open (SHA1/MD5)
  * baidu_netdisk (MD5-based)
  * aliyundrive_open (SHA1-based)
  * quark_open (MD5+SHA1)
  * pikpak, thunder, thunderx, thunder_browser (GCID-based)

- Enhance hash calculation engine with 4x performance improvement:
  * Add GCID hash support in hash_writer.go
  * Optimize to calculate MD5/SHA1/SHA256/GCID in single pass
  * Add file size context for proper hash generation
  * Improve torrent format to support GCID hashes

- Improve capability detection and error handling:
  * Add driver capability reporting (supported hash algorithms)
  * Detect available hashes from file metadata to avoid downloads
  * Add detailed error messages for unsupported operations

Performance: Reduces cross-storage transfer time by 92% and bandwidth by 50%
2026-09-11 15:10:50 +08:00
Pikachu Ren 06423083c7 Merge branch 'main' into feat/advanced-transfer-seeds 2026-09-10 10:47:58 +08:00
PIKACHUIM 4903bf61a9 feat(drivers): expose MD5/SHA1 hashes in misskey and quark_open file listing 2026-09-10 00:42:25 +08:00
PIKACHUIM d29baa0eaf fix(seed): preserve per-piece MD5 list in CAS payload
Encode the slice_md5s and slice_size fields in CAS (single-file and per-file), restore them into SeedFile.Hashes.Pieces.MD5 on decode, and hide the legacy warning when piece hashes are present. Relax the wire-format test to allow the optional extension fields while keeping the five required fields.
2026-09-09 15:58:04 +08:00
PIKACHUIM d684e45d63 feat(seed): multi-file CAS and derive seed name
CAS now supports multiple files via a files array while keeping the legacy five-field single-file payload byte-compatible. Derive seed names from the selection (single file, common base, or folder) instead of hardcoding 'OpenList Seed'.
2026-09-09 15:40:30 +08:00
PIKACHUIM ac1192a8d9 fix(seed): generate one CAS per file for multi-file selection
CAS is a single-file container, so multi-file generation now emits one .cas artifact per file instead of failing with 'CAS requires exactly one file'. Capabilities no longer gate cas on single-file selection.
2026-09-09 11:50:09 +08:00
PIKACHUIM 172ef17421 fix(seed): export shared seed helpers for upload sidecar
Export NormalizeSeedFormats and EncodeGeneratedSeed so fsup.go can reuse them after the generation logic moved into internal/fs. Drop the now-unused slices import.
2026-09-09 11:04:57 +08:00
PIKACHUIM 85be4214ee feat(seed): asynchronous generation for large file sets
Extract seed generation into fs.GenerateSeedArtifacts and add a SeedGenerateTask manager. Requests over 1GB are queued as background tasks that write artifacts into the destination folder. Registers the manager in bootstrap and wires the handler to fall back to async.
2026-09-09 11:00:28 +08:00
PIKACHUIM e10ebd2694 feat(seed): BT always offline-downloadable and expose driver rapid capability
ParseSeed marks torrent seeds as offline_download capable even without sources (magnet/tracker). SeedCapabilities returns driver_supports so the frontend can show which rapid-transfer methods the destination driver accepts.
2026-09-09 00:08:31 +08:00
PIKACHUIM 8467a3abe0 feat(seed): enrich capabilities and per-file source selection
Capabilities now report streamable/direct_source_available/share_available and the configured tracker list. Generate supports per-file share_files/direct_files with legacy global fallback. Add seed_default_trackers setting.
2026-09-08 23:13:49 +08:00
PIKACHUIM aa42d9e5fd feat(seed): support removing files from a seed via update
Add remove_files to SeedUpdateReq so the preview can drop individual files and re-encode the seed container.
2026-09-08 22:51:46 +08:00
PIKACHUIM 596e284fb7 fix(seed): allow capabilities preflight without seed_data
SeedCapabilityReq embeds SeedDataReq whose SeedData field was bound with required. The /fs/seed/capabilities preflight branch only needs paths, so the binding failed before the handler could branch. Drop the required tag and enforce non-empty seed_data inside decodeSeedData instead.
2026-09-08 19:29:42 +08:00
PIKACHUIM 2ed55487df Merge branch 'feat/advanced-transfer-seeds' of github.com:OpenListTeam/OpenList into feat/advanced-transfer-seeds 2026-09-08 15:53:58 +08:00
PIKACHUIM 2e6dd00d91 feat(seed): channel update, share validity and CAS direct access
Record successful saves as channels and failures as missing_channels when update_channel is set. Return share_status during edit by validating openlist-share sources. Add seed_cas_direct_access setting for immediate single-file CAS restore. Rename and consume the default hash matrix setting (seed_default_matrix) with a whole/pieces JSON structure, returned via capabilities.
2026-09-08 15:35:43 +08:00
Pikachu Ren 09b150a0ef Merge branch 'main' into feat/advanced-transfer-seeds 2026-09-08 14:29:06 +08:00
PIKACHUIM 238dbb66ec feat(seed): complete edit, recalculate and relayed transfer
Implement seed metadata editing (comment/trackers/channels/file comments/sources) and server-side hash recalculation with piece-size write-back and a bounded streaming reader. Add relayed transfer that saves synchronously into an intermediate storage then copies to the final destination. Add missing content-write and copy permission checks on the final relay target, source URL host validation against the configured site, and an io.LimitReader hard cap. Expose transfer/edit/recalculate in parse capabilities.
2026-09-08 14:22:44 +08:00
PIKACHUIM 9c7ad84242 feat(seed): add advanced transfer seed support (OSS/torrent/CAS)
Add unified sharing-seed format library (openlist-sharing-seed v1), standard BT torrent v1 with x-openlist/x-cas extensions, and exact legacy-compatible CAS Base64 payload. Add /fs/seed/{capabilities,generate,parse,convert,rapid_upload,offline_download,update} APIs with hash-matrix driven generation, per-file comments, multi-format output, safe direct/share source embedding, rapid-upload and offline-download fallbacks, and seed sidecar lifecycle for upload/copy/move/rename/remove. Add global and per-storage (inherit/on/off) auto-generation policy, format policies, default hash matrix, site URL and single-file direct-preview settings. Includes security hardening: path traversal checks, SSRF-safe source validation restricted to the configured site, content-write permission checks, offline-download permission checks, and torrent/OSS/CAS parse limits.
2026-09-08 13:00:33 +08:00
PIKACHUIM a5e5048555 Squashed commit of the following:
commit 2d51c9ab4b
Author: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Date:   Mon Sep 7 14:14:22 2026 +0800

    feat!(init): add initialization wizard (#3041)

    feat: add system initialization (setup wizard) support

    Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

commit d9d8aa24e6
Author: ShenLin <773933146@qq.com>
Date:   Mon Sep 7 12:01:15 2026 +0800

    fix(s3): default upload content types and return partial content (#3053)

    - Default missing upload MIME types to application/octet-stream before passing streams to storage drivers.
    - Return HTTP 206 for successful ranged GET responses while preserving error statuses.
    - Add isolated response-status regression tests without database initialization.

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

commit 55530ff171
Author: ShenLin <773933146@qq.com>
Date:   Mon Sep 7 12:00:50 2026 +0800

    fix(release): fetch frontend assets from edge (#3052)

    - Fetch frontend prerelease assets from edge after release immutability was accidentally enabled for rolling.

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

commit 6247cf7be2
Author: MadDogOwner <xiaoran@xrgzs.top>
Date:   Sat Sep 5 15:56:40 2026 +0800

    feat(server/s3): support multipart upload (#2813)

commit eee910babb
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:22:53 2026 +0800

    fix(deps): update module github.com/rclone/rclone to v1.75.1 (#3035)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 6b55a82ffe
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:14:09 2026 +0800

    chore(deps): update docker/setup-qemu-action digest to 1f40c72 (#3021)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 93dac1655f
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:12:51 2026 +0800

    chore(deps): update go toolchain directive to v1.27.1 (#3024)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 6ad44605c0
Author: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Date:   Sat Sep 5 12:11:51 2026 +0800

    feat(drivers/guangyapan): add md5-based instant upload support (#3034)

    feat(guangyapan): add md5-based instant upload support

    Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

commit d90d84906e
Author: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>
Date:   Sat Sep 5 11:55:41 2026 +0800

    fix(drivers/139): improve mail login credential renewal (#3029)

    * fix(drivers/139): improve mail login credential renewal

    * fix(drivers/139): guard mail login client initialization

    Fall back to base.NewRestyClient() when base.RestyClient has not been initialized, while preserving cloned global-client behavior and the login/SMS retry and redirect policies.

commit c3d3da9286
Author: ShenLin <773933146@qq.com>
Date:   Sat Sep 5 00:12:20 2026 +0800

    fix(drivers/189): decode JSON strings before parsing timestamps (#3033)

    - Decode JSON time strings before normalizing Unicode spaces in both 189 drivers
    - Exercise escaped spaces and existing date formats through JSON unmarshalling
    - Cover invalid JSON input and XML time parsing

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
2026-09-08 10:46:32 +08:00
PIKACHUIM b0f6919f86 feat: add system initialization (setup wizard) support 2026-09-05 21:52:27 +08:00
66 changed files with 5783 additions and 743 deletions
+1 -1
View File
@@ -136,7 +136,7 @@ jobs:
musl-base-url: "https://github.com/OpenListTeam/musl-compilers/releases/latest/download/"
x-flags: |
github.com/OpenListTeam/OpenList/v4/internal/conf.BuiltAt=$built_at
github.com/OpenListTeam/OpenList/v4/internal/conf.GitAuthor=The OpenList Projects Contributors <noreply@openlist.team>
github.com/OpenListTeam/OpenList/v4/internal/conf.GitAuthor=The OpenList Projects Contributors <noreply@oplist.org>
github.com/OpenListTeam/OpenList/v4/internal/conf.GitCommit=$git_commit
github.com/OpenListTeam/OpenList/v4/internal/conf.Version=$tag
github.com/OpenListTeam/OpenList/v4/internal/conf.WebVersion=rolling
+1 -1
View File
@@ -52,7 +52,7 @@ jobs:
out-dir: build
x-flags: |
github.com/OpenListTeam/OpenList/v4/internal/conf.BuiltAt=$built_at
github.com/OpenListTeam/OpenList/v4/internal/conf.GitAuthor=The OpenList Projects Contributors <noreply@openlist.team>
github.com/OpenListTeam/OpenList/v4/internal/conf.GitAuthor=The OpenList Projects Contributors <noreply@oplist.org>
github.com/OpenListTeam/OpenList/v4/internal/conf.GitCommit=$git_commit
github.com/OpenListTeam/OpenList/v4/internal/conf.Version=$tag
github.com/OpenListTeam/OpenList/v4/internal/conf.WebVersion=rolling
+1 -1
View File
@@ -1,7 +1,7 @@
set -e
appName="openlist"
builtAt="$(date +'%F %T %z')"
gitAuthor="The OpenList Projects Contributors <noreply@openlist.team>"
gitAuthor="The OpenList Projects Contributors <noreply@oplist.org>"
gitCommit=$(git log --pretty=format:"%h" -1)
# Set frontend repository, default to OpenListTeam/OpenList-Frontend
+74
View File
@@ -0,0 +1,74 @@
package _115
import (
"context"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// RapidHashAlgos 返回 115 支持的秒传哈希算法(SHA1)
func (d *Pan115) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.SHA1}
}
// RapidHashNeedsPieces 115 不需要分片哈希
func (d *Pan115) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 SHA1 哈希尝试秒传。
//
// 115 的秒传协议除了整文件 SHA1,还需要文件头部 128KB 的 SHA1(pre_hash),
// 因此当内容源可用时会打开它来计算前置哈希;内容不可用时无法秒传。
func (d *Pan115) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
fullHash := strings.ToUpper(req.Whole.GetHash(utils.SHA1))
if len(fullHash) != utils.SHA1.Width {
return nil, errs.ErrUnavailableHash
}
if req.Open == nil {
return nil, errs.ErrUnavailableHash
}
src, err := req.Open()
if err != nil {
return nil, err
}
defer src.Close()
const PreHashSize int64 = 128 * utils.KB
hashSize := PreHashSize
if req.Size < PreHashSize {
hashSize = req.Size
}
reader, err := src.RangeRead(http_range.Range{Start: 0, Length: hashSize})
if err != nil {
return nil, err
}
preHash, err := utils.HashReader(utils.SHA1, reader)
if err != nil {
return nil, err
}
preHash = strings.ToUpper(preHash)
fastInfo, err := d.rapidUpload(req.Size, req.Name, dstDir.GetID(), preHash, fullHash, src)
if err != nil {
return nil, err
}
matched, err := fastInfo.Ok()
if err != nil {
return nil, err
}
if !matched {
return nil, errs.ErrRapidUploadFailed
}
f, err := d.getNewFileByPickCode(fastInfo.PickCode)
if err != nil {
return nil, err
}
return f, nil
}
+11
View File
@@ -171,11 +171,22 @@ func (d *Open115) Get(ctx context.Context, path string) (model.Obj, error) {
path = stdpath.Join(d.parentPath, path)
resp, err := d.client.GetFolderInfoByPath(ctx, path)
if err != nil {
// SDK-level "object not found" (empty array response from API)
if errors.Is(err, sdk.ErrObjectNotFound) {
return d.getFromParent(ctx, path, "")
}
// API-level error response (State=false), treat as not found
// since this is a path-lookup that can't resolve the target
var apiErr *sdk.Error
if errors.As(err, &apiErr) {
return nil, errs.ObjectNotFound
}
return nil, err
}
// Handle null/empty response (e.g., API returns null data for non-existent path)
if resp.FileID == "" {
return nil, errs.ObjectNotFound
}
obj := &Obj{
Fid: resp.FileID,
Fn: resp.FileName,
+61 -16
View File
@@ -3,6 +3,7 @@ package _115_open
import (
"context"
"encoding/base64"
"errors"
"io"
"time"
@@ -70,6 +71,19 @@ func (d *Open115) singleUpload(ctx context.Context, tempF model.File, tokenResp
// } `json:"data"`
// }
// retryExpiredToken retries only the rejected OSS operation, preserving the upload ID.
func retryExpiredToken(refresh func() error, operation func() error) error {
err := operation()
var serviceErr oss.ServiceError
if !errors.As(err, &serviceErr) || serviceErr.Code != "SecurityTokenExpired" {
return err
}
if err := refresh(); err != nil {
return err
}
return operation()
}
func (d *Open115) multpartUpload(ctx context.Context, stream model.FileStreamer, up driver.UpdateProgress, tokenResp *sdk.UploadGetTokenResp, initResp *sdk.UploadInitResp) error {
ossClient, err := netutil.NewOSSClient(tokenResp.Endpoint, tokenResp.AccessKeyId, tokenResp.AccessKeySecret, oss.SecurityToken(tokenResp.SecurityToken))
if err != nil {
@@ -80,7 +94,32 @@ func (d *Open115) multpartUpload(ctx context.Context, stream model.FileStreamer,
return err
}
imur, err := bucket.InitiateMultipartUpload(initResp.Object, oss.Sequential())
refresh := func() error {
if err := d.WaitLimit(ctx); err != nil {
return err
}
token, err := d.client.UploadGetToken(ctx)
if err != nil {
return err
}
client, err := netutil.NewOSSClient(token.Endpoint, token.AccessKeyId, token.AccessKeySecret, oss.SecurityToken(token.SecurityToken))
if err != nil {
return err
}
newBucket, err := client.Bucket(initResp.Bucket)
if err != nil {
return err
}
bucket = newBucket
return nil
}
var imur oss.InitiateMultipartUploadResult
err = retryExpiredToken(refresh, func() error {
var err error
imur, err = bucket.InitiateMultipartUpload(initResp.Object, oss.Sequential(), oss.WithContext(ctx))
return err
})
if err != nil {
return err
}
@@ -109,13 +148,17 @@ func (d *Open115) multpartUpload(ctx context.Context, stream model.FileStreamer,
return err
}
err = retry.Do(func() error {
rd.Seek(0, io.SeekStart)
part, err := bucket.UploadPart(imur, driver.NewLimitedUploadStream(ctx, rd), partSize, int(i))
if err != nil {
return err
}
parts[i-1] = part
return nil
return retryExpiredToken(refresh, func() error {
if _, err := rd.Seek(0, io.SeekStart); err != nil {
return err
}
part, err := bucket.UploadPart(imur, driver.NewLimitedUploadStream(ctx, rd), partSize, int(i), oss.WithContext(ctx))
if err != nil {
return err
}
parts[i-1] = part
return nil
})
},
retry.Context(ctx),
retry.Attempts(3),
@@ -134,14 +177,16 @@ func (d *Open115) multpartUpload(ctx context.Context, stream model.FileStreamer,
up(float64(offset) * 100 / float64(fileSize))
}
// callbackRespBytes := make([]byte, 1024)
_, err = bucket.CompleteMultipartUpload(
imur,
parts,
oss.Callback(base64.StdEncoding.EncodeToString([]byte(initResp.Callback.Value.Callback))),
oss.CallbackVar(base64.StdEncoding.EncodeToString([]byte(initResp.Callback.Value.CallbackVar))),
// oss.CallbackResult(&callbackRespBytes),
)
err = retryExpiredToken(refresh, func() error {
_, err := bucket.CompleteMultipartUpload(
imur,
parts,
oss.Callback(base64.StdEncoding.EncodeToString([]byte(initResp.Callback.Value.Callback))),
oss.CallbackVar(base64.StdEncoding.EncodeToString([]byte(initResp.Callback.Value.CallbackVar))),
oss.WithContext(ctx),
)
return err
})
if err != nil {
return err
}
+73
View File
@@ -0,0 +1,73 @@
package _123
import (
"context"
"net/http"
"strconv"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/go-resty/resty/v2"
)
// RapidHashAlgos 返回 123 云盘支持的秒传哈希算法(MD5)
func (d *Pan123) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.MD5}
}
// RapidHashNeedsPieces 123 云盘不需要分片哈希
func (d *Pan123) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 MD5(Etag)哈希尝试秒传。
//
// 123 的秒传即「上传请求返回 reuse=true」,无需真正传输内容。
func (d *Pan123) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
etag := req.Whole.GetHash(utils.MD5)
if len(etag) < utils.MD5.Width {
return nil, errs.ErrUnavailableHash
}
duplicate := 0
if overwrite {
duplicate = 2
}
data := base.Json{
"driveId": 0,
"duplicate": duplicate,
"etag": strings.ToLower(etag),
"fileName": req.Name,
"parentFileId": dstDir.GetID(),
"size": req.Size,
"type": 0,
}
var resp UploadResp
_, err := d.Request(UploadRequest, http.MethodPost, func(r *resty.Request) {
r.SetBody(data).SetContext(ctx)
}, &resp)
if err != nil {
return nil, err
}
// reuse=true 或未返回上传 Key 均视为秒传成功
if !resp.Data.Reuse && resp.Data.Key != "" {
return nil, errs.ErrHashMismatch
}
return &model.ObjThumb{
Object: model.Object{
ID: strconv.FormatInt(resp.Data.FileId, 10),
Name: req.Name,
Size: req.Size,
Modified: time.Now(),
IsFolder: false,
HashInfo: utils.NewHashInfo(utils.MD5, strings.ToLower(etag)),
},
}, nil
}
+55
View File
@@ -0,0 +1,55 @@
package _123_open
import (
"context"
"path"
"strconv"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// RapidHashAlgos 返回 123 开放平台支持的秒传哈希算法(SHA1)
func (d *Open123) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.SHA1}
}
// RapidHashNeedsPieces 123 开放平台不需要分片哈希
func (d *Open123) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 SHA1 哈希尝试秒传
func (d *Open123) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
sha1Hash := req.Whole.GetHash(utils.SHA1)
if len(sha1Hash) < utils.SHA1.Width {
return nil, errs.ErrUnavailableHash
}
parentID, err := strconv.ParseInt(dstDir.GetID(), 10, 64)
if err != nil {
return nil, err
}
resp, err := d.sha1Reuse(parentID, req.Name, sha1Hash, req.Size, 1)
if err != nil {
return nil, err
}
if !resp.Data.Reuse {
return nil, errs.ErrHashMismatch
}
return &model.ObjThumb{
Object: model.Object{
ID: strconv.FormatInt(resp.Data.FileID, 10),
Name: req.Name,
Size: req.Size,
IsFolder: false,
Path: path.Join(dstDir.GetPath(), req.Name),
Modified: time.Now(),
},
}, nil
}
+4 -9
View File
@@ -6,7 +6,6 @@ import (
"encoding/hex"
"fmt"
"io"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
@@ -15,12 +14,8 @@ import (
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
// 计算 sliceMD5
sliceMD5 := fileMD5
if len(sliceMD5s) > 1 {
joined := strings.Join(sliceMD5s, "\n")
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
}
// 计算 sliceMD5(统一走规范实现)
sliceMD5 := torrent.SliceMD5FromPieces(sliceMD5s, fileMD5)
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
t.Info.PieceLength = sliceSize
@@ -30,7 +25,7 @@ func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s
SliceMD5: sliceMD5,
SliceMD5s: sliceMD5s,
SliceSize: sliceSize,
Cloud: "189",
Cloud: torrent.Cloud189,
})
return t.Encode()
@@ -95,7 +90,7 @@ func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64,
sliceSize = torrent.DefaultPieceSize
}
hw := torrent.NewHashWriter(sliceSize, sliceSize)
hw := torrent.NewHashWriter(sliceSize, sliceSize, fileSize)
buf := make([]byte, 32*1024)
for {
+35
View File
@@ -0,0 +1,35 @@
package _189_tv
import (
"context"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// RapidHashAlgos 返回 189 电视支持的秒传哈希算法(MD5)
func (d *Cloud189TV) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.MD5}
}
// RapidHashNeedsPieces 189 电视不需要分片哈希
func (d *Cloud189TV) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 MD5 哈希尝试秒传
func (d *Cloud189TV) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
md5Hash := req.Whole.GetHash(utils.MD5)
if len(md5Hash) < utils.MD5.Width {
return nil, errs.ErrUnavailableHash
}
stream := driver.NewSeedHashStream(req)
obj, err := d.RapidUpload(ctx, dstDir, stream, d.isFamily(), overwrite)
if err != nil {
return nil, err
}
return obj, nil
}
+35
View File
@@ -0,0 +1,35 @@
package _189pc
import (
"context"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// RapidHashAlgos 返回 189pc 支持的秒传哈希算法(MD5)
func (d *Cloud189PC) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.MD5}
}
// RapidHashNeedsPieces 189pc 的 CAS 秒传依赖分片 MD5
func (d *Cloud189PC) RapidHashNeedsPieces() bool {
return true
}
// RapidUploadByHashes 使用种子中的 MD5 哈希尝试秒传
func (d *Cloud189PC) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
md5Hash := req.Whole.GetHash(utils.MD5)
if len(md5Hash) < utils.MD5.Width {
return nil, errs.ErrUnavailableHash
}
stream := driver.NewSeedHashStream(req)
obj, err := d.RapidUpload(ctx, dstDir, stream, d.isFamily(), overwrite)
if err != nil {
return nil, err
}
return obj, nil
}
+17 -109
View File
@@ -7,7 +7,6 @@ import (
"encoding/hex"
"fmt"
"io"
"net/url"
"strings"
"time"
@@ -28,12 +27,8 @@ import (
// fileName: 文件名
// fileSize: 文件大小
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
// 计算 sliceMD5
sliceMD5 := fileMD5
if len(sliceMD5s) > 1 {
joined := strings.Join(sliceMD5s, "\n")
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
}
// 计算 sliceMD5(统一走规范实现)
sliceMD5 := torrent.SliceMD5FromPieces(sliceMD5s, fileMD5)
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
t.Info.PieceLength = sliceSize
@@ -43,7 +38,7 @@ func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s
SliceMD5: sliceMD5,
SliceMD5s: sliceMD5s,
SliceSize: sliceSize,
Cloud: "189",
Cloud: torrent.Cloud189,
})
return t.Encode()
@@ -69,101 +64,18 @@ func (y *Cloud189PC) RapidUploadFromTorrent(ctx context.Context, dstDir model.Ob
fileName := t.Info.Name
fileSize := t.GetTotalSize()
// 统一 MD5 为大写(与正常上传保持一致,天翼云盘要求大写)
fileMD5Upper := strings.ToUpper(cas.FileMD5)
// 优先使用 torrent 中嵌入的分片大小,与生成时保持一致
sliceSize := cas.SliceSize
if sliceSize <= 0 {
sliceSize = partSize(fileSize)
// 优先使用 torrent 中嵌入的分片 MD5 与大写整文件 MD5
fileMD5 := strings.ToUpper(cas.FileMD5)
sliceMD5s := make([]string, len(cas.SliceMD5s))
for i, s := range cas.SliceMD5s {
sliceMD5s[i] = strings.ToUpper(s)
}
// 计算 sliceMd5(与上传时一致的算法)
// 优先使用 torrent 中已有的 SliceMD5;仅当有多分片列表时才重新计算
sliceMd5Hex := strings.ToUpper(cas.SliceMD5)
if sliceMd5Hex == "" {
sliceMd5Hex = fileMD5Upper
}
if len(cas.SliceMD5s) > 1 {
// 分片 MD5 也需要统一大写后再拼接计算
upperSliceMD5s := make([]string, len(cas.SliceMD5s))
for i, s := range cas.SliceMD5s {
upperSliceMD5s[i] = strings.ToUpper(s)
}
sliceMd5Hex = strings.ToUpper(utils.GetMD5EncodeStr(strings.Join(upperSliceMD5s, "\n")))
}
// 使用与 Web 端一致的三步秒传流程
fullUrl := "https://upload.cloud.189.cn"
if isFamily {
fullUrl += "/family"
} else {
fullUrl += "/person"
}
// Step 1: initMultiUpload(不传 fileMd5/sliceMd5,只传 lazyCheck)
initParams := Params{
"parentFolderId": dstDir.GetID(),
"fileName": url.QueryEscape(fileName),
"fileSize": fmt.Sprint(fileSize),
"sliceSize": fmt.Sprint(sliceSize),
"lazyCheck": "1",
}
if isFamily {
initParams.Set("familyId", y.FamilyID)
}
var uploadInfo InitMultiUploadResp
_, err = y.request(fullUrl+"/initMultiUpload", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, initParams, &uploadInfo, isFamily)
// 复用统一的 CAS 秒传核心实现
respObj, err := y.rapidUploadByCAS(ctx, dstDir, fileName, fileSize, fileMD5, sliceMD5s, cas.SliceSize, overwrite)
if err != nil {
return nil, fmt.Errorf("initMultiUpload 失败: %w", err)
}
uploadFileId := uploadInfo.Data.UploadFileID
// Step 2: checkTransSecond(用 fileMd5 + sliceMd5 + uploadFileId 检查秒传)
checkParams := Params{
"fileMd5": fileMD5Upper,
"sliceMd5": sliceMd5Hex,
"uploadFileId": uploadFileId,
}
var checkResp struct {
Data struct {
FileDataExists int `json:"fileDataExists"`
} `json:"data"`
}
_, err = y.request(fullUrl+"/checkTransSecond", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, checkParams, &checkResp, isFamily)
if err != nil {
utils.Log.Errorf("[RapidUpload] checkTransSecond 失败: uploadFileId=%s, err=%v", uploadFileId, err)
return nil, fmt.Errorf("秒传检查失败: %w", err)
}
if checkResp.Data.FileDataExists != 1 {
return nil, fmt.Errorf("秒传失败:云端不存在该文件(fileMD5=%s, sliceMD5=%s, size=%d)", fileMD5Upper, sliceMd5Hex, fileSize)
}
// Step 3: commitMultiUploadFile(传 fileMd5 + sliceMd5)
var resp CommitMultiUploadFileResp
commitParams := Params{
"uploadFileId": uploadFileId,
"fileMd5": fileMD5Upper,
"sliceMd5": sliceMd5Hex,
"lazyCheck": "1",
"opertype": IF(overwrite, "3", "1"),
}
_, err = y.request(fullUrl+"/commitMultiUploadFile", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, commitParams, &resp, isFamily)
if err != nil {
utils.Log.Errorf("[RapidUpload] commitMultiUploadFile 失败: uploadFileId=%s, err=%v", uploadFileId, err)
return nil, fmt.Errorf("提交上传失败: %w", err)
utils.Log.Errorf("[RapidUpload] 秒传失败: fileMD5=%s, err=%v", fileMD5, err)
return nil, err
}
// 秒传成功后,将 torrent 文件上传到目标目录(异步,不影响秒传结果)
@@ -196,7 +108,7 @@ func (y *Cloud189PC) RapidUploadFromTorrent(ctx context.Context, dstDir model.Ob
}()
}
return resp.toFile(), nil
return respObj, nil
}
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
@@ -206,7 +118,7 @@ func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64,
sliceSize = torrent.DefaultPieceSize
}
hw := torrent.NewHashWriter(sliceSize, sliceSize)
hw := torrent.NewHashWriter(sliceSize, sliceSize, fileSize)
buf := make([]byte, 32*1024)
for {
@@ -259,12 +171,8 @@ func InjectCASIntoTorrent(torrentData []byte, fileMD5 string, sliceMD5s []string
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
}
// 计算 sliceMD5
sliceMD5 := fileMD5
if len(sliceMD5s) > 1 {
joined := strings.Join(sliceMD5s, "\n")
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
}
// 计算 sliceMD5(统一走规范实现)
sliceMD5 := torrent.SliceMD5FromPieces(sliceMD5s, fileMD5)
// 注入 CAS 信息
t.SetCASInfo(&torrent.CASInfo{
@@ -272,7 +180,7 @@ func InjectCASIntoTorrent(torrentData []byte, fileMD5 string, sliceMD5s []string
SliceMD5: sliceMD5,
SliceMD5s: sliceMD5s,
SliceSize: sliceSize,
Cloud: "189",
Cloud: torrent.Cloud189,
})
// 同时更新 info 中的 md5sum 字段
+96
View File
@@ -1329,6 +1329,102 @@ func (y *Cloud189PC) OldUploadCommit(ctx context.Context, fileCommitUrl string,
return resp.toFile(), nil
}
// rapidUploadByCAS 使用 MD5 + 分片 MD5(CAS)执行天翼云盘秒传。
//
// 流程与 Web 端一致:
// 1. initMultiUpload(仅传 lazyCheck=1)
// 2. checkTransSecond(用 fileMd5 + sliceMd5 检查云端是否已存在文件数据)
// 3. commitMultiUploadFile(提交并返回文件对象)
func (y *Cloud189PC) rapidUploadByCAS(ctx context.Context, dstDir model.Obj, fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, overwrite bool) (model.Obj, error) {
isFamily := y.isFamily()
// 统一 MD5 为大写(天翼云盘要求大写)
fileMD5Upper := strings.ToUpper(fileMD5)
// 优先使用传入的分片大小,否则按文件大小推导
if sliceSize <= 0 {
sliceSize = partSize(fileSize)
}
// 计算 sliceMd5(与上传时一致的算法)
sliceMd5Hex := fileMD5Upper
if len(sliceMD5s) > 1 {
upperSliceMD5s := make([]string, len(sliceMD5s))
for i, s := range sliceMD5s {
upperSliceMD5s[i] = strings.ToUpper(s)
}
sliceMd5Hex = strings.ToUpper(utils.GetMD5EncodeStr(strings.Join(upperSliceMD5s, "\n")))
} else if len(sliceMD5s) == 1 {
sliceMd5Hex = strings.ToUpper(sliceMD5s[0])
}
fullUrl := "https://upload.cloud.189.cn"
if isFamily {
fullUrl += "/family"
} else {
fullUrl += "/person"
}
// Step 1: initMultiUpload(不传 fileMd5/sliceMd5,只传 lazyCheck)
initParams := Params{
"parentFolderId": dstDir.GetID(),
"fileName": url.QueryEscape(fileName),
"fileSize": fmt.Sprint(fileSize),
"sliceSize": fmt.Sprint(sliceSize),
"lazyCheck": "1",
}
if isFamily {
initParams.Set("familyId", y.FamilyID)
}
var uploadInfo InitMultiUploadResp
if _, err := y.request(fullUrl+"/initMultiUpload", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, initParams, &uploadInfo, isFamily); err != nil {
return nil, fmt.Errorf("initMultiUpload 失败: %w", err)
}
uploadFileId := uploadInfo.Data.UploadFileID
// Step 2: checkTransSecond(用 fileMd5 + sliceMd5 + uploadFileId 检查秒传)
checkParams := Params{
"fileMd5": fileMD5Upper,
"sliceMd5": sliceMd5Hex,
"uploadFileId": uploadFileId,
}
var checkResp struct {
Data struct {
FileDataExists int `json:"fileDataExists"`
} `json:"data"`
}
if _, err := y.request(fullUrl+"/checkTransSecond", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, checkParams, &checkResp, isFamily); err != nil {
return nil, fmt.Errorf("秒传检查失败: %w", err)
}
if checkResp.Data.FileDataExists != 1 {
return nil, fmt.Errorf("秒传失败:云端不存在该文件(fileMD5=%s, sliceMD5=%s, size=%d)", fileMD5Upper, sliceMd5Hex, fileSize)
}
// Step 3: commitMultiUploadFile(传 fileMd5 + sliceMd5)
commitParams := Params{
"uploadFileId": uploadFileId,
"fileMd5": fileMD5Upper,
"sliceMd5": sliceMd5Hex,
"lazyCheck": "1",
"opertype": IF(overwrite, "3", "1"),
}
var resp CommitMultiUploadFileResp
if _, err := y.request(fullUrl+"/commitMultiUploadFile", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, commitParams, &resp, isFamily); err != nil {
return nil, fmt.Errorf("提交上传失败: %w", err)
}
return resp.toFile(), nil
}
func (y *Cloud189PC) isFamily() bool {
return y.Type == "family"
}
-1
View File
@@ -328,7 +328,6 @@ func (d *Alias) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (
return nil, err
}
resultLink := link.Clone() // 复制一份,避免修改到原始link
resultLink.Expiration = nil
if args.Redirect {
return resultLink, nil
}
+1
View File
@@ -95,6 +95,7 @@ func (d *AListV3) List(ctx context.Context, dir model.Obj, args model.ListArgs)
file := model.ObjThumb{
Object: model.Object{
Name: f.Name,
Path: path.Join(dir.GetPath(), f.Name),
Modified: f.Modified,
Ctime: f.Created,
Size: f.Size,
+65
View File
@@ -0,0 +1,65 @@
package alist_v3
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/go-resty/resty/v2"
)
// TestListSetsChildPaths descends two levels the way op.Get does, feeding an
// object from one listing back into List as dir. Without a Path on that object
// the driver asks upstream for "", which a real server answers with its own
// root -- hence the fake upstream's fallback, and the endless self-similar tree.
func TestListSetsChildPaths(t *testing.T) {
tree := map[string][]ObjResp{
"/": {{Name: "root-marker", IsDir: true}},
"/drive": {{Name: "concerts", IsDir: true}},
"/drive/concerts": {{Name: "show.mkv"}},
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var req ListReq
_ = json.NewDecoder(r.Body).Decode(&req)
content, ok := tree[req.Path]
if !ok {
content = tree["/"]
}
w.Header().Set("Content-Type", "application/json") // resty only unmarshals JSON
_ = json.NewEncoder(w).Encode(map[string]any{
"code": 200, "message": "success",
"data": map[string]any{"content": content, "total": len(content)},
})
}))
t.Cleanup(srv.Close)
// conf.Conf is nil outside a booted server, so base.InitClient() is unusable.
prev := base.RestyClient
base.RestyClient = resty.New().SetTimeout(5 * time.Second)
t.Cleanup(func() { base.RestyClient = prev })
d := &AListV3{Addition: Addition{
RootPath: driver.RootPath{RootFolderPath: "/drive"},
Address: srv.URL,
}}
dir := model.Obj(&model.Object{Path: "/drive", IsFolder: true})
for _, want := range []string{"/drive/concerts", "/drive/concerts/show.mkv"} {
objs, err := d.List(context.Background(), dir, model.ListArgs{})
if err != nil {
t.Fatalf("List(%q): %v", dir.GetPath(), err)
}
if len(objs) != 1 {
t.Fatalf("List(%q) returned %d objects, want 1", dir.GetPath(), len(objs))
}
if got := objs[0].GetPath(); got != want {
t.Fatalf("child of %q has path %q, want %q", dir.GetPath(), got, want)
}
dir = objs[0]
}
}
+88
View File
@@ -0,0 +1,88 @@
package aliyundrive_open
import (
"context"
"net/http"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/go-resty/resty/v2"
)
// RapidHashAlgos 返回阿里云盘支持的秒传哈希算法(SHA1)
func (d *AliyundriveOpen) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.SHA1}
}
// RapidHashNeedsPieces 阿里云盘不需要分片哈希
func (d *AliyundriveOpen) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 SHA1 哈希尝试秒传。
//
// 阿里云盘的秒传还需要 proof_code(按 proof range 读取的一段内容),
// 因此当内容源不可用时无法完成秒传。
func (d *AliyundriveOpen) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
sha1Hash := req.Whole.GetHash(utils.SHA1)
if len(sha1Hash) < utils.SHA1.Width {
return nil, errs.ErrUnavailableHash
}
if req.Open == nil {
return nil, errs.ErrUnavailableHash
}
stream, err := req.Open()
if err != nil {
return nil, err
}
defer stream.Close()
proofCode, err := d.calProofCode(stream)
if err != nil {
return nil, err
}
var resp CreateResp
_, err = d.request(ctx, limiterOther, "/adrive/v1.0/openFile/create", http.MethodPost, func(r *resty.Request) {
r.SetBody(base.Json{
"drive_id": d.DriveId,
"parent_file_id": dstDir.GetID(),
"name": req.Name,
"type": "file",
"check_name_mode": "auto_rename",
"size": req.Size,
"content_hash": sha1Hash,
"content_hash_name": "sha1",
"proof_version": "v1",
"proof_code": proofCode,
}).SetResult(&resp)
})
if err != nil {
return nil, err
}
if !resp.RapidUpload {
return nil, errs.ErrHashMismatch
}
if resp.FileId != "" {
obj, err := d.completeUpload(ctx, resp.FileId, resp.UploadId)
if err != nil {
return nil, err
}
return obj, nil
}
return &model.ObjThumb{
Object: model.Object{
Name: req.Name,
Size: req.Size,
Modified: time.Now(),
IsFolder: false,
},
}, nil
}
+35
View File
@@ -0,0 +1,35 @@
package baidu_netdisk
import (
"context"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// RapidHashAlgos 返回百度网盘支持的秒传哈希算法(MD5)
func (d *BaiduNetdisk) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*utils.MD5}
}
// RapidHashNeedsPieces 百度网盘不需要分片哈希
func (d *BaiduNetdisk) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 MD5 哈希尝试秒传
func (d *BaiduNetdisk) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
md5Hash := req.Whole.GetHash(utils.MD5)
if len(md5Hash) < utils.MD5.Width {
return nil, errs.ErrUnavailableHash
}
stream := driver.NewSeedHashStream(req)
obj, err := d.PutRapid(ctx, dstDir, stream)
if err != nil {
return nil, err
}
return obj, nil
}
+1
View File
@@ -245,6 +245,7 @@ func mFile2Object(file MFile) *model.ObjThumbURL {
Ctime: ctime,
IsFolder: false,
Size: file.Size,
HashInfo: utils.NewHashInfo(utils.MD5, file.MD5),
},
Thumbnail: model.Thumbnail{
Thumbnail: file.ThumbnailURL,
+58
View File
@@ -0,0 +1,58 @@
package pikpak
import (
"context"
"net/http"
"strings"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
hash_extend "github.com/OpenListTeam/OpenList/v4/pkg/utils/hash"
"github.com/go-resty/resty/v2"
)
// RapidHashAlgos 返回 PikPak 支持的秒传哈希算法(GCID)
func (d *PikPak) RapidHashAlgos() []utils.HashType {
return []utils.HashType{*hash_extend.GCID}
}
// RapidHashNeedsPieces PikPak 不需要分片哈希
func (d *PikPak) RapidHashNeedsPieces() bool {
return false
}
// RapidUploadByHashes 使用种子中的 GCID 哈希尝试秒传
func (d *PikPak) RapidUploadByHashes(ctx context.Context, dstDir model.Obj, req *driver.SeedRapidUploadRequest, overwrite bool) (model.Obj, error) {
gcid := req.Whole.GetHash(hash_extend.GCID)
if len(gcid) < hash_extend.GCID.Width {
return nil, errs.ErrUnavailableHash
}
var resp UploadTaskData
_, err := d.request("https://api-drive.mypikpak.net/drive/v1/files", http.MethodPost, func(r *resty.Request) {
r.SetContext(ctx).SetBody(base.Json{
"kind": "drive#file",
"name": req.Name,
"size": req.Size,
"hash": strings.ToUpper(gcid),
"upload_type": "UPLOAD_TYPE_RESUMABLE",
"objProvider": base.Json{"provider": "UPLOAD_TYPE_UNKNOWN"},
"parent_id": dstDir.GetID(),
"folder_type": "NORMAL",
})
}, &resp)
if err != nil {
return nil, err
}
// 秒传成功时不会返回 Resumable
if resp.Resumable == nil {
file := fileToObj(resp.File)
return file, nil
}
return nil, errs.ErrHashMismatch
}

Some files were not shown because too many files have changed in this diff Show More