Compare commits

..

36 Commits

Author SHA1 Message Date
MadDogOwner 874234449b fix(doubao_share): use new download info (#1890)
fix(doubao_share): update file URL retrieval to use new download info structure

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
2026-01-01 22:29:03 +08:00
Edward 5fe267089a fix(123_open): infinite recursive call (#1854)
fix(123_open): token refresh logic

Fix token handling logic to avoid deadlock. Token method took reference of Alist's implementation.
2025-12-31 00:46:54 +08:00
KirCute 2442e302ad ci(lang): sync only new fields (#1881) 2025-12-30 15:43:30 +08:00
Tron 0612271732 fix(driver): fix file copy failure to 123pan due to incorrect etag (#1874) 2025-12-29 23:54:33 +08:00
我怎么就不是一只猫呢? c261ce78fb fix(s3): use current time as default modified time (#1860) 2025-12-29 23:52:35 +08:00
KirCute 7398e7d45e feat(alias): support load balance (#1767)
* feat(alias): support load balance

* feat(alias): support storage match for load balance

* feat(patch): add alias addition upgrade patch

* fix bugs

* fix(op/balance): optimize compatibility

* chore: change default read conflict policy

* feat(alias): refactor Alias initialization and enhance path handling

* feat(alias): enhance object masking and add support for operation restrictions

* feat(alias): enhance object masking

* feat(fs): add permission checks

* improve parsing

* update object masks

* feat(fs): enhance virtual file handling

* feat(storage): enhance virtual file retrieval and path handling

* refactor(alias): rename path handling functions for clarity and consistency

* fix(alias): update path handling in Other method to use balanced path

* fix bug

* feat(alias): add file size validation

* feat(alias): add hash consistency check

* 移除哈希合并,

* fix(alias): wrong behavior for all_strict/deterministic_or_all

* Revert "fix(alias): wrong behavior for all_strict/deterministic_or_all"

This reverts commit f001f2dcd7.

* fix(alias): wrong behavior for all_strict/deterministic_or_all

* feat(alias): support part-based read load balance

* fix(alias): list panic when leak conflict path

* fix(alias): remove Other load balance

* fix(alias): 修复 Link 方法中 resultLink 的返回类型和内容复制问题

* fix(alias): 更好的下载并发?

* chore(alias): all tips

* fix(alias): moving paths mismatch

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: ShenLin <773933146@qq.com>
2025-12-29 17:16:07 +08:00
KirCute 6e2d499ca9 refactor(bootstrap): fix OpenList-Mobile compile failed (#1857) 2025-12-24 18:46:13 +08:00
绎泽 4680ece2d9 docs(readme): add demo site (#1850)
Last Sync: 2025-12-22 12:39
2025-12-22 12:52:41 +08:00
Seven 8a4f3769d8 feat(strm): add save local mode (#1814)
* feat(strm): add KeepSameNameOnly logic

* chore(strm): skip update strm file when keepLocalDownloadFile

* feat(strm): add save local mode
2025-12-22 10:12:21 +08:00
foxxorcat cc5172e70b fix(weiyun): update sdk and support getDetails (#1845) 2025-12-22 00:20:07 +08:00
XZB-1248 a32ae97860 docs: update README for zh-CN (#1844) 2025-12-22 00:16:33 +08:00
TwoOnefour d6dd62dfe5 fix(s3): incorrect copy key with plus sign (#1820) 2025-12-22 00:15:58 +08:00
XZB-1248 216f071e64 docs: add VPS.Town as sponsor to all README (#1842)
Co-authored-by: XZB-1248 <i@1248.ink>
2025-12-21 12:26:03 +08:00
hshpy f47df5f9b2 feat(115_open): support custom pagesize (#1822) 2025-12-20 13:57:02 +08:00
KirCute ff3c4b885c fix(strm): support generate strm with sign (#1832) 2025-12-20 13:55:51 +08:00
MadDogOwner f86c7c844c feat(cloudreve_v4): add ks3 support (#1828)
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
2025-12-19 18:05:11 +08:00
Mako (XSpy) 5db2172ed6 feat(driver): add personal / business wps drive support (#1802)
* feat(driver): add wps drive support

* feat(driver): add wps drive support

* fix(wps): update personal mode string to English

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* fix(wps): remove trailing slash from drive origin URL

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* fix(wps): correct order of options in mode selection

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* fix(wps): enable local sort and upload overwrite

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* fix(wps): resolve put bugs, fix file op problems and optimize list logic

- Fix uploading bugs. Support all uploading methods based on 8825.85d3c864.js
- Fix issues in delete/copy/move while opearting big folders.
- Use cache to optimize performance of list, especially in a deep path.

---------

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
2025-12-15 21:49:01 +08:00
wongz c4c121befc fix(139): disk-usage unmarshal failed when used capacity overflow (#1718)
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
2025-12-15 21:30:08 +08:00
UcnacDx2 b4542753ba feat(drivers/139): user authentication and file batch operations (#1534)
* feat(139): Enhance 139 driver with password login and root path handling

- Added support for password-based login in the 139 driver.
- Introduced RootPath field to store the root directory path.
- Updated Init method to handle family and group types more effectively.
- Implemented new methods for handling file operations in family and group contexts.
- Enhanced error handling and logging for better debugging.
- Added new request and response structures for batch operations and document modifications.
- Improved encryption and decryption methods for secure communication.

* Update drivers/139/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>

* Update drivers/139/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>

* Update drivers/139/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>

* Update drivers/139/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>

* Update drivers/139/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>

---------

Signed-off-by: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-15 19:38:33 +08:00
KirCute 2a99c97d52 feat(ldap): support webdav, ftp and sftp login (#1746)
* feat(ldap): support webdav, ftp and sftp login

* fix: apply suggestions of Copilot

* feat(ldap) support ftp, sftp and webdav auto-register
2025-12-15 16:53:38 +08:00
MadDogOwner 0a407c3d8b fix(openlist): disable status check for openlist driver (#1757)
* fix(openlist): disable status check to avoid network stability issues

* fix(alist_v3): disable status check to avoid network stability issues

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

---------

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
2025-12-15 16:48:05 +08:00
KirCute b2596fdc24 refactor(bootstrap): move booting logic to bootstrap package (#1773)
* refactor(bootstrap): move booting to bootstrap package

* chore(log): reduce level of some callings of `utils.Log.Fatal`

* fix(s3): no shutdown after SIGTERM received

* fix: add handle hook
2025-12-15 16:47:50 +08:00
zzzhr1990 2dbe1b00d3 fix(halalcloud_open): halal-cloud upload issues (#1800)
fix halal-cloud upload issues
2025-12-15 16:47:09 +08:00
KirCute 1fc9c83df1 fix(ilanzou): parse vip size (#1792) 2025-12-12 12:13:42 +08:00
j2rong4cn d31e1a333d feat(model): add object mask support and enhance cache/task handling (#1743) 2025-12-11 15:10:42 +08:00
jenfonro e1bba7072b fix(task): tasks keep being cancelled (#1745)
* fix_cancel

* update(go.mod): update tache version

* tache v0.2.2

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-12-10 19:09:16 +08:00
MadDogOwner 94c7d68413 feat(utils): add support for ignoring '@eaDir' system files (#1779) 2025-12-10 13:45:14 +08:00
KirCute 9ed77a5875 feat(driver): add AList v3 (#1721)
* feat(driver/openlist): compatible with AList v3

* Revert "feat(driver/openlist): compatible with AList v3"

This reverts commit 90f3f80186.

* feat(driver): add AList v3

* Revert "feat(patch): add migration from Alist V3 driver to OpenList (#919)"

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

---------

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
2025-12-08 22:25:10 +08:00
varg1714 7d6d3b8f55 feat(fs): Support customizing the cache time for a specific path (#1533)
* feat(fs): Support customizing the cache time for a specific path

* feat(fs): Get the cache rule for driver information.

* feat(fs): Support globbing.

* feat(fs): Add log.

---------

Signed-off-by: ShenLin <773933146@qq.com>
Co-authored-by: ShenLin <773933146@qq.com>
2025-12-04 09:59:27 +08:00
ShenLin 5480d61f70 refactor!(userAgent): merge most userAgent into base (#1722)
refactor!(userAgent): merge all userAgent into base

1. change var to const
2. remove duplicated ua definetion after original Resty R
3. upgrade Chrome and OS versions
2025-12-04 09:54:14 +08:00
j2rong4cn 96cd714385 refactor(op): remove automatic Path assignment (#1734)
* refactor: 移除 ObjResp 中的 Id 和 Path 字段

* 移除op.List的自动设置Path
Path和Id只在驱动内使用,不应由op.List设置Path

* cnb_releases:将 Addition 结构体中的 RootPath 字段为 RootID
当List方法加载二级目录时,若使用的是Id,应对使用driver.RootID

* doubao_share: 添加潜在bug注释

* 添加 GetRootPath 方法到多个驱动
2025-12-03 00:55:40 +08:00
VXTLS e29d92f92e fix(mediafire): enable automatic session token acquisition and fix gzip parsing (#1661)
* fix(mediafire): enable automatic session token acquisition and fix gzip parsing

- Fix Init() method to allow automatic session token retrieval from cookie
- Change SessionToken from required to optional in configuration
- Add proper gzip decompression support for API responses
- Improve error handling for session token acquisition failures
- Update help text to clarify authentication requirements

Resolves initialization failure and JSON parsing errors when session token
can be automatically obtained from browser cookie.

* fix(mediafire): ensure driver files end with newline

* chore: gofmt drivers/mediafire/*.go
2025-12-02 12:13:39 +08:00
ShenLin c5f57bbcc5 fix(drivers/crypt): remove hard dependency on RemotePath (#1713) 2025-11-28 01:21:22 +08:00
j2rong4cn 9835afc645 refactor: improve upload handling (#1455)
* fix(quark): refactor upPart to use http.NewRequest

* fix(quark): improved upload handling

* fix(quark_open): improved upload handling

* fix: add retry context to multiple upload functions

* fix: optimize hash calculation in multipart upload to avoid blocking

* fix: update error handling in lifecycle functions for better clarity

* fix: update upload progress calculation to improve accuracy

* fix: simplify error handling in lifecycle functions for improved readability

* fix: remove unnecessary mutex for part uploads to simplify code

* fix(stream): simplify file handling in NewStreamSectionReader and improve error messages

* fix(terabox): optimize chunk count calculation in Put method

* perf(chaoxing): 表单上传文件0拷贝

* fix(cnb_releases): improve file upload progress tracking

* fix(baidu_netdisk): improve upload handling

* fix(upload): optimize buffer initialization for file uploads

* fix(baidu_netdisk): add retry condition to skip ErrUploadIDExpired in upload loop
2025-11-27 19:34:03 +08:00
Seven 1f373eac8d chore(strm): avoid generating empty folders (#1720)
chore(strm): empty folders are not generated locally
2025-11-27 14:02:11 +08:00
jenfonro ede96a314c fix(onedrive_shareurl): Reduce temporary file errors (#1686)
* fix onedrive_shareurl

* .
2025-11-25 21:43:17 +08:00
164 changed files with 6260 additions and 2364 deletions
+6 -1
View File
@@ -122,12 +122,17 @@ Thank you for your support and understanding of the OpenList project.
## Demo
N/A (to be rebuilt)
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## Discussion
Please refer to [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions) for raising general questions, ***Issues* is for bug reports and feature requests only.**
## Sponsor
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## License
The `OpenList` is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
+6 -1
View File
@@ -122,12 +122,17 @@ OpenList 是一个由 OpenList 团队独立维护的开源项目,遵循 AGPL-3
## 演示
N/A(待重建)
- 🇨🇳 [国内演示站](https://demo.oplist.org.cn)
- 🌎 [海外演示站](https://demo.oplist.org)
## 讨论
如有一般性问题请前往 [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions) 讨论区,***Issues* 仅用于错误报告和功能请求。**
## 赞助者
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## 许可证
`OpenList` 是基于 [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) 许可证的开源软件。
+6 -1
View File
@@ -122,12 +122,17 @@ OpenListプロジェクトへのご支援とご理解をありがとうござい
## デモ
N/A(再構築中)
- 🌎 [グローバルデモ](https://demo.oplist.org)
- 🇨🇳 [CNデモ](https://demo.oplist.org.cn)
## ディスカッション
一般的な質問は [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions) をご利用ください。***Issues* はバグ報告と機能リクエスト専用です。**
## スポンサー
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## ライセンス
「OpenList」は [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) ライセンスの下で公開されているオープンソースソフトウェアです。
+6 -1
View File
@@ -122,12 +122,17 @@ Dank u voor uw ondersteuning en begrip
## Demo
N.v.t. (wordt opnieuw opgebouwd)
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## Discussie
Stel algemene vragen in [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions), ***Issues* zijn alleen voor bugmeldingen en feature requests.**
## Sponsoren
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## Licentie
`OpenList` is open-source software onder de [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) licentie.
+7 -6
View File
@@ -6,6 +6,7 @@ package cmd
import (
"fmt"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
"github.com/OpenListTeam/OpenList/v4/internal/conf"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/internal/setting"
@@ -20,8 +21,8 @@ var AdminCmd = &cobra.Command{
Aliases: []string{"password"},
Short: "Show admin user's info and some operations about admin user's password",
Run: func(cmd *cobra.Command, args []string) {
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
admin, err := op.GetAdmin()
if err != nil {
utils.Log.Errorf("failed get admin user: %+v", err)
@@ -61,8 +62,8 @@ var ShowTokenCmd = &cobra.Command{
Use: "token",
Short: "Show admin token",
Run: func(cmd *cobra.Command, args []string) {
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
token := setting.GetStr(conf.Token)
utils.Log.Infof("show admin token from CLI")
fmt.Println("Admin token:", token)
@@ -70,8 +71,8 @@ var ShowTokenCmd = &cobra.Command{
}
func setAdminPassword(pwd string) {
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
admin, err := op.GetAdmin()
if err != nil {
utils.Log.Errorf("failed get admin user: %+v", err)
+3 -2
View File
@@ -6,6 +6,7 @@ package cmd
import (
"fmt"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/spf13/cobra"
@@ -16,8 +17,8 @@ var Cancel2FACmd = &cobra.Command{
Use: "cancel2fa",
Short: "Delete 2FA of admin user",
Run: func(cmd *cobra.Command, args []string) {
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
admin, err := op.GetAdmin()
if err != nil {
utils.Log.Errorf("failed to get admin user: %+v", err)
+2 -10
View File
@@ -6,24 +6,16 @@ import (
"strconv"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap/data"
"github.com/OpenListTeam/OpenList/v4/internal/db"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
log "github.com/sirupsen/logrus"
)
func Init() {
bootstrap.InitConfig()
bootstrap.Log()
bootstrap.InitDB()
data.InitData()
bootstrap.InitStreamLimit()
bootstrap.InitIndex()
bootstrap.InitUpgradePatch()
bootstrap.Init()
}
func Release() {
db.Close()
bootstrap.Release()
}
var pid = -1
+2 -4
View File
@@ -1,19 +1,17 @@
package cmd
import (
log "github.com/sirupsen/logrus"
"io"
"os"
"path"
"path/filepath"
"strings"
"github.com/spf13/cobra"
rcCrypt "github.com/rclone/rclone/backend/crypt"
"github.com/rclone/rclone/fs/config/configmap"
"github.com/rclone/rclone/fs/config/obscure"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
// encryption and decryption command format for Crypt driver
+20 -3
View File
@@ -8,7 +8,6 @@ import (
"fmt"
"io"
"os"
"reflect"
"strings"
_ "github.com/OpenListTeam/OpenList/v4/drivers"
@@ -69,15 +68,33 @@ func writeFile(name string, data interface{}) {
log.Errorf("failed to unmarshal json: %+v", err)
return
}
if reflect.DeepEqual(oldData, newData) {
if mergeJson(newData, oldData) {
log.Infof("%s.json no changed, skip", name)
} else {
log.Infof("%s.json changed, update file", name)
//log.Infof("old: %+v\nnew:%+v", oldData, data)
utils.WriteJsonToFile(fmt.Sprintf("lang/%s.json", name), newData, true)
utils.WriteJsonToFile(fmt.Sprintf("lang/%s.json", name), oldData, true)
}
}
func mergeJson(source, target map[string]interface{}) bool {
equal := true
for k, v := range source {
tgtV, tgtOk := target[k]
if !tgtOk {
equal = false
target[k] = v
} else {
srcMap, srcIsMap := v.(map[string]interface{})
tgtMap, tgtIsMap := tgtV.(map[string]interface{})
if srcIsMap && tgtIsMap {
equal = mergeJson(srcMap, tgtMap) && equal
}
}
}
return equal
}
func generateDriversJson() {
drivers := make(Drivers)
drivers["drivers"] = make(KV[interface{}])
+4 -239
View File
File diff suppressed because it is too large Load Diff
+7 -6
View File
@@ -8,6 +8,7 @@ import (
"os"
"strconv"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
"github.com/OpenListTeam/OpenList/v4/internal/db"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/charmbracelet/bubbles/table"
@@ -30,8 +31,8 @@ var disableStorageCmd = &cobra.Command{
return fmt.Errorf("mount path is required")
}
mountPath := args[0]
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
storage, err := db.GetStorageByMountPath(mountPath)
if err != nil {
return fmt.Errorf("failed to query storage: %+v", err)
@@ -69,8 +70,8 @@ var deleteStorageCmd = &cobra.Command{
}
}
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
err = db.DeleteStorageById(uint(id))
if err != nil {
return fmt.Errorf("failed to delete storage by id: %+v", err)
@@ -123,8 +124,8 @@ var listStorageCmd = &cobra.Command{
Use: "list",
Short: "List all storages",
RunE: func(cmd *cobra.Command, args []string) error {
Init()
defer Release()
bootstrap.Init()
defer bootstrap.Release()
storages, _, err := db.GetStorages(1, -1)
if err != nil {
return fmt.Errorf("failed to query storages: %+v", err)
+7 -1
View File
@@ -53,6 +53,12 @@ func (d *Open115) Init(ctx context.Context) error {
if d.Addition.LimitRate > 0 {
d.limiter = rate.NewLimiter(rate.Limit(d.Addition.LimitRate), 1)
}
if d.PageSize <= 0 {
d.PageSize = 200
} else if d.PageSize > 1150 {
d.PageSize = 1150
}
return nil
}
@@ -69,7 +75,7 @@ func (d *Open115) Drop(ctx context.Context) error {
func (d *Open115) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
var res []model.Obj
pageSize := int64(200)
pageSize := int64(d.PageSize)
offset := int64(0)
for {
if err := d.WaitLimit(ctx); err != nil {
+1
View File
@@ -12,6 +12,7 @@ type Addition struct {
OrderBy string `json:"order_by" type:"select" options:"file_name,file_size,user_utime,file_type"`
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc"`
LimitRate float64 `json:"limit_rate" type:"float" default:"1" help:"limit all api request rate ([limit]r/1s)"`
PageSize int64 `json:"page_size" type:"number" default:"200" help:"list api per page size of 115open driver"`
AccessToken string `json:"access_token" required:"true"`
RefreshToken string `json:"refresh_token" required:"true"`
}
+2 -2
View File
@@ -107,16 +107,16 @@ func (d *Open115) multpartUpload(ctx context.Context, stream model.FileStreamer,
if err != nil {
return err
}
rateLimitedRd := driver.NewLimitedUploadStream(ctx, rd)
err = retry.Do(func() error {
rd.Seek(0, io.SeekStart)
part, err := bucket.UploadPart(imur, rateLimitedRd, partSize, int(i))
part, err := bucket.UploadPart(imur, driver.NewLimitedUploadStream(ctx, rd), partSize, int(i))
if err != nil {
return err
}
parts[i-1] = part
return nil
},
retry.Context(ctx),
retry.Attempts(3),
retry.DelayType(retry.BackOffDelay),
retry.Delay(time.Second))
+7 -16
View File
@@ -125,27 +125,18 @@ func (d *Pan123) newUpload(ctx context.Context, upReq *UploadResp, file model.Fi
curSize = lastChunkSize
}
var reader io.ReadSeeker
var rateLimitedRd io.Reader
threadG.GoWithLifecycle(errgroup.Lifecycle{
Before: func(ctx context.Context) error {
if reader == nil {
var err error
reader, err = ss.GetSectionReader(offset, curSize)
if err != nil {
return err
}
rateLimitedRd = driver.NewLimitedUploadStream(ctx, reader)
}
return nil
Before: func(ctx context.Context) (err error) {
reader, err = ss.GetSectionReader(offset, curSize)
return
},
Do: func(ctx context.Context) error {
Do: func(ctx context.Context) (err error) {
reader.Seek(0, io.SeekStart)
uploadUrl := s3PreSignedUrls.Data.PreSignedUrls[strconv.Itoa(cur)]
if uploadUrl == "" {
return fmt.Errorf("upload url is empty, s3PreSignedUrls: %+v", s3PreSignedUrls)
}
reader.Seek(0, io.SeekStart)
req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadUrl, rateLimitedRd)
req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadUrl, driver.NewLimitedUploadStream(ctx, reader))
if err != nil {
return err
}
@@ -157,7 +148,7 @@ func (d *Pan123) newUpload(ctx context.Context, upReq *UploadResp, file model.Fi
}
defer res.Body.Close()
if res.StatusCode == http.StatusForbidden {
singleflight.AnyGroup.Do(fmt.Sprintf("Pan123.newUpload_%p", threadG), func() (any, error) {
_, err, _ = singleflight.AnyGroup.Do(fmt.Sprintf("Pan123.newUpload_%p", threadG), func() (any, error) {
newS3PreSignedUrls, err := getS3UploadUrl(ctx, upReq, cur, end)
if err != nil {
return nil, err
@@ -177,7 +168,7 @@ func (d *Pan123) newUpload(ctx context.Context, upReq *UploadResp, file model.Fi
}
return fmt.Errorf("upload s3 chunk %d failed, status code: %d, body: %s", cur, res.StatusCode, body)
}
progress := 10.0 + 85.0*float64(threadG.Success())/float64(chunkCount)
progress := 100 * float64(threadG.Success()+1) / float64(chunkCount+1)
up(progress)
return nil
},
+4
View File
@@ -39,6 +39,10 @@ func (d *Pan123Link) Drop(ctx context.Context) error {
return nil
}
func (Addition) GetRootPath() string {
return "/"
}
func (d *Pan123Link) Get(ctx context.Context, path string) (model.Obj, error) {
node := GetNodeFromRootByPath(d.root, path)
return nodeToObj(node, path)
+19
View File
@@ -18,6 +18,7 @@ type Open123 struct {
model.Storage
Addition
UID uint64
tm *tokenManager
}
func (d *Open123) Config() driver.Config {
@@ -33,6 +34,24 @@ func (d *Open123) Init(ctx context.Context) error {
d.UploadThread = 3
}
if d.RefreshToken != "" {
// refresh token 直接主动刷新
d.AccessToken = ""
d.tm = &tokenManager{}
} else {
// 避免个人 token 刷新产生的多个登录,被动刷新
// 默认过期时间90天,jwt exp 不可靠
d.tm = &tokenManager{
// accessToken: d.AccessToken,
expiredAt: time.Now().Add(90 * 24 * time.Hour),
}
}
_, err := d.getAccessToken(false)
if err != nil {
return fmt.Errorf("init get access token error: %w", err)
}
return nil
}
+1 -1
View File
@@ -13,7 +13,7 @@ type Addition struct {
ClientID string `json:"ClientID" required:"false"`
ClientSecret string `json:"ClientSecret" required:"false"`
// 直接写入AccessToken
// 直接写入AccessToken, AccessToken有过期时间,不建议直接填写
AccessToken string `json:"AccessToken" required:"false"`
// 用户名+密码方式登录的AccessToken可以兼容
+115
View File
@@ -0,0 +1,115 @@
package _123_open
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
var (
AccessToken = "https://open-api.123pan.com/api/v1/access_token"
RefreshToken = "https://open-api.123pan.com/api/v1/oauth2/access_token"
)
type tokenManager struct {
// accessToken string
expiredAt time.Time
mu sync.Mutex
blockRefresh bool
}
func (d *Open123) getAccessToken(forceRefresh bool) (string, error) {
tm := d.tm
tm.mu.Lock()
defer tm.mu.Unlock()
if tm.blockRefresh {
return "", errors.New("Authentication expired")
}
if !forceRefresh && d.AccessToken != "" && time.Now().Before(tm.expiredAt.Add(-5*time.Minute)) {
return d.AccessToken, nil
}
if err := d.flushAccessToken(); err != nil {
// token expired and failed to refresh, block further refresh attempts
tm.blockRefresh = true
return "", err
}
return d.AccessToken, nil
}
func (d *Open123) flushAccessToken() error {
// directly send request to avoid deadlock
req := base.RestyClient.R()
req.SetHeaders(map[string]string{
"authorization": "Bearer " + d.AccessToken,
"platform": "open_platform",
"Content-Type": "application/json",
})
if d.ClientID != "" {
if d.RefreshToken != "" {
var resp RefreshTokenResp
req.SetQueryParam("client_id", d.ClientID)
if d.ClientSecret != "" {
req.SetQueryParam("client_secret", d.ClientSecret)
}
req.SetQueryParam("grant_type", "refresh_token")
req.SetQueryParam("refresh_token", d.RefreshToken)
req.SetResult(&resp)
res, err := req.Execute(http.MethodPost, RefreshToken)
if err != nil {
return err
}
body := res.Body()
var baseResp BaseResp
if err = json.Unmarshal(body, &baseResp); err != nil {
return err
}
if baseResp.Code != 0 {
return fmt.Errorf("get access token failed: %s", baseResp.Message)
}
d.AccessToken = resp.AccessToken
// add token expire time
d.tm.expiredAt = time.Now().Add(time.Duration(resp.ExpiresIn) * time.Second)
d.RefreshToken = resp.RefreshToken
op.MustSaveDriverStorage(d)
d.tm.blockRefresh = false
return nil
} else if d.ClientSecret != "" {
var resp AccessTokenResp
req.SetBody(base.Json{
"clientID": d.ClientID,
"clientSecret": d.ClientSecret,
})
req.SetResult(&resp)
res, err := req.Execute(http.MethodPost, AccessToken)
if err != nil {
return err
}
body := res.Body()
var baseResp BaseResp
if err = json.Unmarshal(body, &baseResp); err != nil {
return err
}
if baseResp.Code != 0 {
return fmt.Errorf("get access token failed: %s", baseResp.Message)
}
d.AccessToken = resp.Data.AccessToken
// parse token expire time
d.tm.expiredAt, err = time.Parse(time.RFC3339, resp.Data.ExpiredAt)
if err != nil {
return fmt.Errorf("parse expire time failed: %w", err)
}
op.MustSaveDriverStorage(d)
d.tm.blockRefresh = false
return nil
}
}
return errors.New("no valid authentication method available")
}
+19 -19
View File
@@ -73,25 +73,20 @@ func (d *Open123) Upload(ctx context.Context, file model.FileStreamer, createRes
// 表单
b := bytes.NewBuffer(make([]byte, 0, 2048))
threadG.GoWithLifecycle(errgroup.Lifecycle{
Before: func(ctx context.Context) error {
if reader == nil {
var err error
// 每个分片一个reader
reader, err = ss.GetSectionReader(offset, size)
if err != nil {
return err
}
// 计算当前分片的MD5
Before: func(ctx context.Context) (err error) {
reader, err = ss.GetSectionReader(offset, size)
return
},
Do: func(ctx context.Context) (err error) {
reader.Seek(0, io.SeekStart)
if sliceMD5 == "" {
// 把耗时的计算放在这里,避免阻塞其他协程
sliceMD5, err = utils.HashReader(utils.MD5, reader)
if err != nil {
return err
}
reader.Seek(0, io.SeekStart)
}
return nil
},
Do: func(ctx context.Context) error {
// 重置分片reader位置,因为HashReader、上一次失败已经读取到分片EOF
reader.Seek(0, io.SeekStart)
b.Reset()
w := multipart.NewWriter(b)
@@ -121,6 +116,10 @@ func (d *Open123) Upload(ctx context.Context, file model.FileStreamer, createRes
head := bytes.NewReader(b.Bytes()[:headSize])
tail := bytes.NewReader(b.Bytes()[headSize:])
rateLimitedRd = driver.NewLimitedUploadStream(ctx, io.MultiReader(head, reader, tail))
token, err := d.getAccessToken(false)
if err != nil {
return err
}
// 创建请求并设置header
req, err := http.NewRequestWithContext(ctx, http.MethodPost, uploadDomain+"/upload/v2/file/slice", rateLimitedRd)
if err != nil {
@@ -128,7 +127,7 @@ func (d *Open123) Upload(ctx context.Context, file model.FileStreamer, createRes
}
// 设置请求头
req.Header.Add("Authorization", "Bearer "+d.AccessToken)
req.Header.Add("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", w.FormDataContentType())
req.Header.Add("Platform", "open_platform")
@@ -140,12 +139,13 @@ func (d *Open123) Upload(ctx context.Context, file model.FileStreamer, createRes
if res.StatusCode != 200 {
return fmt.Errorf("slice %d upload failed, status code: %d", partNumber, res.StatusCode)
}
var resp BaseResp
respBody, err := io.ReadAll(res.Body)
b.Reset()
_, err = b.ReadFrom(res.Body)
if err != nil {
return err
}
err = json.Unmarshal(respBody, &resp)
var resp BaseResp
err = json.Unmarshal(b.Bytes(), &resp)
if err != nil {
return err
}
@@ -153,7 +153,7 @@ func (d *Open123) Upload(ctx context.Context, file model.FileStreamer, createRes
return fmt.Errorf("slice %d upload failed: %s", partNumber, resp.Message)
}
progress := 10.0 + 85.0*float64(threadG.Success())/float64(uploadNums)
progress := 100 * float64(threadG.Success()+1) / float64(uploadNums+1)
up(progress)
return nil
},

Some files were not shown because too many files have changed in this diff Show More