fix(doubao_share): use new download info (#1890)

fix(doubao_share): update file URL retrieval to use new download info structure

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

.github/ISSUE_TEMPLATE/00-bug_report_zh.yml

@@ -13,7 +13,7 @@ body:
     attributes:
       label: 请确认以下事项
       description: |
-        您必须勾选以下内容，否则您的问题可能会被直接关闭。
+        您必须确认、同意并勾选以下内容，否则您的问题一定会被直接关闭。
         或者您可以去[讨论区](https://github.com/OpenListTeam/OpenList/discussions)。
       options:
         - label: |
@@ -59,6 +59,14 @@ body:
       label: 问题描述（必填）
     validations:
       required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: 日志（必填）
+      description: |
+        请复制粘贴错误日志，或者截图。(可隐藏隐私字段) [查看方法](https://doc.oplist.org/faq/howto#%E5%A6%82%E4%BD%95%E5%BF%AB%E9%80%9F%E5%AE%9A%E4%BD%8Dbug)
+    validations:
+      required: true
   - type: textarea
     id: config
     attributes:
@@ -67,12 +75,6 @@ body:
         请提供您的`OpenList`应用的配置文件，并截图相关存储配置。(可隐藏隐私字段)
     validations:
       required: true
-  - type: textarea
-    id: logs
-    attributes:
-      label: 日志（可选）
-      description: |
-        请复制粘贴错误日志，或者截图。(可隐藏隐私字段) [查看方法](https://doc.oplist.org/faq/howto#%E5%A6%82%E4%BD%95%E5%BF%AB%E9%80%9F%E5%AE%9A%E4%BD%8Dbug)
   - type: textarea
     id: reproduction
     attributes:

.github/ISSUE_TEMPLATE/01-bug_report_en.yml

@@ -13,7 +13,7 @@ body:
     attributes:
       label: Please confirm the following
       description: |
-        You must check all the following, otherwise your issue may be closed directly.
+        You must confirm, agree, and check all the following, otherwise your issue will definitely be closed directly.
         Or you can go to the [discussions](https://github.com/OpenListTeam/OpenList/discussions).
       options:
         - label: |
@@ -59,6 +59,14 @@ body:
       label: Bug Description (required)
     validations:
       required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: Logs (required)
+      description: |
+        Please copy and paste any relevant log output or screenshots. (You may mask sensitive fields) [Guide](https://doc.oplist.org/faq/howto#how-to-quickly-locate-bugs)
+    validations:
+      required: true
   - type: textarea
     id: config
     attributes:
@@ -67,12 +75,6 @@ body:
         Please provide your `OpenList` application's configuration file and a screenshot of the relevant storage configuration. (You may mask sensitive fields)
     validations:
       required: true
-  - type: textarea
-    id: logs
-    attributes:
-      label: Logs (optional)
-      description: |
-        Please copy and paste any relevant log output or screenshots. (You may mask sensitive fields) [Guide](https://doc.oplist.org/faq/howto#how-to-quickly-locate-bugs)
   - type: textarea
     id: reproduction
     attributes:

README.md

@@ -122,12 +122,17 @@ Thank you for your support and understanding of the OpenList project.
 
 ## Demo
 
-N/A (to be rebuilt)
+- 🌎 [Global Demo](https://demo.oplist.org)
+- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
 
 ## Discussion
 
 Please refer to [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions) for raising general questions, ***Issues* is for bug reports and feature requests only.**
 
+## Sponsor
+
+[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
+
 ## License
 
 The `OpenList` is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.

README_cn.md

@@ -122,12 +122,17 @@ OpenList 是一个由 OpenList 团队独立维护的开源项目，遵循 AGPL-3
 
 ## 演示
 
-N/A（待重建）
+- 🇨🇳 [国内演示站](https://demo.oplist.org.cn)
+- 🌎 [海外演示站](https://demo.oplist.org)
 
 ## 讨论
 
 如有一般性问题请前往 [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions) 讨论区，***Issues* 仅用于错误报告和功能请求。**
 
+## 赞助者
+
+[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
+
 ## 许可证
 
 `OpenList` 是基于 [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) 许可证的开源软件。

README_ja.md

@@ -122,12 +122,17 @@ OpenListプロジェクトへのご支援とご理解をありがとうござい
 
 ## デモ
 
-N/A（再構築中）
+- 🌎 [グローバルデモ](https://demo.oplist.org)
+- 🇨🇳 [CNデモ](https://demo.oplist.org.cn)
 
 ## ディスカッション
 
 一般的な質問は [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions) をご利用ください。***Issues* はバグ報告と機能リクエスト専用です。**
 
+## スポンサー
+
+[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
+
 ## ライセンス
 
 「OpenList」は [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) ライセンスの下で公開されているオープンソースソフトウェアです。

README_nl.md

@@ -122,12 +122,17 @@ Dank u voor uw ondersteuning en begrip
 
 ## Demo
 
-N.v.t. (wordt opnieuw opgebouwd)
+- 🌎 [Global Demo](https://demo.oplist.org)
+- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
 
 ## Discussie
 
 Stel algemene vragen in [*Discussions*](https://github.com/OpenListTeam/OpenList/discussions), ***Issues* zijn alleen voor bugmeldingen en feature requests.**
 
+## Sponsoren
+
+[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
+
 ## Licentie
 
 `OpenList` is open-source software onder de [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) licentie.

cmd/admin.go

@@ -6,6 +6,7 @@ package cmd
 import (
 	"fmt"
 
+	"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
 	"github.com/OpenListTeam/OpenList/v4/internal/conf"
 	"github.com/OpenListTeam/OpenList/v4/internal/op"
 	"github.com/OpenListTeam/OpenList/v4/internal/setting"
@@ -20,8 +21,8 @@ var AdminCmd = &cobra.Command{
 	Aliases: []string{"password"},
 	Short:   "Show admin user's info and some operations about admin user's password",
 	Run: func(cmd *cobra.Command, args []string) {
-		Init()
-		defer Release()
+		bootstrap.Init()
+		defer bootstrap.Release()
 		admin, err := op.GetAdmin()
 		if err != nil {
 			utils.Log.Errorf("failed get admin user: %+v", err)
@@ -61,8 +62,8 @@ var ShowTokenCmd = &cobra.Command{
 	Use:   "token",
 	Short: "Show admin token",
 	Run: func(cmd *cobra.Command, args []string) {
-		Init()
-		defer Release()
+		bootstrap.Init()
+		defer bootstrap.Release()
 		token := setting.GetStr(conf.Token)
 		utils.Log.Infof("show admin token from CLI")
 		fmt.Println("Admin token:", token)
@@ -70,8 +71,8 @@ var ShowTokenCmd = &cobra.Command{
 }
 
 func setAdminPassword(pwd string) {
-	Init()
-	defer Release()
+	bootstrap.Init()
+	defer bootstrap.Release()
 	admin, err := op.GetAdmin()
 	if err != nil {
 		utils.Log.Errorf("failed get admin user: %+v", err)

cmd/cancel2FA.go

@@ -6,6 +6,7 @@ package cmd
 import (
 	"fmt"
 
+	"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
 	"github.com/OpenListTeam/OpenList/v4/internal/op"
 	"github.com/OpenListTeam/OpenList/v4/pkg/utils"
 	"github.com/spf13/cobra"
@@ -16,8 +17,8 @@ var Cancel2FACmd = &cobra.Command{
 	Use:   "cancel2fa",
 	Short: "Delete 2FA of admin user",
 	Run: func(cmd *cobra.Command, args []string) {
-		Init()
-		defer Release()
+		bootstrap.Init()
+		defer bootstrap.Release()
 		admin, err := op.GetAdmin()
 		if err != nil {
 			utils.Log.Errorf("failed to get admin user: %+v", err)

cmd/common.go

@@ -6,24 +6,16 @@ import (
 	"strconv"
 
 	"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
-	"github.com/OpenListTeam/OpenList/v4/internal/bootstrap/data"
-	"github.com/OpenListTeam/OpenList/v4/internal/db"
 	"github.com/OpenListTeam/OpenList/v4/pkg/utils"
 	log "github.com/sirupsen/logrus"
 )
 
 func Init() {
-	bootstrap.InitConfig()
-	bootstrap.Log()
-	bootstrap.InitDB()
-	data.InitData()
-	bootstrap.InitStreamLimit()
-	bootstrap.InitIndex()
-	bootstrap.InitUpgradePatch()
+	bootstrap.Init()
 }
 
 func Release() {
-	db.Close()
+	bootstrap.Release()
 }
 
 var pid = -1

cmd/crypt.go

@@ -1,19 +1,17 @@
 package cmd
 
 import (
-	log "github.com/sirupsen/logrus"
-
 	"io"
 	"os"
 	"path"
 	"path/filepath"
 	"strings"
 
-	"github.com/spf13/cobra"
-
 	rcCrypt "github.com/rclone/rclone/backend/crypt"
 	"github.com/rclone/rclone/fs/config/configmap"
 	"github.com/rclone/rclone/fs/config/obscure"
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/cobra"
 )
 
 // encryption and decryption command format for Crypt driver

cmd/lang.go

@@ -8,7 +8,6 @@ import (
 	"fmt"
 	"io"
 	"os"
-	"reflect"
 	"strings"
 
 	_ "github.com/OpenListTeam/OpenList/v4/drivers"
@@ -69,15 +68,33 @@ func writeFile(name string, data interface{}) {
 		log.Errorf("failed to unmarshal json: %+v", err)
 		return
 	}
-	if reflect.DeepEqual(oldData, newData) {
+	if mergeJson(newData, oldData) {
 		log.Infof("%s.json no changed, skip", name)
 	} else {
 		log.Infof("%s.json changed, update file", name)
 		//log.Infof("old: %+v\nnew:%+v", oldData, data)
-		utils.WriteJsonToFile(fmt.Sprintf("lang/%s.json", name), newData, true)
+		utils.WriteJsonToFile(fmt.Sprintf("lang/%s.json", name), oldData, true)
 	}
 }
 
+func mergeJson(source, target map[string]interface{}) bool {
+	equal := true
+	for k, v := range source {
+		tgtV, tgtOk := target[k]
+		if !tgtOk {
+			equal = false
+			target[k] = v
+		} else {
+			srcMap, srcIsMap := v.(map[string]interface{})
+			tgtMap, tgtIsMap := tgtV.(map[string]interface{})
+			if srcIsMap && tgtIsMap {
+				equal = mergeJson(srcMap, tgtMap) && equal
+			}
+		}
+	}
+	return equal
+}
+
 func generateDriversJson() {
 	drivers := make(Drivers)
 	drivers["drivers"] = make(KV[interface{}])

cmd/storage.go

@@ -8,6 +8,7 @@ import (
 	"os"
 	"strconv"
 
+	"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
 	"github.com/OpenListTeam/OpenList/v4/internal/db"
 	"github.com/OpenListTeam/OpenList/v4/pkg/utils"
 	"github.com/charmbracelet/bubbles/table"
@@ -30,8 +31,8 @@ var disableStorageCmd = &cobra.Command{
 			return fmt.Errorf("mount path is required")
 		}
 		mountPath := args[0]
-		Init()
-		defer Release()
+		bootstrap.Init()
+		defer bootstrap.Release()
 		storage, err := db.GetStorageByMountPath(mountPath)
 		if err != nil {
 			return fmt.Errorf("failed to query storage: %+v", err)
@@ -69,8 +70,8 @@ var deleteStorageCmd = &cobra.Command{
 			}
 		}
 
-		Init()
-		defer Release()
+		bootstrap.Init()
+		defer bootstrap.Release()
 		err = db.DeleteStorageById(uint(id))
 		if err != nil {
 			return fmt.Errorf("failed to delete storage by id: %+v", err)
@@ -123,8 +124,8 @@ var listStorageCmd = &cobra.Command{
 	Use:   "list",
 	Short: "List all storages",
 	RunE: func(cmd *cobra.Command, args []string) error {
-		Init()
-		defer Release()
+		bootstrap.Init()
+		defer bootstrap.Release()
 		storages, _, err := db.GetStorages(1, -1)
 		if err != nil {
 			return fmt.Errorf("failed to query storages: %+v", err)

drivers/115_open/driver.go

@@ -53,6 +53,12 @@ func (d *Open115) Init(ctx context.Context) error {
 	if d.Addition.LimitRate > 0 {
 		d.limiter = rate.NewLimiter(rate.Limit(d.Addition.LimitRate), 1)
 	}
+	if d.PageSize <= 0 {
+		d.PageSize = 200
+	} else if d.PageSize > 1150 {
+		d.PageSize = 1150
+	}
+
 	return nil
 }
 
@@ -69,7 +75,7 @@ func (d *Open115) Drop(ctx context.Context) error {
 
 func (d *Open115) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
 	var res []model.Obj
-	pageSize := int64(200)
+	pageSize := int64(d.PageSize)
 	offset := int64(0)
 	for {
 		if err := d.WaitLimit(ctx); err != nil {

drivers/115_open/meta.go

@@ -12,6 +12,7 @@ type Addition struct {
 	OrderBy        string  `json:"order_by" type:"select" options:"file_name,file_size,user_utime,file_type"`
 	OrderDirection string  `json:"order_direction" type:"select" options:"asc,desc"`
 	LimitRate      float64 `json:"limit_rate" type:"float" default:"1" help:"limit all api request rate ([limit]r/1s)"`
+	PageSize       int64   `json:"page_size" type:"number" default:"200" help:"list api per page size of 115open driver"`
 	AccessToken    string  `json:"access_token" required:"true"`
 	RefreshToken   string  `json:"refresh_token" required:"true"`
 }

drivers/115_open/upload.go

@@ -107,16 +107,16 @@ func (d *Open115) multpartUpload(ctx context.Context, stream model.FileStreamer,
 		if err != nil {
 			return err
 		}
-		rateLimitedRd := driver.NewLimitedUploadStream(ctx, rd)
 		err = retry.Do(func() error {
 			rd.Seek(0, io.SeekStart)
-			part, err := bucket.UploadPart(imur, rateLimitedRd, partSize, int(i))
+			part, err := bucket.UploadPart(imur, driver.NewLimitedUploadStream(ctx, rd), partSize, int(i))
 			if err != nil {
 				return err
 			}
 			parts[i-1] = part
 			return nil
 		},
+			retry.Context(ctx),
 			retry.Attempts(3),
 			retry.DelayType(retry.BackOffDelay),
 			retry.Delay(time.Second))

drivers/123/meta.go

@@ -28,6 +28,7 @@ func init() {
 		return &Pan123{
 			Addition: Addition{
 				UploadThread: 3,
+				Platform: "web",
 			},
 		}
 	})

drivers/123/upload.go

@@ -125,27 +125,18 @@ func (d *Pan123) newUpload(ctx context.Context, upReq *UploadResp, file model.Fi
 				curSize = lastChunkSize
 			}
 			var reader io.ReadSeeker
-			var rateLimitedRd io.Reader
 			threadG.GoWithLifecycle(errgroup.Lifecycle{
-				Before: func(ctx context.Context) error {
-					if reader == nil {
-						var err error
-						reader, err = ss.GetSectionReader(offset, curSize)
-						if err != nil {
-							return err
-						}
-						rateLimitedRd = driver.NewLimitedUploadStream(ctx, reader)
-					}
-					return nil
+				Before: func(ctx context.Context) (err error) {
+					reader, err = ss.GetSectionReader(offset, curSize)
+					return
 				},
-				Do: func(ctx context.Context) error {
+				Do: func(ctx context.Context) (err error) {
 					reader.Seek(0, io.SeekStart)
 					uploadUrl := s3PreSignedUrls.Data.PreSignedUrls[strconv.Itoa(cur)]
 					if uploadUrl == "" {
 						return fmt.Errorf("upload url is empty, s3PreSignedUrls: %+v", s3PreSignedUrls)
 					}
-					reader.Seek(0, io.SeekStart)
-					req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadUrl, rateLimitedRd)
+					req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadUrl, driver.NewLimitedUploadStream(ctx, reader))
 					if err != nil {
 						return err
 					}
@@ -157,7 +148,7 @@ func (d *Pan123) newUpload(ctx context.Context, upReq *UploadResp, file model.Fi
 					}
 					defer res.Body.Close()
 					if res.StatusCode == http.StatusForbidden {
-						singleflight.AnyGroup.Do(fmt.Sprintf("Pan123.newUpload_%p", threadG), func() (any, error) {
+						_, err, _ = singleflight.AnyGroup.Do(fmt.Sprintf("Pan123.newUpload_%p", threadG), func() (any, error) {
 							newS3PreSignedUrls, err := getS3UploadUrl(ctx, upReq, cur, end)
 							if err != nil {
 								return nil, err
@@ -177,7 +168,7 @@ func (d *Pan123) newUpload(ctx context.Context, upReq *UploadResp, file model.Fi
 						}
 						return fmt.Errorf("upload s3 chunk %d failed, status code: %d, body: %s", cur, res.StatusCode, body)
 					}
-					progress := 10.0 + 85.0*float64(threadG.Success())/float64(chunkCount)
+					progress := 100 * float64(threadG.Success()+1) / float64(chunkCount+1)
 					up(progress)
 					return nil
 				},

drivers/123_link/driver.go

@@ -39,6 +39,10 @@ func (d *Pan123Link) Drop(ctx context.Context) error {
 	return nil
 }
 
+func (Addition) GetRootPath() string {
+	return "/"
+}
+
 func (d *Pan123Link) Get(ctx context.Context, path string) (model.Obj, error) {
 	node := GetNodeFromRootByPath(d.root, path)
 	return nodeToObj(node, path)

drivers/123_open/driver.go

@@ -18,6 +18,7 @@ type Open123 struct {
 	model.Storage
 	Addition
 	UID uint64
+	tm  *tokenManager
 }
 
 func (d *Open123) Config() driver.Config {
@@ -33,6 +34,24 @@ func (d *Open123) Init(ctx context.Context) error {
 		d.UploadThread = 3
 	}
 
+	if d.RefreshToken != "" {
+		// refresh token 直接主动刷新
+		d.AccessToken = ""
+		d.tm = &tokenManager{}
+	} else {
+		// 避免个人 token 刷新产生的多个登录，被动刷新
+		// 默认过期时间90天，jwt exp 不可靠
+		d.tm = &tokenManager{
+			// accessToken: d.AccessToken,
+			expiredAt: time.Now().Add(90 * 24 * time.Hour),
+		}
+	}
+
+	_, err := d.getAccessToken(false)
+	if err != nil {
+		return fmt.Errorf("init get access token error: %w", err)
+	}
+
 	return nil
 }