Compare commits

..

47 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 80b0980eac Apply remaining changes 2026-06-30 08:37:46 +00:00
copilot-swe-agent[bot] 75632de2e6 fix(media): return 404 on missing item_id and fix unscraped item query
- Validate GetMediaItemByID result before spawning goroutine in
  StartMediaScrape; return 404 when item_id > 0 but item is not found
- Replace `description` check with `plot` check in GetUnscrappedItems
  since scrapers populate Plot, not Description, to avoid re-selecting
  already-scraped items on every run
2026-06-30 08:35:06 +00:00
Pikachu Ren b85f82c76d Merge branch 'main' into dev-media 2026-06-30 16:32:07 +08:00
风月同天 6eedff851c feat(drivers): add Cloudflare Image Bed support (#2427)
* feat(driver): add Cloudflare Image Bed support

* fix: restore accidentally deleted file and name

* refactor: rename driver to cloudflare_imgbed and fix module structure

- Rename driver identifier and directory to 'cloudflare_imgbed' for consistency.
- Remove invalid 'replace' directive in go.mod.
- Restore accidentally modified/deleted files in public/dist.
- Update driver registration in drivers/all.go.

Co-authored-by: Copilot <copilot@github.com>

* fix: use base.NewRestyClient() and use e.g

Co-authored-by: Copilot <copilot@github.com>

* fix:go fmt

* feat(driver/cloudflare-imgbed): enhance cloudflare_imgbed API integration with improved error handling and pagination

* refactor

* feat(cloudflare_imgbed): implement upload functionality and optimize performance

- Added support for standard multipart form upload with zero-copy streaming.
- Implemented HuggingFace LFS direct upload for large files (>20MB).
- Integrated with OpenList global rate limiter and progress tracking.
- Optimized memory usage using io.MultiReader for request body construction.
- Added configurable upload threads for chunked HF uploads.
- Support auto mkdir dir when in upload

* refactor: simplify path handling logic

* refactor(cloudflare_imgbed): streamline API endpoint constants and improve initialization logic

* refactor(cloudflare_imgbed): clean up upload logic and remove unused functions

* docs: update help descriptions to English in cloudflare_imgbed

* feat(cloudflare_imgbed): add virtual directory support

* refactor(weak_cache): iImprove weak pointer cleanup handling

Store a cleanup handle alongside weak pointers and stop previous cleanups when overwriting entries to avoid stale removals and leaked cleanup handlers.

Ensure Delete signals success and stops associated cleanup. Add Clear to stop all active cleanups. Use entry identity in the cleanup callback to avoid removing newly inserted values.

* fix bug

* Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* Adds uploadFolder param and streams base64 sample

Adds an uploadFolder query parameter to forward the destination path to the backend. Replaces the fixed-size sample read with a streaming base64 encoder to avoid truncation and reduce allocations

Co-authored-by: Copilot <copilot@github.com>

* refactor: add context parameter to doRequest and related calls

* fix: update List method to check args.Refresh before virtual directory mask

* refactor: optimize List method and improve error handling in doRequest

* feat: add public URL support and multi-channel chunked upload

- Support multi-channel chunk size configurations (e.g., Telegram, CFR2, S3, Discord).
- Fix 401 unauthorized error when merging chunks in HuggingFace channel.

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
2026-06-30 16:26:35 +08:00
renovate[bot] aea77c5967 fix(deps): update module github.com/aws/aws-sdk-go to v1.55.8 (#2574)
fix(deps): update aws-sdk-go monorepo to v1.55.8

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-30 16:23:41 +08:00
renovate[bot] 009d834e71 fix(deps): update module github.com/antchfx/htmlquery to v1.3.6 (#2601)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
2026-06-30 16:23:04 +08:00
Yinan Qin e9ea991584 fix(clean_docker): only fetch up to 10 pages of image versions (#2680)
fix(clean_docker): only fetch upto 10 pages of image versions

Signed-off-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>
2026-06-30 15:00:46 +08:00
Pikachu Ren d4aa8b6a15 fix(driver): 123pan url www.123pan.com -> yun.123pan.com (#2678) 2026-06-30 10:36:17 +08:00
Yinan Qin 9e55a0111b feat(actions): add workflow to clean up untagged Docker images (#2657)
* feat(actions): add workflow to clean up untagged Docker images

Signed-off-by: elysia <a.elysia@proton.me>

* fix(actions): suggestions from codex

Signed-off-by: elysia <a.elysia@proton.me>

* chore: remove cron task

---------

Signed-off-by: elysia <a.elysia@proton.me>
2026-06-29 18:48:15 +08:00
pikachuim 3ce3dc4109 feat(media): 刮削修复写入作品名问题 2026-05-15 19:59:05 +08:00
pikachuim 6a46f0e809 feat(media): 刮削支持自定义刮削数量,避免太占性能 2026-05-15 19:34:26 +08:00
pikachuim 609d0a7b84 feat(media): 刮削支持自定义刮削数量,避免太占性能 2026-05-15 19:24:15 +08:00
pikachuim 9a28f7c09c Merge branch 'dev-media' of https://github.com/OpenListTeam/OpenList into dev-media 2026-05-15 18:36:09 +08:00
pikachuim 6646ffe891 feat(media): 提升刮削精准度,支持模糊刮削 2026-05-15 18:32:15 +08:00
j2rong4cn 20d9d39f83 fix(media): clean up imports and improve FetchFileReader logic 2026-05-15 17:12:20 +08:00
pikachuim ccb74025ea feat(media): 提升刮削精准度,支持模糊刮削 2026-05-15 17:05:09 +08:00
pikachuim 94002a2118 Merge branch 'main' into dev-media 2026-05-15 16:31:09 +08:00
pikachuim e750ff502a feat(media): support cloud codec via ffmpeg 2026-05-15 16:26:24 +08:00
pikachuim 97e885fee6 feat(media): support cloud codec via ffmpeg 2026-05-15 16:01:38 +08:00
pikachuim f891d3df1f Merge branch 'dev-media' of https://github.com/OpenListTeam/OpenList into dev-media 2026-05-15 15:48:52 +08:00
pikachuim 6e9878fe61 feat(media): support cloud codec via ffmpeg 2026-05-15 15:48:23 +08:00
pikachuim 08d9e2eb70 Merge branch 'dev-codec' into dev-media
# Conflicts:
#	internal/bootstrap/data/setting.go   resolved by dev-codec version
#	internal/conf/const.go   resolved by dev-codec version
#	internal/model/setting.go   resolved by dev-codec version
#	server/router.go   resolved by dev-codec version
2026-05-15 15:20:09 +08:00
pikachuim 0386f87110 feat(media): support cloud codec via ffmpeg 2026-05-15 15:16:01 +08:00
Pikachu Ren 4c9e63611f Merge branch 'main' into dev-media 2026-05-14 20:36:45 +08:00
pikachuim b00e694f68 fix(media): fix media scan sub path error 2026-05-14 18:25:57 +08:00
pikachuim ae6f05af11 fix(media): fix lrc file display & music id3 2026-05-14 17:07:35 +08:00
pikachuim 8dbcd0fd88 fix(media): fix api proxy path 2026-05-14 16:28:09 +08:00
pikachuim 2182ccff96 feat(media): 清空刮削、删除已失效 2026-05-14 14:27:54 +08:00
pikachuim b5f72ecdcc feat(media): 并发刮削(默认 5)+ 单条落库
增强 TMDB 搜索:模糊匹配 + 多候选关键词
管理页面新增导入/导出媒体库(全部 / 按扫描路径)
缩略图路径默认值改为 imgs
2026-05-14 14:14:58 +08:00
pikachuim 5a27d5fa6d feat(func): media library support change api url 2026-05-14 12:41:52 +08:00
pikachuim ce6bff2ad8 feat(func): media library support change api url 2026-05-14 12:37:07 +08:00
pikachuim cc1fec59ac feat(func): media library support change api url 2026-05-14 12:25:28 +08:00
Pikachu Ren 29f0636833 Merge branch 'main' into dev-media 2026-04-17 14:25:04 +08:00
pikachuim 8a72c78075 feat(func): media library support select 2026-04-08 21:21:14 +08:00
pikachuim 3527cab26b feat(func): media library support select 2026-04-08 20:32:40 +08:00
pikachuim dc92fccf52 feat(func): media library support select 2026-04-07 23:28:55 +08:00
pikachuim e33ff8d9f7 feat(func): media library support select 2026-04-07 23:07:33 +08:00
pikachuim 7571afcd41 Merge branch 'main' into dev-media
# Conflicts:
#	go.sum   resolved by main version
2026-04-07 23:07:07 +08:00
pikachuim 85c7f85b18 Merge branch 'main' into dev-media 2026-04-07 21:26:33 +08:00
pikachuim 9775971eb7 feat(func): media library fix remote fetch error 2026-04-01 19:31:50 +08:00
pikachuim 9096eb60c2 feat(func): media library fix remote fetch error 2026-04-01 19:30:56 +08:00
pikachuim 51c703c0a8 feat(func): media library fix remote fetch error 2026-04-01 19:23:25 +08:00
pikachuim 0e33727675 feat(func): media library build test 2026-03-25 18:07:25 +08:00
pikachuim 879870ef63 feat(func): media library build test 2026-03-25 17:55:07 +08:00
pikachuim bd84607537 feat(func): media library 2026-03-09 20:56:21 +08:00
pikachuim 0f340111da feat(func): media library 2026-03-09 20:52:55 +08:00
pikachuim e776d2ca43 feat(func): media library 2026-03-09 20:43:16 +08:00
53 changed files with 11657 additions and 1056 deletions
+153
View File
@@ -0,0 +1,153 @@
name: Beta Media Release builds
on:
push:
branches: ["dev-media"]
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: write
jobs:
changelog:
name: Beta Media Release Changelog
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Create or update ref
id: create-or-update-ref
uses: ovsds/create-or-update-ref-action@v1
with:
ref: tags/beta-media
sha: ${{ github.sha }}
- name: Delete beta-media tag
run: git tag -d beta-media
continue-on-error: true
- name: changelog
id: changelog
run: |
git tag -l
npx changelogithub --output CHANGELOG.md
- name: Upload assets to beta-media release
uses: softprops/action-gh-release@v2
with:
body_path: CHANGELOG.md
files: CHANGELOG.md
prerelease: true
tag_name: beta-media
- name: Upload assets to github artifact
uses: actions/upload-artifact@v4
with:
name: beta-media changelog
path: ${{ github.workspace }}/CHANGELOG.md
compression-level: 0
if-no-files-found: error
release:
needs:
- changelog
strategy:
matrix:
include:
- target: "!(*musl*|*windows-arm64*|*windows7-*|*android*|*freebsd*)" # xgo and loongarch
hash: "md5"
- target: "linux-!(arm*)-musl*" #musl-not-arm
hash: "md5-linux-musl"
- target: "linux-arm*-musl*" #musl-arm
hash: "md5-linux-musl-arm"
- target: "windows-arm64" #win-arm64
hash: "md5-windows-arm64"
- target: "windows7-*" #win7
hash: "md5-windows7"
- target: "android-*" #android
hash: "md5-android"
- target: "freebsd-*" #freebsd
hash: "md5-freebsd"
name: Beta Media Release
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: "1.25.0"
- name: Setup web
run: |
frontendRepo="${FRONTEND_REPO:-OpenListTeam/OpenList-Frontend}"
release_json=$(curl -fsSL --max-time 10 \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
"https://api.github.com/repos/$frontendRepo/releases/tags/beta-media")
tar_url=$(echo "$release_json" | jq -r '.assets[].browser_download_url' | grep "openlist-frontend-dist" | grep -v "lite" | grep "\.tar\.gz$")
echo "Downloading frontend from: $tar_url"
curl -fsSL "$tar_url" -o dist.tar.gz
rm -rf public/dist && mkdir -p public/dist
tar -zxvf dist.tar.gz -C public/dist
rm -rf dist.tar.gz
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
FRONTEND_REPO: ${{ vars.FRONTEND_REPO }}
- name: Build
uses: OpenListTeam/cgo-actions@v1.2.2
with:
targets: ${{ matrix.target }}
musl-target-format: $os-$musl-$arch
github-token: ${{ secrets.GITHUB_TOKEN }}
out-dir: build
output: openlist-$target$ext
musl-base-url: "https://github.com/OpenListTeam/musl-compilers/releases/latest/download/"
x-flags: |
github.com/OpenListTeam/OpenList/v4/internal/conf.BuiltAt=$built_at
github.com/OpenListTeam/OpenList/v4/internal/conf.GitAuthor=The OpenList Projects Contributors <noreply@openlist.team>
github.com/OpenListTeam/OpenList/v4/internal/conf.GitCommit=$git_commit
github.com/OpenListTeam/OpenList/v4/internal/conf.Version=$tag
github.com/OpenListTeam/OpenList/v4/internal/conf.WebVersion=rolling
- name: Compress
run: |
bash build.sh zip ${{ matrix.hash }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload assets to beta-media release
uses: softprops/action-gh-release@v2
with:
files: build/compress/*
prerelease: true
tag_name: beta-media
- name: Clean illegal characters from matrix.target
id: clean_target_name
run: |
ILLEGAL_CHARS_REGEX='[":<>|*?\\/\r\n]'
CLEANED_TARGET=$(echo "${{ matrix.target }}" | sed -E "s/$ILLEGAL_CHARS_REGEX//g")
echo "Original target: ${{ matrix.target }}"
echo "Cleaned target: $CLEANED_TARGET"
echo "cleaned_target=$CLEANED_TARGET" >> $GITHUB_ENV
- name: Upload assets to github artifact
uses: actions/upload-artifact@v4
with:
name: beta-media builds for ${{ env.cleaned_target }}
path: ${{ github.workspace }}/build/compress/*
compression-level: 0
if-no-files-found: error
File diff suppressed because it is too large Load Diff
+565
View File
File diff suppressed because it is too large Load Diff
+167
View File
@@ -0,0 +1,167 @@
# OpenList 转码协议 v1(自研)
> 用于 OpenList Master 与远程 FFmpeg 转码 Worker 之间的通信。
> 协议本身、所有依赖、推荐编码 (H.264/AV1) 均为开源免费,可商用。
## 总览
```
Player ──► Master(/api/transcode/play) ──► Scheduler.Submit
Worker ──► Master(/api/transcode/worker/register) # 启动一次
Worker ──► Master(/api/transcode/worker/heartbeat) # 周期 10s
Worker ──► Master(/api/transcode/worker/claim) # 长轮询拉任务
Worker ──► Master(/api/transcode/worker/segment) # 流式回推切片
Worker ──► Master(/api/transcode/worker/job/finish) # 上报完成
Player ──► Master(/tc/{job}/{token}/master.m3u8) # 拉取播放列表
Player ──► Master(/tc/{job}/{token}/{profile}/seg-N.ts) # 拉取切片
```
## 通用约定
- 所有请求体均为 JSON。
- Worker 与 Master 间通过两类凭据鉴权:
- **shared_secret**(管理端 `transcode_worker_secret` 设置项):用于 register / heartbeat / claim
- **callback_token**(Master 在 job 中下发,一次性):用于 segment / job/finish
- Header 形式:`Authorization: Bearer <token>`
- 返回包统一使用 OpenList 通用响应:`{"code":200,"message":"success","data":{...}}`
## 端点详细
### POST /api/transcode/worker/register
> 鉴权:Bearer = shared_secret
请求:
```json
{
"name": "gpu-node-01",
"version": "1.0.0",
"capacity": 4,
"hwaccel": ["nvenc"],
"codecs_decode": ["h264","hevc","av1","vp9"],
"codecs_encode": ["h264","hevc"],
"max_resolution": "3840x2160",
"tags": ["linux","amd64"]
}
```
响应:
```json
{
"code":200,"message":"success",
"data": {
"worker_id":"wk_xxx",
"heartbeat_interval":10,
"claim_strategy":"pull",
"protocol_version":"v1"
}
}
```
### POST /api/transcode/worker/heartbeat
> 鉴权:Bearer = shared_secret
> 间隔:register 返回的 `heartbeat_interval`;服务端 30s 未收到则剔除 worker
请求:
```json
{ "worker_id":"wk_xxx", "load":0.4, "running":["job_a"], "free_slots":3 }
```
响应:
```json
{ "code":200,"message":"success","data":{"ok":true,"kick":false,"cancel":["job_x"]}}
```
- `kick=true`:服务端要求 Worker 主动退出
- `cancel`:需要 Worker 立刻终止的 job_ids(异步取消通道)
### POST /api/transcode/worker/claim
> 鉴权:Bearer = shared_secret
> 长轮询:当无任务时,服务端最多挂起 `wait` 秒(最大 30)
请求:
```json
{ "worker_id":"wk_xxx", "slots":1, "wait":20 }
```
响应:
```json
{ "code":200,"message":"success","data":{"jobs":[{
"id":"job_xxx",
"path":"/movies/x.mkv",
"source_url":"https://example.com/d/movies/x.mkv?sign=...",
"profiles":[{
"name":"1080p","video_codec":"h264","video_bitrate":"4000k",
"scale":"1920:-2","audio_codec":"aac","audio_bitrate":"160k",
"hwaccel":"nvenc"
}],
"output":{"format":"hls","segment_duration":6},
"callback_token":"tk_xxx",
"deadline":1731601234
}]}}
```
### PUT /api/transcode/worker/segment
> 鉴权:Bearer = callback_token
> Query 参数:
> - `job` 任务 ID
> - `profile` profile 名(如 `1080p`)
> - `seq` 切片序号(从 0 开始)
> - `duration` 该切片秒数
> - `final` 是否为最后一片(true/false)
>
> Body:原始 MPEG-TS 二进制(`Content-Type: video/mp2t`)
响应:`204 No Content`
### POST /api/transcode/worker/job/finish
> 鉴权:Bearer = callback_token
请求:
```json
{
"job_id":"job_xxx",
"status":"finished",
"error":"",
"stats":{"elapsed":312.4,"fps":120,"speed":2.1,"bytes_out":890123456}
}
```
status 取值:`finished` / `failed` / `cancelled`
## 播放端
### POST /api/fs/transcode/play
> 由前端播放器调用,用户登录态可见
> 请求:`{"path":"/movies/x.mkv"}`
>
> 响应(无需转码):
> `{"transcode":false,"reason":"size below threshold"}`
>
> 响应(已下发转码任务):
> `{"transcode":true,"job_id":"job_xx","master_url":"https://.../tc/.../master.m3u8","profile":"1080p"}`
### GET /tc/:job/:token/master.m3u8
返回多档位主 m3u8。
### GET /tc/:job/:token/:profile/playlist.m3u8
阻塞等待首切片就绪(最多 30s)后返回 HLS 播放列表,未结束时不带 `#EXT-X-ENDLIST`。
### GET /tc/:job/:token/:profile/seg-N.ts
返回 N 号切片;如果尚未生成,最多等待 60s。
## 启动远程 Worker
```bash
# 用 Master 配置中的 transcode_worker_secret 作为密钥
go build -o openlist-worker ./cmd/worker
./openlist-worker \
-master http://openlist.example.com:5244 \
-secret <transcode_worker_secret> \
-capacity 2 \
-hwaccel nvenc \
-ffmpeg /usr/bin/ffmpeg \
-workdir /tmp/openlist-worker
```
## License 说明
- 协议:自研,可自由使用
- FFmpeg:使用系统包提供的 LGPL 版本即可(含 libx264/libx265,libx264 是 GPL,建议在 Worker 镜像中遵循 GPL 公开/动态链接合规)
- 推荐编码:H.264 互联网分发免授权 / AV1 永久免版税
+1 -1
View File
@@ -98,7 +98,7 @@ func (d *Pan123) Link(ctx context.Context, file model.Obj, args model.LinkArgs)
}
log.Debug("download url: ", u_)
res, err := base.NoRedirectClient.R().SetHeader("Referer", "https://www.123pan.com/").Get(u_)
res, err := base.NoRedirectClient.R().SetHeader("Referer", "https://yun.123pan.com/").Get(u_)
if err != nil {
return nil, err
}
+7 -7
View File
@@ -24,9 +24,9 @@ import (
// do others that not defined in Driver interface
const (
Api = "https://www.123pan.com/api"
AApi = "https://www.123pan.com/a/api"
BApi = "https://www.123pan.com/b/api"
Api = "https://yun.123pan.com/api"
AApi = "https://yun.123pan.com/a/api"
BApi = "https://yun.123pan.com/b/api"
LoginApi = "https://login.123pan.com/api"
MainApi = BApi
SignIn = LoginApi + "/user/sign_in"
@@ -169,8 +169,8 @@ func (d *Pan123) login() error {
}
res, err := base.RestyClient.R().
SetHeaders(map[string]string{
"origin": "https://www.123pan.com",
"referer": "https://www.123pan.com/",
"origin": "https://yun.123pan.com",
"referer": "https://yun.123pan.com/",
"user-agent": "Dart/2.19(dart:io)-openlist",
"platform": "web",
"app-version": "3",
@@ -207,8 +207,8 @@ func (d *Pan123) Request(url string, method string, callback base.ReqCallback, r
do:
req := base.RestyClient.R()
req.SetHeaders(map[string]string{
"origin": "https://www.123pan.com",
"referer": "https://www.123pan.com/",
"origin": "https://yun.123pan.com",
"referer": "https://yun.123pan.com/",
"authorization": "Bearer " + d.AccessToken,
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) openlist-client",
"platform": d.Platform,
+1 -1
View File
@@ -101,7 +101,7 @@ func (d *Pan123Share) Link(ctx context.Context, file model.Obj, args model.LinkA
}
log.Debug("download url: ", u_)
res, err := base.NoRedirectClient.R().SetHeader("Referer", "https://www.123pan.com/").Get(u_)
res, err := base.NoRedirectClient.R().SetHeader("Referer", "https://yun.123pan.com/").Get(u_)
if err != nil {
return nil, err
}
+5 -5
View File
@@ -20,9 +20,9 @@ import (
)
const (
Api = "https://www.123pan.com/api"
AApi = "https://www.123pan.com/a/api"
BApi = "https://www.123pan.com/b/api"
Api = "https://yun.123pan.com/api"
AApi = "https://yun.123pan.com/a/api"
BApi = "https://yun.123pan.com/b/api"
MainApi = BApi
FileList = MainApi + "/share/get"
DownloadInfo = MainApi + "/share/download/info"
@@ -58,8 +58,8 @@ func (d *Pan123Share) request(url string, method string, callback base.ReqCallba
}
req := base.RestyClient.R()
req.SetHeaders(map[string]string{
"origin": "https://www.123pan.com",
"referer": "https://www.123pan.com/",
"origin": "https://yun.123pan.com",
"referer": "https://yun.123pan.com/",
"authorization": "Bearer " + d.AccessToken,
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) openlist-client",
"platform": "web",
+1
View File
@@ -24,6 +24,7 @@ import (
_ "github.com/OpenListTeam/OpenList/v4/drivers/baidu_photo"
_ "github.com/OpenListTeam/OpenList/v4/drivers/chaoxing"
_ "github.com/OpenListTeam/OpenList/v4/drivers/chunk"
_ "github.com/OpenListTeam/OpenList/v4/drivers/cloudflare_imgbed"
_ "github.com/OpenListTeam/OpenList/v4/drivers/cloudreve"
_ "github.com/OpenListTeam/OpenList/v4/drivers/cloudreve_v4"
_ "github.com/OpenListTeam/OpenList/v4/drivers/cnb_releases"
+168
View File
@@ -0,0 +1,168 @@
package cloudflare_imgbed
import (
"context"
"fmt"
"net/http"
"path"
"strings"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/cache"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/go-resty/resty/v2"
)
type CFImgBed struct {
model.Storage
Addition
client *resty.Client
virtualDir *cache.WeakCacheMap[string, model.Object]
publicUrlPrefix string
}
func (d *CFImgBed) Config() driver.Config { return config }
func (d *CFImgBed) GetAddition() driver.Additional { return &d.Addition }
func (d *CFImgBed) Init(ctx context.Context) error {
d.UploadThread = min(d.UploadThread, 32)
if d.UploadThread < 1 {
d.UploadThread = 3
}
d.Address = strings.TrimRight(d.Address, "/")
d.client = base.NewRestyClient().
SetBaseURL(d.Address).
SetHeader("Authorization", "Bearer "+d.Token).
SetDebug(false)
// 连通性测试:尝试获取根目录单条数据
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"start": "0",
"count": "1",
"dir": "/",
})
}, nil)
if err != nil {
return fmt.Errorf("init verification failed: %w", err)
}
d.virtualDir = cache.NewWeakCacheMap[string, model.Object]()
return nil
}
func (d *CFImgBed) Drop(ctx context.Context) error {
if d.virtualDir != nil {
d.virtualDir.Clear()
}
return nil
}
func (d *CFImgBed) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
// if !args.Refresh && model.ObjHasMask(dir, model.Virtual) {
// if _, ok := d.virtualDir.Load(dir.GetPath()); ok {
// return nil, nil
// }
// }
var dirSeen map[string]bool
var fileSeen map[string]bool
var objs []model.Obj
start := 0
for {
var resp ListResponse
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"dir": dir.GetPath(),
"start": fmt.Sprintf("%d", start),
"count": fmt.Sprintf("%d", listPageSize),
})
}, &resp)
if err != nil {
return nil, err
}
if len(resp.Files) == 0 && len(resp.Directories) == 0 {
break
}
if start == 0 {
dirSeen = make(map[string]bool, len(resp.Directories))
fileSeen = make(map[string]bool, len(resp.Files))
objs = make([]model.Obj, 0, len(resp.Directories)+len(resp.Files))
}
for _, rawDir := range resp.Directories {
rawDir = "/" + strings.TrimRight(rawDir, "/")
if !dirSeen[rawDir] {
dirSeen[rawDir] = true
objs = append(objs, &model.Object{
Path: rawDir,
Name: path.Base(rawDir),
Modified: d.Modified,
IsFolder: true,
})
}
}
for _, item := range resp.Files {
if !fileSeen[item.Name] {
fileSeen[item.Name] = true
objs = append(objs, parseFile(item))
}
}
// 如果当前获取的数量少于分页大小,说明已加载完毕
if len(resp.Files)+len(resp.Directories) < listPageSize {
break
}
start += listPageSize
}
return objs, nil
}
func (d *CFImgBed) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if d.publicUrlPrefix != "" {
return &model.Link{URL: d.publicUrlPrefix + utils.EncodePath(file.GetPath())}, nil
}
return &model.Link{URL: d.Address + "/file" + utils.EncodePath(file.GetPath())}, nil
}
func (d *CFImgBed) Get(ctx context.Context, pathStr string) (model.Obj, error) {
fullPath := path.Join(d.RootFolderPath, pathStr)
if obj, found := d.virtualDir.Load(fullPath); found {
return obj, nil
}
return nil, errs.NotSupport
}
// MakeDir 在图床中通常是虚拟的,此处返回虚拟目录对象以支持上传时的路径展示
func (d *CFImgBed) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
fullPath := path.Join(parentDir.GetPath(), dirName)
temp := &model.Object{
Path: fullPath,
Name: dirName,
IsFolder: true,
Modified: d.Modified,
Mask: model.Virtual,
}
d.virtualDir.Store(fullPath, temp)
return temp, nil
}
func (d *CFImgBed) Remove(ctx context.Context, obj model.Obj) error {
reqPath := obj.GetPath()
if model.ObjHasMask(obj, model.Virtual) {
d.virtualDir.Delete(reqPath)
return nil
}
_, err := d.doRequest(ctx, http.MethodPost, deleteApi+utils.EncodePath(reqPath), func(req *resty.Request) {
req.SetQueryParam("folder", fmt.Sprintf("%t", obj.IsDir()))
}, nil)
return err
}
var _ driver.Driver = (*CFImgBed)(nil)
+27
View File
@@ -0,0 +1,27 @@
package cloudflare_imgbed
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
Address string `json:"address" required:"true" help:"Backend API address of the image hosting service, e.g., https://img.example.com"`
Token string `json:"token" required:"true" help:"Authentication Token"`
SmallChannelName string `json:"smallChannelName" help:"Channel name for regular files (typically <20MB)"`
LargeChannelName string `json:"largeChannelName" help:"Channel name for large files"`
LargeChannelType string `json:"largeChannelType" type:"select" options:",huggingface,telegram,cfr2,s3,discord" help:"Large File Channel Type: Hugging Face (Direct Upload)、telegram/cfr2/s3/discord(Multipart Upload)"`
UploadThread int `json:"uploadThread" type:"number" default:"3" help:"Concurrent thread count for HuggingFace chunked direct upload"`
}
var config = driver.Config{
Name: "cloudflare_imgbed",
LocalSort: true,
NoUpload: false,
DefaultRoot: "/",
}
func init() {
op.RegisterDriver(func() driver.Driver { return &CFImgBed{} })
}
+117
View File
@@ -0,0 +1,117 @@
package cloudflare_imgbed
import (
"fmt"
"path"
"strconv"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
)
const listPageSize = 1000
// ListResponse 列表接口响应
type ListResponse struct {
Files []FileItem `json:"files"`
Directories []string `json:"directories"`
}
type FileItem struct {
Name string `json:"name"`
Metadata map[string]interface{} `json:"metadata"` // 存储文件大小、哈希、时间戳等
}
type apiError struct {
Error string `json:"error"`
Message string `json:"message"`
}
// standardUploadResp 标准上传成功返回的数组
type standardUploadResp []struct {
Src string `json:"src"`
PublicUrl string `json:"publicUrl"`
}
// hfGetUrlResp 获取 HF 直传授权地址的响应
type hfGetUrlResp struct {
Success bool `json:"success"`
FullID string `json:"fullId"`
FilePath string `json:"filePath"`
ChannelName string `json:"channelName"`
Repo string `json:"repo"`
NeedsLfs bool `json:"needsLfs"` // 是否需要进行 LFS 物理上传
AlreadyExists bool `json:"alreadyExists"` // 是否秒传成功
Oid string `json:"oid"` // Git LFS 对象 ID (SHA256)
UploadAction *UploadAction `json:"uploadAction"`
}
type UploadAction struct {
Href string `json:"href"`
Header map[string]string `json:"header"`
}
type hfCommitResp struct {
Success bool `json:"success"`
Src string `json:"src"`
PublicUrl string `json:"publicUrl"`
FileUrl string `json:"fileUrl"`
FullID string `json:"fullId"`
}
// 辅助函数:从 map 中安全提取字符串/数值
func getString(m map[string]interface{}, keys ...string) string {
for _, k := range keys {
if v, ok := m[k]; ok {
switch val := v.(type) {
case string:
return val
case float64:
return strconv.FormatInt(int64(val), 10)
default:
return fmt.Sprintf("%v", val)
}
}
}
return ""
}
func getInt64(m map[string]interface{}, keys ...string) int64 {
for _, k := range keys {
if v, ok := m[k]; ok {
switch val := v.(type) {
case string:
n, _ := strconv.ParseInt(val, 10, 64)
return n
case float64:
return int64(val)
case int64:
return val
}
}
}
return 0
}
func parseFile(item FileItem) *model.Object {
name := path.Base(item.Name)
var size int64
var modTime time.Time
if item.Metadata != nil {
size = getInt64(item.Metadata, "FileSizeBytes", "File-Size")
ts := getInt64(item.Metadata, "TimeStamp")
if ts > 0 {
modTime = time.UnixMilli(ts)
}
}
return &model.Object{
Path: "/" + strings.TrimRight(item.Name, "/"),
Name: name,
Size: size,
Modified: modTime,
IsFolder: false,
}
}
File diff suppressed because it is too large Load Diff
+70
View File
@@ -0,0 +1,70 @@
package cloudflare_imgbed
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/go-resty/resty/v2"
log "github.com/sirupsen/logrus"
)
const (
listApi = "/api/manage/list"
deleteApi = "/api/manage/delete"
uploadApi = "/upload"
hfGetUrlApi = "/upload/huggingface/getUploadUrl"
hfCommitApi = "/upload/huggingface/commitUpload"
hfDirectThreshold int64 = 20 * 1024 * 1024
fileSampleSize = 512 // HF 申请上传地址时需提供文件前 512 字节的 Sample
)
// doRequest 通用请求封装,包含重试和 API 错误解析
func (d *CFImgBed) doRequest(ctx context.Context, method, urlPath string, callback func(*resty.Request), resp interface{}) ([]byte, error) {
maxRetries := 3
for i := 0; i < maxRetries; i++ {
req := d.client.R()
req.SetContext(ctx)
if callback != nil {
callback(req)
}
if resp != nil {
req.SetResult(resp)
}
res, err := req.Execute(method, urlPath)
if err != nil {
log.WithError(err).Warnf("request %s %s failed, attempt %d/%d", method, urlPath, i+1, maxRetries)
if i < maxRetries-1 {
time.Sleep(time.Duration(i+1) * time.Second)
continue
}
return nil, err
}
// Retry on rate limit before attempting to interpret the body as an API error.
if res.StatusCode() == 429 {
time.Sleep(time.Duration(i+1) * 2 * time.Second)
continue
}
body := res.Body()
var apiErr apiError
if err := json.Unmarshal(body, &apiErr); err == nil {
if apiErr.Error != "" || apiErr.Message != "" {
msg := apiErr.Error
if msg == "" {
msg = apiErr.Message
}
return nil, fmt.Errorf("API error: %s", msg)
}
}
if res.IsError() {
return nil, fmt.Errorf("HTTP %d", res.StatusCode())
}
return body, nil
}
return nil, fmt.Errorf("max retries exceeded")
}
+7 -6
View File
@@ -18,10 +18,10 @@ require (
github.com/ProtonMail/gopenpgp/v2 v2.9.0
github.com/SheltonZhu/115driver v1.3.3
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible
github.com/antchfx/htmlquery v1.3.5
github.com/antchfx/xpath v1.3.5
github.com/antchfx/htmlquery v1.3.6
github.com/antchfx/xpath v1.3.6
github.com/avast/retry-go v3.0.0+incompatible
github.com/aws/aws-sdk-go v1.55.7
github.com/aws/aws-sdk-go v1.55.8
github.com/blevesearch/bleve/v2 v2.5.2
github.com/bmatcuk/doublestar/v4 v4.9.1
github.com/caarlos0/env/v9 v9.0.0
@@ -65,6 +65,7 @@ require (
github.com/pquerna/otp v1.5.0
github.com/quic-go/quic-go v0.54.1
github.com/rclone/rclone v1.70.3
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd
github.com/shirou/gopsutil/v4 v4.25.5
github.com/sirupsen/logrus v1.9.3
github.com/spf13/afero v1.15.0
@@ -85,8 +86,8 @@ require (
gopkg.in/ldap.v3 v3.1.0
gorm.io/driver/mysql v1.5.7
gorm.io/driver/postgres v1.5.9
gorm.io/driver/sqlite v1.5.6
gorm.io/gorm v1.25.11
gorm.io/driver/sqlite v1.6.0
gorm.io/gorm v1.30.0
)
require (
@@ -120,6 +121,7 @@ require (
github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
github.com/lanrat/extsort v1.0.2 // indirect
github.com/mattn/go-sqlite3 v1.14.22 // indirect
github.com/mikelolasagasti/xz v1.0.1 // indirect
github.com/minio/minlz v1.0.0 // indirect
github.com/minio/xxml v0.0.3 // indirect
@@ -250,7 +252,6 @@ require (
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mattn/go-sqlite3 v1.14.22 // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
+12 -12
View File
@@ -27,8 +27,6 @@ github.com/Masterminds/semver/v3 v3.2.0 h1:3MEsd0SM6jqZojhjLWWeBY+Kcjy9i6MQAeY7Y
github.com/Masterminds/semver/v3 v3.2.0/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd h1:nzE1YQBdx1bq9IlZinHa+HVffy+NmVRoKr+wHN8fpLE=
github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd/go.mod h1:C8yoIfvESpM3GD07OCHU7fqI7lhwyZ2Td1rbNbTAhnc=
github.com/OpenListTeam/115-sdk-go v0.2.3 h1:nDNz0GxgliW+nT2Ds486k/rp/GgJj7Ngznc98ZBUwZo=
github.com/OpenListTeam/115-sdk-go v0.2.3/go.mod h1:cfvitk2lwe6036iNi2h+iNxwxWDifKZsSvNtrur5BqU=
github.com/OpenListTeam/115-sdk-go v0.2.4 h1:dVoEz3Pm996n/ZCuRckQvz36w938uNHjPrS7E/SVpBM=
github.com/OpenListTeam/115-sdk-go v0.2.4/go.mod h1:cfvitk2lwe6036iNi2h+iNxwxWDifKZsSvNtrur5BqU=
github.com/OpenListTeam/go-cache v0.1.0 h1:eV2+FCP+rt+E4OCJqLUW7wGccWZNJMV0NNkh+uChbAI=
@@ -80,15 +78,15 @@ github.com/andybalholm/brotli v1.1.2-0.20250424173009-453214e765f3 h1:8PmGpDEZl9
github.com/andybalholm/brotli v1.1.2-0.20250424173009-453214e765f3/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/antchfx/htmlquery v1.3.5 h1:aYthDDClnG2a2xePf6tys/UyyM/kRcsFRm+ifhFKoU0=
github.com/antchfx/htmlquery v1.3.5/go.mod h1:5oyIPIa3ovYGtLqMPNjBF2Uf25NPCKsMjCnQ8lvjaoA=
github.com/antchfx/xpath v1.3.5 h1:PqbXLC3TkfeZyakF5eeh3NTWEbYl4VHNVeufANzDbKQ=
github.com/antchfx/xpath v1.3.5/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/htmlquery v1.3.6 h1:RNHHL7YehO5XdO8IM8CynwLKONwRHWkrghbYhQIk9ag=
github.com/antchfx/htmlquery v1.3.6/go.mod h1:kcVUqancxPygm26X2rceEcagZFFVkLEE7xgLkGSDl/4=
github.com/antchfx/xpath v1.3.6 h1:s0y+ElRRtTQdfHP609qFu0+c6bglDv20pqOViQjjdPI=
github.com/antchfx/xpath v1.3.6/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/avast/retry-go v3.0.0+incompatible h1:4SOWQ7Qs+oroOTQOYnAHqelpCO0biHSxpiH9JdtuBj0=
github.com/avast/retry-go v3.0.0+incompatible/go.mod h1:XtSnn+n/sHqQIpZ10K1qAevBhOOCWBLXXy3hyiqqBrY=
github.com/aws/aws-sdk-go v1.38.20/go.mod h1:hcU610XS61/+aQV88ixoOzUoG7v3b31pl2zKMmprdro=
github.com/aws/aws-sdk-go v1.55.7 h1:UJrkFq7es5CShfBwlWAC8DA077vp8PyVbQd3lqLiztE=
github.com/aws/aws-sdk-go v1.55.7/go.mod h1:eRwEWoyTWFMVYVQzKMNHWP5/RV4xIUGMQfXQHfHkpNU=
github.com/aws/aws-sdk-go v1.55.8 h1:JRmEUbU52aJQZ2AjX4q4Wu7t4uZjOu71uyNmaWlUkJQ=
github.com/aws/aws-sdk-go v1.55.8/go.mod h1:ZkViS9AqA6otK+JBBNH2++sx1sgxrPKcSzPPvQkUtXk=
github.com/aws/aws-sdk-go-v2 v1.36.3 h1:mJoei2CxPutQVxaATCzDUjcZEjVRdpsiiXi2o38yqWM=
github.com/aws/aws-sdk-go-v2 v1.36.3/go.mod h1:LLXuLpgzEbD766Z5ECcRmi8AzSwfZItDtmABVkRLGzg=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.10 h1:zAybnyUQXIZ5mok5Jqwlf58/TFE7uvd3IAsa1aF9cXs=
@@ -595,6 +593,8 @@ github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUc
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd h1:CmH9+J6ZSsIjUK3dcGsnCnO41eRBOnY12zwkn5qVwgc=
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk=
github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46 h1:GHRpF1pTW19a8tTFrMLUcfWwyC0pnifVo2ClaLq+hP8=
github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46/go.mod h1:uAQ5PCi+MFsC7HjREoAz1BU+Mq60+05gifQSsHSDG/8=
github.com/secsy/goftp v0.0.0-20200609142545-aa2de14babf4 h1:PT+ElG/UUFMfqy5HrxJxNzj3QBOf7dZwupeVC+mG1Lo=
@@ -855,11 +855,11 @@ gorm.io/driver/mysql v1.5.7 h1:MndhOPYOfEp2rHKgkZIhJ16eVUIRf2HmzgoPmh7FCWo=
gorm.io/driver/mysql v1.5.7/go.mod h1:sEtPWMiqiN1N1cMXoXmBbd8C6/l+TESwriotuRRpkDM=
gorm.io/driver/postgres v1.5.9 h1:DkegyItji119OlcaLjqN11kHoUgZ/j13E0jkJZgD6A8=
gorm.io/driver/postgres v1.5.9/go.mod h1:DX3GReXH+3FPWGrrgffdvCk3DQ1dwDPdmbenSkweRGI=
gorm.io/driver/sqlite v1.5.6 h1:fO/X46qn5NUEEOZtnjJRWRzZMe8nqJiQ9E+0hi+hKQE=
gorm.io/driver/sqlite v1.5.6/go.mod h1:U+J8craQU6Fzkcvu8oLeAQmi50TkwPEhHDEjQZXDah4=
gorm.io/driver/sqlite v1.6.0 h1:WHRRrIiulaPiPFmDcod6prc4l2VGVWHz80KspNsxSfQ=
gorm.io/driver/sqlite v1.6.0/go.mod h1:AO9V1qIQddBESngQUKWL9yoH93HIeA1X6V633rBwyT8=
gorm.io/gorm v1.25.7/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
gorm.io/gorm v1.25.11 h1:/Wfyg1B/je1hnDx3sMkX+gAlxrlZpn6X0BXRlwXlvHg=
gorm.io/gorm v1.25.11/go.mod h1:xh7N7RHfYlNc5EmcI/El95gXusucDrQnHXe0+CgWcLQ=
gorm.io/gorm v1.30.0 h1:qbT5aPv1UH8gI99OsRlvDToLxW5zR7FzS9acZDOZcgs=
gorm.io/gorm v1.30.0/go.mod h1:8Z33v652h4//uMA76KjeDH8mJXPm1QNCYrMeatR0DOE=
lukechampine.com/blake3 v1.1.7 h1:GgRMhmdsuK8+ii6UZFDL8Nb+VyMwadAgcJyfYHxG6n0=
lukechampine.com/blake3 v1.1.7/go.mod h1:tkKEOtDkNtklkXtLNEOGNq5tcV90tJiA1vAA12R78LA=
modernc.org/libc v1.22.5 h1:91BNch/e5B0uPbJFgqbxXuOnxBQjlS//icfQEGmvyjE=
+73 -6
View File
@@ -162,12 +162,6 @@ func InitialSettings() []model.SettingItem {
{Key: conf.ReadMeAutoRender, Value: "true", Type: conf.TypeBool, Group: model.PREVIEW},
{Key: conf.FilterReadMeScripts, Value: "true", Type: conf.TypeBool, Group: model.PREVIEW}, // frontend
{Key: conf.NonEFSZipEncoding, Value: "IBM437", Type: conf.TypeString, Group: model.PREVIEW},
// wopi settings
{Key: conf.WopiEnabled, Value: "false", Type: conf.TypeBool, Group: model.PREVIEW, Flag: model.PUBLIC},
{Key: conf.WopiServices, Value: "[]", Type: conf.TypeText, Group: model.PREVIEW, Flag: model.PUBLIC, Help: "WOPI services JSON array"},
{Key: conf.WopiMaxSize, Value: "52428800", Type: conf.TypeNumber, Group: model.PREVIEW, Flag: model.PRIVATE, Help: "Max file size for WOPI editing in bytes (default 50MB)"},
// global settings
{Key: conf.HideFiles, Value: "/\\/README.md/i", Type: conf.TypeText, Group: model.GLOBAL},
{Key: "package_download", Value: "true", Type: conf.TypeBool, Group: model.GLOBAL},
@@ -255,6 +249,79 @@ func InitialSettings() []model.SettingItem {
{Key: conf.StreamMaxClientUploadSpeed, Value: "-1", Type: conf.TypeNumber, Group: model.TRAFFIC, Flag: model.PRIVATE},
{Key: conf.StreamMaxServerDownloadSpeed, Value: "-1", Type: conf.TypeNumber, Group: model.TRAFFIC, Flag: model.PRIVATE},
{Key: conf.StreamMaxServerUploadSpeed, Value: "-1", Type: conf.TypeNumber, Group: model.TRAFFIC, Flag: model.PRIVATE},
// media settings
{Key: conf.MediaTMDBKey, Value: "", Type: conf.TypeString, Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaTMDBAPIURL, Value: "api.themoviedb.org", Type: conf.TypeString, Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaDiscogsToken, Value: "", Type: conf.TypeString, Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaDiscogsAPIURL, Value: "api.discogs.com", Type: conf.TypeString, Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaStoreThumbnail, Value: "false", Type: conf.TypeBool, Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaThumbnailMode, Value: "base64", Type: conf.TypeSelect, Options: "base64,local", Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaThumbnailPath, Value: "/imgs", Type: conf.TypeString, Group: model.MEDIA, Flag: model.PRIVATE},
{Key: conf.MediaScrapeConcurrency, Value: "5", Type: conf.TypeNumber, Group: model.MEDIA, Flag: model.PRIVATE},
// transcode settings (FFmpeg 云端/本地转码) - 默认全部关闭
{Key: conf.TranscodeEnabled, Value: "false", Type: conf.TypeBool, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `开启后,超过阈值的媒体文件将通过 FFmpeg 转码后再播放;默认关闭`},
{Key: conf.TranscodeRunMode, Value: "local", Type: conf.TypeSelect, Options: "local,remote,hybrid", Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `local=仅使用本机内置 worker;remote=只使用远程 Worker 节点;hybrid=本地优先,超载后派发到远程`},
{Key: conf.TranscodeMinSizeGB, Value: "5", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `文件大于该 GB 数才走转码(小于则直链播放),0=任意大小都转码`},
{Key: conf.TranscodeMinBitrateMbps, Value: "20", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `视频码率超过该 Mbps 才转码,0=不限制`},
{Key: conf.TranscodeSourceCodecs, Value: "hevc,h265,av1,vvc,vp9", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `仅对这些源视频编码进行转码(逗号分隔)。常见高码率/兼容性差的编码:hevc,av1,vvc,vp9`},
{Key: conf.TranscodeSourceExtensions, Value: "mkv,ts,m2ts,mov,avi,wmv,flv,rmvb,webm", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `仅对这些后缀进行转码(逗号分隔,不带点),mp4 默认不转码可直接播放`},
{Key: conf.TranscodeOutputFormat, Value: "hls", Type: conf.TypeSelect, Options: "hls,dash,mp4", Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `输出封装格式,HLS 兼容性最好`},
{Key: conf.TranscodeOutputCodec, Value: "h264", Type: conf.TypeSelect, Options: "h264,hevc,av1", Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `重新编码后的视频编码,推荐 h264(最广兼容、互联网分发免授权费)`},
{Key: conf.TranscodeOutputBitrate, Value: "4000k", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `输出视频码率,例如 4000k / 6M。建议 1080p:4000k、720p:2500k`},
{Key: conf.TranscodeOutputAudioCodec, Value: "aac", Type: conf.TypeSelect, Options: "aac,mp3,opus,copy", Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `输出音频编码,aac 兼容性最好;copy=直接复制源音频流`},
{Key: conf.TranscodeOutputAudioBitrate, Value: "160k", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `输出音频码率`},
{Key: conf.TranscodeOutputResolution, Value: "1920x1080", Type: conf.TypeSelect, Options: "source,3840x2160,2560x1440,1920x1080,1280x720,854x480", Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `输出分辨率上限,超过该分辨率会下采样;source=保持源分辨率`},
{Key: conf.TranscodeSegmentDuration, Value: "6", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `HLS/DASH 切片时长(秒),越小首帧越快但请求数变多,推荐 4-10`},
{Key: conf.TranscodeHWAccel, Value: "none", Type: conf.TypeSelect,
Options: "none,auto,nvenc,qsv,vaapi,amf,videotoolbox",
Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `GPU 硬件加速:
none = 纯 CPU(libx264)
auto = 自动探测可用加速器
nvenc = NVIDIA GPU(GeForce/Tesla/Quadro/RTX,需 NVIDIA 驱动)
qsv = Intel 集显/独显 QuickSync(免费,功耗低)
vaapi = Linux 通用 VA-API(支持 Intel/AMD)
amf = AMD GPU(Windows AMF)
videotoolbox = macOS 硬件加速`},
{Key: conf.TranscodeFFmpegPath, Value: "ffmpeg", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `FFmpeg 可执行路径,留空则使用 PATH 中的 ffmpeg`},
{Key: conf.TranscodeFFprobePath, Value: "ffprobe", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `FFprobe 可执行路径`},
{Key: conf.TranscodeWorkerSecret, Value: "", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `远程 Worker 注册时使用的共享密钥,留空则禁用远程 Worker 注册`},
{Key: conf.TranscodeCachePath, Value: "data/transcode_cache", Type: conf.TypeString, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `转码切片缓存目录`},
{Key: conf.TranscodeCacheMaxGB, Value: "20", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `切片缓存最大容量(GB),超过后按 LRU 清理`},
{Key: conf.TranscodeJobTimeoutMin, Value: "120", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `单个转码任务超时分钟数,超时自动失败`},
{Key: conf.TranscodeLocalConcurrency, Value: "1", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `本机内置 worker 同时执行的转码任务数(local/hybrid 模式生效)`},
{Key: conf.TranscodeIdleTimeoutSec, Value: "90", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `播放端无请求多少秒后自动停止转码(默认90秒,0=禁用)`},
{Key: conf.TranscodeChunkDurationSec, Value: "60", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `每个 chunk 时长(秒),越大每段缓冲越长但 ffmpeg 进程切换越少(默认60秒)`},
{Key: conf.TranscodeMaxChunkConcurrency, Value: "2", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `单个视频任务最多同时运行的 chunk 数(默认2,越大并发越高但 CPU/GPU 占用越多)`},
{Key: conf.TranscodeChunkIdleSec, Value: "60", Type: conf.TypeNumber, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `chunk 多久没被访问就 kill ffmpeg 释放资源(默认60秒)`},
{Key: conf.TranscodeChunkPrefetch, Value: "true", Type: conf.TypeBool, Group: model.TRANSCODE, Flag: model.PRIVATE,
Help: `当前 chunk 转码进度过半时,是否预启动下一个 chunk(默认开启,可让顺序播放无缝衔接)`},
}
additionalSettingItems := tool.Tools.Items()
// 固定顺序
+2 -2
View File
@@ -20,12 +20,12 @@ import (
func InitDB() {
logLevel := logger.Silent
if flags.Debug || flags.Dev {
logLevel = logger.Info
logLevel = logger.Warn // Warn 级别:只输出慢查询和错误,不输出每条 SQL
}
newLogger := logger.New(
stdlog.New(log.StandardLogger().Out, "\r\n", stdlog.LstdFlags),
logger.Config{
SlowThreshold: time.Second,
SlowThreshold: 200 * time.Millisecond, // 超过 200ms 才记录慢查询
LogLevel: logLevel,
IgnoreRecordNotFoundError: true,
Colorful: true,
+5
View File
@@ -15,6 +15,7 @@ import (
"github.com/OpenListTeam/OpenList/v4/internal/conf"
"github.com/OpenListTeam/OpenList/v4/internal/db"
"github.com/OpenListTeam/OpenList/v4/internal/fs"
"github.com/OpenListTeam/OpenList/v4/internal/transcode"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/OpenListTeam/OpenList/v4/server"
"github.com/OpenListTeam/OpenList/v4/server/middlewares"
@@ -92,6 +93,10 @@ func Start() {
InitOfflineDownloadTools()
LoadStorages()
InitTaskManager()
// 初始化转码模块(总开关关时也会启动 manager,但本地 worker 仅在 enabled 时由 Manager.Start 内部按 run_mode 启动)
if transcode.IsEnabled() {
transcode.Default().Start()
}
if !flags.Debug && !flags.Dev {
gin.SetMode(gin.ReleaseMode)
}
+78
View File
@@ -0,0 +1,78 @@
package cache
import (
"runtime"
"sync"
"weak"
)
// WeakCacheMap is a map that holds weak references to values.
// Use for shared expensive objects and automatic cleanup when no longer used.
// This object can be GC and no goroutine is used for cleanup.
type WeakCacheMap[K comparable, V any] struct {
mu sync.Mutex
m map[K]*weakCacheEntry[V]
}
type weakCacheEntry[V any] struct {
weakPtr weak.Pointer[V]
cleanup runtime.Cleanup
}
func NewWeakCacheMap[K comparable, V any]() *WeakCacheMap[K, V] {
return &WeakCacheMap[K, V]{
m: make(map[K]*weakCacheEntry[V]),
}
}
func (c *WeakCacheMap[K, V]) Load(key K) (value *V, ok bool) {
c.mu.Lock()
defer c.mu.Unlock()
entry, exists := c.m[key]
if !exists {
return nil, false
}
value = entry.weakPtr.Value()
return value, value != nil
}
func (c *WeakCacheMap[K, V]) Store(key K, value *V) {
c.mu.Lock()
defer c.mu.Unlock()
entry, exists := c.m[key]
if exists {
entry.cleanup.Stop()
} else {
entry = &weakCacheEntry[V]{}
c.m[key] = entry
}
entry.weakPtr = weak.Make(value)
entry.cleanup = runtime.AddCleanup(value, func(struct{}) {
c.mu.Lock()
defer c.mu.Unlock()
if c.m[key] == entry {
delete(c.m, key)
}
}, struct{}{})
}
func (c *WeakCacheMap[K, V]) Delete(key K) bool {
c.mu.Lock()
defer c.mu.Unlock()
entry, exists := c.m[key]
if !exists {
return false
}
entry.cleanup.Stop()
delete(c.m, key)
return true
}
func (c *WeakCacheMap[K, V]) Clear() {
c.mu.Lock()
defer c.mu.Unlock()
for _, entry := range c.m {
entry.cleanup.Stop()
}
clear(c.m)
}

Some files were not shown because too many files have changed in this diff Show More