Compare commits

..

11 Commits

Author SHA1 Message Date
pikachuim 55d3827dee add(interface): driver&mamage 2025-08-14 22:16:19 +08:00
pikachuim 1fbc9427df add(interface): driver&mamage 2025-08-14 22:16:01 +08:00
pikachuim bb3d139a47 add(interface): driver&mamage 2025-08-14 21:59:44 +08:00
pikachuim d227ab85d6 add(trunk): base interface 2025-08-14 21:44:34 +08:00
pikachuim 5342ae96d0 add(trunk): base interface 2025-08-14 21:39:00 +08:00
pikachuim 273e15a050 add(trunk): base interface 2025-08-14 21:30:18 +08:00
pikachuim 13aad2c2fa add(trunk): base interface 2025-08-14 19:56:43 +08:00
j2rong4cn 368dc65a6e feat: Implement plugin architecture with gRPC support
- Added driver initialization for gRPC plugins in internal/bootstrap/driver.go.
- Introduced configuration structures and protobuf definitions for driver plugins in proto/driver/config.proto and proto/driver/driver.proto.
- Implemented gRPC server and client interfaces for driver plugins in shared/driver/grpc.go.
- Created common response handling utilities in server/common/common.go and server/common/resp.go.
- Developed plugin registration endpoint in server/handles/plugin.go.
- Added test cases for plugin functionality in shared/driver/plugin_test.go.
- Defined plugin reattachment configuration model in shared/model/plugin.go.
2025-08-13 19:04:38 +08:00
j2rong4cn 8b4b6ba970 feat(config): enhance configuration management and add CORS support
feat(server): implement server initialization with context and graceful shutdown
feat(utils): add utility functions for file and JSON operations
refactor(conf): restructure configuration types and improve default settings
2025-08-13 10:03:22 +08:00
j2rong4cn 4d28e838ce feat(cmd): initialize command structure and configuration management 2025-08-12 22:15:25 +08:00
j2rong4cn 3930d4789a add(trunk): next branch 2025-08-12 21:20:33 +08:00
912 changed files with 12409 additions and 44182 deletions
-114
View File
@@ -1,114 +0,0 @@
<!--
PR title / PR 标题:
- Use Conventional Commits: `type(scope): summary`
- Allowed types: `feat`, `docs`, `fix`, `style`, `refactor`, `chore`
- Scope is required by the current PR title check.
- For breaking changes, add `!`: `feat(driver)!: change auth flow`
-->
## Summary / 摘要
<!--
Briefly describe what changed and why.
简要说明改了什么,以及为什么需要改。
-->
<!--
- List user-visible behavior changes.
- List important implementation changes.
- Mention config, storage, API, or compatibility changes if any.
- 列出用户可感知的行为变化。
- 列出重要实现变化。
- 如涉及配置、存储、API 或兼容性变化,请明确说明。
-->
- [ ] This PR has breaking changes.
/ 此 PR 包含破坏性变更。
- [ ] This PR changes public API, config, storage format, or migration behavior.
/ 此 PR 修改了公开 API、配置、存储格式或迁移行为。
- [ ] This PR requires corresponding changes in related repositories.
/ 此 PR 需要关联仓库同步修改。
Related repository PRs / 关联仓库 PR:
- OpenList-Frontend:
- OpenList-Docs:
## Related Issues / 关联 Issue
<!--
Use `Closes #123`, `Fixes #123`, or `Relates to #123`.
Remove this section if not applicable.
使用 `Closes #123`、`Fixes #123` 或 `Relates to #123`。
不适用时请删除本节。
-->
## Testing / 测试
<!--
Describe commands, platforms, and manual checks.
If not tested, explain why.
说明执行过的命令、测试平台和手动验证。
如果未测试,请说明原因。
-->
- [ ] `go test ./...`
- [ ] Manual test / 手动测试:
## Checklist / 检查清单
- [ ] I have read [CONTRIBUTING](https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md).
/ 我已阅读 [CONTRIBUTING](https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md)。
- [ ] I confirm this contribution follows the repository license, contribution policy, and code of conduct.
/ 我确认此贡献符合仓库许可证、贡献规范和行为准则。
- [ ] I have formatted the changed code with `gofmt`, `go fmt`, or `prettier` where applicable.
/ 我已按适用情况使用 `gofmt`、`go fmt` 或 `prettier` 格式化变更代码。
- [ ] I have requested review from relevant maintainers or code owners where applicable.
/ 我已在适用情况下请求相关维护者或代码所有者审查。
## AI Disclosure / AI 使用声明
<!--
Please disclose any substantial AI assistance used in this PR.
Minor AI assistance, such as typo fixes, autocomplete, formatting suggestions,
or wording polish, does not need to be disclosed.
Remove this section if not applicable.
请披露此 PR 中使用的重要 AI 辅助内容。
轻微 AI 辅助,例如拼写修正、自动补全、格式建议或文字润色,无需披露。
如不适用,请删除本节。
Deliberate non-disclosure may be treated as a trust and compliance issue.
故意隐瞒 AI 使用情况可能被视为信任与合规问题。
-->
- [ ] This PR includes AI-assisted content.
/ 此 PR 包含 AI 辅助内容。
Tools used / 使用工具:
- [ ] ChatGPT
- [ ] Codex
- [ ] GitHub Copilot
- [ ] Claude
- [ ] Gemini
- [ ] Other (please specify) / 其他(请注明):
Usage scope / 使用范围:
- [ ] Code generation / 代码生成
- [ ] Refactoring / 重构
- [ ] Documentation / 文档
- [ ] Tests / 测试
- [ ] Translation / 翻译
- [ ] Review assistance / 审查辅助
- [ ] I have reviewed and validated all AI-assisted content included in this PR.
/ 我已审核并验证此 PR 中的所有 AI 辅助内容。
- [ ] I have ensured that all AI-assisted commits include `Co-Authored-By` attribution.
/ 我已确保所有 AI 辅助提交都包含 `Co-Authored-By` 归属信息。
- [ ] I can reproduce all AI-assisted content included in this PR without any AI tools.
/ 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。
-215
View File
@@ -1,215 +0,0 @@
name: Container Registry Cleanup
permissions:
contents: read
packages: write
on:
workflow_dispatch:
inputs:
package:
description: 'The name of the package to clean up'
required: true
default: 'openlist-git'
owner:
description: 'The owner of the package (user or organization)'
required: false
default: 'openlistteam'
older_than:
description: 'Delete untagged images older than this many days'
required: false
default: '-1'
untagged_timestamp_tolerance:
description: 'Tolerance in milliseconds for untagged images close to tagged ones'
required: false
# We do this because multi-arch docker containers will be pushed as separate untagged images
# to the container registry and GitHub cannot recognize them as part of a tagged image.
# Setting this option large enough prevents accidental deletion of one of these images.
# Here we set it to 10 seconds
default: '10000'
token:
description: 'GitHub token with permissions to delete packages (leave empty to use default GITHUB_TOKEN)'
required: false
default: ''
jobs:
cleanup:
runs-on: ubuntu-slim
steps:
- name: Check inputs and tools
id: setup
shell: bash
run: |
PACKAGE="${{ github.event.inputs.package || 'openlist-git' }}"
OLDER="${{ github.event.inputs.older_than || '-1' }}"
TOLERANCE_VAL="${{ github.event.inputs.untagged_timestamp_tolerance || '10000' }}"
TOKEN_INPUT="${{ github.event.inputs.token || '' }}"
OWNER="${{ github.event.inputs.owner || 'openlistteam' }}"
echo "package_name=$PACKAGE" >> "$GITHUB_OUTPUT"
echo "older_than=$OLDER" >> "$GITHUB_OUTPUT"
echo "tolerance=$TOLERANCE_VAL" >> "$GITHUB_OUTPUT"
echo "owner=$OWNER" >> "$GITHUB_OUTPUT"
if [[ -z "$TOKEN_INPUT" ]]; then
echo "::notice:: No token provided, using GITHUB_TOKEN instead."
echo "token=${{ secrets.GITHUB_TOKEN }}" >> "$GITHUB_OUTPUT"
else
echo "::add-mask::$TOKEN_INPUT"
echo "token=$TOKEN_INPUT" >> "$GITHUB_OUTPUT"
fi
if [[ -z "$PACKAGE" || -z "$OWNER" || ( -z "$TOKEN_INPUT" && -z "${{ secrets.GITHUB_TOKEN }}" ) ]]; then
echo "::error:: Missing required inputs (package, token, or owner context)."
exit 1
fi
command -v curl >/dev/null 2>&1 || { echo "::error::curl is required but not installed."; exit 1; }
command -v jq >/dev/null 2>&1 || { echo "::error::jq is required but not installed."; exit 1; }
- name: Determine owner type (user or organization)
id: owner_type
shell: bash
run: |
OWNER="${{ steps.setup.outputs.owner }}"
TOKEN="${{ steps.setup.outputs.token }}"
TYPE=$(curl -s -H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
"https://api.github.com/users/${OWNER}" | jq -r '.type')
if [[ "$TYPE" == "Organization" ]]; then
OWNER_PATH="orgs"
else
OWNER_PATH="users"
fi
echo "owner_path=$OWNER_PATH" >> "$GITHUB_OUTPUT"
echo "Detected owner type: $TYPE (using /${OWNER_PATH}/)"
- name: Access all versions from GitHub
id: fetch
shell: bash
run: |
PACKAGE_NAME="${{ steps.setup.outputs.package_name }}"
OWNER="${{ steps.setup.outputs.owner }}"
TOKEN="${{ steps.setup.outputs.token }}"
OWNER_PATH="${{ steps.owner_type.outputs.owner_path }}"
BASE_URL="https://api.github.com/${OWNER_PATH}/${OWNER}/packages/container/${PACKAGE_NAME}/versions"
URL="${BASE_URL}?per_page=100"
TEMP_FILE="/tmp/all_versions.ndjson"
> "$TEMP_FILE"
PAGE=1
while [[ -n "$URL" && $PAGE -le 10 ]]; do
echo "Fetching: $URL"
curl -s -H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
-D headers.txt \
"$URL" > page.json
jq -c '.[]' page.json >> "$TEMP_FILE"
URL=$(grep -i '^link:' headers.txt | grep -o '<[^>]*>;\s*rel="next"' | sed 's/<\(.*\)>.*/\1/' || true)
PAGE=$((PAGE + 1))
done
rm -f headers.txt page.json
echo "temp_file=$TEMP_FILE" >> "$GITHUB_OUTPUT"
- name: Filter untagged versions
id: filter
shell: bash
run: |
OLDER_THAN="${{ steps.setup.outputs.older_than }}"
TOLERANCE="${{ steps.setup.outputs.tolerance }}"
TEMP_FILE="${{ steps.fetch.outputs.temp_file }}"
TO_DELETE=$(jq -c -s \
--arg older_than "$OLDER_THAN" \
--arg tolerance "$TOLERANCE" \
'
def abs: if . < 0 then -. else . end;
def to_ms: sub("\\.[0-9]+Z$"; "Z") | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime * 1000;
. as $all |
($older_than | tonumber) as $older_than_val |
($tolerance | tonumber) as $tolerance_val |
now as $now |
(if $older_than_val > 0 then ($now - ($older_than_val * 86400)) * 1000 else 0 end) as $cutoff_ms |
[ $all[] | select((.metadata.container.tags // []) | length > 0) | .created_at | to_ms ] as $tagged_times |
$all[] |
select((.metadata.container.tags // []) | length == 0) |
. as $item |
(.created_at | to_ms) as $created_ms |
(if $older_than_val > 0 then $created_ms < $cutoff_ms else true end) as $pass_age |
(if $tolerance_val <= 0 then true
else [ $tagged_times[] | select(($created_ms - .) | abs < $tolerance_val) ] | length == 0
end) as $pass_tolerance |
select($pass_age and $pass_tolerance) |
{id, name}
' "$TEMP_FILE")
rm -f "$TEMP_FILE"
if [[ -z "$TO_DELETE" ]]; then
DELETE_COUNT=0
else
DELETE_COUNT=$(echo "$TO_DELETE" | wc -l | tr -d ' ')
fi
echo "Found $DELETE_COUNT untagged images no longer necessary"
{
echo 'to_delete<<EOF'
echo "$TO_DELETE"
echo 'EOF'
} >> "$GITHUB_OUTPUT"
echo "delete_count=$DELETE_COUNT" >> "$GITHUB_OUTPUT"
- name: Delete untagged versions
id: delete
shell: bash
run: |
PACKAGE_NAME="${{ steps.setup.outputs.package_name }}"
OWNER="${{ steps.setup.outputs.owner }}"
TOKEN="${{ steps.setup.outputs.token }}"
OWNER_PATH="${{ steps.owner_type.outputs.owner_path }}"
DELETE_COUNT="${{ steps.filter.outputs.delete_count }}"
TO_DELETE='${{ steps.filter.outputs.to_delete }}'
if [[ "$DELETE_COUNT" -gt 0 && -n "$TO_DELETE" ]]; then
while IFS= read -r item; do
[[ -z "$item" ]] && continue
# $item 是完整的单行 JSON,如 {"id":123,"name":"sha256:..."}
ID=$(echo "$item" | jq -r '.id')
NAME=$(echo "$item" | jq -r '.name')
DELETE_URL="https://api.github.com/${OWNER_PATH}/${OWNER}/packages/container/${PACKAGE_NAME}/versions/${ID}"
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
-H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
"$DELETE_URL")
if [[ "$HTTP_CODE" == "204" ]]; then
echo "Deleted untagged container image '${NAME}'"
else
echo "::warning:: Failed to delete ${NAME} (ID: ${ID}). HTTP Status: ${HTTP_CODE}"
fi
done <<< "$TO_DELETE"
else
echo "No images to delete."
fi
echo "Cleanup completed."
-101
View File
@@ -1,101 +0,0 @@
name: Issue or PR Auto Reply
on:
issues:
types: [opened]
pull_request:
types: [opened]
permissions:
issues: write
pull-requests: write
jobs:
auto-reply:
runs-on: ubuntu-latest
if: github.event_name == 'issues'
steps:
- name: Check issue for unchecked tasks and reply
uses: actions/github-script@v7
with:
script: |
let comment = "";
const issueTitle = context.payload.issue.title || "";
const titleNotEdited = /(请修改标题|Please modify the title)/i.test(issueTitle);
if (titleNotEdited) {
comment = "⚠️ 请修改标题以更好地描述您的问题或需求,并删除示例提示。当前 Issue 将被自动关闭。如需继续提交,请创建新的 Issue。\n";
comment += "⚠️ Please modify the title to better describe your issue or request, and remove the example prompt. This issue will be automatically closed. If you wish to proceed, please create a new issue.\n";
await github.rest.issues.createComment({
...context.repo,
issue_number: context.issue.number,
body: comment
});
await github.rest.issues.update({
...context.repo,
issue_number: context.issue.number,
state: 'closed',
state_reason: 'not_planned',
labels: ['invalid']
});
return;
}
const issueBody = context.payload.issue.body || "";
const confirmHasRead = /- \[ \] (?!我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody);
const confirmNotRead = /- \[[xX]\] (?:我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody);
if (confirmNotRead) {
comment = "⚠️ 你的 Issue 不符合提交规则。请先阅读相关规范后再重新提交。当前 Issue 将被自动关闭。如需继续提交,请确认已了解规则后重新打开或创建新的 Issue。\n";
comment += "⚠️ Your issue does not comply with the submission rules. Please read the guidelines before submitting again. This issue will be automatically closed. If you wish to proceed, please confirm that you have reviewed the rules before reopening or creating a new issue.\n";
await github.rest.issues.createComment({
...context.repo,
issue_number: context.issue.number,
body: comment
});
await github.rest.issues.update({
...context.repo,
issue_number: context.issue.number,
state: 'closed',
state_reason: 'not_planned',
labels: ['invalid']
});
return;
}
if (confirmHasRead) {
comment = "感谢您联系OpenList。我们会尽快回复您。\n";
comment += "Thanks for contacting OpenList. We will reply to you as soon as possible.\n\n";
comment += "由于您提出的 Issue 中包含部分未确认的项目,为了更好地管理项目,在人工审核后可能会直接关闭此问题。\n";
comment += "如果您能确认并补充相关未确认项目的信息,欢迎随时重新提交。我们会及时关注并处理。感谢您的理解与支持!\n";
comment += "Since your issue contains some unchecked tasks, it may be closed after manual review.\n";
comment += "If you can confirm and provide information for the unchecked tasks, feel free to resubmit.\n";
comment += "We will pay attention and handle it in a timely manner.\n\n";
comment += "感谢您的理解与支持!\n";
comment += "Thank you for your understanding and support!\n";
await github.rest.issues.createComment({
...context.repo,
issue_number: context.issue.number,
body: comment
});
}
pr-title-check:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Check PR title for required prefix and comment
uses: actions/github-script@v7
with:
script: |
const title = context.payload.pull_request.title || "";
const ok = /^(feat|docs|fix|style|refactor|chore)\(.+?\)!?: /i.test(title);
if (!ok) {
let comment = "⚠️ PR 标题需以 `feat(): `, `docs(): `, `fix(): `, `style(): `, `refactor(): `, `chore(): ` 其中之一开头,例如:`feat(component): 新增功能`。\n";
comment += "⚠️ The PR title must start with `feat(): `, `docs(): `, `fix(): `, `style(): `, or `refactor(): `, `chore(): `. For example: `feat(component): add new feature`.\n\n";
comment += "如果跨多个组件,请使用主要组件作为前缀,并在标题中枚举、描述中说明。\n";
comment += "If it spans multiple components, use the main component as the prefix and enumerate in the title, describe in the body.\n\n";
comment += "如果是破坏性变更,请在类型后添加 `!`,例如 `feat(component)!: 破坏性变更`。\n";
comment += "For breaking changes, add `!` after the type, e.g., `feat(component)!: breaking change`.\n\n";
await github.rest.issues.createComment({
...context.repo,
issue_number: context.issue.number,
body: comment
});
}
-38
View File
@@ -1,38 +0,0 @@
name: Sync to Gitee
on:
push:
branches:
- main
workflow_dispatch:
jobs:
sync:
runs-on: ubuntu-latest
name: Sync GitHub to Gitee
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup SSH
run: |
mkdir -p ~/.ssh
echo "${{ secrets.GITEE_SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-keyscan gitee.com >> ~/.ssh/known_hosts
- name: Create single commit and push
run: |
git config user.name "GitHub Actions"
git config user.email "actions@github.com"
# Create a new branch
git checkout --orphan new-main
git add .
git commit -m "Sync from GitHub: $(date)"
# Add Gitee remote and force push
git remote add gitee ${{ vars.GITEE_REPO_URL }}
git push --force gitee new-main:main
-60
View File
@@ -1,60 +0,0 @@
# Agent Instructions
## Issues
Before creating an issue, review the available issue templates in the `.github` directory.
When drafting the issue:
- Use the most appropriate template.
- Follow the template structure.
- Fill in all required sections.
- Remove sections that the template explicitly marks as optional or not applicable.
- Do not invent reproduction steps, logs, screenshots, or expected behavior.
## Pull Requests
Before creating a pull request, read `.github/PULL_REQUEST_TEMPLATE.md`.
When drafting the pull request:
- Follow the template structure.
- Use the title format required by the template.
- Fill in or remove each section according to the template guidance.
- Include testing details, or explain why testing was not run.
- Do not invent testing results.
- Do not claim validation, verification, or review steps that were not actually performed.
## Automated Contributions
Fully automated contributions are not considered equivalent to normal community participation.
A contribution may be considered fully automated if it is submitted through an automated agent, or if the submitting account participates in project discussions through an automated agent, without meaningful human review or intervention.
When making this determination, maintainers may consider the overall behavior of the account, including but not limited to disclosed agent usage, interaction patterns, response characteristics, and other available evidence. No single factor is determinative.
Maintainers reserve the right to accept, reject, modify, or reimplement any contribution independently of any action taken against the submitting account. Acceptance of a contribution does not imply acceptance of the submitting account or its contribution method. If an account is determined to be primarily operated through automated processes, we may need to restrict its future participation in contributions until that determination is rescinded.
## Git Commits
When creating commits, follow the repository `git-commit` skill rules:
- Use Conventional Commits title format: `type(scope): subject`.
- Allowed types: `feat`, `fix`, `refactor`, `perf`, `docs`, `style`, `test`, `build`, `ci`, `chore`, `revert`.
- Use a meaningful scope based on the main module, package, or feature.
- Write the subject in imperative mood and describe the actual change.
- Use a concise Markdown list in the commit body, with each item describing one key change.
- Do not invent changes that are not present in the diff.
- Do not describe behavior, refactors, fixes, or tests that are not reflected in the commit.
Include at most one `Co-authored-by` trailer that matches the AI assistant actually used to produce the change.
Examples:
- `Co-authored-by: Codex <267193182+codex@users.noreply.github.com>`
- `Co-authored-by: GitHub Copilot <copilot@github.com>`
- `Co-authored-by: Claude <81847+claude@users.noreply.github.com>`
If you are not one of the listed assistants, do not add a `Co-authored-by` trailer.
Instead, ask the human collaborator to provide the exact `Co-authored-by` trailer to use. Do not invent, infer, or generate one yourself.
-110
View File
@@ -1,110 +0,0 @@
# Contributing
## Setup your machine
`OpenList` is written in [Go](https://golang.org/) and [SolidJS](https://www.solidjs.com/).
Prerequisites:
- [git](https://git-scm.com)
- [Go](https://golang.org/doc/install) version declared in [`go.mod`](./go.mod)
- [gcc](https://gcc.gnu.org/)
- [nodejs](https://nodejs.org/)
## Cloning a fork
Fork and clone `OpenList` and `OpenList-Frontend` anywhere:
```shell
git clone https://github.com/<your-username>/OpenList.git
git clone --recurse-submodules https://github.com/<your-username>/OpenList-Frontend.git
```
## Creating a branch
Create a new branch from the `main` branch, with an appropriate name.
```shell
git checkout -b <branch-name>
```
## Preview your change
### backend
```shell
go run main.go
```
### frontend
```shell
pnpm dev
```
## Add a new driver
Copy `drivers/template` folder and rename it, and follow the comments in it.
## Community and policies
By contributing, you agree to follow the repository's code of conduct and license terms.
- Code of conduct: [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md)
- License: [LICENSE](./LICENSE)
- Security issues: please report privately according to [SECURITY.md](./SECURITY.md)
If your contribution includes substantial AI-assisted content, disclose the tools used and the scope of assistance in the pull request.
## Create a commit
Commit messages should be well formatted, and to make that "standardized".
Submit your pull request. For PR titles, follow [Conventional Commits](https://www.conventionalcommits.org).
<https://github.com/OpenListTeam/OpenList/issues/376>
It's suggested to sign your commits. See: [How to sign commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
## Submit a pull request
Please make sure your code has been formatted with `go fmt` or [prettier](https://prettier.io/) before submitting.
Push your branch to your `openlist` fork and open a pull request against the `main` branch.
## Merge your pull request
Your pull request will be merged after review. Please wait for the maintainer to merge your pull request after review.
At least 1 approving review is required by reviewers with write access. You can also request a review from maintainers.
## Delete your branch
(Optional) After your pull request is merged, you can delete your branch.
## AI Disclosure
If your pull request includes substantial AI-assisted content, disclose it in the PR description.
The pull request description must follow the repository's pull request template.
Fully automated contributions are not considered equivalent to normal community participation.
A contribution may be considered fully automated if it is submitted through an automated agent, or if the submitting account participates in project discussions through an automated agent, without meaningful human review or intervention.
When making this determination, maintainers may consider the overall behavior of the account, including but not limited to disclosed agent usage, interaction patterns, response characteristics, and other available evidence. No single factor is determinative.
Maintainers reserve the right to accept, reject, modify, or reimplement any contribution independently of any action taken against the submitting account. Acceptance of a contribution does not imply acceptance of the submitting account or its contribution method. If an account is determined to be primarily operated through automated processes, we may need to restrict its future participation in contributions until that determination is rescinded.
Please include:
- Tools used, such as ChatGPT, GitHub Copilot, Claude, Cursor, or other AI tools.
- Usage scope, such as code generation, refactoring, documentation, tests, translation, or review assistance.
- Confirmation that you have reviewed and validated all AI-assisted content before submission.
- Confirmation that the submitted content complies with this repository's license and contribution policies.
Minor AI assistance, such as typo fixes, autocomplete, formatting suggestions, or wording polish, does not need to be disclosed.
---
Thank you for your contribution! Let's make OpenList better together!
-136
View File
@@ -1,136 +0,0 @@
<div align="center">
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
<p><em>OpenList هو تفرّع من AList يتميز بالمرونة والحوكمة طويلة الأمد ويقوده المجتمع، وقد بُني للدفاع عن البرمجيات مفتوحة المصدر ضد الهجمات القائمة على الثقة.</em></p>
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
</div>
---
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | العربية
- [Contributing](../CONTRIBUTING.md)
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
- [LICENSE](../LICENSE)
## إخلاء المسؤولية
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
## الميزات
- [x] Multiple storages
- [x] Local storage
- [x] [Aliyundrive](https://www.alipan.com)
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
- [x] [GoogleDrive](https://drive.google.com)
- [x] [123pan](https://www.123pan.com)
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
- [x] [PikPak](https://www.mypikpak.com)
- [x] [S3](https://aws.amazon.com/s3)
- [x] [Seafile](https://seafile.com)
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [ProtonDrive](https://proton.me/drive)
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
- [x] [YandexDisk](https://disk.yandex.com)
- [x] [BaiduNetdisk](http://pan.baidu.com)
- [x] [Terabox](https://www.terabox.com/main)
- [x] [UC](https://drive.uc.cn)
- [x] [Quark](https://pan.quark.cn)
- [x] [Thunder](https://pan.xunlei.com)
- [x] [Lanzou](https://www.lanzou.com)
- [x] [ILanzou](https://www.ilanzou.com)
- [x] [Google photo](https://photos.google.com)
- [x] [Mega.nz](https://mega.nz)
- [x] [Baidu photo](https://photo.baidu.com)
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
- [x] [115](https://115.com)
- [x] [Cloudreve](https://cloudreve.org)
- [x] [Dropbox](https://www.dropbox.com)
- [x] [FeijiPan](https://www.feijipan.com)
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
- [x] [Chaoxing](https://www.chaoxing.com)
- [x] [CNB](https://cnb.cool/)
- [x] [Degoo](https://degoo.com)
- [x] [Doubao](https://www.doubao.com)
- [x] [Febbox](https://www.febbox.com)
- [x] [GitHub](https://github.com)
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
- [x] [Weiyun](https://www.weiyun.com)
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
- [x] Easy to deploy and out-of-the-box
- [x] File preview (PDF, markdown, code, plain text, ...)
- [x] Image preview in gallery mode
- [x] Video and audio preview, with lyrics and subtitles support
- [x] Office document preview (docx, pptx, xlsx, ...)
- [x] `README.md` preview rendering
- [x] File permalink copy and direct file download
- [x] Dark mode and i18n
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
- [x] File/folder package download and web upload operations
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
## الوثائق
- 📘 [Docs](https://doc.oplist.org)
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
## العرض التجريبي
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## النقاش
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
## الراعي
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## المتبرعون
Thanks to the following donors for their generous support:
- [HisAtri](https://github.com/HisAtri)
- 爱发电用户_7jTh
- suka
## الترخيص
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
## تواصل معنا
- [@GitHub](https://github.com/OpenListTeam)
- [Telegram Group](https://t.me/OpenListTeam)
- [Telegram Channel](https://t.me/OpenListOfficial)
## المساهمون
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
[![Contributors](https://contrib.rocks/image?repo=OpenListTeam/OpenList)](https://github.com/OpenListTeam/OpenList/graphs/contributors)
-136
View File
@@ -1,136 +0,0 @@
<div align="center">
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
<p><em>OpenList ist ein widerstandsfähiger, langfristig verwalteter und Community-getriebener Fork von AList, entwickelt zum Schutz von Open Source vor vertrauensbasierten Angriffen.</em></p>
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
</div>
---
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | Deutsch | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
- [Contributing](../CONTRIBUTING.md)
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
- [LICENSE](../LICENSE)
## Haftungsausschluss
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
## Funktionen
- [x] Multiple storages
- [x] Local storage
- [x] [Aliyundrive](https://www.alipan.com)
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
- [x] [GoogleDrive](https://drive.google.com)
- [x] [123pan](https://www.123pan.com)
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
- [x] [PikPak](https://www.mypikpak.com)
- [x] [S3](https://aws.amazon.com/s3)
- [x] [Seafile](https://seafile.com)
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [ProtonDrive](https://proton.me/drive)
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
- [x] [YandexDisk](https://disk.yandex.com)
- [x] [BaiduNetdisk](http://pan.baidu.com)
- [x] [Terabox](https://www.terabox.com/main)
- [x] [UC](https://drive.uc.cn)
- [x] [Quark](https://pan.quark.cn)
- [x] [Thunder](https://pan.xunlei.com)
- [x] [Lanzou](https://www.lanzou.com)
- [x] [ILanzou](https://www.ilanzou.com)
- [x] [Google photo](https://photos.google.com)
- [x] [Mega.nz](https://mega.nz)
- [x] [Baidu photo](https://photo.baidu.com)
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
- [x] [115](https://115.com)
- [x] [Cloudreve](https://cloudreve.org)
- [x] [Dropbox](https://www.dropbox.com)
- [x] [FeijiPan](https://www.feijipan.com)
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
- [x] [Chaoxing](https://www.chaoxing.com)
- [x] [CNB](https://cnb.cool/)
- [x] [Degoo](https://degoo.com)
- [x] [Doubao](https://www.doubao.com)
- [x] [Febbox](https://www.febbox.com)
- [x] [GitHub](https://github.com)
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
- [x] [Weiyun](https://www.weiyun.com)
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
- [x] Easy to deploy and out-of-the-box
- [x] File preview (PDF, markdown, code, plain text, ...)
- [x] Image preview in gallery mode
- [x] Video and audio preview, with lyrics and subtitles support
- [x] Office document preview (docx, pptx, xlsx, ...)
- [x] `README.md` preview rendering
- [x] File permalink copy and direct file download
- [x] Dark mode and i18n
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
- [x] File/folder package download and web upload operations
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
## Dokumentation
- 📘 [Docs](https://doc.oplist.org)
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
## Demo
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## Diskussion
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
## Sponsor
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## Spender
Thanks to the following donors for their generous support:
- [HisAtri](https://github.com/HisAtri)
- 爱发电用户_7jTh
- suka
## Lizenz
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
## Kontakt
- [@GitHub](https://github.com/OpenListTeam)
- [Telegram Group](https://t.me/OpenListTeam)
- [Telegram Channel](https://t.me/OpenListOfficial)
## Mitwirkende
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
[![Contributors](https://contrib.rocks/image?repo=OpenListTeam/OpenList)](https://github.com/OpenListTeam/OpenList/graphs/contributors)
-136
View File
@@ -1,136 +0,0 @@
<div align="center">
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
<p><em>OpenList es un fork de AList resiliente, con gobernanza a largo plazo e impulsado por la comunidad, creado para defender el código abierto frente a ataques basados en la confianza.</em></p>
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
</div>
---
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | Español | [العربية](./README_ar.md)
- [Contributing](../CONTRIBUTING.md)
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
- [LICENSE](../LICENSE)
## Aviso legal
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
## Características
- [x] Multiple storages
- [x] Local storage
- [x] [Aliyundrive](https://www.alipan.com)
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
- [x] [GoogleDrive](https://drive.google.com)
- [x] [123pan](https://www.123pan.com)
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
- [x] [PikPak](https://www.mypikpak.com)
- [x] [S3](https://aws.amazon.com/s3)
- [x] [Seafile](https://seafile.com)
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [ProtonDrive](https://proton.me/drive)
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
- [x] [YandexDisk](https://disk.yandex.com)
- [x] [BaiduNetdisk](http://pan.baidu.com)
- [x] [Terabox](https://www.terabox.com/main)
- [x] [UC](https://drive.uc.cn)
- [x] [Quark](https://pan.quark.cn)
- [x] [Thunder](https://pan.xunlei.com)
- [x] [Lanzou](https://www.lanzou.com)
- [x] [ILanzou](https://www.ilanzou.com)
- [x] [Google photo](https://photos.google.com)
- [x] [Mega.nz](https://mega.nz)
- [x] [Baidu photo](https://photo.baidu.com)
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
- [x] [115](https://115.com)
- [x] [Cloudreve](https://cloudreve.org)
- [x] [Dropbox](https://www.dropbox.com)
- [x] [FeijiPan](https://www.feijipan.com)
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
- [x] [Chaoxing](https://www.chaoxing.com)
- [x] [CNB](https://cnb.cool/)
- [x] [Degoo](https://degoo.com)
- [x] [Doubao](https://www.doubao.com)
- [x] [Febbox](https://www.febbox.com)
- [x] [GitHub](https://github.com)
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
- [x] [Weiyun](https://www.weiyun.com)
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
- [x] Easy to deploy and out-of-the-box
- [x] File preview (PDF, markdown, code, plain text, ...)
- [x] Image preview in gallery mode
- [x] Video and audio preview, with lyrics and subtitles support
- [x] Office document preview (docx, pptx, xlsx, ...)
- [x] `README.md` preview rendering
- [x] File permalink copy and direct file download
- [x] Dark mode and i18n
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
- [x] File/folder package download and web upload operations
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
## Documentación
- 📘 [Docs](https://doc.oplist.org)
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
## Demo
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## Discusión
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
## Patrocinador
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## Donantes
Thanks to the following donors for their generous support:
- [HisAtri](https://github.com/HisAtri)
- 爱发电用户_7jTh
- suka
## Licencia
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
## Contacto
- [@GitHub](https://github.com/OpenListTeam)
- [Telegram Group](https://t.me/OpenListTeam)
- [Telegram Channel](https://t.me/OpenListOfficial)
## Colaboradores
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
[![Contributors](https://contrib.rocks/image?repo=OpenListTeam/OpenList)](https://github.com/OpenListTeam/OpenList/graphs/contributors)
-136
View File
@@ -1,136 +0,0 @@
<div align="center">
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
<p><em>OpenList est un fork d’AList résilient, gouverné sur le long terme et porté par la communauté, conçu pour protéger l’open source contre les attaques fondées sur la confiance.</em></p>
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
</div>
---
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | Français | [Español](./README_es.md) | [العربية](./README_ar.md)
- [Contributing](../CONTRIBUTING.md)
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
- [LICENSE](../LICENSE)
## Avertissement
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
## Fonctionnalités
- [x] Multiple storages
- [x] Local storage
- [x] [Aliyundrive](https://www.alipan.com)
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
- [x] [GoogleDrive](https://drive.google.com)
- [x] [123pan](https://www.123pan.com)
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
- [x] [PikPak](https://www.mypikpak.com)
- [x] [S3](https://aws.amazon.com/s3)
- [x] [Seafile](https://seafile.com)
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [ProtonDrive](https://proton.me/drive)
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
- [x] [YandexDisk](https://disk.yandex.com)
- [x] [BaiduNetdisk](http://pan.baidu.com)
- [x] [Terabox](https://www.terabox.com/main)
- [x] [UC](https://drive.uc.cn)
- [x] [Quark](https://pan.quark.cn)
- [x] [Thunder](https://pan.xunlei.com)
- [x] [Lanzou](https://www.lanzou.com)
- [x] [ILanzou](https://www.ilanzou.com)
- [x] [Google photo](https://photos.google.com)
- [x] [Mega.nz](https://mega.nz)
- [x] [Baidu photo](https://photo.baidu.com)
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
- [x] [115](https://115.com)
- [x] [Cloudreve](https://cloudreve.org)
- [x] [Dropbox](https://www.dropbox.com)
- [x] [FeijiPan](https://www.feijipan.com)
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
- [x] [Chaoxing](https://www.chaoxing.com)
- [x] [CNB](https://cnb.cool/)
- [x] [Degoo](https://degoo.com)
- [x] [Doubao](https://www.doubao.com)
- [x] [Febbox](https://www.febbox.com)
- [x] [GitHub](https://github.com)
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
- [x] [Weiyun](https://www.weiyun.com)
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
- [x] Easy to deploy and out-of-the-box
- [x] File preview (PDF, markdown, code, plain text, ...)
- [x] Image preview in gallery mode
- [x] Video and audio preview, with lyrics and subtitles support
- [x] Office document preview (docx, pptx, xlsx, ...)
- [x] `README.md` preview rendering
- [x] File permalink copy and direct file download
- [x] Dark mode and i18n
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
- [x] File/folder package download and web upload operations
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
## Documentation
- 📘 [Docs](https://doc.oplist.org)
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
## Démo
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## Discussion
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
## Sponsor
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## Donateurs
Thanks to the following donors for their generous support:
- [HisAtri](https://github.com/HisAtri)
- 爱发电用户_7jTh
- suka
## Licence
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
## Nous contacter
- [@GitHub](https://github.com/OpenListTeam)
- [Telegram Group](https://t.me/OpenListTeam)
- [Telegram Channel](https://t.me/OpenListOfficial)
## Contributeurs
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
[![Contributors](https://contrib.rocks/image?repo=OpenListTeam/OpenList)](https://github.com/OpenListTeam/OpenList/graphs/contributors)
-136
View File
@@ -1,136 +0,0 @@
<div align="center">
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
<p><em>OpenList는 신뢰 기반 공격으로부터 오픈 소스를 보호하기 위해 만들어진, 탄력적이고 장기 거버넌스를 지향하는 커뮤니티 주도 AList 포크입니다.</em></p>
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
</div>
---
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | 한국어 | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
- [Contributing](../CONTRIBUTING.md)
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
- [LICENSE](../LICENSE)
## 면책 조항
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
## 기능
- [x] Multiple storages
- [x] Local storage
- [x] [Aliyundrive](https://www.alipan.com)
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
- [x] [GoogleDrive](https://drive.google.com)
- [x] [123pan](https://www.123pan.com)
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
- [x] [PikPak](https://www.mypikpak.com)
- [x] [S3](https://aws.amazon.com/s3)
- [x] [Seafile](https://seafile.com)
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [ProtonDrive](https://proton.me/drive)
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
- [x] [YandexDisk](https://disk.yandex.com)
- [x] [BaiduNetdisk](http://pan.baidu.com)
- [x] [Terabox](https://www.terabox.com/main)
- [x] [UC](https://drive.uc.cn)
- [x] [Quark](https://pan.quark.cn)
- [x] [Thunder](https://pan.xunlei.com)
- [x] [Lanzou](https://www.lanzou.com)
- [x] [ILanzou](https://www.ilanzou.com)
- [x] [Google photo](https://photos.google.com)
- [x] [Mega.nz](https://mega.nz)
- [x] [Baidu photo](https://photo.baidu.com)
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
- [x] [115](https://115.com)
- [x] [Cloudreve](https://cloudreve.org)
- [x] [Dropbox](https://www.dropbox.com)
- [x] [FeijiPan](https://www.feijipan.com)
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
- [x] [Chaoxing](https://www.chaoxing.com)
- [x] [CNB](https://cnb.cool/)
- [x] [Degoo](https://degoo.com)
- [x] [Doubao](https://www.doubao.com)
- [x] [Febbox](https://www.febbox.com)
- [x] [GitHub](https://github.com)
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
- [x] [Weiyun](https://www.weiyun.com)
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
- [x] Easy to deploy and out-of-the-box
- [x] File preview (PDF, markdown, code, plain text, ...)
- [x] Image preview in gallery mode
- [x] Video and audio preview, with lyrics and subtitles support
- [x] Office document preview (docx, pptx, xlsx, ...)
- [x] `README.md` preview rendering
- [x] File permalink copy and direct file download
- [x] Dark mode and i18n
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
- [x] File/folder package download and web upload operations
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
## 문서
- 📘 [Docs](https://doc.oplist.org)
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
## 데모
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## 토론
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
## 후원
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## 후원자
Thanks to the following donors for their generous support:
- [HisAtri](https://github.com/HisAtri)
- 爱发电用户_7jTh
- suka
## 라이선스
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
## 문의
- [@GitHub](https://github.com/OpenListTeam)
- [Telegram Group](https://t.me/OpenListTeam)
- [Telegram Channel](https://t.me/OpenListOfficial)
## 기여자
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
[![Contributors](https://contrib.rocks/image?repo=OpenListTeam/OpenList)](https://github.com/OpenListTeam/OpenList/graphs/contributors)
-136
View File
@@ -1,136 +0,0 @@
<div align="center">
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
<p><em>OpenList — устойчивый, управляемый сообществом форк AList с долгосрочным управлением, созданный для защиты open source от атак, основанных на доверии.</em></p>
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
</div>
---
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | Русский | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
- [Contributing](../CONTRIBUTING.md)
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
- [LICENSE](../LICENSE)
## Отказ от ответственности
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
## Возможности
- [x] Multiple storages
- [x] Local storage
- [x] [Aliyundrive](https://www.alipan.com)
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
- [x] [GoogleDrive](https://drive.google.com)
- [x] [123pan](https://www.123pan.com)
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
- [x] [PikPak](https://www.mypikpak.com)
- [x] [S3](https://aws.amazon.com/s3)
- [x] [Seafile](https://seafile.com)
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
- [x] [MediaFire](https://www.mediafire.com)
- [x] [Mediatrack](https://www.mediatrack.cn)
- [x] [ProtonDrive](https://proton.me/drive)
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
- [x] [YandexDisk](https://disk.yandex.com)
- [x] [BaiduNetdisk](http://pan.baidu.com)
- [x] [Terabox](https://www.terabox.com/main)
- [x] [UC](https://drive.uc.cn)
- [x] [Quark](https://pan.quark.cn)
- [x] [Thunder](https://pan.xunlei.com)
- [x] [Lanzou](https://www.lanzou.com)
- [x] [ILanzou](https://www.ilanzou.com)
- [x] [Google photo](https://photos.google.com)
- [x] [Mega.nz](https://mega.nz)
- [x] [Baidu photo](https://photo.baidu.com)
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
- [x] [115](https://115.com)
- [x] [Cloudreve](https://cloudreve.org)
- [x] [Dropbox](https://www.dropbox.com)
- [x] [FeijiPan](https://www.feijipan.com)
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
- [x] [Chaoxing](https://www.chaoxing.com)
- [x] [CNB](https://cnb.cool/)
- [x] [Degoo](https://degoo.com)
- [x] [Doubao](https://www.doubao.com)
- [x] [Febbox](https://www.febbox.com)
- [x] [GitHub](https://github.com)
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
- [x] [Weiyun](https://www.weiyun.com)
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
- [x] Easy to deploy and out-of-the-box
- [x] File preview (PDF, markdown, code, plain text, ...)
- [x] Image preview in gallery mode
- [x] Video and audio preview, with lyrics and subtitles support
- [x] Office document preview (docx, pptx, xlsx, ...)
- [x] `README.md` preview rendering
- [x] File permalink copy and direct file download
- [x] Dark mode and i18n
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
- [x] File/folder package download and web upload operations
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
## Документация
- 📘 [Docs](https://doc.oplist.org)
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
## Демо
- 🌎 [Global Demo](https://demo.oplist.org)
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
## Обсуждение
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
## Спонсор
[![VPS.Town](https://vps.town/static/images/sponsor.png)](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
## Доноры
Thanks to the following donors for their generous support:
- [HisAtri](https://github.com/HisAtri)
- 爱发电用户_7jTh
- suka
## Лицензия
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
## Связаться с нами
- [@GitHub](https://github.com/OpenListTeam)
- [Telegram Group](https://t.me/OpenListTeam)
- [Telegram Channel](https://t.me/OpenListOfficial)
## Участники
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
[![Contributors](https://contrib.rocks/image?repo=OpenListTeam/OpenList)](https://github.com/OpenListTeam/OpenList/graphs/contributors)
-89
View File
@@ -1,89 +0,0 @@
# Security Policy
## Supported Versions
Only the latest stable release receives security patches. We strongly recommend always keeping OpenList up to date.
| Version | Supported |
| -------------------- | ------------------ |
| Latest stable (v4.x) | :white_check_mark: |
| Older versions | :x: |
## Reporting a Vulnerability
**Please do NOT report security vulnerabilities through public GitHub Issues.**
If you discover a security vulnerability in OpenList, please report it responsibly by using one of the following channels:
- **GitHub Private Security Advisory** (preferred): [Submit here](https://github.com/OpenListTeam/OpenList/security/advisories/new)
- **Telegram**: Contact a maintainer privately via [@OpenListTeam](https://t.me/OpenListTeam)
When reporting, please include as much of the following as possible:
- A description of the vulnerability and its potential impact
- The affected version(s)
- Step-by-step instructions to reproduce the issue
- Any proof-of-concept code or screenshots (if applicable)
- Suggested mitigation or fix (optional but appreciated)
## Security Best Practices for Users
To keep your OpenList instance secure:
- Always update to the latest release.
- Use a strong, unique admin password and change it after first login.
- Enable HTTPS (TLS) for your deployment — do **not** expose OpenList over plain HTTP on the public internet.
- Limit exposed ports using a reverse proxy (e.g., Nginx, Caddy).
- Set up access controls and avoid enabling guest access unless necessary.
- Regularly review mounted storage permissions and revoke unused API tokens.
- When using Docker, avoid running the container as root if possible.
## Acknowledgments
We sincerely thank all security researchers and community members who responsibly disclose vulnerabilities and help make OpenList safer for everyone.
---
# 安全政策
## 支持的版本
我们仅对最新稳定版本提供安全补丁。强烈建议始终保持 OpenList 为最新版本。
| 版本 | 是否支持 |
| ------------------ | ------------------ |
| 最新稳定版(v4.x) | :white_check_mark: |
| 旧版本 | :x: |
## 报告漏洞
**请勿通过公开的 GitHub Issues 报告安全漏洞。**
如果您在 OpenList 中发现安全漏洞,请通过以下渠道之一负责任地进行报告:
- **GitHub 私密安全公告**(推荐):[点击提交](https://github.com/OpenListTeam/OpenList/security/advisories/new)
- **Telegram**:通过 [@OpenListTeam](https://t.me/OpenListTeam) 私信联系维护者
报告时,请尽量提供以下信息:
- 漏洞描述及其潜在影响
- 受影响的版本
- 复现问题的详细步骤
- 概念验证代码或截图(如有)
- 建议的缓解措施或修复方案(可选,但非常欢迎)
## 用户安全最佳实践
为保障您的 OpenList 实例安全:
- 始终更新至最新版本。
- 使用强且唯一的管理员密码,并在首次登录后立即修改。
- 为您的部署启用 HTTPS(TLS)—— **请勿**在公网上以明文 HTTP 方式暴露 OpenList。
- 使用反向代理(如 Nginx、Caddy)限制对外暴露的端口。
- 配置访问控制,非必要情况下不要开启访客访问。
- 定期检查已挂载存储的权限,并撤销未使用的 API 令牌。
- 使用 Docker 部署时,尽可能避免以 root 用户运行容器。
## 致谢
我们衷心感谢所有负责任地披露漏洞、帮助 OpenList 变得更加安全的安全研究人员和社区成员。
+11
View File
@@ -0,0 +1,11 @@
version: v1
plugins:
- plugin: buf.build/protocolbuffers/go:v1.36.7
out: .
opt:
- paths=source_relative
- plugin: buf.build/grpc/go:v1.5.1
out: .
opt:
- paths=source_relative
- require_unimplemented_servers=false
+1
View File
@@ -0,0 +1 @@
version: v1
+29 -30
View File
@@ -1,43 +1,42 @@
package cmd
import (
"os"
"path/filepath"
"strconv"
"context"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
log "github.com/sirupsen/logrus"
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
"github.com/OpenListTeam/OpenList/v5/internal/bootstrap"
"github.com/sirupsen/logrus"
)
func Init() {
bootstrap.Init()
func Init(ctx context.Context) {
if flags.Dev {
flags.Debug = true
}
initLogrus()
bootstrap.InitConfig()
bootstrap.InitDriverPlugins()
}
func Release() {
bootstrap.Release()
}
var pid = -1
var pidFile string
func initDaemon() {
ex, err := os.Executable()
if err != nil {
log.Fatal(err)
}
exPath := filepath.Dir(ex)
_ = os.MkdirAll(filepath.Join(exPath, "daemon"), 0700)
pidFile = filepath.Join(exPath, "daemon/pid")
if utils.Exists(pidFile) {
bytes, err := os.ReadFile(pidFile)
if err != nil {
log.Fatal("failed to read pid file", err)
}
id, err := strconv.Atoi(string(bytes))
if err != nil {
log.Fatal("failed to parse pid data", err)
}
pid = id
func initLog(l *logrus.Logger) {
if flags.Debug {
l.SetLevel(logrus.DebugLevel)
l.SetReportCaller(true)
} else {
l.SetLevel(logrus.InfoLevel)
l.SetReportCaller(false)
}
}
func initLogrus() {
formatter := logrus.TextFormatter{
ForceColors: true,
EnvironmentOverrideColors: true,
TimestampFormat: "2006-01-02 15:04:05",
FullTimestamp: true,
}
logrus.SetFormatter(&formatter)
initLog(logrus.StandardLogger())
}
+31 -2
View File
@@ -1,11 +1,40 @@
package flags
import (
"os"
"path/filepath"
"github.com/sirupsen/logrus"
)
var (
DataDir string
ConfigPath string
ConfigFile string
Debug bool
NoPrefix bool
Dev bool
ForceBinDir bool
LogStd bool
pwd string
)
// Program working directory
func PWD() string {
if pwd != "" {
return pwd
}
if ForceBinDir {
ex, err := os.Executable()
if err != nil {
logrus.Fatal(err)
}
pwd = filepath.Dir(ex)
return pwd
}
d, err := os.Getwd()
if err != nil {
logrus.Fatal(err)
}
pwd = d
return d
}
+4 -8
View File
@@ -4,10 +4,7 @@ import (
"fmt"
"os"
"github.com/OpenListTeam/OpenList/v4/cmd/flags"
_ "github.com/OpenListTeam/OpenList/v4/drivers"
_ "github.com/OpenListTeam/OpenList/v4/internal/archive"
_ "github.com/OpenListTeam/OpenList/v4/internal/offline_download"
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
"github.com/spf13/cobra"
)
@@ -27,11 +24,10 @@ func Execute() {
}
func init() {
RootCmd.PersistentFlags().StringVar(&flags.DataDir, "data", "data", "data directory (relative paths are resolved against the current working directory)")
RootCmd.PersistentFlags().StringVar(&flags.ConfigPath, "config", "", "path to config.json (relative to current working directory; defaults to [data directory]/config.json, where [data directory] is set by --data)")
RootCmd.PersistentFlags().StringVarP(&flags.ConfigFile, "config", "c", "data/config.json", "config file")
RootCmd.PersistentFlags().BoolVar(&flags.Debug, "debug", false, "start with debug mode")
RootCmd.PersistentFlags().BoolVar(&flags.NoPrefix, "no-prefix", false, "disable env prefix")
RootCmd.PersistentFlags().BoolVar(&flags.Dev, "dev", false, "start with dev mode")
RootCmd.PersistentFlags().BoolVar(&flags.ForceBinDir, "force-bin-dir", false, "Force to use the directory where the binary file is located as data directory")
RootCmd.PersistentFlags().BoolVar(&flags.LogStd, "log-std", false, "Force to log to std")
RootCmd.PersistentFlags().BoolVarP(&flags.ForceBinDir, "force-bin-dir", "f", false, "force to use the directory where the binary file is located as data directory")
RootCmd.PersistentFlags().BoolVar(&flags.LogStd, "log-std", false, "force to log to std")
}
+125 -19
View File
@@ -1,13 +1,26 @@
package cmd
import (
"context"
"errors"
"fmt"
"net"
"net/http"
"os"
"os/signal"
"strconv"
"sync"
"syscall"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
"github.com/OpenListTeam/OpenList/v5/internal/conf"
"github.com/OpenListTeam/OpenList/v5/server"
"github.com/gin-gonic/gin"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
)
// ServerCmd represents the server command
@@ -16,34 +29,127 @@ var ServerCmd = &cobra.Command{
Short: "Start the server at the specified address",
Long: `Start the server at the specified address
the address is defined in config file`,
Run: func(cmd *cobra.Command, args []string) {
bootstrap.Init()
defer bootstrap.Release()
bootstrap.Start()
// Wait for interrupt signal to gracefully shutdown the server with
// a timeout of 1 second.
Run: func(_ *cobra.Command, args []string) {
serverCtx, serverCancel := context.WithCancel(context.Background())
defer serverCancel()
Init(serverCtx)
if !flags.Debug {
gin.SetMode(gin.ReleaseMode)
}
r := gin.New()
r.Use(gin.LoggerWithWriter(log.StandardLogger().Out))
r.Use(gin.RecoveryWithWriter(log.StandardLogger().Out))
server.Init(r)
var httpHandler http.Handler = r
if conf.Conf.Scheme.EnableH2c {
httpHandler = h2c.NewHandler(r, &http2.Server{})
}
var httpSrv, httpsSrv, unixSrv *http.Server
if conf.Conf.Scheme.HttpPort > 0 {
httpBase := fmt.Sprintf("%s:%d", conf.Conf.Scheme.Address, conf.Conf.Scheme.HttpPort)
log.Infoln("start HTTP server", "@", httpBase)
httpSrv = &http.Server{Addr: httpBase, Handler: httpHandler}
go func() {
err := httpSrv.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Errorln("start HTTP server", ":", err)
serverCancel()
}
}()
}
if conf.Conf.Scheme.HttpsPort > 0 {
httpsBase := fmt.Sprintf("%s:%d", conf.Conf.Scheme.Address, conf.Conf.Scheme.HttpsPort)
log.Infoln("start HTTPS server", "@", httpsBase)
httpsSrv = &http.Server{Addr: httpsBase, Handler: r}
go func() {
err := httpsSrv.ListenAndServeTLS(conf.Conf.Scheme.CertFile, conf.Conf.Scheme.KeyFile)
if err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Errorln("start HTTPS server", ":", err)
serverCancel()
}
}()
}
if conf.Conf.Scheme.UnixFile != "" {
log.Infoln("start Unix server", "@", conf.Conf.Scheme.UnixFile)
unixSrv = &http.Server{Handler: httpHandler}
go func() {
listener, err := net.Listen("unix", conf.Conf.Scheme.UnixFile)
if err != nil {
log.Errorln("start Unix server", ":", err)
serverCancel()
return
}
mode, err := strconv.ParseUint(conf.Conf.Scheme.UnixFilePerm, 8, 32)
if err != nil {
log.Errorln("parse unix_file_perm", ":", err)
} else {
err = os.Chmod(conf.Conf.Scheme.UnixFile, os.FileMode(mode))
if err != nil {
log.Errorln("chmod socket file", ":", err)
}
}
err = unixSrv.Serve(listener)
if err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Errorln("start Unix server", ":", err)
serverCancel()
}
}()
}
quit := make(chan os.Signal, 1)
// kill (no param) default send syscanll.SIGTERM
// kill -2 is syscall.SIGINT
// kill -9 is syscall. SIGKILL but can"t be catch, so don't need add it
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
bootstrap.Shutdown(1 * time.Second)
select {
case <-quit:
case <-serverCtx.Done():
}
log.Println("shutdown server...")
Release()
quitCtx, quitCancel := context.WithTimeout(context.Background(), time.Second)
defer quitCancel()
var wg sync.WaitGroup
if httpSrv != nil {
wg.Add(1)
go func() {
defer wg.Done()
if err := httpSrv.Shutdown(quitCtx); err != nil {
log.Errorln("shutdown HTTP server", ":", err)
}
}()
}
if httpsSrv != nil {
wg.Add(1)
go func() {
defer wg.Done()
if err := httpsSrv.Shutdown(quitCtx); err != nil {
log.Errorln("shutdown HTTPS server", ":", err)
}
}()
}
if unixSrv != nil {
wg.Add(1)
go func() {
defer wg.Done()
if err := unixSrv.Shutdown(quitCtx); err != nil {
log.Errorln("shutdown Unix server", ":", err)
}
}()
}
wg.Wait()
log.Println("server exit")
},
}
func init() {
RootCmd.AddCommand(ServerCmd)
// Here you will define your flags and configuration settings.
// Cobra supports Persistent Flags which will work for this command
// and all subcommands, e.g.:
// serverCmd.PersistentFlags().String("foo", "", "A help for foo")
// Cobra supports local flags which will only run when this command
// is called directly, e.g.:
// serverCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
}
// OutOpenListInit 暴露用于外部启动server的函数
-60
View File
@@ -1,60 +0,0 @@
package _115
import (
"errors"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
driver115 "github.com/SheltonZhu/115driver/pkg/driver"
log "github.com/sirupsen/logrus"
)
var (
md5Salt = "Qclm8MGWUv59TnrR0XPg"
appVer = "35.6.0.3"
)
func (d *Pan115) getAppVersion() (string, error) {
result := VersionResp{}
res, err := base.RestyClient.R().Get(driver115.ApiGetVersion)
if err != nil {
return "", err
}
err = utils.Json.Unmarshal(res.Body(), &result)
if err != nil {
return "", err
}
if len(result.Error) > 0 {
return "", errors.New(result.Error)
}
return result.Data.Win.Version, nil
}
func (d *Pan115) getAppVer() string {
ver, err := d.getAppVersion()
if err != nil {
log.Warnf("[115] get app version failed: %v", err)
return appVer
}
if len(ver) > 0 {
return ver
}
return appVer
}
func (d *Pan115) initAppVer() {
appVer = d.getAppVer()
log.Debugf("use app version: %v", appVer)
}
type VersionResp struct {
Error string `json:"error,omitempty"`
Data Versions `json:"data"`
}
type Versions struct {
Win Version `json:"win"`
}
type Version struct {
Version string `json:"version_code"`
}
-15
View File
@@ -1,15 +0,0 @@
package _115_open
import "encoding/json"
func ParseInt64(v json.Number) (int64, error) {
i, err := v.Int64()
if err == nil {
return i, nil
}
f, e1 := v.Float64()
if e1 == nil {
return int64(f), nil
}
return int64(0), err
}
-129
View File
@@ -1,129 +0,0 @@
package _123_open
import (
"errors"
"fmt"
"net/http"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
var (
AccessToken = "https://open-api.123pan.com/api/v1/access_token"
)
func expiresInToExpiredAt(expiresIn int64) (time.Time, error) {
if expiresIn <= 0 {
return time.Time{}, errors.New("invalid expires_in from official API")
}
return time.Now().UTC().Add(time.Duration(expiresIn) * time.Second), nil
}
type tokenManager struct {
// accessToken string
expiredAt time.Time
mu sync.Mutex
blockRefresh bool
}
func (d *Open123) getAccessToken(forceRefresh bool) (string, error) {
tm := d.tm
tm.mu.Lock()
defer tm.mu.Unlock()
if tm.blockRefresh {
return "", errors.New("Authentication expired")
}
if !forceRefresh && d.AccessToken != "" && time.Now().Before(tm.expiredAt.Add(-5*time.Minute)) {
return d.AccessToken, nil
}
if err := d.flushAccessToken(); err != nil {
// token expired and failed to refresh, block further refresh attempts
tm.blockRefresh = true
return "", err
}
return d.AccessToken, nil
}
func (d *Open123) flushAccessToken() error {
// Official app renewapi response contains access_token, refresh_token and expires_in.
if d.UseOnlineAPI && d.RefreshToken != "" && len(d.APIAddress) > 0 {
var resp RefreshTokenResp
_, err := base.RestyClient.R().
SetResult(&resp).
SetQueryParams(map[string]string{
"refresh_ui": d.RefreshToken,
"server_use": "true",
"driver_txt": "123cloud_oa",
}).
Get(d.APIAddress)
if err != nil {
return err
}
if resp.AccessToken == "" || resp.RefreshToken == "" {
errMessage := resp.ErrorDescription
if errMessage == "" {
errMessage = resp.Text
}
if errMessage == "" {
errMessage = resp.Message
}
if errMessage == "" {
errMessage = resp.Error
}
if errMessage != "" {
return fmt.Errorf("failed to refresh token: %s", errMessage)
}
return fmt.Errorf("empty access_token or refresh_token returned from official API")
}
expiredAt, err := expiresInToExpiredAt(resp.ExpiresIn)
if err != nil {
return err
}
d.AccessToken = resp.AccessToken
d.RefreshToken = resp.RefreshToken
d.tm.expiredAt = expiredAt
op.MustSaveDriverStorage(d)
d.tm.blockRefresh = false
return nil
}
// Developer API response contains code/message/data(accessToken, expiredAt).
if d.ClientID != "" && d.ClientSecret != "" {
req := base.RestyClient.R()
req.SetHeaders(map[string]string{
"platform": "open_platform",
"Content-Type": "application/json",
})
var resp AccessTokenResp
req.SetBody(base.Json{
"clientID": d.ClientID,
"clientSecret": d.ClientSecret,
})
req.SetResult(&resp)
_, err := req.Execute(http.MethodPost, AccessToken)
if err != nil {
return err
}
if resp.Code != 0 {
return fmt.Errorf("get access token failed: %s", resp.Message)
}
if resp.Data.AccessToken == "" || resp.Data.ExpiredAt == "" {
return errors.New("invalid token payload from developer API")
}
expiredAt, err := time.Parse(time.RFC3339, resp.Data.ExpiredAt)
if err != nil {
return fmt.Errorf("parse expire time failed: %w", err)
}
d.AccessToken = resp.Data.AccessToken
d.tm.expiredAt = expiredAt.UTC()
op.MustSaveDriverStorage(d)
d.tm.blockRefresh = false
return nil
}
return errors.New("no valid authentication method available")
}
-278
View File
@@ -1,278 +0,0 @@
package _123_open
import (
"context"
"crypto/md5"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/go-resty/resty/v2"
"github.com/google/uuid"
log "github.com/sirupsen/logrus"
)
var ( // 不同情况下获取的AccessTokenQPS限制不同 如下模块化易于拓展
Api = "https://open-api.123pan.com"
UserInfo = InitApiInfo(Api+"/api/v1/user/info", 1)
FileList = InitApiInfo(Api+"/api/v2/file/list", 3)
DownloadInfo = InitApiInfo(Api+"/api/v1/file/download_info", 5)
DirectLink = InitApiInfo(Api+"/api/v1/direct-link/url", 5)
Mkdir = InitApiInfo(Api+"/upload/v1/file/mkdir", 2)
Move = InitApiInfo(Api+"/api/v1/file/move", 1)
Rename = InitApiInfo(Api+"/api/v1/file/name", 1)
Trash = InitApiInfo(Api+"/api/v1/file/trash", 2)
UploadCreate = InitApiInfo(Api+"/upload/v2/file/create", 2)
UploadComplete = InitApiInfo(Api+"/upload/v2/file/upload_complete", 0)
UploadSHA1Reuse = InitApiInfo(Api+"/upload/v2/file/sha1_reuse", 2)
OfflineDownload = InitApiInfo(Api+"/api/v1/offline/download", 1)
OfflineDownloadProcess = InitApiInfo(Api+"/api/v1/offline/download/process", 5)
)
func (d *Open123) Request(apiInfo *ApiInfo, method string, callback base.ReqCallback, resp interface{}) ([]byte, error) {
for {
token, err := d.getAccessToken(false)
if err != nil {
return nil, err
}
req := base.RestyClient.R()
req.SetHeaders(map[string]string{
"authorization": "Bearer " + token,
"platform": "open_platform",
"Content-Type": "application/json",
})
if callback != nil {
callback(req)
}
if resp != nil {
req.SetResult(resp)
}
log.Debugf("API: %s, QPS: %d, NowLen: %d", apiInfo.url, apiInfo.qps, apiInfo.NowLen())
apiInfo.Require()
defer apiInfo.Release()
res, err := req.Execute(method, apiInfo.url)
if err != nil {
return nil, err
}
body := res.Body()
// 解析为通用响应
var baseResp BaseResp
if err = json.Unmarshal(body, &baseResp); err != nil {
return nil, err
}
if baseResp.Code == 0 {
return body, nil
} else if baseResp.Code == 401 {
// 强制刷新Token, 有小概率会 race condition 导致多次刷新Token,但不影响正确运行
if _, err := d.getAccessToken(true); err != nil {
return nil, err
}
} else if baseResp.Code == 429 {
time.Sleep(500 * time.Millisecond)
log.Warningf("API: %s, QPS: %d, 请求太频繁,对应API提示过多请减小QPS", apiInfo.url, apiInfo.qps)
} else {
return nil, errors.New(baseResp.Message)
}
}
}
func (d *Open123) SignURL(originURL, privateKey string, uid uint64, validDuration time.Duration) (newURL string, err error) {
// 生成Unix时间戳
ts := time.Now().Add(validDuration).Unix()
// 生成随机数(建议使用UUID,不能包含中划线(-))
rand := strings.ReplaceAll(uuid.New().String(), "-", "")
// 解析URL
objURL, err := url.Parse(originURL)
if err != nil {
return "", err
}
// 待签名字符串,格式:path-timestamp-rand-uid-privateKey
unsignedStr := fmt.Sprintf("%s-%d-%s-%d-%s", objURL.Path, ts, rand, uid, privateKey)
md5Hash := md5.Sum([]byte(unsignedStr))
// 生成鉴权参数,格式:timestamp-rand-uid-md5hash
authKey := fmt.Sprintf("%d-%s-%d-%x", ts, rand, uid, md5Hash)
// 添加鉴权参数到URL查询参数
v := objURL.Query()
v.Add("auth_key", authKey)
objURL.RawQuery = v.Encode()
return objURL.String(), nil
}
func (d *Open123) getUserInfo(ctx context.Context) (*UserInfoResp, error) {
var resp UserInfoResp
if _, err := d.Request(UserInfo, http.MethodGet, func(req *resty.Request) {
req.SetContext(ctx)
}, &resp); err != nil {
return nil, err
}
return &resp, nil
}
func (d *Open123) getUID(ctx context.Context) (uint64, error) {
if d.UID != 0 {
return d.UID, nil
}
resp, err := d.getUserInfo(ctx)
if err != nil {
return 0, err
}
d.UID = resp.Data.UID
return resp.Data.UID, nil
}
func (d *Open123) getFiles(parentFileId int64, limit int, lastFileId int64) (*FileListResp, error) {
var resp FileListResp
_, err := d.Request(FileList, http.MethodGet, func(req *resty.Request) {
req.SetQueryParams(
map[string]string{
"parentFileId": strconv.FormatInt(parentFileId, 10),
"limit": strconv.Itoa(limit),
"lastFileId": strconv.FormatInt(lastFileId, 10),
"trashed": "false",
"searchMode": "",
"searchData": "",
})
}, &resp)
if err != nil {
return nil, err
}
return &resp, nil
}
func (d *Open123) getDownloadInfo(fileId int64) (*DownloadInfoResp, error) {
var resp DownloadInfoResp
_, err := d.Request(DownloadInfo, http.MethodGet, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"fileId": strconv.FormatInt(fileId, 10),
})
}, &resp)
if err != nil {
return nil, err
}
return &resp, nil
}
func (d *Open123) getDirectLink(fileId int64) (*DirectLinkResp, error) {
var resp DirectLinkResp
_, err := d.Request(DirectLink, http.MethodGet, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"fileID": strconv.FormatInt(fileId, 10),
})
}, &resp)
if err != nil {
return nil, err
}
return &resp, nil
}
func (d *Open123) mkdir(parentID int64, name string) error {
_, err := d.Request(Mkdir, http.MethodPost, func(req *resty.Request) {
req.SetBody(base.Json{
"parentID": strconv.FormatInt(parentID, 10),
"name": name,
})
}, nil)
if err != nil {
return err
}
return nil
}
func (d *Open123) move(fileID, toParentFileID int64) error {
_, err := d.Request(Move, http.MethodPost, func(req *resty.Request) {
req.SetBody(base.Json{
"fileIDs": []int64{fileID},
"toParentFileID": toParentFileID,
})
}, nil)
if err != nil {
return err
}
return nil
}
func (d *Open123) rename(fileId int64, fileName string) error {
_, err := d.Request(Rename, http.MethodPut, func(req *resty.Request) {
req.SetBody(base.Json{
"fileId": fileId,
"fileName": fileName,
})
}, nil)
if err != nil {
return err
}
return nil
}
func (d *Open123) trash(fileId int64) error {
_, err := d.Request(Trash, http.MethodPost, func(req *resty.Request) {
req.SetBody(base.Json{
"fileIDs": []int64{fileId},
})
}, nil)
if err != nil {
return err
}
return nil
}
func (d *Open123) createOfflineDownloadTask(ctx context.Context, url string, dirID, callback string) (taskID int, err error) {
body := base.Json{
"url": url,
"dirID": dirID,
}
if len(callback) > 0 {
body["callBackUrl"] = callback
}
var resp OfflineDownloadResp
_, err = d.Request(OfflineDownload, http.MethodPost, func(req *resty.Request) {
req.SetBody(body)
}, &resp)
if err != nil {
return 0, err
}
return resp.Data.TaskID, nil
}
func (d *Open123) queryOfflineDownloadStatus(ctx context.Context, taskID int) (process float64, status int, err error) {
var resp OfflineDownloadProcessResp
_, err = d.Request(OfflineDownloadProcess, http.MethodGet, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"taskID": strconv.Itoa(taskID),
})
}, &resp)
if err != nil {
return .0, 0, err
}
return resp.Data.Process, resp.Data.Status, nil
}
-1377
View File
File diff suppressed because it is too large Load Diff
-149
View File
@@ -1,149 +0,0 @@
package _189
import (
"context"
"crypto/sha1"
"encoding/hex"
"fmt"
"io"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
// 计算 sliceMD5
sliceMD5 := fileMD5
if len(sliceMD5s) > 1 {
joined := strings.Join(sliceMD5s, "\n")
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
}
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
t.Info.PieceLength = sliceSize
t.SetPieces(pieceHashes)
t.SetCASInfo(&torrent.CASInfo{
FileMD5: fileMD5,
SliceMD5: sliceMD5,
SliceMD5s: sliceMD5s,
SliceSize: sliceSize,
Cloud: "189",
})
return t.Encode()
}
// RapidUploadFromTorrent 从 torrent 文件中提取 CAS 信息进行秒传
func (d *Cloud189) RapidUploadFromTorrent(ctx context.Context, dstDir model.Obj, torrentData []byte) error {
// 解析 torrent
t, err := torrent.Decode(torrentData)
if err != nil {
return fmt.Errorf("解析 torrent 失败: %w", err)
}
// 检查是否包含 CAS 扩展信息
if !t.HasCASInfo() {
return fmt.Errorf("torrent 不包含 CAS 扩展信息,无法秒传")
}
cas := t.CAS
fileName := t.Info.Name
fileSize := t.GetTotalSize()
// 获取 sessionKey
sessionKey, err := d.getSessionKey()
if err != nil {
return err
}
d.sessionKey = sessionKey
// 初始化上传
res, err := d.uploadRequest("/person/initMultiUpload", map[string]string{
"parentFolderId": dstDir.GetID(),
"fileName": encode(fileName),
"fileSize": fmt.Sprint(fileSize),
"sliceSize": fmt.Sprint(cas.SliceSize),
"lazyCheck": "1",
}, nil)
if err != nil {
return fmt.Errorf("初始化上传失败: %w", err)
}
uploadFileId := utils.Json.Get(res, "data", "uploadFileId").ToString()
// 提交上传(使用 CAS 信息秒传)
_, err = d.uploadRequest("/person/commitMultiUploadFile", map[string]string{
"uploadFileId": uploadFileId,
"fileMd5": cas.FileMD5,
"sliceMd5": cas.SliceMD5,
"lazyCheck": "1",
"opertype": "3",
}, nil)
if err != nil {
return fmt.Errorf("秒传提交失败: %w", err)
}
return nil
}
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64, sliceSize int64) ([]byte, error) {
if sliceSize <= 0 {
sliceSize = torrent.DefaultPieceSize
}
hw := torrent.NewHashWriter(sliceSize, sliceSize)
buf := make([]byte, 32*1024)
for {
n, err := reader.Read(buf)
if n > 0 {
hw.Write(buf[:n])
}
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
}
hw.Finish()
fileMD5 := hw.GetFileMD5()
sliceMD5s := hw.GetSliceMD5s()
pieceHashes := hw.GetPieceHashes()
return GenerateTorrent(fileName, fileSize, fileMD5, sliceMD5s, sliceSize, pieceHashes)
}
// ComputePieceSHA1 计算单个分片的 SHA-1 哈希
func ComputePieceSHA1(data []byte) []byte {
h := sha1.Sum(data)
return h[:]
}
// ExtractCASFromTorrent 从 torrent 数据中提取 CAS 信息
func ExtractCASFromTorrent(torrentData []byte) (*torrent.CASInfo, string, int64, error) {
t, err := torrent.Decode(torrentData)
if err != nil {
return nil, "", 0, fmt.Errorf("解析 torrent 失败: %w", err)
}
if !t.HasCASInfo() {
return nil, "", 0, fmt.Errorf("torrent 不包含 CAS 扩展信息")
}
return t.CAS, t.Info.Name, t.GetTotalSize(), nil
}
// GetInfoHashHex 获取 torrent 的 info_hash(十六进制字符串)
func GetInfoHashHex(torrentData []byte) (string, error) {
t, err := torrent.Decode(torrentData)
if err != nil {
return "", err
}
return hex.EncodeToString(t.InfoHash), nil
}
-38
View File
@@ -1,38 +0,0 @@
package _189pc
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
LoginType string `json:"login_type" type:"select" options:"password,qrcode" default:"password" required:"true"`
Username string `json:"username" required:"true"`
Password string `json:"password" required:"true"`
VCode string `json:"validate_code"`
AccessToken string `json:"access_token" required:"false"`
RefreshToken string `json:"refresh_token" help:"To switch accounts, please clear this field"`
driver.RootID
OrderBy string `json:"order_by" type:"select" options:"filename,filesize,lastOpTime" default:"filename"`
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc" default:"asc"`
Type string `json:"type" type:"select" options:"personal,family" default:"personal"`
FamilyID string `json:"family_id"`
UploadMethod string `json:"upload_method" type:"select" options:"stream,rapid,old" default:"stream"`
UploadThread string `json:"upload_thread" default:"3" help:"1<=thread<=32"`
FamilyTransfer bool `json:"family_transfer"`
RapidUpload bool `json:"rapid_upload"`
NoUseOcr bool `json:"no_use_ocr"`
GenerateTorrent bool `json:"generate_torrent" help:"Generate torrent file with CAS extension after upload"`
}
var config = driver.Config{
Name: "189CloudPC",
DefaultRoot: "-11",
CheckStatus: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Cloud189PC{}
})
}
-449
View File
@@ -1,449 +0,0 @@
package _189pc
import (
"bytes"
"context"
"crypto/sha1"
"encoding/hex"
"fmt"
"io"
"net/url"
"strings"
"time"
"github.com/go-resty/resty/v2"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
// fileMD5: 整文件 MD5(大写十六进制)
// sliceMD5s: 每个分片的 MD5 列表(大写十六进制)
// sliceSize: 分片大小
// pieceHashes: SHA-1 piece hashes 拼接(每 20 字节一个)
// fileName: 文件名
// fileSize: 文件大小
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
// 计算 sliceMD5
sliceMD5 := fileMD5
if len(sliceMD5s) > 1 {
joined := strings.Join(sliceMD5s, "\n")
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
}
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
t.Info.PieceLength = sliceSize
t.SetPieces(pieceHashes)
t.SetCASInfo(&torrent.CASInfo{
FileMD5: fileMD5,
SliceMD5: sliceMD5,
SliceMD5s: sliceMD5s,
SliceSize: sliceSize,
Cloud: "189",
})
return t.Encode()
}
// RapidUploadFromTorrent 从 torrent 文件中提取 CAS 信息进行秒传
// 返回值:上传成功的文件对象、错误
func (y *Cloud189PC) RapidUploadFromTorrent(ctx context.Context, dstDir model.Obj, torrentData []byte, overwrite bool) (model.Obj, error) {
isFamily := y.isFamily()
// 解析 torrent
t, err := torrent.Decode(torrentData)
if err != nil {
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
}
// 检查是否包含 CAS 扩展信息
if !t.HasCASInfo() {
return nil, fmt.Errorf("torrent 不包含 CAS 扩展信息,无法秒传")
}
cas := t.CAS
fileName := t.Info.Name
fileSize := t.GetTotalSize()
// 统一 MD5 为大写(与正常上传保持一致,天翼云盘要求大写)
fileMD5Upper := strings.ToUpper(cas.FileMD5)
// 优先使用 torrent 中嵌入的分片大小,与生成时保持一致
sliceSize := cas.SliceSize
if sliceSize <= 0 {
sliceSize = partSize(fileSize)
}
// 计算 sliceMd5(与上传时一致的算法)
// 优先使用 torrent 中已有的 SliceMD5;仅当有多分片列表时才重新计算
sliceMd5Hex := strings.ToUpper(cas.SliceMD5)
if sliceMd5Hex == "" {
sliceMd5Hex = fileMD5Upper
}
if len(cas.SliceMD5s) > 1 {
// 分片 MD5 也需要统一大写后再拼接计算
upperSliceMD5s := make([]string, len(cas.SliceMD5s))
for i, s := range cas.SliceMD5s {
upperSliceMD5s[i] = strings.ToUpper(s)
}
sliceMd5Hex = strings.ToUpper(utils.GetMD5EncodeStr(strings.Join(upperSliceMD5s, "\n")))
}
// 使用与 Web 端一致的三步秒传流程
fullUrl := "https://upload.cloud.189.cn"
if isFamily {
fullUrl += "/family"
} else {
fullUrl += "/person"
}
// Step 1: initMultiUpload(不传 fileMd5/sliceMd5,只传 lazyCheck)
initParams := Params{
"parentFolderId": dstDir.GetID(),
"fileName": url.QueryEscape(fileName),
"fileSize": fmt.Sprint(fileSize),
"sliceSize": fmt.Sprint(sliceSize),
"lazyCheck": "1",
}
if isFamily {
initParams.Set("familyId", y.FamilyID)
}
var uploadInfo InitMultiUploadResp
_, err = y.request(fullUrl+"/initMultiUpload", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, initParams, &uploadInfo, isFamily)
if err != nil {
return nil, fmt.Errorf("initMultiUpload 失败: %w", err)
}
uploadFileId := uploadInfo.Data.UploadFileID
// Step 2: checkTransSecond(用 fileMd5 + sliceMd5 + uploadFileId 检查秒传)
checkParams := Params{
"fileMd5": fileMD5Upper,
"sliceMd5": sliceMd5Hex,
"uploadFileId": uploadFileId,
}
var checkResp struct {
Data struct {
FileDataExists int `json:"fileDataExists"`
} `json:"data"`
}
_, err = y.request(fullUrl+"/checkTransSecond", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, checkParams, &checkResp, isFamily)
if err != nil {
utils.Log.Errorf("[RapidUpload] checkTransSecond 失败: uploadFileId=%s, err=%v", uploadFileId, err)
return nil, fmt.Errorf("秒传检查失败: %w", err)
}
if checkResp.Data.FileDataExists != 1 {
return nil, fmt.Errorf("秒传失败:云端不存在该文件(fileMD5=%s, sliceMD5=%s, size=%d)", fileMD5Upper, sliceMd5Hex, fileSize)
}
// Step 3: commitMultiUploadFile(传 fileMd5 + sliceMd5)
var resp CommitMultiUploadFileResp
commitParams := Params{
"uploadFileId": uploadFileId,
"fileMd5": fileMD5Upper,
"sliceMd5": sliceMd5Hex,
"lazyCheck": "1",
"opertype": IF(overwrite, "3", "1"),
}
_, err = y.request(fullUrl+"/commitMultiUploadFile", "GET", func(req *resty.Request) {
req.SetContext(ctx)
}, commitParams, &resp, isFamily)
if err != nil {
utils.Log.Errorf("[RapidUpload] commitMultiUploadFile 失败: uploadFileId=%s, err=%v", uploadFileId, err)
return nil, fmt.Errorf("提交上传失败: %w", err)
}
// 秒传成功后,将 torrent 文件上传到目标目录(异步,不影响秒传结果)
if y.Addition.GenerateTorrent {
capturedDstDir := dstDir
capturedIsFamily := isFamily
go func() {
torrentName := fileName + ".cas.torrent"
infoHash, _ := GetInfoHashHex(torrentData)
utils.Log.Infof("秒传成功,上传 torrent: %s (info_hash: %s, size: %d bytes)",
torrentName, infoHash, len(torrentData))
torrentFileStream := &stream.FileStream{
Ctx: context.Background(),
Obj: &model.Object{
Name: torrentName,
Size: int64(len(torrentData)),
IsFolder: false,
},
Reader: bytes.NewReader(torrentData),
Mimetype: "application/x-bittorrent",
}
_, uploadErr := y.fastUpload(context.Background(), capturedDstDir, torrentFileStream, func(p float64) {}, capturedIsFamily, false, false)
if uploadErr != nil {
utils.Log.Warnf("上传 torrent 文件失败: %v", uploadErr)
} else {
utils.Log.Infof("torrent 文件已上传: %s", torrentName)
op.Cache.DeleteDirectory(y, capturedDstDir.GetPath())
}
}()
}
return resp.toFile(), nil
}
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
// 适用于:已有文件需要生成 torrent 的场景(如下载完成后生成)
func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64, sliceSize int64) ([]byte, error) {
if sliceSize <= 0 {
sliceSize = torrent.DefaultPieceSize
}
hw := torrent.NewHashWriter(sliceSize, sliceSize)
buf := make([]byte, 32*1024)
for {
n, err := reader.Read(buf)
if n > 0 {
hw.Write(buf[:n])
}
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
}
hw.Finish()
fileMD5 := hw.GetFileMD5()
sliceMD5s := hw.GetSliceMD5s()
pieceHashes := hw.GetPieceHashes()
return GenerateTorrent(fileName, fileSize, fileMD5, sliceMD5s, sliceSize, pieceHashes)
}
// ComputePieceSHA1 计算单个分片的 SHA-1 哈希
func ComputePieceSHA1(data []byte) []byte {
h := sha1.Sum(data)
return h[:]
}
// ExtractCASFromTorrent 从 torrent 数据中提取 CAS 信息
// 返回:CAS 信息、文件名、文件大小、错误
func ExtractCASFromTorrent(torrentData []byte) (*torrent.CASInfo, string, int64, error) {
t, err := torrent.Decode(torrentData)
if err != nil {
return nil, "", 0, fmt.Errorf("解析 torrent 失败: %w", err)
}
if !t.HasCASInfo() {
return nil, "", 0, fmt.Errorf("torrent 不包含 CAS 扩展信息")
}
return t.CAS, t.Info.Name, t.GetTotalSize(), nil
}
// InjectCASIntoTorrent 向已有的 torrent 文件注入 CAS 扩展信息
// 用于:下载完成后,计算了 MD5 信息,写回到 torrent 中
func InjectCASIntoTorrent(torrentData []byte, fileMD5 string, sliceMD5s []string, sliceSize int64) ([]byte, error) {
t, err := torrent.Decode(torrentData)
if err != nil {
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
}
// 计算 sliceMD5
sliceMD5 := fileMD5
if len(sliceMD5s) > 1 {
joined := strings.Join(sliceMD5s, "\n")
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
}
// 注入 CAS 信息
t.SetCASInfo(&torrent.CASInfo{
FileMD5: fileMD5,
SliceMD5: sliceMD5,
SliceMD5s: sliceMD5s,
SliceSize: sliceSize,
Cloud: "189",
})
// 同时更新 info 中的 md5sum 字段
if t.Info.MD5Sum == "" {
t.Info.MD5Sum = fileMD5
}
return t.Encode()
}
// GetInfoHashHex 获取 torrent 的 info_hash(十六进制字符串)
func GetInfoHashHex(torrentData []byte) (string, error) {
t, err := torrent.Decode(torrentData)
if err != nil {
return "", err
}
return hex.EncodeToString(t.InfoHash), nil
}
func isCASTorrentFile(fileName string) bool {
return strings.HasSuffix(fileName, ".cas.torrent")
}
// ComputeSliceMD5sFromReader 从 reader 中计算每个 10MB 分片的 MD5
// 返回:整文件 MD5、分片 MD5 列表
func ComputeSliceMD5sFromReader(reader io.Reader, sliceSize int64) (string, []string, error) {
if sliceSize <= 0 {
sliceSize = torrent.DefaultPieceSize
}
fileMD5Hash := utils.MD5.NewFunc()
sliceMD5s := make([]string, 0)
buf := make([]byte, sliceSize)
for {
n, err := io.ReadFull(reader, buf)
if n > 0 {
chunk := buf[:n]
fileMD5Hash.Write(chunk)
// 计算该分片的 MD5
sliceMD5 := strings.ToUpper(utils.HashData(utils.MD5, chunk))
sliceMD5s = append(sliceMD5s, sliceMD5)
}
if err == io.EOF || err == io.ErrUnexpectedEOF {
break
}
if err != nil {
return "", nil, err
}
}
fileMD5Hex := strings.ToUpper(hex.EncodeToString(fileMD5Hash.Sum(nil)))
return fileMD5Hex, sliceMD5s, nil
}
// torrentFollowCopy 跟随复制 torrent 文件(异步,不影响主操作)
// srcFolderId: 源文件所在目录 ID
// srcFileName: 源文件名
// dstDir: 目标目录
func (y *Cloud189PC) torrentFollowCopy(srcFolderId string, srcFileName string, dstDir model.Obj) {
if !y.Addition.GenerateTorrent {
return
}
if srcFolderId == "" {
return
}
torrentName := srcFileName + ".cas.torrent"
isFamily := y.isFamily()
go func() {
torrentFile, err := y.findFileByName(context.Background(), torrentName, srcFolderId, isFamily)
if err != nil {
utils.Log.Debugf("查找 torrent 文件失败(跟随复制): %v", err)
return
}
// 复制 torrent 文件到目标目录
resp, copyErr := y.CreateBatchTask("COPY", IF(isFamily, y.FamilyID, ""), dstDir.GetID(),
map[string]string{"targetFileName": dstDir.GetName()},
BatchTaskInfo{
FileId: torrentFile.GetID(),
FileName: torrentFile.GetName(),
IsFolder: 0,
})
if copyErr != nil {
utils.Log.Warnf("跟随复制 torrent 文件失败: %v", copyErr)
return
}
if err = y.WaitBatchTask("COPY", resp.TaskID, time.Second); err != nil {
utils.Log.Warnf("等待跟随复制 torrent 文件失败: %v", err)
}
}()
}
// torrentFollowMove 跟随移动 torrent 文件(异步,不影响主操作)
// srcFolderId: 源文件所在目录 ID
// srcFileName: 源文件名
// dstDir: 目标目录
func (y *Cloud189PC) torrentFollowMove(srcFolderId string, srcFileName string, dstDir model.Obj) {
if !y.Addition.GenerateTorrent {
return
}
if srcFolderId == "" {
return
}
torrentName := srcFileName + ".cas.torrent"
isFamily := y.isFamily()
go func() {
torrentFile, err := y.findFileByName(context.Background(), torrentName, srcFolderId, isFamily)
if err != nil {
utils.Log.Debugf("查找 torrent 文件失败(跟随移动): %v", err)
return
}
// 移动 torrent 文件到目标目录
resp, moveErr := y.CreateBatchTask("MOVE", IF(isFamily, y.FamilyID, ""), dstDir.GetID(),
map[string]string{"targetFileName": dstDir.GetName()},
BatchTaskInfo{
FileId: torrentFile.GetID(),
FileName: torrentFile.GetName(),
IsFolder: 0,
})
if moveErr != nil {
utils.Log.Warnf("跟随移动 torrent 文件失败: %v", moveErr)
return
}
if err = y.WaitBatchTask("MOVE", resp.TaskID, time.Millisecond*400); err != nil {
utils.Log.Warnf("等待跟随移动 torrent 文件失败: %v", err)
}
}()
}
// torrentFollowRename 跟随重命名 torrent 文件(异步,不影响主操作)
// folderId: 文件所在目录 ID
// oldFileName: 原文件名
// newFileName: 新文件名
func (y *Cloud189PC) torrentFollowRename(folderId string, oldFileName string, newFileName string) {
if !y.Addition.GenerateTorrent {
return
}
if folderId == "" {
return
}
oldTorrentName := oldFileName + ".cas.torrent"
newTorrentName := newFileName + ".cas.torrent"
isFamily := y.isFamily()
go func() {
torrentFile, err := y.findFileByName(context.Background(), oldTorrentName, folderId, isFamily)
if err != nil {
utils.Log.Debugf("查找 torrent 文件失败(跟随重命名): %v", err)
return
}
// 重命名 torrent 文件
queryParam := make(map[string]string)
fullUrl := API_URL
method := "POST"
if isFamily {
fullUrl += "/family/file"
method = "GET"
queryParam["familyId"] = y.FamilyID
}
fullUrl += "/renameFile.action"
queryParam["fileId"] = torrentFile.GetID()
queryParam["destFileName"] = newTorrentName
_, renameErr := y.request(fullUrl, method, func(req *resty.Request) {
req.SetContext(context.Background()).SetQueryParams(queryParam)
}, nil, &RenameResp{}, isFamily)
if renameErr != nil {
utils.Log.Warnf("跟随重命名 torrent 文件失败: %v", renameErr)
}
}()
}
-579
View File
@@ -1,579 +0,0 @@
package alias
import (
"context"
"errors"
"fmt"
"io"
"math/rand"
"net/url"
stdpath "path"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/fs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/internal/sign"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/OpenListTeam/OpenList/v4/server/common"
)
type Alias struct {
model.Storage
Addition
rootOrder []string
pathMap map[string][]string
root model.Obj
}
func (d *Alias) Config() driver.Config {
return config
}
func (d *Alias) GetAddition() driver.Additional {
return &d.Addition
}
func (d *Alias) Init(ctx context.Context) error {
paths := strings.Split(d.Paths, "\n")
d.rootOrder = make([]string, 0, len(paths))
d.pathMap = make(map[string][]string)
for _, path := range paths {
path = strings.TrimSpace(path)
if path == "" {
continue
}
k, v := getPair(path)
temp, ok := d.pathMap[k]
if !ok {
d.rootOrder = append(d.rootOrder, k)
}
d.pathMap[k] = append(temp, v)
}
switch len(d.rootOrder) {
case 0:
return errors.New("paths is required")
case 1:
paths := d.pathMap[d.rootOrder[0]]
roots := make(BalancedObjs, 0, len(paths))
roots = append(roots, &model.Object{
Name: "root",
Path: paths[0],
IsFolder: true,
Modified: d.Modified,
Mask: model.Locked,
})
for _, path := range paths[1:] {
roots = append(roots, &model.Object{
Path: path,
})
}
d.root = roots
default:
d.root = &model.Object{
Name: "root",
Path: "/",
IsFolder: true,
Modified: d.Modified,
Mask: model.ReadOnly,
}
}
if !utils.SliceContains(ValidReadConflictPolicy, d.ReadConflictPolicy) {
d.ReadConflictPolicy = FirstRWP
}
if !utils.SliceContains(ValidWriteConflictPolicy, d.WriteConflictPolicy) {
d.WriteConflictPolicy = DisabledWP
}
if !utils.SliceContains(ValidPutConflictPolicy, d.PutConflictPolicy) {
d.PutConflictPolicy = DisabledWP
}
return nil
}
func (d *Alias) Drop(ctx context.Context) error {
d.rootOrder = nil
d.pathMap = nil
d.root = nil
return nil
}
func (d *Alias) GetRoot(ctx context.Context) (model.Obj, error) {
if d.root == nil {
return nil, errs.StorageNotInit
}
return d.root, nil
}
// 通过op.Get调用的话,path一定是子路径(/开头)
func (d *Alias) Get(ctx context.Context, path string) (model.Obj, error) {
roots, sub := d.getRootsAndPath(path)
if len(roots) == 0 {
return nil, errs.ObjectNotFound
}
for idx, root := range roots {
rawPath := stdpath.Join(root, sub)
obj, err := fs.Get(ctx, rawPath, &fs.GetArgs{NoLog: true})
if err != nil {
continue
}
mask := model.GetObjMask(obj) &^ model.Temp
if sub == "" {
// 根目录
mask |= model.Locked | model.Virtual
}
ret := model.Object{
Path: rawPath,
Name: obj.GetName(),
Size: obj.GetSize(),
Modified: obj.ModTime(),
IsFolder: obj.IsDir(),
HashInfo: obj.GetHash(),
Mask: mask,
}
obj = &ret
if d.ProviderPassThrough && !obj.IsDir() {
if storage, err := fs.GetStorage(rawPath, &fs.GetStoragesArgs{}); err == nil {
obj = &model.ObjectProvider{
Object: ret,
Provider: model.Provider{
Provider: storage.Config().Name,
},
}
}
}
roots = roots[idx+1:]
var objs BalancedObjs
if idx > 0 {
objs = make(BalancedObjs, 0, len(roots)+2)
} else {
objs = make(BalancedObjs, 0, len(roots)+1)
}
objs = append(objs, obj)
if idx > 0 {
objs = append(objs, nil)
}
for _, d := range roots {
objs = append(objs, &tempObj{model.Object{
Path: stdpath.Join(d, sub),
}})
}
return objs, nil
}
return nil, errs.ObjectNotFound
}
func (d *Alias) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
dirs, ok := dir.(BalancedObjs)
if !ok {
return d.listRoot(ctx, args.WithStorageDetails && d.DetailsPassThrough, args.Refresh), nil
}
// 因为alias是NoCache且Get方法不会返回NotSupport或NotImplement错误
// 所以这里对象不会传回到alias,也就不需要返回BalancedObjs了
objMap := make(map[string]model.Obj)
for _, dir := range dirs {
if dir == nil {
continue
}
dirPath := dir.GetPath()
tmp, err := fs.List(ctx, dirPath, &fs.ListArgs{
NoLog: true,
Refresh: args.Refresh,
WithStorageDetails: args.WithStorageDetails && d.DetailsPassThrough,
})
if err != nil {
continue
}
for _, obj := range tmp {
name := obj.GetName()
if _, exists := objMap[name]; exists {
continue
}
mask := model.GetObjMask(obj) &^ model.Temp
objRes := model.Object{
Name: name,
Path: stdpath.Join(dirPath, name),
Size: obj.GetSize(),
Modified: obj.ModTime(),
IsFolder: obj.IsDir(),
Mask: mask,
}
var objRet model.Obj
if thumb, ok := model.GetThumb(obj); ok {
objRet = &model.ObjThumb{
Object: objRes,
Thumbnail: model.Thumbnail{
Thumbnail: thumb,
},
}
} else {
objRet = &objRes
}
if details, ok := model.GetStorageDetails(obj); ok {
objRet = &model.ObjStorageDetails{
Obj: objRet,
StorageDetails: details,
}
}
objMap[name] = objRet
}
}
objs := make([]model.Obj, 0, len(objMap))
for _, obj := range objMap {
objs = append(objs, obj)
}
if d.OrderBy == "" {
sort := getAllSort(dirs)
if sort.OrderBy != "" {
model.SortFiles(objs, sort.OrderBy, sort.OrderDirection)
}
if d.ExtractFolder == "" && sort.ExtractFolder != "" {
model.ExtractFolder(objs, sort.ExtractFolder)
}
}
return objs, nil
}
func (d *Alias) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if d.ReadConflictPolicy == AllRWP && !args.Redirect {
files, err := d.getAllObjs(ctx, file, getWriteAndPutFilterFunc(AllRWP))
if err != nil {
return nil, err
}
linkClosers := make([]io.Closer, 0, len(files))
rrf := make([]model.RangeReaderIF, 0, len(files))
requireReference := false
for _, f := range files {
link, fi, err := d.link(ctx, f.GetPath(), args)
if err != nil {
continue
}
if fi.GetSize() != files.GetSize() {
_ = link.Close()
continue
}
l := &model.Link{
URL: link.URL,
Header: link.Header,
RangeReader: link.RangeReader,
Concurrency: link.Concurrency,
PartSize: link.PartSize,
ContentLength: link.ContentLength,
}
if d.DownloadConcurrency > 0 {
l.Concurrency = d.DownloadConcurrency
}
if d.DownloadPartSize > 0 {
l.PartSize = d.DownloadPartSize * utils.KB
}
if l.ContentLength == 0 {
l.ContentLength = fi.GetSize()
}
rr, err := stream.GetRangeReaderFromLink(l.ContentLength, l)
if err != nil {
_ = link.Close()
continue
}
linkClosers = append(linkClosers, link)
if link.RequireReference {
requireReference = true
}
rrf = append(rrf, rr)
}
rr := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
return rrf[rand.Intn(len(rrf))].RangeRead(ctx, httpRange)
}
return &model.Link{
RangeReader: stream.RangeReaderFunc(rr),
SyncClosers: utils.NewSyncClosers(linkClosers...),
RequireReference: requireReference,
}, nil
}
var link *model.Link
var fi model.Obj
var err error
files := file.(BalancedObjs)
if d.ReadConflictPolicy == RandomBalancedRP || d.ReadConflictPolicy == AllRWP {
rand.Shuffle(len(files), func(i, j int) {
files[i], files[j] = files[j], files[i]
})
}
for _, f := range files {
if f == nil {
continue
}
link, fi, err = d.link(ctx, f.GetPath(), args)
if err == nil {
if link == nil {
// 重定向且需要通过代理
return &model.Link{
URL: fmt.Sprintf("%s/p%s?sign=%s",
common.GetApiUrl(ctx),
utils.EncodePath(f.GetPath(), true),
sign.Sign(f.GetPath())),
}, nil
}
break
}
}
if err != nil {
return nil, err
}
resultLink := link.Clone() // 复制一份,避免修改到原始link
resultLink.Expiration = nil
if args.Redirect {
return resultLink, nil
}
if d.DownloadConcurrency > 0 {
resultLink.Concurrency = d.DownloadConcurrency
}
if d.DownloadPartSize > 0 {
resultLink.PartSize = d.DownloadPartSize * utils.KB
}
if resultLink.ContentLength == 0 {
resultLink.ContentLength = fi.GetSize()
}
return resultLink, nil
}
func (d *Alias) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
// Other 不应负载均衡,这是因为前端是否调用 /fs/other 的判断条件是返回的 provider 的值
// 而 ProviderPassThrough 开启时,返回的 provider 固定为第一个 obj 的后端驱动
storage, actualPath, err := op.GetStorageAndActualPath(args.Obj.GetPath())
if err != nil {
return nil, err
}
return op.Other(ctx, storage, model.FsOtherArgs{
Path: actualPath,
Method: args.Method,
Data: args.Data,
})
}
func (d *Alias) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
objs, err := d.getWriteObjs(ctx, parentDir)
if err == nil {
for _, obj := range objs {
err = errors.Join(err, fs.MakeDir(ctx, stdpath.Join(obj.GetPath(), dirName)))
}
}
return err
}
func (d *Alias) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
srcs, dsts, err := d.getMoveObjs(ctx, srcObj, dstDir)
if err == nil {
for i, dst := range dsts {
src := srcs[i]
_, e := fs.Move(ctx, src.GetPath(), dst.GetPath())
err = errors.Join(err, e)
}
srcs = srcs[len(dsts):]
for _, src := range srcs {
e := fs.Remove(ctx, src.GetPath())
err = errors.Join(err, e)
}
}
return err
}
func (d *Alias) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
objs, err := d.getWriteObjs(ctx, srcObj)
if err == nil {
for _, obj := range objs {
err = errors.Join(err, fs.Rename(ctx, obj.GetPath(), newName))
}
}
return err
}
func (d *Alias) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
srcs, dsts, err := d.getCopyObjs(ctx, srcObj, dstDir)
if err == nil {
for i, src := range srcs {
dst := dsts[i]
_, e := fs.Copy(ctx, src.GetPath(), dst.GetPath())
err = errors.Join(err, e)
}
}
return err
}
func (d *Alias) Remove(ctx context.Context, obj model.Obj) error {
objs, err := d.getWriteObjs(ctx, obj)
if err == nil {
for _, obj := range objs {
err = errors.Join(err, fs.Remove(ctx, obj.GetPath()))
}
}
return err
}
func (d *Alias) Put(ctx context.Context, dstDir model.Obj, s model.FileStreamer, up driver.UpdateProgress) error {
objs, err := d.getPutObjs(ctx, dstDir)
if err == nil {
if len(objs) == 1 {
storage, reqActualPath, err := op.GetStorageAndActualPath(objs.GetPath())
if err != nil {
return err
}
return op.Put(ctx, storage, reqActualPath, &stream.FileStream{
Obj: s,
Mimetype: s.GetMimetype(),
Reader: s,
}, up)
} else {
file, err := s.CacheFullAndWriter(nil, nil)
if err != nil {
return err
}
count := float64(len(objs) + 1)
up(100 / count)
for i, obj := range objs {
err = errors.Join(err, fs.PutDirectly(ctx, obj.GetPath(), &stream.FileStream{
Obj: s,
Mimetype: s.GetMimetype(),
Reader: file,
}))
up(float64(i+2) / float64(count) * 100)
_, e := file.Seek(0, io.SeekStart)
if e != nil {
return errors.Join(err, e)
}
}
return err
}
}
return err
}
func (d *Alias) PutURL(ctx context.Context, dstDir model.Obj, name, url string) error {
objs, err := d.getPutObjs(ctx, dstDir)
if err == nil {
for _, obj := range objs {
err = errors.Join(err, fs.PutURL(ctx, obj.GetPath(), name, url))
}
return err
}
return err
}
func (d *Alias) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
reqPath := d.getBalancedPath(ctx, obj)
if reqPath == "" {
return nil, errs.NotFile
}
meta, err := d.getArchiveMeta(ctx, reqPath, args)
if err == nil {
return meta, nil
}
return nil, errs.NotImplement
}
func (d *Alias) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
reqPath := d.getBalancedPath(ctx, obj)
if reqPath == "" {
return nil, errs.NotFile
}
l, err := d.listArchive(ctx, reqPath, args)
if err == nil {
return l, nil
}
return nil, errs.NotImplement
}
func (d *Alias) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
// alias的两个驱动,一个支持驱动提取,一个不支持,如何兼容?
// 如果访问的是不支持驱动提取的驱动内的压缩文件,GetArchiveMeta就会返回errs.NotImplement,提取URL前缀就会是/ae,Extract就不会被调用
// 如果访问的是支持驱动提取的驱动内的压缩文件,GetArchiveMeta就会返回有效值,提取URL前缀就会是/ad,Extract就会被调用
reqPath := d.getBalancedPath(ctx, obj)
if reqPath == "" {
return nil, errs.NotFile
}
link, err := d.extract(ctx, reqPath, args)
if err != nil {
return nil, errs.NotImplement
}
if link == nil {
return &model.Link{
URL: fmt.Sprintf("%s/ap%s?inner=%s&pass=%s&sign=%s",
common.GetApiUrl(ctx),
utils.EncodePath(reqPath, true),
utils.EncodePath(args.InnerPath, true),
url.QueryEscape(args.Password),
sign.SignArchive(reqPath)),
}, nil
}
return link.Clone(), nil
}
func (d *Alias) ArchiveDecompress(ctx context.Context, srcObj, dstDir model.Obj, args model.ArchiveDecompressArgs) error {
srcs, dsts, err := d.getCopyObjs(ctx, srcObj, dstDir)
if err == nil {
for i, src := range srcs {
dst := dsts[i]
_, e := fs.ArchiveDecompress(ctx, src.GetPath(), dst.GetPath(), args)
err = errors.Join(err, e)
}
}
return err
}
func (d *Alias) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
if !d.DetailsPassThrough {
return nil, errs.NotImplement
}
if len(d.rootOrder) != 1 {
return nil, errs.NotImplement
}
backends := d.pathMap[d.rootOrder[0]]
var storage driver.Driver
for _, backend := range backends {
s, err := fs.GetStorage(backend, &fs.GetStoragesArgs{})
if err != nil {
return nil, errs.NotImplement
}
if storage == nil {
storage = s
} else if storage.GetStorage().MountPath != s.GetStorage().MountPath {
return nil, errs.NotImplement
}
}
if storage == nil { // should never access
return nil, errs.NotImplement
}
return op.GetStorageDetails(ctx, storage)
}
func (d *Alias) ResolveLinkCacheMode(path string) driver.LinkCacheMode {
roots, sub := d.getRootsAndPath(path)
if len(roots) == 0 {
return 0
}
for _, root := range roots {
storage, actualPath, err := op.GetStorageAndActualPath(stdpath.Join(root, sub))
if err != nil {
continue
}
if storage.Config().CheckStatus && storage.GetStorage().Status != op.WORK {
continue
}
mode := storage.Config().LinkCacheMode
if mode == -1 {
return storage.(driver.LinkCacheModeResolver).ResolveLinkCacheMode(actualPath)
} else {
return mode
}
}
return 0
}
var _ driver.Driver = (*Alias)(nil)
-34
View File
@@ -1,34 +0,0 @@
package alias
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
Paths string `json:"paths" required:"true" type:"text"`
ReadConflictPolicy string `json:"read_conflict_policy" type:"select" options:"first,random,all" default:"first"`
WriteConflictPolicy string `json:"write_conflict_policy" type:"select" options:"disabled,first,deterministic,deterministic_or_all,all,all_strict" default:"disabled" help:"How the driver handles identical backend paths when renaming, removing, or making directories."`
PutConflictPolicy string `json:"put_conflict_policy" type:"select" options:"disabled,first,deterministic,deterministic_or_all,all,all_strict,random,quota,quota_strict" default:"disabled" help:"How the driver handles identical backend paths when uploading, copying, moving, or decompressing."`
FileConsistencyCheck bool `json:"file_consistency_check" type:"bool" default:"false"`
DownloadConcurrency int `json:"download_concurrency" default:"0" required:"false" type:"number" help:"Need to enable proxy"`
DownloadPartSize int `json:"download_part_size" default:"0" type:"number" required:"false" help:"Need to enable proxy. Unit: KB"`
ProviderPassThrough bool `json:"provider_pass_through" type:"bool" default:"false"`
DetailsPassThrough bool `json:"details_pass_through" type:"bool" default:"false"`
}
var config = driver.Config{
Name: "Alias",
LocalSort: true,
NoCache: true,
NoUpload: false,
DefaultRoot: "/",
ProxyRangeOption: true,
LinkCacheMode: driver.LinkCacheAuto,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Alias{}
})
}
-78
View File
@@ -1,78 +0,0 @@
package alias
import (
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/pkg/errors"
)
const (
DisabledWP = "disabled"
FirstRWP = "first"
DeterministicWP = "deterministic"
DeterministicOrAllWP = "deterministic_or_all"
AllRWP = "all"
AllStrictWP = "all_strict"
RandomBalancedRP = "random"
BalancedByQuotaP = "quota"
BalancedByQuotaStrictP = "quota_strict"
)
var (
ValidReadConflictPolicy = []string{FirstRWP, RandomBalancedRP, AllRWP}
ValidWriteConflictPolicy = []string{DisabledWP, FirstRWP, DeterministicWP, DeterministicOrAllWP, AllRWP,
AllStrictWP}
ValidPutConflictPolicy = []string{DisabledWP, FirstRWP, DeterministicWP, DeterministicOrAllWP, AllRWP,
AllStrictWP, RandomBalancedRP, BalancedByQuotaP, BalancedByQuotaStrictP}
)
var (
ErrPathConflict = errors.New("path conflict")
ErrSamePathLeak = errors.New("leak some of same-name dirs")
ErrNoEnoughSpace = errors.New("none of same-name dirs has enough space")
ErrNotEnoughSrcObjs = errors.New("cannot move fewer objs to more paths, please try copying")
)
type BalancedObjs []model.Obj
func (b BalancedObjs) GetSize() int64 {
return b[0].GetSize()
}
func (b BalancedObjs) ModTime() time.Time {
return b[0].ModTime()
}
func (b BalancedObjs) CreateTime() time.Time {
return b[0].CreateTime()
}
func (b BalancedObjs) IsDir() bool {
return b[0].IsDir()
}
func (b BalancedObjs) GetHash() utils.HashInfo {
return b[0].GetHash()
}
func (b BalancedObjs) GetName() string {
return b[0].GetName()
}
func (b BalancedObjs) GetPath() string {
return b[0].GetPath()
}
func (b BalancedObjs) GetID() string {
return b[0].GetID()
}
func (b BalancedObjs) Unwrap() model.Obj {
return b[0]
}
var _ model.Obj = (BalancedObjs)(nil)
type tempObj struct{ model.Object }
-532
View File
@@ -1,532 +0,0 @@
package alias
import (
"context"
"math/rand"
stdpath "path"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/fs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/server/common"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
type detailWithIndex struct {
idx int
val *model.StorageDetails
}
func (d *Alias) listRoot(ctx context.Context, withDetails, refresh bool) []model.Obj {
var objs []model.Obj
detailsChan := make(chan detailWithIndex, len(d.pathMap))
workerCount := 0
for _, k := range d.rootOrder {
obj := &model.Object{
Name: k,
Path: "/" + k,
IsFolder: true,
Modified: d.Modified,
Mask: model.Locked | model.Virtual,
}
idx := len(objs)
objs = append(objs, obj)
v := d.pathMap[k]
if !withDetails || len(v) != 1 {
continue
}
remoteDriver, err := op.GetStorageByMountPath(v[0])
if err != nil {
continue
}
obj.Modified = remoteDriver.GetStorage().Modified
_, ok := remoteDriver.(driver.WithDetails)
if !ok {
continue
}
objs[idx] = &model.ObjStorageDetails{
Obj: objs[idx],
StorageDetails: nil,
}
workerCount++
go func(dri driver.Driver, i int) {
details, e := op.GetStorageDetails(ctx, dri, refresh)
if e != nil {
if !errors.Is(e, errs.NotImplement) && !errors.Is(e, errs.StorageNotInit) {
log.Errorf("failed get %s storage details: %+v", dri.GetStorage().MountPath, e)
}
}
detailsChan <- detailWithIndex{idx: i, val: details}
}(remoteDriver, idx)
}
for workerCount > 0 {
select {
case r := <-detailsChan:
objs[r.idx].(*model.ObjStorageDetails).StorageDetails = r.val
workerCount--
case <-time.After(time.Second):
workerCount = 0
}
}
return objs
}
// do others that not defined in Driver interface
func getPair(path string) (string, string) {
if name, path, ok := strings.Cut(path, ":"); ok && !strings.Contains(name, "/") {
return name, path
}
return stdpath.Base(path), path
}
func (d *Alias) getRootsAndPath(path string) (roots []string, sub string) {
if len(d.rootOrder) == 1 {
return d.pathMap[d.rootOrder[0]], path
}
path = strings.TrimPrefix(path, "/")
before, after, ok := strings.Cut(path, "/")
if !ok {
return d.pathMap[path], ""
}
return d.pathMap[before], after
}
func (d *Alias) link(ctx context.Context, reqPath string, args model.LinkArgs) (*model.Link, model.Obj, error) {
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
if err != nil {
return nil, nil, err
}
if args.Redirect && common.ShouldProxy(storage, stdpath.Base(reqPath)) {
return nil, nil, nil
}
return op.Link(ctx, storage, reqActualPath, args)
}
func isConsistent(a, b model.Obj) bool {
if a.GetSize() != b.GetSize() {
return false
}
for ht, v := range a.GetHash().All() {
ah := b.GetHash().GetHash(ht)
if ah != "" && ah != v {
return false
}
}
return true
}
func (d *Alias) getAllObjs(ctx context.Context, bObj model.Obj, ifContinue func(err error) (bool, error)) (BalancedObjs, error) {
objs := bObj.(BalancedObjs)
length := 0
for _, o := range objs {
var err error
var obj model.Obj
temp, isTemp := o.(*tempObj)
if isTemp {
obj, err = fs.Get(ctx, o.GetPath(), &fs.GetArgs{NoLog: true})
if err == nil {
if !bObj.IsDir() {
if obj.IsDir() {
err = errs.NotFile
} else if d.FileConsistencyCheck && !isConsistent(bObj, obj) {
err = errs.ObjectNotFound
}
} else if !obj.IsDir() {
err = errs.NotFolder
}
}
} else if o == nil {
err = errs.ObjectNotFound
}
cont, err := ifContinue(err)
if err != nil {
if cont {
continue
}
return nil, err
}
if isTemp {
objRes := temp.Object
// objRes.Name = obj.GetName()
// objRes.Size = obj.GetSize()
// objRes.Modified = obj.ModTime()
// objRes.HashInfo = obj.GetHash()
objs[length] = &objRes
} else {
objs[length] = o
}
length++
if !cont {
break
}
}
if length == 0 {
return nil, errs.ObjectNotFound
}
return objs[:length], nil
}
func (d *Alias) getBalancedPath(ctx context.Context, file model.Obj) string {
if d.ReadConflictPolicy == FirstRWP {
return file.GetPath()
}
files := file.(BalancedObjs)
if rand.Intn(len(files)) == 0 {
return file.GetPath()
}
files, _ = d.getAllObjs(ctx, file, getWriteAndPutFilterFunc(AllRWP))
return files[rand.Intn(len(files))].GetPath()
}
func getWriteAndPutFilterFunc(policy string) func(error) (bool, error) {
if policy == AllRWP {
return func(err error) (bool, error) {
return true, err
}
}
all := true
l := 0
return func(err error) (bool, error) {
if err != nil {
switch policy {
case AllStrictWP:
return false, ErrSamePathLeak
case DeterministicOrAllWP:
if l >= 2 {
return false, ErrSamePathLeak
}
}
all = false
} else {
switch policy {
case FirstRWP:
return false, nil
case DeterministicWP:
if l > 0 {
return false, ErrPathConflict
}
case DeterministicOrAllWP:
if l > 0 && !all {
return false, ErrSamePathLeak
}
}
l += 1
}
return true, err
}
}
func (d *Alias) getWriteObjs(ctx context.Context, obj model.Obj) (BalancedObjs, error) {
if d.WriteConflictPolicy == DisabledWP {
return nil, errs.PermissionDenied
}
return d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.WriteConflictPolicy))
}
func (d *Alias) getPutObjs(ctx context.Context, obj model.Obj) (BalancedObjs, error) {
if d.PutConflictPolicy == DisabledWP {
return nil, errs.PermissionDenied
}
objs, err := d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.PutConflictPolicy))
if err != nil {
return nil, err
}
strict := false
switch d.PutConflictPolicy {
case RandomBalancedRP:
ri := rand.Intn(len(objs))
return objs[ri : ri+1], nil
case BalancedByQuotaStrictP:
strict = true
fallthrough
case BalancedByQuotaP:
objs, ok := getRandomObjByQuotaBalanced(ctx, objs, strict, obj.GetSize())
if !ok {
return nil, ErrNoEnoughSpace
}
return objs, nil
default:
return objs, nil
}
}
func getRandomObjByQuotaBalanced(ctx context.Context, reqPath BalancedObjs, strict bool, objSize int64) (BalancedObjs, bool) {
// Get all space
details := make([]*model.StorageDetails, len(reqPath))
detailsChan := make(chan detailWithIndex, len(reqPath))
workerCount := 0
for i, p := range reqPath {
s, err := fs.GetStorage(p.GetPath(), &fs.GetStoragesArgs{})
if err != nil {
continue
}
if _, ok := s.(driver.WithDetails); !ok {
continue
}
workerCount++
go func(dri driver.Driver, i int) {
d, e := op.GetStorageDetails(ctx, dri)
if e != nil {
if !errors.Is(e, errs.NotImplement) && !errors.Is(e, errs.StorageNotInit) {
log.Errorf("failed get %s storage details: %+v", dri.GetStorage().MountPath, e)
}
}
detailsChan <- detailWithIndex{idx: i, val: d}
}(s, i)
}
for workerCount > 0 {
select {
case r := <-detailsChan:
details[r.idx] = r.val
workerCount--
case <-time.After(time.Second):
workerCount = 0
}
}
// Try select one that has space info
selected, ok := selectRandom(details, func(d *model.StorageDetails) uint64 {
if d == nil || d.FreeSpace() < objSize {
return 0
}
return uint64(d.FreeSpace())
})
if !ok {
if strict {
return nil, false
} else {
// No strict mode, return any of non-details ones
noDetails := make([]int, 0, len(details))
for i, d := range details {
if d == nil {
noDetails = append(noDetails, i)
}
}
if len(noDetails) == 0 {
return nil, false
}
selected = noDetails[rand.Intn(len(noDetails))]
}
}
return reqPath[selected : selected+1], true
}
func selectRandom[Item any](arr []Item, getWeight func(Item) uint64) (int, bool) {
var totalWeight uint64 = 0
for _, i := range arr {
totalWeight += getWeight(i)
}
if totalWeight == 0 {
return 0, false
}
r := rand.Uint64() % totalWeight
for i, item := range arr {
w := getWeight(item)
if r < w {
return i, true
}
r -= w
}
return 0, false
}
func (d *Alias) getCopyObjs(ctx context.Context, srcObj, dstDir model.Obj) (BalancedObjs, BalancedObjs, error) {
if d.PutConflictPolicy == DisabledWP {
return nil, nil, errs.PermissionDenied
}
dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy))
if err != nil {
return nil, nil, err
}
dstStorageMap := make(map[string][]model.Obj)
allocatingDst := make(map[model.Obj]struct{})
for _, o := range dstObjs {
storage, e := fs.GetStorage(o.GetPath(), &fs.GetStoragesArgs{})
if e != nil {
return nil, nil, errors.WithMessagef(e, "cannot copy to virtual path [%s]", o.GetPath())
}
mp := storage.GetStorage().MountPath
dstStorageMap[mp] = append(dstStorageMap[mp], o)
allocatingDst[o] = struct{}{}
}
tmpSrcObjs, err := d.getAllObjs(ctx, srcObj, getWriteAndPutFilterFunc(AllRWP))
if err != nil {
return nil, nil, err
}
srcObjs := make(BalancedObjs, 0, len(dstObjs))
for _, src := range tmpSrcObjs {
storage, e := fs.GetStorage(src.GetPath(), &fs.GetStoragesArgs{})
if e != nil {
continue
}
mp := storage.GetStorage().MountPath
if tmp, ok := dstStorageMap[mp]; ok {
for _, dst := range tmp {
dstObjs[len(srcObjs)] = dst
srcObjs = append(srcObjs, src)
delete(allocatingDst, dst)
}
delete(dstStorageMap, mp)
}
}
dstObjs = dstObjs[:len(srcObjs)]
for dst := range allocatingDst {
src := tmpSrcObjs[0]
if d.ReadConflictPolicy == RandomBalancedRP || d.ReadConflictPolicy == AllRWP {
src = tmpSrcObjs[rand.Intn(len(tmpSrcObjs))]
}
srcObjs = append(srcObjs, src)
dstObjs = append(dstObjs, dst)
}
return srcObjs, dstObjs, nil
}
func (d *Alias) getMoveObjs(ctx context.Context, srcObj, dstDir model.Obj) (BalancedObjs, BalancedObjs, error) {
if d.PutConflictPolicy == DisabledWP {
return nil, nil, errs.PermissionDenied
}
dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy))
if err != nil {
return nil, nil, err
}
tmpSrcObjs, err := d.getAllObjs(ctx, srcObj, getWriteAndPutFilterFunc(AllRWP))
if err != nil {
return nil, nil, err
}
if len(tmpSrcObjs) < len(dstObjs) {
return nil, nil, ErrNotEnoughSrcObjs
}
dstStorageMap := make(map[string][]model.Obj)
allocatingDst := make(map[model.Obj]struct{})
for _, o := range dstObjs {
storage, e := fs.GetStorage(o.GetPath(), &fs.GetStoragesArgs{})
if e != nil {
return nil, nil, errors.WithMessagef(e, "cannot move to virtual path [%s]", o.GetPath())
}
mp := storage.GetStorage().MountPath
dstStorageMap[mp] = append(dstStorageMap[mp], o)
allocatingDst[o] = struct{}{}
}
srcObjs := make(BalancedObjs, 0, len(tmpSrcObjs))
restSrcObjs := make(BalancedObjs, 0, len(tmpSrcObjs)-len(dstObjs))
for _, src := range tmpSrcObjs {
storage, e := fs.GetStorage(src.GetPath(), &fs.GetStoragesArgs{})
if e != nil {
continue
}
mp := storage.GetStorage().MountPath
if tmp, ok := dstStorageMap[mp]; ok {
dst := tmp[0]
if len(tmp) == 1 {
delete(dstStorageMap, mp)
} else {
dstStorageMap[mp] = tmp[1:]
}
dstObjs[len(srcObjs)] = dst
srcObjs = append(srcObjs, src)
delete(allocatingDst, dst)
} else {
restSrcObjs = append(restSrcObjs, src)
}
}
dstObjs = dstObjs[:len(srcObjs)]
// len(restSrcObjs) >= len(allocatingDst)
srcObjs = append(srcObjs, restSrcObjs...)
for dst := range allocatingDst {
dstObjs = append(dstObjs, dst)
}
return srcObjs, dstObjs, nil
}
func (d *Alias) getArchiveMeta(ctx context.Context, reqPath string, args model.ArchiveArgs) (model.ArchiveMeta, error) {
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
if err != nil {
return nil, err
}
if _, ok := storage.(driver.ArchiveReader); ok {
return op.GetArchiveMeta(ctx, storage, reqActualPath, model.ArchiveMetaArgs{
ArchiveArgs: args,
Refresh: true,
})
}
return nil, errs.NotImplement
}
func (d *Alias) listArchive(ctx context.Context, reqPath string, args model.ArchiveInnerArgs) ([]model.Obj, error) {
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
if err != nil {
return nil, err
}
if _, ok := storage.(driver.ArchiveReader); ok {
return op.ListArchive(ctx, storage, reqActualPath, model.ArchiveListArgs{
ArchiveInnerArgs: args,
Refresh: true,
})
}
return nil, errs.NotImplement
}
func (d *Alias) extract(ctx context.Context, reqPath string, args model.ArchiveInnerArgs) (*model.Link, error) {
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
if err != nil {
return nil, err
}
if _, ok := storage.(driver.ArchiveReader); !ok {
return nil, errs.NotImplement
}
if args.Redirect && common.ShouldProxy(storage, stdpath.Base(reqPath)) {
_, err := fs.Get(ctx, reqPath, &fs.GetArgs{NoLog: true})
if err == nil {
return nil, err
}
return nil, nil
}
link, _, err := op.DriverExtract(ctx, storage, reqActualPath, args)
return link, err
}
func getAllSort(dirs []model.Obj) model.Sort {
ret := model.Sort{}
noSort := false
noExtractFolder := false
for _, dir := range dirs {
if dir == nil {
continue
}
storage, err := fs.GetStorage(dir.GetPath(), &fs.GetStoragesArgs{})
if err != nil {
continue
}
if !noSort && storage.GetStorage().OrderBy != "" {
if ret.OrderBy == "" {
ret.OrderBy = storage.GetStorage().OrderBy
ret.OrderDirection = storage.GetStorage().OrderDirection
if ret.OrderDirection == "" {
ret.OrderDirection = "asc"
}
} else if ret.OrderBy != storage.GetStorage().OrderBy || ret.OrderDirection != storage.GetStorage().OrderDirection {
ret.OrderBy = ""
ret.OrderDirection = ""
noSort = true
}
}
if !noExtractFolder && storage.GetStorage().ExtractFolder != "" {
if ret.ExtractFolder == "" {
ret.ExtractFolder = storage.GetStorage().ExtractFolder
} else if ret.ExtractFolder != storage.GetStorage().ExtractFolder {
ret.ExtractFolder = ""
noExtractFolder = true
}
}
if noSort && noExtractFolder {
break
}
}
return ret
}
-147
View File
@@ -1,147 +0,0 @@
package alidoc
import (
"context"
"fmt"
"net/http"
"strings"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/go-resty/resty/v2"
)
type AliDoc struct {
model.Storage
Addition
client *resty.Client
}
func (d *AliDoc) Config() driver.Config {
return config
}
func (d *AliDoc) GetAddition() driver.Additional {
return &d.Addition
}
func (d *AliDoc) Init(ctx context.Context) error {
d.Cookie = strings.TrimSpace(d.Cookie)
d.RootFolderID = strings.TrimSpace(d.RootFolderID)
if d.Cookie == "" {
return fmt.Errorf("cookie is empty")
}
if d.RootFolderID == "" {
return fmt.Errorf("root folder id is empty")
}
d.client = newClient()
if err := d.checkCookie(ctx); err != nil {
return err
}
return nil
}
func (d *AliDoc) Drop(ctx context.Context) error {
d.client = nil
return nil
}
func (d *AliDoc) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
items, err := d.list(ctx, dir.GetID())
if err != nil {
return nil, err
}
objs := make([]model.Obj, 0, len(items))
for _, item := range items {
if strings.TrimSpace(item.DentryUUID) == "" || strings.TrimSpace(item.Name) == "" {
continue
}
objs = append(objs, toObj(item))
}
return objs, nil
}
func (d *AliDoc) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if file.IsDir() {
return nil, fmt.Errorf("alidoc does not support directory links")
}
resp, err := d.download(ctx, file.GetID())
if err != nil {
return nil, err
}
url, err := firstDownloadURL(resp)
if err != nil {
return nil, err
}
return &model.Link{
URL: url,
Header: http.Header{
"User-Agent": []string{base.UserAgent},
"Referer": []string{apiBase + "/"},
},
}, nil
}
func (d *AliDoc) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
err := d.post(ctx, "/box/api/v2/dentry/createfolder", map[string]string{
"dentryType": "folder",
"name": dirName,
"parentDentryUuid": parentDir.GetID(),
"conflictHandleStrategy": "auto_rename",
})
if err != nil {
return nil, err
}
return nil, nil
}
func (d *AliDoc) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
err := d.post(ctx, "/box/api/v2/dentry/move", map[string]interface{}{
"targetParentDentryUuid": dstDir.GetID(),
"sourceDentryUuid": srcObj.GetID(),
"operateFrom": 1,
})
if err != nil {
return nil, err
}
return srcObj, nil
}
func (d *AliDoc) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
err := d.post(ctx, "/box/api/v2/dentry/rename", map[string]string{
"dentryUuid": srcObj.GetID(),
"name": newName,
})
if err != nil {
return nil, err
}
srcObj.(*model.Object).Name = newName
return srcObj, nil
}
func (d *AliDoc) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
return d.post(ctx, "/box/api/v2/dentry/copy", map[string]interface{}{
"sourceDentryUuid": srcObj.GetID(),
"targetParentDentryUuid": dstDir.GetID(),
"operateFrom": 1,
"onlyCopyMeta": false,
})
}
func (d *AliDoc) Remove(ctx context.Context, obj model.Obj) error {
return d.post(ctx, "/box/api/v1/dentry/recycle", map[string]string{
"dentryUuid": obj.GetID(),
})
}
var (
_ driver.Driver = (*AliDoc)(nil)
_ driver.MkdirResult = (*AliDoc)(nil)
_ driver.MoveResult = (*AliDoc)(nil)
_ driver.RenameResult = (*AliDoc)(nil)
_ driver.Copy = (*AliDoc)(nil)
_ driver.Remove = (*AliDoc)(nil)
)
-23
View File
@@ -1,23 +0,0 @@
package alidoc
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootID
Cookie string `json:"cookie" type:"text" required:"true" help:"钉钉文档网页 Cookie"`
}
var config = driver.Config{
Name: "AliDoc",
LocalSort: true,
DefaultRoot: "",
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &AliDoc{}
})
}
-87
View File
@@ -1,87 +0,0 @@
package alidoc
type apiResp struct {
Status int `json:"status"`
IsSuccess bool `json:"isSuccess"`
Message string `json:"message"`
Msg string `json:"msg"`
}
func (r apiResp) ErrMessage() string {
if r.Message != "" {
return r.Message
}
if r.Msg != "" {
return r.Msg
}
return ""
}
type listResp struct {
apiResp
Data listData `json:"data"`
}
type listData struct {
Children []dentry `json:"children"`
}
type dentry struct {
DentryType string `json:"dentryType"`
DentryUUID string `json:"dentryUuid"`
ParentDentryUUID string `json:"parentDentryUuid"`
Name string `json:"name"`
Path string `json:"path"`
FileSize int64 `json:"fileSize"`
CreatedTime int64 `json:"createdTime"`
UpdatedTime int64 `json:"updatedTime"`
ContentType string `json:"contentType"`
Extension string `json:"extension"`
DentryStatistic struct {
ChildrenCount int `json:"childrenCount"`
} `json:"dentryStatistic"`
URL struct {
PCChildAppPreviewURL string `json:"pcChildAppPreviewUrl"`
PCChildAppURL string `json:"pcChildAppUrl"`
} `json:"url"`
}
type downloadResp struct {
apiResp
Data downloadData `json:"data"`
}
type downloadData struct {
OSSURLPreSignatureInfo struct {
PreSignURLs []string `json:"preSignUrls"`
} `json:"ossUrlPreSignatureInfo"`
}
type uploadInfoResp struct {
apiResp
Data uploadInfoData `json:"data"`
}
type uploadInfoData struct {
CurrentTimestamp int64 `json:"currentTimestamp"`
FileUploadProtocolConfig uploadProtocolConfig `json:"fileUploadProtocolConfig"`
STSSignatureInfo uploadSTSSignatureInfo `json:"stsSignatureInfo"`
UploadKey string `json:"uploadKey"`
UploadType string `json:"uploadType"`
}
type uploadProtocolConfig struct {
MinPartSize int64 `json:"minPartSize"`
}
type uploadSTSSignatureInfo struct {
AccelerateCname string `json:"accelerateCname"`
AccessKeyID string `json:"accessKeyId"`
AccessKeySecret string `json:"accessKeySecret"`
AccessToken string `json:"accessToken"`
AccessTokenExpiration int64 `json:"accessTokenExpiration"`
Bucket string `json:"bucket"`
Cname string `json:"cname"`
EndPoint string `json:"endPoint"`
ObjectKey string `json:"objectKey"`
}
-263
View File
@@ -1,263 +0,0 @@
package alidoc
import (
"context"
"fmt"
"io"
"math"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
netutil "github.com/OpenListTeam/OpenList/v4/internal/net"
streamPkg "github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
"github.com/avast/retry-go"
"github.com/google/uuid"
)
const (
defaultAliDocMultipartThreshold = 16 * 1024 * 1024
defaultAliDocPartSize = 100 * 1024
maxAliDocMultipartParts = 10000
)
func (d *AliDoc) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
size := file.GetSize()
useMultipart := size > defaultAliDocMultipartThreshold
info, err := d.getUploadInfo(ctx, dstDir.GetID(), file.GetName(), size, useMultipart)
if err != nil {
return err
}
if size > 0 {
partSize := calcAliDocPartSize(size, info.Data.FileUploadProtocolConfig.MinPartSize)
if size > partSize && !useMultipart {
useMultipart = true
info, err = d.getUploadInfo(ctx, dstDir.GetID(), file.GetName(), size, true)
if err != nil {
return err
}
}
}
if useMultipart {
err = d.multipartUpload(ctx, file, size, info, up)
} else {
err = d.singleUpload(ctx, file, size, info, up)
}
if err != nil {
return err
}
if err := d.commitUpload(ctx, dstDir.GetID(), file.GetName(), size, info.Data.UploadKey); err != nil {
return err
}
return nil
}
func (d *AliDoc) getUploadInfo(ctx context.Context, parentDentryUUID, name string, fileSize int64, multipart bool) (uploadInfoResp, error) {
var result uploadInfoResp
body := map[string]interface{}{
"uploadType": "STS_SIGNATURE",
"supportUploadTypes": []string{"STS_SIGNATURE", "HTTP_TO_CENTER"},
"parentDentryUuid": parentDentryUUID,
"fileSize": fileSize,
"name": name,
"multipart": multipart,
}
resp, err := d.request(ctx).
SetBody(body).
SetResult(&result).
SetError(&result).
Post(apiBase + "/box/api/v2/file/uploadinfo")
if err != nil {
return result, err
}
if err := checkResp(resp, result.apiResp); err != nil {
return result, err
}
if strings.TrimSpace(result.Data.STSSignatureInfo.Bucket) == "" {
return result, fmt.Errorf("empty upload bucket")
}
return result, nil
}
func (d *AliDoc) commitUpload(ctx context.Context, parentDentryUUID, name string, fileSize int64, uploadKey string) error {
uploadKey = strings.TrimSpace(uploadKey)
if uploadKey == "" {
return fmt.Errorf("empty upload key")
}
var result apiResp
body := map[string]interface{}{
"parentDentryUuid": parentDentryUUID,
"uploadKey": uploadKey,
"fileSize": fileSize,
"name": name,
"toPrevDentryUuid": nil,
"toNextDentryUuid": nil,
"batchId": uuid.NewString(),
"batchUploadType": 1,
"batchParentDentryUuid": parentDentryUUID,
}
resp, err := d.request(ctx).
SetBody(body).
SetResult(&result).
SetError(&result).
Post(apiBase + "/box/api/v2/file/commit")
if err != nil {
return err
}
return checkResp(resp, result)
}
func calcAliDocPartSize(fileSize, minPartSize int64) int64 {
partSize := minPartSize
if partSize <= 0 {
partSize = defaultAliDocPartSize
}
if fileSize <= 0 {
return partSize
}
minRequired := int64(math.Ceil(float64(fileSize) / maxAliDocMultipartParts))
if minRequired > partSize {
partSize = minRequired
}
return partSize
}
func (d *AliDoc) singleUpload(ctx context.Context, src model.FileStreamer, size int64, info uploadInfoResp, up driver.UpdateProgress) error {
bucket, objectKey, err := d.newOSSBucket(info)
if err != nil {
return err
}
err = bucket.PutObject(
objectKey,
driver.NewLimitedUploadStream(ctx, io.TeeReader(src, driver.NewProgress(size, up))),
)
if err != nil {
return err
}
up(100)
return nil
}
func (d *AliDoc) multipartUpload(ctx context.Context, src model.FileStreamer, size int64, info uploadInfoResp, up driver.UpdateProgress) error {
bucket, objectKey, err := d.newOSSBucket(info)
if err != nil {
return err
}
imur, err := bucket.InitiateMultipartUpload(objectKey, oss.Sequential())
if err != nil {
return err
}
partSize := calcAliDocPartSize(size, info.Data.FileUploadProtocolConfig.MinPartSize)
partNum := int((size + partSize - 1) / partSize)
parts := make([]oss.UploadPart, 0, partNum)
ss, err := streamPkg.NewStreamSectionReader(src, int(partSize), &up)
if err != nil {
return err
}
var offset int64
for partNumber := 1; partNumber <= partNum; partNumber++ {
if err := ctx.Err(); err != nil {
return err
}
length := partSize
if remain := size - offset; remain < length {
length = remain
}
reader, err := ss.GetSectionReader(offset, length)
if err != nil {
return err
}
var part oss.UploadPart
var uploadErr error
err = retry.Do(func() error {
if _, err := reader.Seek(0, io.SeekStart); err != nil {
return err
}
part, uploadErr = bucket.UploadPart(
imur,
driver.NewLimitedUploadStream(ctx, reader),
length,
partNumber,
)
return uploadErr
},
retry.Context(ctx),
retry.Attempts(3),
retry.DelayType(retry.BackOffDelay),
retry.Delay(time.Second),
)
ss.FreeSectionReader(reader)
if err != nil {
return err
}
parts = append(parts, part)
up(100 * float64(len(parts)) / float64(partNum+1))
offset += length
}
_, err = bucket.CompleteMultipartUpload(imur, parts)
if err != nil {
return err
}
up(100)
return nil
}
func (d *AliDoc) newOSSBucket(info uploadInfoResp) (*oss.Bucket, string, error) {
sts := info.Data.STSSignatureInfo
objectKey := strings.TrimSpace(sts.ObjectKey)
if objectKey == "" {
objectKey = strings.TrimSpace(info.Data.UploadKey)
}
if objectKey == "" {
return nil, "", fmt.Errorf("empty upload object key")
}
endpoint, useCname := pickAliDocOSSEndpoint(sts)
if endpoint == "" {
return nil, "", fmt.Errorf("empty upload endpoint")
}
if !strings.HasPrefix(endpoint, "http://") && !strings.HasPrefix(endpoint, "https://") {
endpoint = "https://" + endpoint
}
options := []oss.ClientOption{oss.SecurityToken(sts.AccessToken)}
if useCname {
options = append(options, oss.UseCname(true))
}
client, err := netutil.NewOSSClient(
endpoint,
sts.AccessKeyID,
sts.AccessKeySecret,
options...,
)
if err != nil {
return nil, "", err
}
bucket, err := client.Bucket(sts.Bucket)
if err != nil {
return nil, "", err
}
return bucket, objectKey, nil
}
func pickAliDocOSSEndpoint(sts uploadSTSSignatureInfo) (endpoint string, useCname bool) {
if endpoint = strings.TrimSpace(sts.EndPoint); endpoint != "" {
return endpoint, false
}
if endpoint = strings.TrimSpace(sts.Cname); endpoint != "" {
return endpoint, true
}
if endpoint = strings.TrimSpace(sts.AccelerateCname); endpoint != "" {
return endpoint, true
}
return "", false
}
-135
View File
@@ -1,135 +0,0 @@
package alidoc
import (
"context"
"fmt"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/go-resty/resty/v2"
)
const apiBase = "https://alidocs.dingtalk.com"
func (d *AliDoc) request(ctx context.Context) *resty.Request {
return d.client.R().
SetContext(ctx).
SetHeader("Cookie", d.Cookie).
SetHeader("Accept", "application/json, text/plain, */*").
SetHeader("Referer", apiBase+"/").
SetHeader("Origin", apiBase)
}
func msToTime(v int64) time.Time {
if v <= 0 {
return time.Time{}
}
return time.UnixMilli(v)
}
func checkResp(resp *resty.Response, result apiResp) error {
if resp != nil && resp.IsError() {
if msg := result.ErrMessage(); msg != "" {
return fmt.Errorf("%s", msg)
}
return fmt.Errorf("http error: %d", resp.StatusCode())
}
if !result.IsSuccess || result.Status != 200 {
msg := result.ErrMessage()
if msg == "" {
msg = "request failed"
}
return fmt.Errorf("%s", msg)
}
return nil
}
func toObj(item dentry) model.Obj {
return &model.Object{
ID: item.DentryUUID,
Name: item.Name,
Size: item.FileSize,
Modified: msToTime(item.UpdatedTime),
Ctime: msToTime(item.CreatedTime),
IsFolder: item.DentryType == "folder",
}
}
func firstDownloadURL(resp downloadResp) (string, error) {
if len(resp.Data.OSSURLPreSignatureInfo.PreSignURLs) == 0 {
return "", fmt.Errorf("empty download url")
}
return resp.Data.OSSURLPreSignatureInfo.PreSignURLs[0], nil
}
func newClient() *resty.Client {
client := base.NewRestyClient()
client.SetHeader("User-Agent", base.UserAgent)
return client
}
func (d *AliDoc) post(ctx context.Context, path string, body interface{}) error {
var result apiResp
resp, err := d.request(ctx).
SetBody(body).
SetResult(&result).
SetError(&result).
Post(apiBase + path)
if err != nil {
return err
}
return checkResp(resp, result)
}
func (d *AliDoc) checkCookie(ctx context.Context) error {
var result apiResp
resp, err := d.request(ctx).
SetResult(&result).
SetError(&result).
Get(apiBase + "/portal/api/v1/mine/info")
if err != nil {
return err
}
return checkResp(resp, result)
}
func (d *AliDoc) list(ctx context.Context, dentryUUID string) ([]dentry, error) {
var result listResp
resp, err := d.request(ctx).
SetQueryParam("dentryUuid", dentryUUID).
SetQueryParam("withParentAncestors", "true").
SetQueryParam("orderType", "SORT_KEY").
SetQueryParam("sortType", "desc").
SetQueryParam("listDentrySource", "2").
SetQueryParam("pageSize", "1000").
SetResult(&result).
SetError(&result).
Get(apiBase + "/box/api/v2/dentry/list")
if err != nil {
return nil, err
}
if err := checkResp(resp, result.apiResp); err != nil {
return nil, err
}
return result.Data.Children, nil
}
func (d *AliDoc) download(ctx context.Context, dentryUUID string) (downloadResp, error) {
var result downloadResp
resp, err := d.request(ctx).
SetQueryParam("dentryUuid", dentryUUID).
SetQueryParam("version", "1").
SetQueryParam("supportDownloadTypes", "URL_PRE_SIGNATURE,HTTP_TO_CENTER").
SetQueryParam("downloadType", "URL_PRE_SIGNATURE").
SetResult(&result).
SetError(&result).
Get(apiBase + "/box/api/v2/file/download")
if err != nil {
return result, err
}
if err := checkResp(resp, result.apiResp); err != nil {
return result, err
}
return result, nil
}
-379
View File
@@ -1,379 +0,0 @@
package alist_v3
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"path"
"strings"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/conf"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/OpenListTeam/OpenList/v4/server/common"
"github.com/go-resty/resty/v2"
log "github.com/sirupsen/logrus"
)
type AListV3 struct {
model.Storage
Addition
}
func (d *AListV3) Config() driver.Config {
return config
}
func (d *AListV3) GetAddition() driver.Additional {
return &d.Addition
}
func (d *AListV3) Init(ctx context.Context) error {
d.Addition.Address = strings.TrimSuffix(d.Addition.Address, "/")
var resp common.Resp[MeResp]
_, _, err := d.request("/me", http.MethodGet, func(req *resty.Request) {
req.SetResult(&resp)
})
if err != nil {
return err
}
// if the username is not empty and the username is not the same as the current username, then login again
if d.Username != resp.Data.Username {
err = d.login()
if err != nil {
return err
}
}
// re-get the user info
_, _, err = d.request("/me", http.MethodGet, func(req *resty.Request) {
req.SetResult(&resp)
})
if err != nil {
return err
}
if utils.SliceContains(resp.Data.Role, model.GUEST) {
u := d.Address + "/api/public/settings"
res, err := base.RestyClient.R().Get(u)
if err != nil {
return err
}
allowMounted := utils.Json.Get(res.Body(), "data", conf.AllowMounted).ToString() == "true"
if !allowMounted {
return fmt.Errorf("the site does not allow mounted")
}
}
return err
}
func (d *AListV3) Drop(ctx context.Context) error {
return nil
}
func (d *AListV3) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
var resp common.Resp[FsListResp]
_, _, err := d.request("/fs/list", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ListReq{
PageReq: model.PageReq{
Page: 1,
PerPage: 0,
},
Path: dir.GetPath(),
Password: d.MetaPassword,
Refresh: false,
})
})
if err != nil {
return nil, err
}
var files []model.Obj
for _, f := range resp.Data.Content {
file := model.ObjThumb{
Object: model.Object{
Name: f.Name,
Modified: f.Modified,
Ctime: f.Created,
Size: f.Size,
IsFolder: f.IsDir,
HashInfo: utils.FromString(f.HashInfo),
},
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
}
files = append(files, &file)
}
return files, nil
}
func (d *AListV3) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
var resp common.Resp[FsGetResp]
headers := map[string]string{
"User-Agent": base.UserAgent,
}
// if PassUAToUpsteam is true, then pass the user-agent to the upstream
if d.PassUAToUpsteam {
userAgent := args.Header.Get("user-agent")
if userAgent != "" {
headers["User-Agent"] = userAgent
}
}
// if PassIPToUpsteam is true, then pass the ip address to the upstream
if d.PassIPToUpsteam {
ip := args.IP
if ip != "" {
headers["X-Forwarded-For"] = ip
headers["X-Real-Ip"] = ip
}
}
_, _, err := d.request("/fs/get", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(FsGetReq{
Path: file.GetPath(),
Password: d.MetaPassword,
}).SetHeaders(headers)
})
if err != nil {
return nil, err
}
return &model.Link{
URL: resp.Data.RawURL,
}, nil
}
func (d *AListV3) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
_, _, err := d.request("/fs/mkdir", http.MethodPost, func(req *resty.Request) {
req.SetBody(MkdirOrLinkReq{
Path: path.Join(parentDir.GetPath(), dirName),
})
})
return err
}
func (d *AListV3) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
_, _, err := d.request("/fs/move", http.MethodPost, func(req *resty.Request) {
req.SetBody(MoveCopyReq{
SrcDir: path.Dir(srcObj.GetPath()),
DstDir: dstDir.GetPath(),
Names: []string{srcObj.GetName()},
})
})
return err
}
func (d *AListV3) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
_, _, err := d.request("/fs/rename", http.MethodPost, func(req *resty.Request) {
req.SetBody(RenameReq{
Path: srcObj.GetPath(),
Name: newName,
})
})
return err
}
func (d *AListV3) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
_, _, err := d.request("/fs/copy", http.MethodPost, func(req *resty.Request) {
req.SetBody(MoveCopyReq{
SrcDir: path.Dir(srcObj.GetPath()),
DstDir: dstDir.GetPath(),
Names: []string{srcObj.GetName()},
})
})
return err
}
func (d *AListV3) Remove(ctx context.Context, obj model.Obj) error {
_, _, err := d.request("/fs/remove", http.MethodPost, func(req *resty.Request) {
req.SetBody(RemoveReq{
Dir: path.Dir(obj.GetPath()),
Names: []string{obj.GetName()},
})
})
return err
}
func (d *AListV3) Put(ctx context.Context, dstDir model.Obj, s model.FileStreamer, up driver.UpdateProgress) error {
reader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
Reader: s,
UpdateProgress: up,
})
req, err := http.NewRequestWithContext(ctx, http.MethodPut, d.Address+"/api/fs/put", reader)
if err != nil {
return err
}
req.Header.Set("Authorization", d.Token)
req.Header.Set("File-Path", path.Join(dstDir.GetPath(), s.GetName()))
req.Header.Set("Password", d.MetaPassword)
if md5 := s.GetHash().GetHash(utils.MD5); len(md5) > 0 {
req.Header.Set("X-File-Md5", md5)
}
if sha1 := s.GetHash().GetHash(utils.SHA1); len(sha1) > 0 {
req.Header.Set("X-File-Sha1", sha1)
}
if sha256 := s.GetHash().GetHash(utils.SHA256); len(sha256) > 0 {
req.Header.Set("X-File-Sha256", sha256)
}
req.ContentLength = s.GetSize()
// client := base.NewHttpClient()
// client.Timeout = time.Hour * 6
res, err := base.HttpClient.Do(req)
if err != nil {
return err
}
bytes, err := io.ReadAll(res.Body)
if err != nil {
return err
}
log.Debugf("[openlist] response body: %s", string(bytes))
if res.StatusCode >= 400 {
return fmt.Errorf("request failed, status: %s", res.Status)
}
code := utils.Json.Get(bytes, "code").ToInt()
if code != 200 {
if code == 401 || code == 403 {
err = d.login()
if err != nil {
return err
}
}
return fmt.Errorf("request failed,code: %d, message: %s", code, utils.Json.Get(bytes, "message").ToString())
}
return nil
}
func (d *AListV3) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
if !d.ForwardArchiveReq {
return nil, errs.NotImplement
}
var resp common.Resp[ArchiveMetaResp]
_, code, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ArchiveMetaReq{
ArchivePass: args.Password,
Password: d.MetaPassword,
Path: obj.GetPath(),
Refresh: false,
})
})
if code == 202 {
return nil, errs.WrongArchivePassword
}
if err != nil {
return nil, err
}
var tree []model.ObjTree
if resp.Data.Content != nil {
tree = make([]model.ObjTree, 0, len(resp.Data.Content))
for _, content := range resp.Data.Content {
tree = append(tree, &content)
}
}
return &model.ArchiveMetaInfo{
Comment: resp.Data.Comment,
Encrypted: resp.Data.Encrypted,
Tree: tree,
}, nil
}
func (d *AListV3) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
if !d.ForwardArchiveReq {
return nil, errs.NotImplement
}
var resp common.Resp[ArchiveListResp]
_, code, err := d.request("/fs/archive/list", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ArchiveListReq{
ArchiveMetaReq: ArchiveMetaReq{
ArchivePass: args.Password,
Password: d.MetaPassword,
Path: obj.GetPath(),
Refresh: false,
},
PageReq: model.PageReq{
Page: 1,
PerPage: 0,
},
InnerPath: args.InnerPath,
})
})
if code == 202 {
return nil, errs.WrongArchivePassword
}
if err != nil {
return nil, err
}
var files []model.Obj
for _, f := range resp.Data.Content {
file := model.ObjThumb{
Object: model.Object{
Name: f.Name,
Modified: f.Modified,
Ctime: f.Created,
Size: f.Size,
IsFolder: f.IsDir,
HashInfo: utils.FromString(f.HashInfo),
},
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
}
files = append(files, &file)
}
return files, nil
}
func (d *AListV3) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
if !d.ForwardArchiveReq {
return nil, errs.NotSupport
}
var resp common.Resp[ArchiveMetaResp]
_, _, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ArchiveMetaReq{
ArchivePass: args.Password,
Password: d.MetaPassword,
Path: obj.GetPath(),
Refresh: false,
})
})
if err != nil {
return nil, err
}
return &model.Link{
URL: fmt.Sprintf("%s?inner=%s&pass=%s&sign=%s",
resp.Data.RawURL,
utils.EncodePath(args.InnerPath, true),
url.QueryEscape(args.Password),
resp.Data.Sign),
}, nil
}
func (d *AListV3) ArchiveDecompress(ctx context.Context, srcObj, dstDir model.Obj, args model.ArchiveDecompressArgs) error {
if !d.ForwardArchiveReq {
return errs.NotImplement
}
dir, name := path.Split(srcObj.GetPath())
_, _, err := d.request("/fs/archive/decompress", http.MethodPost, func(req *resty.Request) {
req.SetBody(DecompressReq{
ArchivePass: args.Password,
CacheFull: args.CacheFull,
DstDir: dstDir.GetPath(),
InnerPath: args.InnerPath,
Name: []string{name},
PutIntoNewDir: args.PutIntoNewDir,
SrcDir: dir,
})
})
return err
}
func (d *AListV3) ResolveLinkCacheMode(_ string) driver.LinkCacheMode {
var mode driver.LinkCacheMode
if d.PassIPToUpsteam {
mode |= driver.LinkCacheIP
}
if d.PassUAToUpsteam {
mode |= driver.LinkCacheUA
}
return mode
}
var _ driver.Driver = (*AListV3)(nil)
-182
View File
@@ -1,182 +0,0 @@
package alist_v3
import (
"encoding/json"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
type ListReq struct {
model.PageReq
Path string `json:"path" form:"path"`
Password string `json:"password" form:"password"`
Refresh bool `json:"refresh"`
}
type ObjResp struct {
Name string `json:"name"`
Size int64 `json:"size"`
IsDir bool `json:"is_dir"`
Modified time.Time `json:"modified"`
Created time.Time `json:"created"`
Sign string `json:"sign"`
Thumb string `json:"thumb"`
Type int `json:"type"`
HashInfo string `json:"hashinfo"`
}
type FsListResp struct {
Content []ObjResp `json:"content"`
Total int64 `json:"total"`
Readme string `json:"readme"`
Write bool `json:"write"`
Provider string `json:"provider"`
}
type FsGetReq struct {
Path string `json:"path" form:"path"`
Password string `json:"password" form:"password"`
}
type FsGetResp struct {
ObjResp
RawURL string `json:"raw_url"`
Readme string `json:"readme"`
Provider string `json:"provider"`
Related []ObjResp `json:"related"`
}
type MkdirOrLinkReq struct {
Path string `json:"path" form:"path"`
}
type MoveCopyReq struct {
SrcDir string `json:"src_dir"`
DstDir string `json:"dst_dir"`
Names []string `json:"names"`
}
type RenameReq struct {
Path string `json:"path"`
Name string `json:"name"`
}
type RemoveReq struct {
Dir string `json:"dir"`
Names []string `json:"names"`
}
type LoginResp struct {
Token string `json:"token"`
}
type MeResp struct {
Id int `json:"id"`
Username string `json:"username"`
Password string `json:"password"`
BasePath string `json:"base_path"`
Role IntSlice `json:"role"`
Disabled bool `json:"disabled"`
Permission int `json:"permission"`
SsoId string `json:"sso_id"`
Otp bool `json:"otp"`
}
type IntSlice []int
func (s *IntSlice) UnmarshalJSON(b []byte) error {
var i int
if json.Unmarshal(b, &i) == nil {
*s = []int{i}
return nil
}
return json.Unmarshal(b, (*[]int)(s))
}
type ArchiveMetaReq struct {
ArchivePass string `json:"archive_pass"`
Password string `json:"password"`
Path string `json:"path"`
Refresh bool `json:"refresh"`
}
type TreeResp struct {
ObjResp
Children []TreeResp `json:"children"`
hashCache *utils.HashInfo
}
func (t *TreeResp) GetSize() int64 {
return t.Size
}
func (t *TreeResp) GetName() string {
return t.Name
}
func (t *TreeResp) ModTime() time.Time {
return t.Modified
}
func (t *TreeResp) CreateTime() time.Time {
return t.Created
}
func (t *TreeResp) IsDir() bool {
return t.ObjResp.IsDir
}
func (t *TreeResp) GetHash() utils.HashInfo {
return utils.FromString(t.HashInfo)
}
func (t *TreeResp) GetID() string {
return ""
}
func (t *TreeResp) GetPath() string {
return ""
}
func (t *TreeResp) GetChildren() []model.ObjTree {
ret := make([]model.ObjTree, 0, len(t.Children))
for _, child := range t.Children {
ret = append(ret, &child)
}
return ret
}
func (t *TreeResp) Thumb() string {
return t.ObjResp.Thumb
}
type ArchiveMetaResp struct {
Comment string `json:"comment"`
Encrypted bool `json:"encrypted"`
Content []TreeResp `json:"content"`
RawURL string `json:"raw_url"`
Sign string `json:"sign"`
}
type ArchiveListReq struct {
model.PageReq
ArchiveMetaReq
InnerPath string `json:"inner_path"`
}
type ArchiveListResp struct {
Content []ObjResp `json:"content"`
Total int64 `json:"total"`
}
type DecompressReq struct {
ArchivePass string `json:"archive_pass"`
CacheFull bool `json:"cache_full"`
DstDir string `json:"dst_dir"`
InnerPath string `json:"inner_path"`
Name []string `json:"name"`
PutIntoNewDir bool `json:"put_into_new_dir"`
SrcDir string `json:"src_dir"`
}
-65
View File
@@ -1,65 +0,0 @@
package alist_v3
import (
"fmt"
"net/http"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/OpenListTeam/OpenList/v4/server/common"
"github.com/go-resty/resty/v2"
log "github.com/sirupsen/logrus"
)
func (d *AListV3) login() error {
if d.Username == "" {
return nil
}
var resp common.Resp[LoginResp]
_, _, err := d.request("/auth/login", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(base.Json{
"username": d.Username,
"password": d.Password,
})
})
if err != nil {
return err
}
d.Token = resp.Data.Token
op.MustSaveDriverStorage(d)
return nil
}
func (d *AListV3) request(api, method string, callback base.ReqCallback, retry ...bool) ([]byte, int, error) {
url := d.Address + "/api" + api
req := base.RestyClient.R()
req.SetHeader("Authorization", d.Token)
if callback != nil {
callback(req)
}
res, err := req.Execute(method, url)
if err != nil {
code := 0
if res != nil {
code = res.StatusCode()
}
return nil, code, err
}
log.Debugf("[openlist] response body: %s", res.String())
if res.StatusCode() >= 400 {
return nil, res.StatusCode(), fmt.Errorf("request failed, status: %s", res.Status())
}
code := utils.Json.Get(res.Body(), "code").ToInt()
if code != 200 {
if (code == 401 || code == 403) && !utils.IsBool(retry...) {
err = d.login()
if err != nil {
return nil, code, err
}
return d.request(api, method, callback, true)
}
return nil, code, fmt.Errorf("request failed,code: %d, message: %s", code, utils.Json.Get(res.Body(), "message").ToString())
}
return res.Body(), 200, nil
}
-169
View File
@@ -1,169 +0,0 @@
package autoindex
import (
"context"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/antchfx/htmlquery"
"github.com/antchfx/xpath"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
type AutoIndex struct {
model.Storage
Addition
itemXPath *xpath.Expr
nameXPath *xpath.Expr
modifiedXPath *xpath.Expr
sizeXPath *xpath.Expr
ignores map[string]any
}
func (d *AutoIndex) Config() driver.Config {
return config
}
func (d *AutoIndex) GetAddition() driver.Additional {
return &d.Addition
}
func (d *AutoIndex) Init(ctx context.Context) error {
var err error
d.itemXPath, err = xpath.Compile(d.ItemXPath)
if err != nil {
return errors.WithMessage(err, "failed to compile Item XPath")
}
d.nameXPath, err = xpath.Compile(d.NameXPath)
if err != nil {
return errors.WithMessage(err, "failed to compile Name XPath")
}
if len(d.ModifiedXPath) > 0 {
d.modifiedXPath, err = xpath.Compile(d.ModifiedXPath)
if err != nil {
return errors.WithMessage(err, "failed to compile Modified XPath")
}
}
if len(d.SizeXPath) > 0 {
d.sizeXPath, err = xpath.Compile(d.SizeXPath)
if err != nil {
return errors.WithMessage(err, "failed to compile Size XPath")
}
}
ignores := strings.Split(d.IgnoreFileNames, "\n")
d.ignores = make(map[string]any, len(ignores))
for _, i := range ignores {
i = strings.TrimSpace(i)
if len(i) == 0 {
continue
}
d.ignores[i] = struct{}{}
}
hasScheme := strings.Contains(d.URL, "://")
hasSuffix := strings.HasSuffix(d.URL, "/")
if !hasScheme || !hasSuffix {
if !hasSuffix {
d.URL = d.URL + "/"
}
if !hasScheme {
d.URL = "https://" + d.URL
}
op.MustSaveDriverStorage(d)
}
return nil
}
func (d *AutoIndex) Drop(ctx context.Context) error {
return nil
}
func (d *AutoIndex) GetRoot(ctx context.Context) (model.Obj, error) {
return &model.Object{
Name: op.RootName,
Path: d.URL,
Modified: d.Modified,
Mask: model.Locked,
IsFolder: true,
}, nil
}
func (d *AutoIndex) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
res, err := base.RestyClient.R().
SetContext(ctx).
SetDoNotParseResponse(true).
Get(dir.GetPath())
if err != nil {
return nil, errors.WithMessagef(err, "failed to get url [%s]", dir.GetPath())
}
defer res.RawResponse.Body.Close()
doc, err := htmlquery.Parse(res.RawBody())
if err != nil {
return nil, errors.WithMessagef(err, "failed to parse [%s]", dir.GetPath())
}
itemsIter := d.itemXPath.Select(htmlquery.CreateXPathNavigator(doc))
var objs []model.Obj
for itemsIter.MoveNext() {
nameFull, err := parseString(d.nameXPath.Evaluate(itemsIter.Current().Copy()))
if err != nil {
log.Warnf("skip invalid name evaluating result: %v", err)
continue
}
nameFull = strings.TrimSpace(nameFull)
name, isDir := strings.CutSuffix(nameFull, "/")
if _, ok := d.ignores[name]; ok {
continue
}
var size int64 = 0
exact := false
modified := time.Now()
if d.sizeXPath != nil {
size, exact, err = parseSize(d.sizeXPath.Evaluate(itemsIter.Current().Copy()))
if err != nil {
log.Errorf("failed to parse size of %s: %v", name, err)
}
}
if d.modifiedXPath != nil {
modified, err = parseTime(d.modifiedXPath.Evaluate(itemsIter.Current().Copy()), d.ModifiedTimeFormat)
if err != nil {
log.Errorf("failed to parse modified time of %s: %v", name, err)
}
}
var o model.Obj = &model.Object{
Name: name,
IsFolder: isDir,
Path: dir.GetPath() + nameFull,
Modified: modified,
Size: size,
}
if exact {
o = &exactSizeObj{Obj: o}
}
objs = append(objs, o)
}
return objs, nil
}
func (d *AutoIndex) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if _, ok := file.(*exactSizeObj); ok || args.Redirect {
return &model.Link{URL: file.GetPath()}, nil
}
res, err := base.RestyClient.R().
SetContext(ctx).
SetDoNotParseResponse(true).
Head(file.GetPath())
if err != nil {
return nil, errors.WithMessagef(err, "failed to head [%s]", file.GetPath())
}
_ = res.RawResponse.Body.Close()
return &model.Link{
URL: file.GetPath(),
ContentLength: res.RawResponse.ContentLength,
}, nil
}
var _ driver.Driver = (*AutoIndex)(nil)
-29
View File
@@ -1,29 +0,0 @@
package autoindex
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
URL string `json:"url" required:"true"`
ItemXPath string `json:"item_xpath" required:"true"`
NameXPath string `json:"name_xpath" required:"true"`
ModifiedXPath string `json:"modified_xpath"`
SizeXPath string `json:"size_xpath"`
IgnoreFileNames string `json:"ignore_file_names" type:"text" default:".\n..\nParent Directory\nUp"`
ModifiedTimeFormat string `json:"modified_time_format" default:"02-Jan-2006 15:04" help:"Must be based on the time point Mon Jan 2 15:04:05 -0700 MST 2006"`
}
var config = driver.Config{
Name: "AutoIndex",
LocalSort: true,
CheckStatus: true,
NoUpload: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &AutoIndex{}
})
}
-13
View File
@@ -1,13 +0,0 @@
package autoindex
import (
"fmt"
"github.com/OpenListTeam/OpenList/v4/internal/model"
)
var (
errEmptyEvaluateResult = fmt.Errorf("empty result")
)
type exactSizeObj struct{ model.Obj }
-116
View File
@@ -1,116 +0,0 @@
package autoindex
import (
"fmt"
"strconv"
"strings"
"time"
"github.com/antchfx/xpath"
"github.com/pkg/errors"
)
var units = map[string]int64{
"": 1,
"b": 1,
"byte": 1,
"bytes": 1,
"k": 1 << 10,
"kb": 1 << 10,
"kib": 1 << 10,
"m": 1 << 20,
"mb": 1 << 20,
"mib": 1 << 20,
"g": 1 << 30,
"gb": 1 << 30,
"gib": 1 << 30,
"t": 1 << 40,
"tb": 1 << 40,
"tib": 1 << 40,
"p": 1 << 50,
"pb": 1 << 50,
"pib": 1 << 50,
}
func splitUnit(s string) (string, string) {
for i := len(s) - 1; i >= 0; i-- {
if s[i] >= '0' && s[i] <= '9' {
return strings.TrimSpace(s[:i+1]), strings.TrimSpace(s[i+1:])
}
}
return "", s
}
func parseSize(a any) (int64, bool, error) {
// 第二个返回值exact表示大小是否精确
if f, ok := a.(float64); ok {
return int64(f), false, nil
}
s, err := parseString(a)
if errors.Is(err, errEmptyEvaluateResult) {
// 可能是错误,也可能确实大小为0
// 如果确实大小为0,大概率不会下载,exact返回false也不会有什么性能损失
// 如果是错误,exact返回true会导致本地代理出错,综合来看返回false更好
return 0, false, nil
}
if err != nil {
return 0, false, err
}
s = strings.TrimSpace(s)
if s == "-" {
return 0, false, nil
}
nbs, unit := splitUnit(s)
mul, ok := units[strings.ToLower(unit)]
exact := mul == 1
if !ok {
mul = 1
// 推测无单位,exact应为false
}
nb, err := strconv.ParseInt(nbs, 10, 64)
if err != nil {
fnb, err := strconv.ParseFloat(nbs, 64)
if err != nil {
return 0, false, fmt.Errorf("failed to convert %s to number", nbs)
}
nb = int64(fnb * float64(mul))
exact = false
} else {
nb = nb * mul
}
return nb, exact, nil
}
func parseString(res any) (string, error) {
if r, ok := res.(string); ok {
if len(r) == 0 {
return "", errEmptyEvaluateResult
}
return r, nil
}
n, ok := res.(*xpath.NodeIterator)
if !ok {
return "", fmt.Errorf("unsupported evaluating result")
}
if !n.MoveNext() {
return "", fmt.Errorf("no matched nodes")
}
ns := n.Current().Value()
if len(ns) == 0 {
return "", errEmptyEvaluateResult
}
return ns, nil
}
func parseTime(res any, format string) (time.Time, error) {
s, err := parseString(res)
if err != nil {
return time.Now(), err
}
s = strings.TrimSpace(s)
t, err := time.Parse(format, s)
if err != nil {
return time.Now(), errors.WithMessagef(err, "failed to convert %s to time", s)
}
return t, nil
}
-49
View File
@@ -1,49 +0,0 @@
package autoindex
import (
"testing"
)
type wantType struct {
v int64
exact bool
error bool
}
func TestParseSize(t *testing.T) {
tests := []struct {
input string
want wantType
}{
{"100", wantType{100, true, false}},
{"1k", wantType{1024, false, false}},
{"1kb", wantType{1024, false, false}},
{"1K", wantType{1024, false, false}}, // case insensitive
{"1.5m", wantType{1572864, false, false}}, // 1.5 * 1024^2
{"500 bytes", wantType{500, true, false}},
{"-", wantType{0, false, false}},
{"", wantType{0, false, false}},
{"abc", wantType{0, false, true}},
{"1.5GB", wantType{1610612736, false, false}}, // 1.5 * 1024^3
{"2t", wantType{2199023255552, false, false}}, // 2 * 1024^4
{"1p", wantType{1125899906842624, false, false}}, // 1 * 1024^5
{"0", wantType{0, true, false}},
{" 100 ", wantType{100, true, false}}, // trimmed
{"100b", wantType{100, true, false}},
{"1gib", wantType{1073741824, false, false}}, // 1024^3
{"1z", wantType{1, false, false}}, // invalid unit, mul=1
{"1.5", wantType{1, false, false}}, // float without unit, truncated
{"2.7k", wantType{2764, false, false}}, // 2.7 * 1024 truncated
{"1.0g", wantType{1073741824, false, false}}, // 1.0 * 1024^3
{"invalid", wantType{0, false, true}},
{"123xyz", wantType{123, false, false}}, // unit not found, mul=1
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
got, exact, err := parseSize(tt.input)
if got != tt.want.v || exact != tt.want.exact || (err != nil) != tt.want.error {
t.Errorf("ParseSize(%q) = (%d, %t, %t), want (%d, %t, %t)", tt.input, got, exact, err != nil, tt.want.v, tt.want.exact, tt.want.error)
}
})
}
}
-545
View File
@@ -1,545 +0,0 @@
package chunk
import (
"bytes"
"context"
"errors"
"fmt"
"io"
stdpath "path"
"strconv"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/conf"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/fs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/internal/sign"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/OpenListTeam/OpenList/v4/server/common"
"github.com/avast/retry-go"
)
type Chunk struct {
model.Storage
Addition
}
func (d *Chunk) Config() driver.Config {
return config
}
func (d *Chunk) GetAddition() driver.Additional {
return &d.Addition
}
func (d *Chunk) Init(ctx context.Context) error {
if d.PartSize <= 0 {
return errors.New("part size must be positive")
}
if len(d.ChunkPrefix) <= 0 {
return errors.New("chunk folder prefix must not be empty")
}
d.RemotePath = utils.FixAndCleanPath(d.RemotePath)
return nil
}
func (d *Chunk) Drop(ctx context.Context) error {
return nil
}
func (*Chunk) GetRootPath() string {
return ""
}
func (d *Chunk) Get(ctx context.Context, path string) (model.Obj, error) {
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
if err != nil {
return nil, err
}
remoteActualPath = stdpath.Join(remoteActualPath, path)
if remoteObj, err := op.Get(ctx, remoteStorage, remoteActualPath); err == nil {
return &model.Object{
Path: path,
Name: remoteObj.GetName(),
Size: remoteObj.GetSize(),
Modified: remoteObj.ModTime(),
IsFolder: remoteObj.IsDir(),
HashInfo: remoteObj.GetHash(),
}, nil
}
remoteActualDir, name := stdpath.Split(remoteActualPath)
chunkName := d.ChunkPrefix + name
chunkObjs, err := op.List(ctx, remoteStorage, stdpath.Join(remoteActualDir, chunkName), model.ListArgs{})
if err != nil {
return nil, err
}
var totalSize int64
// 0号块默认为-1 以支持空文件
chunkSizes := []int64{-1}
h := make(map[*utils.HashType]string)
var first model.Obj
for _, o := range chunkObjs {
if o.IsDir() {
continue
}
if after, ok := strings.CutPrefix(o.GetName(), "hash_"); ok {
hn, value, ok := strings.Cut(strings.TrimSuffix(after, d.CustomExt), "_")
if ok {
ht, ok := utils.GetHashByName(hn)
if ok {
h[ht] = value
}
}
continue
}
idx, err := strconv.Atoi(strings.TrimSuffix(o.GetName(), d.CustomExt))
if err != nil {
continue
}
totalSize += o.GetSize()
if len(chunkSizes) > idx {
if idx == 0 {
first = o
}
chunkSizes[idx] = o.GetSize()
} else if len(chunkSizes) == idx {
chunkSizes = append(chunkSizes, o.GetSize())
} else {
newChunkSizes := make([]int64, idx+1)
copy(newChunkSizes, chunkSizes)
chunkSizes = newChunkSizes
chunkSizes[idx] = o.GetSize()
}
}
reqDir, _ := stdpath.Split(path)
// 文件块不完整时,返回文件夹对象
if chunkSizes[0] == -1 {
return &model.Object{
Path: stdpath.Join(reqDir, chunkName),
Name: chunkName,
IsFolder: true,
Modified: d.Modified,
}, nil
}
for i, l := 1, len(chunkSizes)-1; i < l; i++ {
if chunkSizes[i] == 0 {
return &model.Object{
Path: stdpath.Join(reqDir, chunkName),
Name: chunkName,
IsFolder: true,
Modified: d.Modified,
}, nil
}
}
objRes := chunkObject{
Object: model.Object{
Path: stdpath.Join(reqDir, chunkName),
Name: name,
Size: totalSize,
Modified: first.ModTime(),
Ctime: first.CreateTime(),
},
chunkSizes: chunkSizes,
}
if len(h) > 0 {
objRes.HashInfo = utils.NewHashInfoByMap(h)
}
return &objRes, nil
}
func (d *Chunk) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
if err != nil {
return nil, err
}
remoteActualDir := stdpath.Join(remoteActualPath, dir.GetPath())
remoteObjs, err := op.List(ctx, remoteStorage, remoteActualDir, model.ListArgs{
ReqPath: args.ReqPath,
Refresh: args.Refresh,
})
if err != nil {
return nil, err
}
result := make([]model.Obj, 0, len(remoteObjs))
listG, listCtx := errgroup.NewGroupWithContext(ctx, d.NumListWorkers, retry.Attempts(3))
for _, obj := range remoteObjs {
if utils.IsCanceled(listCtx) {
break
}
rawName := obj.GetName()
if obj.IsDir() {
if name, ok := strings.CutPrefix(rawName, d.ChunkPrefix); ok {
resultIdx := len(result)
result = append(result, nil)
listG.Go(func(ctx context.Context) error {
chunkObjs, err := op.List(ctx, remoteStorage, stdpath.Join(remoteActualDir, rawName), model.ListArgs{
Refresh: args.Refresh,
})
if err != nil {
return err
}
var totalSize int64
// 0号块默认为-1 以支持空文件
chunkSizes := []int64{-1}
h := make(map[*utils.HashType]string)
var first model.Obj
for _, o := range chunkObjs {
if o.IsDir() {
continue
}
if after, ok := strings.CutPrefix(o.GetName(), "hash_"); ok {
hn, value, ok := strings.Cut(strings.TrimSuffix(after, d.CustomExt), "_")
if ok {
ht, ok := utils.GetHashByName(hn)
if ok {
h[ht] = value
}
}
continue
}
idx, err := strconv.Atoi(strings.TrimSuffix(o.GetName(), d.CustomExt))
if err != nil {
continue
}
totalSize += o.GetSize()
if len(chunkSizes) > idx {
if idx == 0 {
first = o
}
chunkSizes[idx] = o.GetSize()
} else if len(chunkSizes) == idx {
chunkSizes = append(chunkSizes, o.GetSize())
} else {
newChunkSizes := make([]int64, idx+1)
copy(newChunkSizes, chunkSizes)
chunkSizes = newChunkSizes
chunkSizes[idx] = o.GetSize()
}
}
// 文件块不完整时,返回文件夹对象
if chunkSizes[0] == -1 {
result[resultIdx] = &model.Object{
Name: rawName,
Size: obj.GetSize(),
Modified: obj.ModTime(),
IsFolder: true,
}
return nil
}
for i, l := 1, len(chunkSizes)-1; i < l; i++ {
if chunkSizes[i] == 0 {
result[resultIdx] = &model.Object{
Name: rawName,
Size: obj.GetSize(),
Modified: obj.ModTime(),
IsFolder: true,
}
return nil
}
}
objRes := model.Object{
Name: name,
Size: totalSize,
Modified: first.ModTime(),
Ctime: first.CreateTime(),
}
if len(h) > 0 {
objRes.HashInfo = utils.NewHashInfoByMap(h)
}
if !d.Thumbnail {
result[resultIdx] = &objRes
} else {
thumbPath := stdpath.Join(args.ReqPath, ".thumbnails", name+".webp")
thumb := fmt.Sprintf("%s/d%s?sign=%s",
common.GetApiUrl(ctx),
utils.EncodePath(thumbPath, true),
sign.Sign(thumbPath))
result[resultIdx] = &model.ObjThumb{
Object: objRes,
Thumbnail: model.Thumbnail{
Thumbnail: thumb,
},
}
}
return nil
})
continue
}
}
if !d.ShowHidden && strings.HasPrefix(rawName, ".") {
continue
}
thumb, ok := model.GetThumb(obj)
objRes := model.Object{
Name: rawName,
Size: obj.GetSize(),
Modified: obj.ModTime(),
IsFolder: obj.IsDir(),
HashInfo: obj.GetHash(),
}
if !ok {
result = append(result, &objRes)
} else {
result = append(result, &model.ObjThumb{
Object: objRes,
Thumbnail: model.Thumbnail{
Thumbnail: thumb,
},
})
}
}
if err = listG.Wait(); err != nil {
return nil, err
}
return result, nil
}
func (d *Chunk) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
if err != nil {
return nil, err
}
chunkFile, ok := file.(*chunkObject)
remoteActualPath = stdpath.Join(remoteActualPath, file.GetPath())
if !ok {
l, _, err := op.Link(ctx, remoteStorage, remoteActualPath, args)
if err != nil {
return nil, err
}
return l.Clone(), nil
}
// 检查0号块不等于-1 以支持空文件
// 如果块数量大于1 最后一块不可能为0
// 只检查中间块是否有0
if chunkFile.chunkSizes[0] == -1 {
return nil, fmt.Errorf("chunk part[%d] are missing", 0)
}
for i, l := 1, len(chunkFile.chunkSizes)-1; i < l; i++ {
if chunkFile.chunkSizes[i] == 0 {
return nil, fmt.Errorf("chunk part[%d] are missing", i)
}
}
fileSize := chunkFile.GetSize()
mergedRrf := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
start := httpRange.Start
length := httpRange.Length
if length < 0 || start+length > fileSize {
length = fileSize - start
}
if length == 0 {
return io.NopCloser(strings.NewReader("")), nil
}
rs := make([]io.Reader, 0)
cs := make(utils.Closers, 0)
var (
rc io.ReadCloser
readFrom bool
)
for idx, chunkSize := range chunkFile.chunkSizes {
if readFrom {
l, o, err := op.Link(ctx, remoteStorage, stdpath.Join(remoteActualPath, d.getPartName(idx)), args)
if err != nil {
_ = cs.Close()
return nil, err
}
cs = append(cs, l)
chunkSize2 := l.ContentLength
if chunkSize2 <= 0 {
chunkSize2 = o.GetSize()
}
if chunkSize2 != chunkSize {
_ = cs.Close()
return nil, fmt.Errorf("chunk part[%d] size not match", idx)
}
rrf, err := stream.GetRangeReaderFromLink(chunkSize2, l)
if err != nil {
_ = cs.Close()
return nil, err
}
newLength := length - chunkSize2
if newLength >= 0 {
length = newLength
rc, err = rrf.RangeRead(ctx, http_range.Range{Length: -1})
} else {
rc, err = rrf.RangeRead(ctx, http_range.Range{Length: length})
}
if err != nil {
_ = cs.Close()
return nil, err
}
rs = append(rs, rc)
cs = append(cs, rc)
if newLength <= 0 {
return utils.ReadCloser{
Reader: io.MultiReader(rs...),
Closer: &cs,
}, nil
}
} else if newStart := start - chunkSize; newStart >= 0 {
start = newStart
} else {
l, o, err := op.Link(ctx, remoteStorage, stdpath.Join(remoteActualPath, d.getPartName(idx)), args)
if err != nil {
_ = cs.Close()
return nil, err
}
cs = append(cs, l)
chunkSize2 := l.ContentLength
if chunkSize2 <= 0 {
chunkSize2 = o.GetSize()
}
if chunkSize2 != chunkSize {
_ = cs.Close()
return nil, fmt.Errorf("chunk part[%d] size not match", idx)
}
rrf, err := stream.GetRangeReaderFromLink(chunkSize2, l)
if err != nil {
_ = cs.Close()
return nil, err
}
rc, err = rrf.RangeRead(ctx, http_range.Range{Start: start, Length: -1})
if err != nil {
_ = cs.Close()
return nil, err
}
length -= chunkSize2 - start
cs = append(cs, rc)
if length <= 0 {
return utils.ReadCloser{
Reader: rc,
Closer: &cs,
}, nil
}
rs = append(rs, rc)
readFrom = true
}
}
return nil, fmt.Errorf("invalid range: start=%d,length=%d,fileSize=%d", httpRange.Start, httpRange.Length, fileSize)
}
return &model.Link{
RangeReader: stream.RangeReaderFunc(mergedRrf),
}, nil
}
func (d *Chunk) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
path := stdpath.Join(d.RemotePath, parentDir.GetPath(), dirName)
return fs.MakeDir(ctx, path)
}
func (d *Chunk) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
src := stdpath.Join(d.RemotePath, srcObj.GetPath())
dst := stdpath.Join(d.RemotePath, dstDir.GetPath())
_, err := fs.Move(ctx, src, dst)
return err
}
func (d *Chunk) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
if _, ok := srcObj.(*chunkObject); ok {
newName = d.ChunkPrefix + newName
}
return fs.Rename(ctx, stdpath.Join(d.RemotePath, srcObj.GetPath()), newName)
}
func (d *Chunk) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
dst := stdpath.Join(d.RemotePath, dstDir.GetPath())
src := stdpath.Join(d.RemotePath, srcObj.GetPath())
_, err := fs.Copy(ctx, src, dst)
return err
}
func (d *Chunk) Remove(ctx context.Context, obj model.Obj) error {
return fs.Remove(ctx, stdpath.Join(d.RemotePath, obj.GetPath()))
}
func (d *Chunk) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
if err != nil {
return err
}
if (d.Thumbnail && dstDir.GetName() == ".thumbnails") || (d.ChunkLargeFileOnly && file.GetSize() <= d.PartSize) {
return op.Put(ctx, remoteStorage, stdpath.Join(remoteActualPath, dstDir.GetPath()), file, up)
}
upReader := &driver.ReaderUpdatingProgress{
Reader: file,
UpdateProgress: up,
}
dst := stdpath.Join(remoteActualPath, dstDir.GetPath(), d.ChunkPrefix+file.GetName())
skipHookCtx := context.WithValue(ctx, conf.SkipHookKey, struct{}{})
if d.StoreHash {
for ht, value := range file.GetHash().All() {
_ = op.Put(skipHookCtx, remoteStorage, dst, &stream.FileStream{
Obj: &model.Object{
Name: fmt.Sprintf("hash_%s_%s%s", ht.Name, value, d.CustomExt),
Size: 1,
Modified: file.ModTime(),
},
Mimetype: "application/octet-stream",
Reader: bytes.NewReader([]byte{0}), // 兼容不支持空文件的驱动
}, nil)
}
}
fullPartCount := int(file.GetSize() / d.PartSize)
tailSize := file.GetSize() % d.PartSize
if tailSize == 0 && fullPartCount > 0 {
fullPartCount--
tailSize = d.PartSize
}
partIndex := 0
for partIndex < fullPartCount {
err = op.Put(skipHookCtx, remoteStorage, dst, &stream.FileStream{
Obj: &model.Object{
Name: d.getPartName(partIndex),
Size: d.PartSize,
Modified: file.ModTime(),
},
Mimetype: file.GetMimetype(),
Reader: io.LimitReader(upReader, d.PartSize),
}, nil)
if err != nil {
_ = op.Remove(ctx, remoteStorage, dst)
return err
}
partIndex++
}
err = op.Put(ctx, remoteStorage, dst, &stream.FileStream{
Obj: &model.Object{
Name: d.getPartName(fullPartCount),
Size: tailSize,
Modified: file.ModTime(),
},
Mimetype: file.GetMimetype(),
Reader: upReader,
}, nil)
if err != nil {
_ = op.Remove(ctx, remoteStorage, dst)
}
return err
}
func (d *Chunk) getPartName(part int) string {
return fmt.Sprintf("%d%s", part, d.CustomExt)
}
func (d *Chunk) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
remoteStorage, err := fs.GetStorage(d.RemotePath, &fs.GetStoragesArgs{})
if err != nil {
return nil, errs.NotImplement
}
remoteDetails, err := op.GetStorageDetails(ctx, remoteStorage)
if err != nil {
return nil, err
}
return &model.StorageDetails{
DiskUsage: remoteDetails.DiskUsage,
}, nil
}
var _ driver.Driver = (*Chunk)(nil)
-39
View File
@@ -1,39 +0,0 @@
package chunk
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
RemotePath string `json:"remote_path" required:"true"`
PartSize int64 `json:"part_size" required:"true" type:"number" help:"bytes"`
ChunkLargeFileOnly bool `json:"chunk_large_file_only" default:"false" help:"chunk only if file size > part_size"`
ChunkPrefix string `json:"chunk_prefix" type:"string" default:"[openlist_chunk]" help:"the prefix of chunk folder"`
CustomExt string `json:"custom_ext" type:"string"`
StoreHash bool `json:"store_hash" type:"bool" default:"true"`
NumListWorkers int `json:"num_list_workers" required:"true" type:"number" default:"5"`
Thumbnail bool `json:"thumbnail" required:"true" default:"false" help:"enable thumbnail which pre-generated under .thumbnails folder"`
ShowHidden bool `json:"show_hidden" default:"true" required:"false" help:"show hidden directories and files"`
}
var config = driver.Config{
Name: "Chunk",
LocalSort: true,
OnlyProxy: true,
NoCache: true,
DefaultRoot: "/",
NoLinkURL: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Chunk{
Addition: Addition{
ChunkPrefix: "[openlist_chunk]",
NumListWorkers: 5,
},
}
})
}
-8
View File
@@ -1,8 +0,0 @@
package chunk
import "github.com/OpenListTeam/OpenList/v4/internal/model"
type chunkObject struct {
model.Object
chunkSizes []int64
}
-168
View File
@@ -1,168 +0,0 @@
package cloudflare_imgbed
import (
"context"
"fmt"
"net/http"
"path"
"strings"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/cache"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/go-resty/resty/v2"
)
type CFImgBed struct {
model.Storage
Addition
client *resty.Client
virtualDir *cache.WeakCacheMap[string, model.Object]
publicUrlPrefix string
}
func (d *CFImgBed) Config() driver.Config { return config }
func (d *CFImgBed) GetAddition() driver.Additional { return &d.Addition }
func (d *CFImgBed) Init(ctx context.Context) error {
d.UploadThread = min(d.UploadThread, 32)
if d.UploadThread < 1 {
d.UploadThread = 3
}
d.Address = strings.TrimRight(d.Address, "/")
d.client = base.NewRestyClient().
SetBaseURL(d.Address).
SetHeader("Authorization", "Bearer "+d.Token).
SetDebug(false)
// 连通性测试:尝试获取根目录单条数据
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"start": "0",
"count": "1",
"dir": "/",
})
}, nil)
if err != nil {
return fmt.Errorf("init verification failed: %w", err)
}
d.virtualDir = cache.NewWeakCacheMap[string, model.Object]()
return nil
}
func (d *CFImgBed) Drop(ctx context.Context) error {
if d.virtualDir != nil {
d.virtualDir.Clear()
}
return nil
}
func (d *CFImgBed) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
// if !args.Refresh && model.ObjHasMask(dir, model.Virtual) {
// if _, ok := d.virtualDir.Load(dir.GetPath()); ok {
// return nil, nil
// }
// }
var dirSeen map[string]bool
var fileSeen map[string]bool
var objs []model.Obj
start := 0
for {
var resp ListResponse
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"dir": dir.GetPath(),
"start": fmt.Sprintf("%d", start),
"count": fmt.Sprintf("%d", listPageSize),
})
}, &resp)
if err != nil {
return nil, err
}
if len(resp.Files) == 0 && len(resp.Directories) == 0 {
break
}
if start == 0 {
dirSeen = make(map[string]bool, len(resp.Directories))
fileSeen = make(map[string]bool, len(resp.Files))
objs = make([]model.Obj, 0, len(resp.Directories)+len(resp.Files))
}
for _, rawDir := range resp.Directories {
rawDir = "/" + strings.TrimRight(rawDir, "/")
if !dirSeen[rawDir] {
dirSeen[rawDir] = true
objs = append(objs, &model.Object{
Path: rawDir,
Name: path.Base(rawDir),
Modified: d.Modified,
IsFolder: true,
})
}
}
for _, item := range resp.Files {
if !fileSeen[item.Name] {
fileSeen[item.Name] = true
objs = append(objs, parseFile(item))
}
}
// 如果当前获取的数量少于分页大小,说明已加载完毕
if len(resp.Files)+len(resp.Directories) < listPageSize {
break
}
start += listPageSize
}
return objs, nil
}
func (d *CFImgBed) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if d.publicUrlPrefix != "" {
return &model.Link{URL: d.publicUrlPrefix + utils.EncodePath(file.GetPath())}, nil
}
return &model.Link{URL: d.Address + "/file" + utils.EncodePath(file.GetPath())}, nil
}
func (d *CFImgBed) Get(ctx context.Context, pathStr string) (model.Obj, error) {
fullPath := path.Join(d.RootFolderPath, pathStr)
if obj, found := d.virtualDir.Load(fullPath); found {
return obj, nil
}
return nil, errs.NotSupport
}
// MakeDir 在图床中通常是虚拟的,此处返回虚拟目录对象以支持上传时的路径展示
func (d *CFImgBed) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
fullPath := path.Join(parentDir.GetPath(), dirName)
temp := &model.Object{
Path: fullPath,
Name: dirName,
IsFolder: true,
Modified: d.Modified,
Mask: model.Virtual,
}
d.virtualDir.Store(fullPath, temp)
return temp, nil
}
func (d *CFImgBed) Remove(ctx context.Context, obj model.Obj) error {
reqPath := obj.GetPath()
if model.ObjHasMask(obj, model.Virtual) {
d.virtualDir.Delete(reqPath)
return nil
}
_, err := d.doRequest(ctx, http.MethodPost, deleteApi+utils.EncodePath(reqPath), func(req *resty.Request) {
req.SetQueryParam("folder", fmt.Sprintf("%t", obj.IsDir()))
}, nil)
return err
}
var _ driver.Driver = (*CFImgBed)(nil)
-27
View File
@@ -1,27 +0,0 @@
package cloudflare_imgbed
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
Address string `json:"address" required:"true" help:"Backend API address of the image hosting service, e.g., https://img.example.com"`
Token string `json:"token" required:"true" help:"Authentication Token"`
SmallChannelName string `json:"smallChannelName" help:"Channel name for regular files (typically <20MB)"`
LargeChannelName string `json:"largeChannelName" help:"Channel name for large files"`
LargeChannelType string `json:"largeChannelType" type:"select" options:",huggingface,telegram,cfr2,s3,discord" help:"Large File Channel Type: Hugging Face (Direct Upload)、telegram/cfr2/s3/discord(Multipart Upload)"`
UploadThread int `json:"uploadThread" type:"number" default:"3" help:"Concurrent thread count for HuggingFace chunked direct upload"`
}
var config = driver.Config{
Name: "cloudflare_imgbed",
LocalSort: true,
NoUpload: false,
DefaultRoot: "/",
}
func init() {
op.RegisterDriver(func() driver.Driver { return &CFImgBed{} })
}
-117
View File
@@ -1,117 +0,0 @@
package cloudflare_imgbed
import (
"fmt"
"path"
"strconv"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
)
const listPageSize = 1000
// ListResponse 列表接口响应
type ListResponse struct {
Files []FileItem `json:"files"`
Directories []string `json:"directories"`
}
type FileItem struct {
Name string `json:"name"`
Metadata map[string]interface{} `json:"metadata"` // 存储文件大小、哈希、时间戳等
}
type apiError struct {
Error string `json:"error"`
Message string `json:"message"`
}
// standardUploadResp 标准上传成功返回的数组
type standardUploadResp []struct {
Src string `json:"src"`
PublicUrl string `json:"publicUrl"`
}
// hfGetUrlResp 获取 HF 直传授权地址的响应
type hfGetUrlResp struct {
Success bool `json:"success"`
FullID string `json:"fullId"`
FilePath string `json:"filePath"`
ChannelName string `json:"channelName"`
Repo string `json:"repo"`
NeedsLfs bool `json:"needsLfs"` // 是否需要进行 LFS 物理上传
AlreadyExists bool `json:"alreadyExists"` // 是否秒传成功
Oid string `json:"oid"` // Git LFS 对象 ID (SHA256)
UploadAction *UploadAction `json:"uploadAction"`
}
type UploadAction struct {
Href string `json:"href"`
Header map[string]string `json:"header"`
}
type hfCommitResp struct {
Success bool `json:"success"`
Src string `json:"src"`
PublicUrl string `json:"publicUrl"`
FileUrl string `json:"fileUrl"`
FullID string `json:"fullId"`
}
// 辅助函数:从 map 中安全提取字符串/数值
func getString(m map[string]interface{}, keys ...string) string {
for _, k := range keys {
if v, ok := m[k]; ok {
switch val := v.(type) {
case string:
return val
case float64:
return strconv.FormatInt(int64(val), 10)
default:
return fmt.Sprintf("%v", val)
}
}
}
return ""
}
func getInt64(m map[string]interface{}, keys ...string) int64 {
for _, k := range keys {
if v, ok := m[k]; ok {
switch val := v.(type) {
case string:
n, _ := strconv.ParseInt(val, 10, 64)
return n
case float64:
return int64(val)
case int64:
return val
}
}
}
return 0
}
func parseFile(item FileItem) *model.Object {
name := path.Base(item.Name)
var size int64
var modTime time.Time
if item.Metadata != nil {
size = getInt64(item.Metadata, "FileSizeBytes", "File-Size")
ts := getInt64(item.Metadata, "TimeStamp")
if ts > 0 {
modTime = time.UnixMilli(ts)
}
}
return &model.Object{
Path: "/" + strings.TrimRight(item.Name, "/"),
Name: name,
Size: size,
Modified: modTime,
IsFolder: false,
}
}
-560
View File
@@ -1,560 +0,0 @@
package cloudflare_imgbed
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"mime/multipart"
"net/http"
"net/url"
"path"
"strconv"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/avast/retry-go"
"github.com/go-resty/resty/v2"
log "github.com/sirupsen/logrus"
)
func (d *CFImgBed) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (newObj model.Obj, err error) {
if file.GetSize() < hfDirectThreshold {
newObj, err = d.standardUpload(ctx, dstDir, file, up)
} else {
switch d.LargeChannelType {
case "huggingface":
newObj, err = d.hfDirectUpload(ctx, dstDir, file, up)
case "telegram", "cfr2", "s3", "discord":
newObj, err = d.chunkedUpload(ctx, dstDir, file, up, d.LargeChannelType, d.LargeChannelName)
default:
newObj, err = d.standardUpload(ctx, dstDir, file, up)
}
}
if newObj != nil && model.ObjHasMask(dstDir, model.Virtual) {
key := dstDir.GetPath()
for d.virtualDir.Delete(key) {
key = path.Dir(key)
}
}
return
}
// standardUpload 通过普通 multipart 表单上传。
// 使用 io.MultiReader 实现虚拟拼接,避免将整个大文件读入内存构建表单。
func (d *CFImgBed) standardUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
channelName := d.SmallChannelName
if file.GetSize() >= hfDirectThreshold {
channelName = d.LargeChannelName
log.WithField("size", file.GetSize()).Warn("large file falls back to standard upload, consider configuring LargeChannelType")
}
if channelName == "" {
return nil, fmt.Errorf("channel name not configured")
}
// 1. 将参数放入 Query String
reqUrl, err := url.Parse(d.Address + uploadApi)
if err != nil {
return nil, err
}
q := reqUrl.Query()
q.Set("returnFormat", "default")
q.Set("channelName", channelName)
q.Set("uploadFolder", dstDir.GetPath())
q.Set("autoRetry", "true")
reqUrl.RawQuery = q.Encode()
// 2. 构建 multipart 表单的头部
b := bytes.NewBuffer(make([]byte, 0, 164+len(file.GetName()))) // 预估头部大小,避免频繁扩容
w := multipart.NewWriter(b)
_, err = w.CreateFormFile("file", file.GetName())
if err != nil {
return nil, err
}
headSize := b.Len()
err = w.Close()
if err != nil {
return nil, err
}
head := bytes.NewReader(b.Bytes()[:headSize])
tail := bytes.NewReader(b.Bytes()[headSize:])
// 3. 将 [表单头 + 文件流 + 表单尾] 组合成单一 Reader
rateLimitedReader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
Reader: &driver.SimpleReaderWithSize{
Reader: io.MultiReader(head, file, tail),
Size: int64(b.Len()) + file.GetSize(),
},
UpdateProgress: up,
})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, reqUrl.String(), rateLimitedReader)
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", w.FormDataContentType())
req.Header.Set("Authorization", "Bearer "+d.Token)
req.ContentLength = int64(b.Len()) + file.GetSize()
res, err := base.HttpClient.Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
b.Reset()
_, err = b.ReadFrom(res.Body)
if err != nil {
return nil, err
}
if res.StatusCode != http.StatusOK {
return nil, fmt.Errorf("upload failed %d: %s", res.StatusCode, b.String())
}
var resp standardUploadResp
if err := json.Unmarshal(b.Bytes(), &resp); err != nil {
return nil, err
}
if len(resp) == 0 || resp[0].Src == "" {
return nil, fmt.Errorf("no src returned")
}
srcPath := strings.TrimPrefix(resp[0].Src, "/file/")
srcPath = strings.TrimPrefix(srcPath, "/")
if resp[0].PublicUrl != "" {
if u, err := url.Parse(resp[0].PublicUrl); err == nil {
d.publicUrlPrefix = u.Scheme + "://" + u.Host
}
}
return &model.Object{
Path: srcPath,
Name: file.GetName(),
Size: file.GetSize(),
Modified: file.ModTime(),
IsFolder: false,
}, nil
}
func (d *CFImgBed) chunkedUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress, channelType, channelName string) (model.Obj, error) {
if channelName == "" {
return nil, fmt.Errorf("channel name not configured for chunked upload")
}
fileSize := file.GetSize()
fileMime := file.GetMimetype()
fileName := file.GetName()
var chunkSizeMap = map[string]int64{
}
chunkSize := chunkSizeMap[channelType]
if chunkSize == 0 {
chunkSize = 5 * 1024 * 1024
}
totalChunks := int((fileSize + chunkSize - 1) / chunkSize)
// 第一步:initChunked
var initResp struct {
Success bool `json:"success"`
UploadId string `json:"uploadId"`
}
_, err := d.doRequest(ctx, http.MethodPost, uploadApi, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"initChunked": "true",
"uploadChannel": channelType,
"channelName": channelName,
})
req.SetFormData(map[string]string{
"originalFileName": fileName,
"originalFileType": fileMime,
"totalChunks": strconv.Itoa(totalChunks),
})
}, &initResp)
if err != nil {
return nil, fmt.Errorf("initChunked failed: %w", err)
}
if !initResp.Success || initResp.UploadId == "" {
return nil, fmt.Errorf("initChunked returned no uploadId")
}
uploadId := initResp.UploadId
// 第二步:逐块上传
ss, err := stream.NewStreamSectionReader(file, int(chunkSize), nil)
if err != nil {
return nil, err
}
reqUrl := d.Address + uploadApi + "?chunked=true&uploadChannel=" + channelType + "&channelName=" + channelName
b := bytes.NewBuffer(make([]byte, 0, 2048))
g, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, min(d.UploadThread, totalChunks),
retry.Attempts(3),
retry.Delay(time.Second),
retry.DelayType(retry.BackOffDelay))
for i := 0; i < totalChunks; i++ {
if utils.IsCanceled(uploadCtx) {
break
}
chunkIndex := i
offset := int64(chunkIndex) * chunkSize
sizeToRead := chunkSize
if offset+sizeToRead > fileSize {
sizeToRead = fileSize - offset
}
var reader io.ReadSeeker
g.GoWithLifecycle(errgroup.Lifecycle{
Before: func(ctx context.Context) (err error) {
reader, err = ss.GetSectionReader(offset, sizeToRead)
return
},
After: func(err error) {
ss.FreeSectionReader(reader)
},
Do: func(ctx context.Context) (err error) {
_, err = reader.Seek(0, io.SeekStart)
if err != nil {
return err
}
b.Reset()
w := multipart.NewWriter(b)
_ = w.WriteField("uploadId", uploadId)
_ = w.WriteField("chunkIndex", strconv.Itoa(chunkIndex))
_ = w.WriteField("totalChunks", strconv.Itoa(totalChunks))
_ = w.WriteField("originalFileName", fileName)
_ = w.WriteField("originalFileType", fileMime)
_, _ = w.CreateFormFile("file", fileName)
headSize := b.Len()
_ = w.Close()
head := bytes.NewReader(b.Bytes()[:headSize])
tail := bytes.NewReader(b.Bytes()[headSize:])
rateLimitedRd := driver.NewLimitedUploadStream(ctx, io.MultiReader(head, reader, tail))
req, err := http.NewRequestWithContext(ctx, http.MethodPost, reqUrl, rateLimitedRd)
if err != nil {
return err
}
req.Header.Set("Content-Type", w.FormDataContentType())
req.Header.Set("Authorization", "Bearer "+d.Token)
req.ContentLength = int64(headSize) + sizeToRead + int64(b.Len()-headSize)
res, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return fmt.Errorf("chunk %d upload failed: %d", chunkIndex, res.StatusCode)
}
up(90 * float64(chunkIndex+1) / float64(totalChunks))
return nil
},
})
}
if err := g.Wait(); err != nil {
return nil, err
}
// 第三步:merge
var mergeResp standardUploadResp
_, err = d.doRequest(ctx, http.MethodPost, uploadApi, func(req *resty.Request) {
req.SetQueryParams(map[string]string{
"chunked": "true",
"merge": "true",
"uploadChannel": channelType,
"channelName": channelName,
"returnFormat": "default",
"uploadFolder": dstDir.GetPath(),
})
req.SetFormData(map[string]string{
"uploadId": uploadId,
"totalChunks": strconv.Itoa(totalChunks),
"originalFileName": fileName,
"originalFileType": fileMime,
})
}, &mergeResp)
if err != nil {
return nil, fmt.Errorf("merge failed: %w", err)
}
if len(mergeResp) == 0 || mergeResp[0].Src == "" {
return nil, fmt.Errorf("merge returned no src")
}
up(95)
srcPath := strings.TrimPrefix(mergeResp[0].Src, "/file/")
srcPath = strings.TrimPrefix(srcPath, "/")
if mergeResp[0].PublicUrl != "" {
if u, err := url.Parse(mergeResp[0].PublicUrl); err == nil {
d.publicUrlPrefix = u.Scheme + "://" + u.Host
}
}
return &model.Object{
Path: srcPath,
Name: fileName,
Size: fileSize,
Modified: file.ModTime(),
IsFolder: false,
}, nil
}
// hfDirectUpload 处理 HuggingFace 的 LFS 直传逻辑(申请授权 -> 物理上传 -> 后端 Commit)
func (d *CFImgBed) hfDirectUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
channelName := d.LargeChannelName
if channelName == "" {
return nil, errors.New("LargeChannelName not configured")
}
sha256Hash := file.GetHash().GetHash(utils.SHA256)
if len(sha256Hash) != utils.SHA256.Width {
var err error
_, sha256Hash, err = stream.CacheFullAndHash(file, &up, utils.SHA256)
if err != nil {
return nil, err
}
}
fileSize := file.GetSize()
sampleSize := min(fileSize, fileSampleSize)
sampleRd, err := file.RangeRead(http_range.Range{Start: 0, Length: sampleSize})
if err != nil {
return nil, err
}
sampleBuf := make([]byte, sampleSize)
_, err = io.ReadFull(sampleRd, sampleBuf)
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
return nil, err
}
fileSample := base64.StdEncoding.EncodeToString(sampleBuf)
fileMime := file.GetMimetype()
// 1. 请求图床后端获取 HF 授权地址
reqBody := map[string]interface{}{
"fileName": file.GetName(),
"fileType": fileMime,
"fileSize": fileSize,
"sha256": sha256Hash,
"fileSample": fileSample,
"channelName": channelName,
"uploadFolder": dstDir.GetPath(),
}
var getUrlResp hfGetUrlResp
_, err = d.doRequest(ctx, http.MethodPost, hfGetUrlApi, func(req *resty.Request) {
req.SetBody(reqBody)
req.SetHeader("Content-Type", "application/json")
}, &getUrlResp)
if err != nil {
return nil, err
}
// 秒传逻辑
if getUrlResp.AlreadyExists || !getUrlResp.NeedsLfs {
up(100)
return d.hfCommit(ctx, getUrlResp, file.GetName(), fileSize, fileMime, file.ModTime())
}
if getUrlResp.UploadAction == nil {
return nil, fmt.Errorf("HF upload action is nil")
}
headers := getUrlResp.UploadAction.Header
href := getUrlResp.UploadAction.Href
// 2. 根据响应判断是执行分片上传还是单文件上传
chunkSizeStr, needChunk := headers["chunk_size"]
if needChunk {
// 分片直传 (AWS S3 Multipart 风格)
chunkSize, _ := strconv.ParseInt(chunkSizeStr, 10, 64)
if chunkSize <= 0 {
chunkSize = 20 * 1024 * 1024
}
partUrls := make(map[int]string)
for k, v := range headers {
if k != "chunk_size" {
if idx, err := strconv.Atoi(k); err == nil {
partUrls[idx] = v
}
}
}
totalParts := len(partUrls)
ss, err := stream.NewStreamSectionReader(file, int(chunkSize), &up)
if err != nil {
return nil, err
}
g, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, min(d.UploadThread, totalParts),
retry.Attempts(3),
retry.Delay(time.Second),
retry.DelayType(retry.BackOffDelay))
parts := make([]map[string]any, totalParts)
for partNumber := range partUrls {
if utils.IsCanceled(uploadCtx) {
break
}
partUrl := partUrls[partNumber]
offset := int64(partNumber-1) * chunkSize
sizeToRead := chunkSize
if offset+sizeToRead > fileSize {
sizeToRead = fileSize - offset
}
var reader io.ReadSeeker
g.GoWithLifecycle(errgroup.Lifecycle{
Before: func(ctx context.Context) (err error) {
reader, err = ss.GetSectionReader(offset, sizeToRead)
return
},
After: func(err error) {
ss.FreeSectionReader(reader)
},
Do: func(ctx context.Context) (err error) {
_, err = reader.Seek(0, io.SeekStart)
if err != nil {
return err
}
limitedReader := driver.NewLimitedUploadStream(ctx, reader)
req, err := http.NewRequestWithContext(ctx, http.MethodPut, partUrl, limitedReader)
if err != nil {
return err
}
req.ContentLength = sizeToRead
res, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return fmt.Errorf("chunk %d failed: %d", partNumber, res.StatusCode)
}
etag := res.Header.Get("ETag")
parts[partNumber-1] = map[string]any{"partNumber": partNumber, "etag": etag}
up(95 * float64(g.Success()+1) / float64(totalParts))
return nil
},
})
}
if err := g.Wait(); err != nil {
return nil, err
}
// 合并分片
// sort.Slice(parts, func(i, j int) bool { return parts[i]["partNumber"].(int) < parts[j]["partNumber"].(int) })
mergeBody, _ := json.Marshal(map[string]any{"oid": getUrlResp.Oid, "parts": parts})
mergeReq, err := http.NewRequestWithContext(ctx, http.MethodPost, href, bytes.NewReader(mergeBody))
if err != nil {
return nil, err
}
mergeReq.Header.Set("Content-Type", "application/vnd.git-lfs+json")
if d.Token != "" {
mergeReq.Header.Set("Authorization", "Bearer "+d.Token)
}
res, err := base.HttpClient.Do(mergeReq)
if err != nil {
return nil, err
}
defer res.Body.Close()
body, _ := io.ReadAll(res.Body) // 读取 HF 返回的错误详情
if res.StatusCode != http.StatusOK {
log.WithField("status", res.StatusCode).WithField("response", string(body)).Error("HF merge chunks failed")
return nil, fmt.Errorf("merge chunks failed: %s", string(body))
}
up(97)
} else {
// 单文件直传 (PUT)
limitedReader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
Reader: file,
UpdateProgress: model.UpdateProgressWithRange(up, 0, 97),
})
req, err := http.NewRequestWithContext(ctx, http.MethodPut, href, limitedReader)
if err != nil {
return nil, err
}
req.ContentLength = fileSize
for k, v := range headers {
req.Header.Set(k, v)
}
res, err := base.HttpClient.Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return nil, fmt.Errorf("direct upload failed")
}
}
defer up(100)
// 3. 通知图床后端完成文件登记
return d.hfCommit(ctx, getUrlResp, file.GetName(), fileSize, fileMime, file.ModTime())
}
func (d *CFImgBed) hfCommit(ctx context.Context, getUrlResp hfGetUrlResp, fileName string, fileSize int64, fileMime string, modTime time.Time) (model.Obj, error) {
commitBody := map[string]interface{}{
"fullId": getUrlResp.FullID,
"filePath": getUrlResp.FilePath,
"sha256": getUrlResp.Oid,
"fileSize": fileSize,
"fileName": fileName,
"fileType": fileMime,
"channelName": getUrlResp.ChannelName,
}
var commitResp hfCommitResp
_, err := d.doRequest(ctx, http.MethodPost, hfCommitApi, func(req *resty.Request) {
req.SetBody(commitBody)
}, &commitResp)
if err != nil {
return nil, fmt.Errorf("HF commit request failed: %w", err)
}
if !commitResp.Success {
return nil, fmt.Errorf("HF commit failed: success=false")
}
srcPath := strings.TrimPrefix(commitResp.Src, "/file/")
srcPath = strings.TrimPrefix(srcPath, "/")
if commitResp.PublicUrl != "" {
if u, err := url.Parse(commitResp.PublicUrl); err == nil {
d.publicUrlPrefix = u.Scheme + "://" + u.Host
}
}
return &model.Object{
Path: srcPath,
Name: fileName,
Size: fileSize,
Modified: modTime,
IsFolder: false,
}, nil
}
-70
View File
@@ -1,70 +0,0 @@
package cloudflare_imgbed
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/go-resty/resty/v2"
log "github.com/sirupsen/logrus"
)
const (
listApi = "/api/manage/list"
deleteApi = "/api/manage/delete"
uploadApi = "/upload"
hfGetUrlApi = "/upload/huggingface/getUploadUrl"
hfCommitApi = "/upload/huggingface/commitUpload"
hfDirectThreshold int64 = 20 * 1024 * 1024
fileSampleSize = 512 // HF 申请上传地址时需提供文件前 512 字节的 Sample
)
// doRequest 通用请求封装,包含重试和 API 错误解析
func (d *CFImgBed) doRequest(ctx context.Context, method, urlPath string, callback func(*resty.Request), resp interface{}) ([]byte, error) {
maxRetries := 3
for i := 0; i < maxRetries; i++ {
req := d.client.R()
req.SetContext(ctx)
if callback != nil {
callback(req)
}
if resp != nil {
req.SetResult(resp)
}
res, err := req.Execute(method, urlPath)
if err != nil {
log.WithError(err).Warnf("request %s %s failed, attempt %d/%d", method, urlPath, i+1, maxRetries)
if i < maxRetries-1 {
time.Sleep(time.Duration(i+1) * time.Second)
continue
}
return nil, err
}
// Retry on rate limit before attempting to interpret the body as an API error.
if res.StatusCode() == 429 {
time.Sleep(time.Duration(i+1) * 2 * time.Second)
continue
}
body := res.Body()
var apiErr apiError
if err := json.Unmarshal(body, &apiErr); err == nil {
if apiErr.Error != "" || apiErr.Message != "" {
msg := apiErr.Error
if msg == "" {
msg = apiErr.Message
}
return nil, fmt.Errorf("API error: %s", msg)
}
}
if res.IsError() {
return nil, fmt.Errorf("HTTP %d", res.StatusCode())
}
return body, nil
}
return nil, fmt.Errorf("max retries exceeded")
}
-232
View File
@@ -1,232 +0,0 @@
package cnb_releases
import (
"bytes"
"context"
"fmt"
"io"
"mime/multipart"
"net/http"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/go-resty/resty/v2"
)
type CnbReleases struct {
model.Storage
Addition
ref *CnbReleases
}
func (d *CnbReleases) Config() driver.Config {
return config
}
func (d *CnbReleases) GetAddition() driver.Additional {
return &d.Addition
}
func (d *CnbReleases) Init(ctx context.Context) error {
return nil
}
func (d *CnbReleases) InitReference(storage driver.Driver) error {
refStorage, ok := storage.(*CnbReleases)
if ok {
d.ref = refStorage
return nil
}
return fmt.Errorf("ref: storage is not CnbReleases")
}
func (d *CnbReleases) Drop(ctx context.Context) error {
d.ref = nil
return nil
}
func (d *CnbReleases) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
dirID := dir.GetID()
if dirID == "" {
// get all releases for root dir
var resp ReleaseList
err := d.Request(http.MethodGet, "/{repo}/-/releases", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
}, &resp)
if err != nil {
return nil, err
}
return utils.SliceConvert(resp, func(src Release) (model.Obj, error) {
name := src.Name
if d.UseTagName {
name = src.TagName
}
return &model.Object{
ID: src.ID,
Name: name,
Size: d.sumAssetsSize(src.Assets),
Ctime: src.CreatedAt,
Modified: src.UpdatedAt,
IsFolder: true,
}, nil
})
}
var resp Release
err := d.Request(http.MethodGet, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
req.SetPathParam("release_id", dirID)
}, &resp)
if err != nil {
return nil, err
}
return utils.SliceConvert(resp.Assets, func(src ReleaseAsset) (model.Obj, error) {
return &Object{
Object: model.Object{
ID: src.ID,
Path: src.Path,
Name: src.Name,
Size: src.Size,
Ctime: src.CreatedAt,
Modified: src.UpdatedAt,
IsFolder: false,
},
ParentID: dirID,
}, nil
})
}
func (d *CnbReleases) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
return &model.Link{
URL: "https://cnb.cool" + file.GetPath(),
}, nil
}
func (d *CnbReleases) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
if parentDir.GetPath() == "/" {
// create a new release
branch := d.DefaultBranch
if branch == "" {
branch = "main" // fallback to "main" if not set
}
return d.Request(http.MethodPost, "/{repo}/-/releases", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
req.SetBody(base.Json{
"name": dirName,
"tag_name": dirName,
"target_commitish": branch,
})
}, nil)
}
return errs.NotImplement
}
func (d *CnbReleases) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
return nil, errs.NotImplement
}
func (d *CnbReleases) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
if srcObj.IsDir() && !d.UseTagName {
return d.Request(http.MethodPatch, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
req.SetPathParam("release_id", srcObj.GetID())
req.SetFormData(map[string]string{
"name": newName,
})
}, nil)
}
return errs.NotImplement
}
func (d *CnbReleases) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
return nil, errs.NotImplement
}
func (d *CnbReleases) Remove(ctx context.Context, obj model.Obj) error {
if obj.IsDir() {
return d.Request(http.MethodDelete, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
req.SetPathParam("release_id", obj.GetID())
}, nil)
}
if o, ok := obj.(*Object); ok {
return d.Request(http.MethodDelete, "/{repo}/-/releases/{release_id}/assets/{asset_id}", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
req.SetPathParam("release_id", o.ParentID)
req.SetPathParam("asset_id", obj.GetID())
}, nil)
} else {
return fmt.Errorf("unable to get release ID")
}
}
func (d *CnbReleases) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
// 1. get upload info
var resp ReleaseAssetUploadURL
err := d.Request(http.MethodPost, "/{repo}/-/releases/{release_id}/asset-upload-url", func(req *resty.Request) {
req.SetPathParam("repo", d.Repo)
req.SetPathParam("release_id", dstDir.GetID())
req.SetBody(base.Json{
"asset_name": file.GetName(),
"overwrite": true,
"size": file.GetSize(),
})
}, &resp)
if err != nil {
return err
}
// 2. upload file
// use multipart to create form file
var b bytes.Buffer
w := multipart.NewWriter(&b)
_, err = w.CreateFormFile("file", file.GetName())
if err != nil {
return err
}
headSize := b.Len()
err = w.Close()
if err != nil {
return err
}
head := bytes.NewReader(b.Bytes()[:headSize])
tail := bytes.NewReader(b.Bytes()[headSize:])
r := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
Reader: &driver.SimpleReaderWithSize{
Reader: io.MultiReader(head, file, tail),
Size: int64(b.Len()) + file.GetSize(),
},
UpdateProgress: up,
})
// use net/http to upload file
ctxWithTimeout, cancel := context.WithTimeout(ctx, time.Duration(resp.ExpiresInSec+1)*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctxWithTimeout, http.MethodPost, resp.UploadURL, r)
if err != nil {
return err
}
req.Header.Set("Content-Type", w.FormDataContentType())
req.Header.Set("User-Agent", base.UserAgent)
httpResp, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer httpResp.Body.Close()
if httpResp.StatusCode != http.StatusNoContent {
return fmt.Errorf("upload file failed: %s", httpResp.Status)
}
// 3. verify upload
return d.Request(http.MethodPost, resp.VerifyURL, nil, nil)
}
var _ driver.Driver = (*CnbReleases)(nil)
-25
View File
@@ -1,25 +0,0 @@
package cnb_releases
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootID
Repo string `json:"repo" type:"string" required:"true"`
Token string `json:"token" type:"string" required:"true"`
UseTagName bool `json:"use_tag_name" type:"bool" default:"false" help:"Use tag name instead of release name"`
DefaultBranch string `json:"default_branch" type:"string" default:"main" help:"Default branch for new releases"`
}
var config = driver.Config{
Name: "CNB Releases",
LocalSort: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &CnbReleases{}
})
}
-100
View File
@@ -1,100 +0,0 @@
package cnb_releases
import (
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
)
type Object struct {
model.Object
ParentID string
}
type TagList []Tag
type Tag struct {
Commit struct {
Author UserInfo `json:"author"`
Commit CommitObject `json:"commit"`
Committer UserInfo `json:"committer"`
Parents []CommitParent `json:"parents"`
Sha string `json:"sha"`
} `json:"commit"`
Name string `json:"name"`
Target string `json:"target"`
TargetType string `json:"target_type"`
Verification TagObjectVerification `json:"verification"`
}
type UserInfo struct {
Freeze bool `json:"freeze"`
Nickname string `json:"nickname"`
Username string `json:"username"`
}
type CommitObject struct {
Author Signature `json:"author"`
CommentCount int `json:"comment_count"`
Committer Signature `json:"committer"`
Message string `json:"message"`
Tree CommitObjectTree `json:"tree"`
Verification CommitObjectVerification `json:"verification"`
}
type Signature struct {
Date time.Time `json:"date"`
Email string `json:"email"`
Name string `json:"name"`
}
type CommitObjectTree struct {
Sha string `json:"sha"`
}
type CommitObjectVerification struct {
Payload string `json:"payload"`
Reason string `json:"reason"`
Signature string `json:"signature"`
Verified bool `json:"verified"`
VerifiedAt string `json:"verified_at"`
}
type CommitParent = CommitObjectTree
type TagObjectVerification = CommitObjectVerification
type ReleaseList []Release
type Release struct {
Assets []ReleaseAsset `json:"assets"`
Author UserInfo `json:"author"`
Body string `json:"body"`
CreatedAt time.Time `json:"created_at"`
Draft bool `json:"draft"`
ID string `json:"id"`
IsLatest bool `json:"is_latest"`
Name string `json:"name"`
Prerelease bool `json:"prerelease"`
PublishedAt time.Time `json:"published_at"`
TagCommitish string `json:"tag_commitish"`
TagName string `json:"tag_name"`
UpdatedAt time.Time `json:"updated_at"`
}
type ReleaseAsset struct {
ContentType string `json:"content_type"`
CreatedAt time.Time `json:"created_at"`
ID string `json:"id"`
Name string `json:"name"`
Path string `json:"path"`
Size int64 `json:"size"`
UpdatedAt time.Time `json:"updated_at"`
Uploader UserInfo `json:"uploader"`
}
type ReleaseAssetUploadURL struct {
UploadURL string `json:"upload_url"`
ExpiresInSec int `json:"expires_in_sec"`
VerifyURL string `json:"verify_url"`
}
-58
View File
@@ -1,58 +0,0 @@
package cnb_releases
import (
"encoding/json"
"fmt"
"net/http"
"strings"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
log "github.com/sirupsen/logrus"
)
// do others that not defined in Driver interface
func (d *CnbReleases) Request(method string, path string, callback base.ReqCallback, resp any) error {
if d.ref != nil {
return d.ref.Request(method, path, callback, resp)
}
var url string
if strings.HasPrefix(path, "http") {
url = path
} else {
url = "https://api.cnb.cool" + path
}
req := base.RestyClient.R()
req.SetHeader("Accept", "application/json")
req.SetAuthScheme("Bearer")
req.SetAuthToken(d.Token)
if callback != nil {
callback(req)
}
res, err := req.Execute(method, url)
log.Debugln(res.String())
if err != nil {
return err
}
if res.StatusCode() != http.StatusOK && res.StatusCode() != http.StatusCreated && res.StatusCode() != http.StatusNoContent {
return fmt.Errorf("failed to request %s, status code: %d, message: %s", url, res.StatusCode(), res.String())
}
if resp != nil {
err = json.Unmarshal(res.Body(), resp)
if err != nil {
return err
}
}
return nil
}
func (d *CnbReleases) sumAssetsSize(assets []ReleaseAsset) int64 {
var size int64
for _, asset := range assets {
size += asset.Size
}
return size
}
-29
View File
@@ -1,29 +0,0 @@
package crypt
import (
stdpath "path"
"path/filepath"
"strings"
)
// will give the best guessing based on the path
func guessPath(path string) (isFolder, secondTry bool) {
if strings.HasSuffix(path, "/") {
//confirmed a folder
return true, false
}
lastSlash := strings.LastIndex(path, "/")
if !strings.Contains(path[lastSlash:], ".") {
//no dot, try folder then try file
return true, true
}
return false, true
}
func (d *Crypt) encryptPath(path string, isFolder bool) string {
if isFolder {
return d.cipher.EncryptDirName(path)
}
dir, fileName := filepath.Split(path)
return stdpath.Join(d.cipher.EncryptDirName(dir), d.cipher.EncryptFileName(fileName))
}
-224
View File
@@ -1,224 +0,0 @@
package degoo
import (
"context"
"fmt"
"net/http"
"strconv"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
type Degoo struct {
model.Storage
Addition
client *http.Client
}
func (d *Degoo) Config() driver.Config {
return config
}
func (d *Degoo) GetAddition() driver.Additional {
return &d.Addition
}
func (d *Degoo) Init(ctx context.Context) error {
d.client = base.HttpClient
// Ensure we have a valid token (will login if needed or refresh if expired)
if err := d.ensureValidToken(ctx); err != nil {
return fmt.Errorf("failed to initialize token: %w", err)
}
return d.getDevices(ctx)
}
func (d *Degoo) Drop(ctx context.Context) error {
return nil
}
func (d *Degoo) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
items, err := d.getAllFileChildren5(ctx, dir.GetID())
if err != nil {
return nil, err
}
return utils.MustSliceConvert(items, func(s DegooFileItem) model.Obj {
isFolder := s.Category == 2 || s.Category == 1 || s.Category == 10
createTime, modTime, _ := humanReadableTimes(s.CreationTime, s.LastModificationTime, s.LastUploadTime)
size, err := strconv.ParseInt(s.Size, 10, 64)
if err != nil {
size = 0 // Default to 0 if size parsing fails
}
return &model.Object{
ID: s.ID,
Path: s.FilePath,
Name: s.Name,
Size: size,
Modified: modTime,
Ctime: createTime,
IsFolder: isFolder,
}
}), nil
}
func (d *Degoo) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
item, err := d.getOverlay4(ctx, file.GetID())
if err != nil {
return nil, err
}
return &model.Link{URL: item.URL}, nil
}
func (d *Degoo) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
// This is done by calling the setUploadFile3 API with a special checksum and size.
const query = `mutation SetUploadFile3($Token: String!, $FileInfos: [FileInfoUpload3]!) { setUploadFile3(Token: $Token, FileInfos: $FileInfos) }`
variables := map[string]interface{}{
"Token": d.AccessToken,
"FileInfos": []map[string]interface{}{
{
"Checksum": folderChecksum,
"Name": dirName,
"CreationTime": time.Now().UnixMilli(),
"ParentID": parentDir.GetID(),
"Size": 0,
},
},
}
_, err := d.apiCall(ctx, "SetUploadFile3", query, variables)
if err != nil {
return err
}
return nil
}
func (d *Degoo) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
const query = `mutation SetMoveFile($Token: String!, $Copy: Boolean, $NewParentID: String!, $FileIDs: [String]!) { setMoveFile(Token: $Token, Copy: $Copy, NewParentID: $NewParentID, FileIDs: $FileIDs) }`
variables := map[string]interface{}{
"Token": d.AccessToken,
"Copy": false,
"NewParentID": dstDir.GetID(),
"FileIDs": []string{srcObj.GetID()},
}
_, err := d.apiCall(ctx, "SetMoveFile", query, variables)
if err != nil {
return nil, err
}
return srcObj, nil
}
func (d *Degoo) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
const query = `mutation SetRenameFile($Token: String!, $FileRenames: [FileRenameInfo]!) { setRenameFile(Token: $Token, FileRenames: $FileRenames) }`
variables := map[string]interface{}{
"Token": d.AccessToken,
"FileRenames": []DegooFileRenameInfo{
{
ID: srcObj.GetID(),
NewName: newName,
},
},
}
_, err := d.apiCall(ctx, "SetRenameFile", query, variables)
if err != nil {
return err
}
return nil
}
func (d *Degoo) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
// Copy is not implemented, Degoo API does not support direct copy.
return nil, errs.NotImplement
}
func (d *Degoo) Remove(ctx context.Context, obj model.Obj) error {
// Remove deletes a file or folder (moves to trash).
const query = `mutation SetDeleteFile5($Token: String!, $IsInRecycleBin: Boolean!, $IDs: [IDType]!) { setDeleteFile5(Token: $Token, IsInRecycleBin: $IsInRecycleBin, IDs: $IDs) }`
variables := map[string]interface{}{
"Token": d.AccessToken,
"IsInRecycleBin": false,
"IDs": []map[string]string{{"FileID": obj.GetID()}},
}
_, err := d.apiCall(ctx, "SetDeleteFile5", query, variables)
return err
}
func (d *Degoo) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
tmpF, err := file.CacheFullAndWriter(&up, nil)
if err != nil {
return err
}
parentID := dstDir.GetID()
// Calculate the checksum for the file.
checksum, err := d.checkSum(tmpF)
if err != nil {
return err
}
// 1. Get upload authorization via getBucketWriteAuth4.
auths, err := d.getBucketWriteAuth4(ctx, file, parentID, checksum)
if err != nil {
return err
}
// 2. Upload file.
// support rapid upload
if auths.GetBucketWriteAuth4[0].Error != "Already exist!" {
err = d.uploadS3(ctx, auths, tmpF, file, checksum)
if err != nil {
return err
}
}
// 3. Register metadata with setUploadFile3.
data, err := d.SetUploadFile3(ctx, file, parentID, checksum)
if err != nil {
return err
}
if !data.SetUploadFile3 {
return fmt.Errorf("setUploadFile3 failed: %v", data)
}
return nil
}
func (d *Degoo) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
quota, err := d.getUserInfo(ctx)
if err != nil {
return nil, err
}
used, err := strconv.ParseInt(quota.GetUserInfo3.UsedQuota, 10, 64)
if err != nil {
return nil, fmt.Errorf("failed to parse used quota: %v", err)
}
total, err := strconv.ParseInt(quota.GetUserInfo3.TotalQuota, 10, 64)
if err != nil {
return nil, fmt.Errorf("failed to parse total quota: %v", err)
}
return &model.StorageDetails{
DiskUsage: model.DiskUsage{
TotalSpace: total,
UsedSpace: used,
},
}, nil
}
-27
View File
@@ -1,27 +0,0 @@
package degoo
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootID
Username string `json:"username" help:"Your Degoo account email"`
Password string `json:"password" help:"Your Degoo account password"`
RefreshToken string `json:"refresh_token" help:"Refresh token for automatic token renewal, obtained automatically"`
AccessToken string `json:"access_token" help:"Access token for Degoo API, obtained automatically"`
}
var config = driver.Config{
Name: "Degoo",
LocalSort: true,
DefaultRoot: "0",
NoOverwriteUpload: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Degoo{}
})
}
-133
View File
@@ -1,133 +0,0 @@
package degoo
import (
"encoding/json"
)
// DegooLoginRequest represents the login request body.
type DegooLoginRequest struct {
GenerateToken bool `json:"GenerateToken"`
Username string `json:"Username"`
Password string `json:"Password"`
}
// DegooLoginResponse represents a successful login response.
type DegooLoginResponse struct {
Token string `json:"Token"`
RefreshToken string `json:"RefreshToken"`
}
// DegooAccessTokenRequest represents the token refresh request body.
type DegooAccessTokenRequest struct {
RefreshToken string `json:"RefreshToken"`
}
// DegooAccessTokenResponse represents the token refresh response.
type DegooAccessTokenResponse struct {
AccessToken string `json:"AccessToken"`
}
// DegooFileItem represents a Degoo file or folder.
type DegooFileItem struct {
ID string `json:"ID"`
ParentID string `json:"ParentID"`
Name string `json:"Name"`
Category int `json:"Category"`
Size string `json:"Size"`
URL string `json:"URL"`
CreationTime string `json:"CreationTime"`
LastModificationTime string `json:"LastModificationTime"`
LastUploadTime string `json:"LastUploadTime"`
MetadataID string `json:"MetadataID"`
DeviceID int64 `json:"DeviceID"`
FilePath string `json:"FilePath"`
IsInRecycleBin bool `json:"IsInRecycleBin"`
}
type DegooErrors struct {
Path []string `json:"path"`
Data interface{} `json:"data"`
ErrorType string `json:"errorType"`
ErrorInfo interface{} `json:"errorInfo"`
Message string `json:"message"`
}
// DegooGraphqlResponse is the common structure for GraphQL API responses.
type DegooGraphqlResponse struct {
Data json.RawMessage `json:"data"`
Errors []DegooErrors `json:"errors,omitempty"`
}
// DegooGetChildren5Data is the data field for getFileChildren5.
type DegooGetChildren5Data struct {
GetFileChildren5 struct {
Items []DegooFileItem `json:"Items"`
NextToken string `json:"NextToken"`
} `json:"getFileChildren5"`
}
// DegooGetOverlay4Data is the data field for getOverlay4.
type DegooGetOverlay4Data struct {
GetOverlay4 DegooFileItem `json:"getOverlay4"`
}
// DegooFileRenameInfo represents a file rename operation.
type DegooFileRenameInfo struct {
ID string `json:"ID"`
NewName string `json:"NewName"`
}
// DegooFileIDs represents a list of file IDs for move operations.
type DegooFileIDs struct {
FileIDs []string `json:"FileIDs"`
}
// DegooGetBucketWriteAuth4Data is the data field for GetBucketWriteAuth4.
type DegooGetBucketWriteAuth4Data struct {
GetBucketWriteAuth4 []struct {
AuthData struct {
PolicyBase64 string `json:"PolicyBase64"`
Signature string `json:"Signature"`
BaseURL string `json:"BaseURL"`
KeyPrefix string `json:"KeyPrefix"`
AccessKey struct {
Key string `json:"Key"`
Value string `json:"Value"`
} `json:"AccessKey"`
ACL string `json:"ACL"`
AdditionalBody []struct {
Key string `json:"Key"`
Value string `json:"Value"`
} `json:"AdditionalBody"`
} `json:"AuthData"`
Error interface{} `json:"Error"`
} `json:"getBucketWriteAuth4"`
}
// DegooSetUploadFile3Data is the data field for SetUploadFile3.
type DegooSetUploadFile3Data struct {
SetUploadFile3 bool `json:"setUploadFile3"`
}
type DegooGetUserInfo3Data struct {
GetUserInfo3 struct {
// ID string
// FirstName string
// LastName string
// Email string
// AvatarURL string
// CountryCode string = CN
// LanguageCode string = zh-cn
// Phone string
// AccountType int
UsedQuota string `json:"UsedQuota"`
TotalQuota string `json:"TotalQuota"`
// OAuth2Provider
// GPMigrationStatus int
// FeatureNoAds bool
// FeatureTopSecret bool
// FeatureDownsampling bool
// FeatureAutomaticVideoUploads bool
// FileSizeLimit string
} `json:"getUserInfo3"`
}
-198
View File
@@ -1,198 +0,0 @@
package degoo
import (
"bytes"
"context"
"crypto/sha1"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"mime/multipart"
"net/http"
"strconv"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
func (d *Degoo) getBucketWriteAuth4(ctx context.Context, file model.FileStreamer, parentID string, checksum string) (*DegooGetBucketWriteAuth4Data, error) {
const query = `query GetBucketWriteAuth4(
$Token: String!
$ParentID: String!
$StorageUploadInfos: [StorageUploadInfo2]
) {
getBucketWriteAuth4(
Token: $Token
ParentID: $ParentID
StorageUploadInfos: $StorageUploadInfos
) {
AuthData {
PolicyBase64
Signature
BaseURL
KeyPrefix
AccessKey {
Key
Value
}
ACL
AdditionalBody {
Key
Value
}
}
Error
}
}`
variables := map[string]interface{}{
"Token": d.AccessToken,
"ParentID": parentID,
"StorageUploadInfos": []map[string]string{{
"FileName": file.GetName(),
"Checksum": checksum,
"Size": strconv.FormatInt(file.GetSize(), 10),
}}}
data, err := d.apiCall(ctx, "GetBucketWriteAuth4", query, variables)
if err != nil {
return nil, err
}
var resp DegooGetBucketWriteAuth4Data
err = json.Unmarshal(data, &resp)
if err != nil {
return nil, err
}
return &resp, nil
}
// checkSum calculates the SHA1-based checksum for Degoo upload API.
func (d *Degoo) checkSum(file io.Reader) (string, error) {
seed := []byte{13, 7, 2, 2, 15, 40, 75, 117, 13, 10, 19, 16, 29, 23, 3, 36}
hasher := sha1.New()
hasher.Write(seed)
if _, err := utils.CopyWithBuffer(hasher, file); err != nil {
return "", err
}
cs := hasher.Sum(nil)
csBytes := []byte{10, byte(len(cs))}
csBytes = append(csBytes, cs...)
csBytes = append(csBytes, 16, 0)
return strings.ReplaceAll(base64.StdEncoding.EncodeToString(csBytes), "/", "_"), nil
}
func (d *Degoo) uploadS3(ctx context.Context, auths *DegooGetBucketWriteAuth4Data, tmpF model.File, file model.FileStreamer, checksum string) error {
a := auths.GetBucketWriteAuth4[0].AuthData
_, err := tmpF.Seek(0, io.SeekStart)
if err != nil {
return err
}
ext := utils.Ext(file.GetName())
key := fmt.Sprintf("%s%s/%s.%s", a.KeyPrefix, ext, checksum, ext)
var b bytes.Buffer
w := multipart.NewWriter(&b)
err = w.WriteField("key", key)
if err != nil {
return err
}
err = w.WriteField("acl", a.ACL)
if err != nil {
return err
}
err = w.WriteField("policy", a.PolicyBase64)
if err != nil {
return err
}
err = w.WriteField("signature", a.Signature)
if err != nil {
return err
}
err = w.WriteField(a.AccessKey.Key, a.AccessKey.Value)
if err != nil {
return err
}
for _, additional := range a.AdditionalBody {
err = w.WriteField(additional.Key, additional.Value)
if err != nil {
return err
}
}
err = w.WriteField("Content-Type", "")
if err != nil {
return err
}
_, err = w.CreateFormFile("file", key)
if err != nil {
return err
}
headSize := b.Len()
err = w.Close()
if err != nil {
return err
}
head := bytes.NewReader(b.Bytes()[:headSize])
tail := bytes.NewReader(b.Bytes()[headSize:])
rateLimitedRd := driver.NewLimitedUploadStream(ctx, io.MultiReader(head, tmpF, tail))
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.BaseURL, rateLimitedRd)
if err != nil {
return err
}
req.Header.Add("ngsw-bypass", "1")
req.Header.Add("Content-Type", w.FormDataContentType())
res, err := d.client.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode != http.StatusNoContent {
return fmt.Errorf("upload failed with status code %d", res.StatusCode)
}
return nil
}
var _ driver.Driver = (*Degoo)(nil)
func (d *Degoo) SetUploadFile3(ctx context.Context, file model.FileStreamer, parentID string, checksum string) (*DegooSetUploadFile3Data, error) {
const query = `mutation SetUploadFile3($Token: String!, $FileInfos: [FileInfoUpload3]!) {
setUploadFile3(Token: $Token, FileInfos: $FileInfos)
}`
variables := map[string]interface{}{
"Token": d.AccessToken,
"FileInfos": []map[string]string{{
"Checksum": checksum,
"CreationTime": strconv.FormatInt(file.CreateTime().UnixMilli(), 10),
"Name": file.GetName(),
"ParentID": parentID,
"Size": strconv.FormatInt(file.GetSize(), 10),
}}}
data, err := d.apiCall(ctx, "SetUploadFile3", query, variables)
if err != nil {
return nil, err
}
var resp DegooSetUploadFile3Data
err = json.Unmarshal(data, &resp)
if err != nil {
return nil, err
}
return &resp, nil
}
-478
View File
@@ -1,478 +0,0 @@
package degoo
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
// Thanks to https://github.com/bernd-wechner/Degoo for API research.
const (
// API endpoints
loginURL = "https://rest-api.degoo.com/login"
accessTokenURL = "https://rest-api.degoo.com/access-token/v2"
apiURL = "https://production-appsync.degoo.com/graphql"
// API configuration
apiKey = "da2-vs6twz5vnjdavpqndtbzg3prra"
folderChecksum = "CgAQAg"
// Token management
tokenRefreshThreshold = 5 * time.Minute
// Rate limiting
minRequestInterval = 1 * time.Second
// Error messages
errRateLimited = "rate limited (429), please try again later"
errUnauthorized = "unauthorized access"
)
var (
// Global rate limiting - protects against concurrent API calls
lastRequestTime time.Time
requestMutex sync.Mutex
)
// JWT payload structure for token expiration checking
type JWTPayload struct {
UserID string `json:"userID"`
Exp int64 `json:"exp"`
Iat int64 `json:"iat"`
}
// Rate limiting helper functions
// applyRateLimit ensures minimum interval between API requests
func applyRateLimit() {
requestMutex.Lock()
defer requestMutex.Unlock()
if !lastRequestTime.IsZero() {
if elapsed := time.Since(lastRequestTime); elapsed < minRequestInterval {
time.Sleep(minRequestInterval - elapsed)
}
}
lastRequestTime = time.Now()
}
// HTTP request helper functions
// createJSONRequest creates a new HTTP request with JSON body
func createJSONRequest(ctx context.Context, method, url string, body interface{}) (*http.Request, error) {
jsonBody, err := json.Marshal(body)
if err != nil {
return nil, fmt.Errorf("failed to marshal request body: %w", err)
}
req, err := http.NewRequestWithContext(ctx, method, url, bytes.NewBuffer(jsonBody))
if err != nil {
return nil, fmt.Errorf("failed to create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", base.UserAgent)
return req, nil
}
// checkHTTPResponse checks for common HTTP error conditions
func checkHTTPResponse(resp *http.Response, operation string) error {
if resp.StatusCode == http.StatusTooManyRequests {
return fmt.Errorf("%s %s", operation, errRateLimited)
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("%s failed: %s", operation, resp.Status)
}
return nil
}
// isTokenExpired checks if the JWT token is expired or will expire soon
func (d *Degoo) isTokenExpired() bool {
if d.AccessToken == "" {
return true
}
payload, err := extractJWTPayload(d.AccessToken)
if err != nil {
return true // Invalid token format
}
// Check if token expires within the threshold
expireTime := time.Unix(payload.Exp, 0)
return time.Now().Add(tokenRefreshThreshold).After(expireTime)
}
// extractJWTPayload extracts and parses JWT payload
func extractJWTPayload(token string) (*JWTPayload, error) {
parts := strings.Split(token, ".")
if len(parts) != 3 {
return nil, fmt.Errorf("invalid JWT format")
}
// Decode the payload (second part)
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return nil, fmt.Errorf("failed to decode JWT payload: %w", err)
}
var jwtPayload JWTPayload
if err := json.Unmarshal(payload, &jwtPayload); err != nil {
return nil, fmt.Errorf("failed to parse JWT payload: %w", err)
}
return &jwtPayload, nil
}
// refreshToken attempts to refresh the access token using the refresh token
func (d *Degoo) refreshToken(ctx context.Context) error {
if d.RefreshToken == "" {
return fmt.Errorf("no refresh token available")
}
// Create request
tokenReq := DegooAccessTokenRequest{RefreshToken: d.RefreshToken}
req, err := createJSONRequest(ctx, "POST", accessTokenURL, tokenReq)
if err != nil {
return fmt.Errorf("failed to create refresh token request: %w", err)
}
// Execute request
resp, err := d.client.Do(req)
if err != nil {
return fmt.Errorf("refresh token request failed: %w", err)
}
defer resp.Body.Close()
// Check response
if err := checkHTTPResponse(resp, "refresh token"); err != nil {
return err
}
var accessTokenResp DegooAccessTokenResponse
if err := json.NewDecoder(resp.Body).Decode(&accessTokenResp); err != nil {
return fmt.Errorf("failed to parse access token response: %w", err)
}
if accessTokenResp.AccessToken == "" {
return fmt.Errorf("empty access token received")
}
d.AccessToken = accessTokenResp.AccessToken
// Save the updated token to storage
op.MustSaveDriverStorage(d)
return nil
}
// ensureValidToken ensures we have a valid, non-expired token
func (d *Degoo) ensureValidToken(ctx context.Context) error {
// Check if token is expired or will expire soon
if d.isTokenExpired() {
// Try to refresh token first if we have a refresh token
if d.RefreshToken != "" {
if refreshErr := d.refreshToken(ctx); refreshErr == nil {
return nil // Successfully refreshed
} else {
// If refresh failed, fall back to full login
fmt.Printf("Token refresh failed, falling back to full login: %v\n", refreshErr)
}
}
// Perform full login
if d.Username != "" && d.Password != "" {
return d.login(ctx)
}
}
return nil
}
// login performs the login process and retrieves the access token.
func (d *Degoo) login(ctx context.Context) error {
if d.Username == "" || d.Password == "" {
return fmt.Errorf("username or password not provided")
}
creds := DegooLoginRequest{
GenerateToken: true,
Username: d.Username,
Password: d.Password,
}
jsonCreds, err := json.Marshal(creds)
if err != nil {
return fmt.Errorf("failed to serialize login credentials: %w", err)
}
req, err := http.NewRequestWithContext(ctx, "POST", loginURL, bytes.NewBuffer(jsonCreds))
if err != nil {
return fmt.Errorf("failed to create login request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", base.UserAgent)
req.Header.Set("Origin", "https://app.degoo.com")
resp, err := d.client.Do(req)
if err != nil {
return fmt.Errorf("login request failed: %w", err)
}
defer resp.Body.Close()
// Handle rate limiting (429 Too Many Requests)
if resp.StatusCode == http.StatusTooManyRequests {
return fmt.Errorf("login rate limited (429), please try again later")
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("login failed: %s", resp.Status)
}
var loginResp DegooLoginResponse
if err := json.NewDecoder(resp.Body).Decode(&loginResp); err != nil {
return fmt.Errorf("failed to parse login response: %w", err)
}
if loginResp.RefreshToken != "" {
tokenReq := DegooAccessTokenRequest{RefreshToken: loginResp.RefreshToken}
jsonTokenReq, err := json.Marshal(tokenReq)
if err != nil {
return fmt.Errorf("failed to serialize access token request: %w", err)
}
tokenReqHTTP, err := http.NewRequestWithContext(ctx, "POST", accessTokenURL, bytes.NewBuffer(jsonTokenReq))
if err != nil {
return fmt.Errorf("failed to create access token request: %w", err)
}
tokenReqHTTP.Header.Set("User-Agent", base.UserAgent)
tokenResp, err := d.client.Do(tokenReqHTTP)
if err != nil {
return fmt.Errorf("failed to get access token: %w", err)
}
defer tokenResp.Body.Close()
var accessTokenResp DegooAccessTokenResponse
if err := json.NewDecoder(tokenResp.Body).Decode(&accessTokenResp); err != nil {
return fmt.Errorf("failed to parse access token response: %w", err)
}
d.AccessToken = accessTokenResp.AccessToken
d.RefreshToken = loginResp.RefreshToken // Save refresh token
} else if loginResp.Token != "" {
d.AccessToken = loginResp.Token
d.RefreshToken = "" // Direct token, no refresh token available
} else {
return fmt.Errorf("login failed, no valid token returned")
}
// Save the updated tokens to storage
op.MustSaveDriverStorage(d)
return nil
}
// apiCall performs a Degoo GraphQL API request.
func (d *Degoo) apiCall(ctx context.Context, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
// Apply rate limiting
applyRateLimit()
// Ensure we have a valid token before making the API call
if err := d.ensureValidToken(ctx); err != nil {
return nil, fmt.Errorf("failed to ensure valid token: %w", err)
}
// Update the Token in variables if it exists (after potential refresh)
d.updateTokenInVariables(variables)
return d.executeGraphQLRequest(ctx, operationName, query, variables)
}
// updateTokenInVariables updates the Token field in GraphQL variables
func (d *Degoo) updateTokenInVariables(variables map[string]interface{}) {
if variables != nil {
if _, hasToken := variables["Token"]; hasToken {
variables["Token"] = d.AccessToken
}
}
}
// executeGraphQLRequest executes a GraphQL request with retry logic
func (d *Degoo) executeGraphQLRequest(ctx context.Context, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
reqBody := map[string]interface{}{
"operationName": operationName,
"query": query,
"variables": variables,
}
// Create and configure request
req, err := createJSONRequest(ctx, "POST", apiURL, reqBody)
if err != nil {
return nil, err
}
// Set Degoo-specific headers
req.Header.Set("x-api-key", apiKey)
if d.AccessToken != "" {
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", d.AccessToken))
}
// Execute request
resp, err := d.client.Do(req)
if err != nil {
return nil, fmt.Errorf("GraphQL API request failed: %w", err)
}
defer resp.Body.Close()
// Check for HTTP errors
if err := checkHTTPResponse(resp, "GraphQL API"); err != nil {
return nil, err
}
// Parse GraphQL response
var degooResp DegooGraphqlResponse
if err := json.NewDecoder(resp.Body).Decode(&degooResp); err != nil {
return nil, fmt.Errorf("failed to decode GraphQL response: %w", err)
}
// Handle GraphQL errors
if len(degooResp.Errors) > 0 {
return d.handleGraphQLError(ctx, degooResp.Errors[0], operationName, query, variables)
}
return degooResp.Data, nil
}
// handleGraphQLError handles GraphQL-level errors with retry logic
func (d *Degoo) handleGraphQLError(ctx context.Context, gqlError DegooErrors, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
if gqlError.ErrorType == "Unauthorized" {
// Re-login and retry
if err := d.login(ctx); err != nil {
return nil, fmt.Errorf("%s, login failed: %w", errUnauthorized, err)
}
// Update token in variables and retry
d.updateTokenInVariables(variables)
return d.apiCall(ctx, operationName, query, variables)
}
return nil, fmt.Errorf("GraphQL API error: %s", gqlError.Message)
}
// humanReadableTimes converts Degoo timestamps to Go time.Time.
func humanReadableTimes(creation, modification, upload string) (cTime, mTime, uTime time.Time) {
cTime, _ = time.Parse(time.RFC3339, creation)
if modification != "" {
modMillis, _ := strconv.ParseInt(modification, 10, 64)
mTime = time.Unix(0, modMillis*int64(time.Millisecond))
}
if upload != "" {
upMillis, _ := strconv.ParseInt(upload, 10, 64)
uTime = time.Unix(0, upMillis*int64(time.Millisecond))
}
return cTime, mTime, uTime
}
// getDevices fetches and caches top-level devices and folders.
func (d *Degoo) getDevices(ctx context.Context) error {
const query = `query GetFileChildren5($Token: String! $ParentID: String $AllParentIDs: [String] $Limit: Int! $Order: Int! $NextToken: String ) { getFileChildren5(Token: $Token ParentID: $ParentID AllParentIDs: $AllParentIDs Limit: $Limit Order: $Order NextToken: $NextToken) { Items { ParentID } NextToken } }`
variables := map[string]interface{}{
"Token": d.AccessToken,
"ParentID": "0",
"Limit": 10,
"Order": 3,
}
data, err := d.apiCall(ctx, "GetFileChildren5", query, variables)
if err != nil {
return err
}
var resp DegooGetChildren5Data
if err := json.Unmarshal(data, &resp); err != nil {
return fmt.Errorf("failed to parse device list: %w", err)
}
if d.RootFolderID == "0" {
if len(resp.GetFileChildren5.Items) > 0 {
d.RootFolderID = resp.GetFileChildren5.Items[0].ParentID
}
op.MustSaveDriverStorage(d)
}
return nil
}
// getAllFileChildren5 fetches all children of a directory with pagination.
func (d *Degoo) getAllFileChildren5(ctx context.Context, parentID string) ([]DegooFileItem, error) {
const query = `query GetFileChildren5($Token: String! $ParentID: String $AllParentIDs: [String] $Limit: Int! $Order: Int! $NextToken: String ) { getFileChildren5(Token: $Token ParentID: $ParentID AllParentIDs: $AllParentIDs Limit: $Limit Order: $Order NextToken: $NextToken) { Items { ID ParentID Name Category Size CreationTime LastModificationTime LastUploadTime FilePath IsInRecycleBin DeviceID MetadataID } NextToken } }`
var allItems []DegooFileItem
nextToken := ""
for {
variables := map[string]interface{}{
"Token": d.AccessToken,
"ParentID": parentID,
"Limit": 1000,
"Order": 3,
}
if nextToken != "" {
variables["NextToken"] = nextToken
}
data, err := d.apiCall(ctx, "GetFileChildren5", query, variables)
if err != nil {
return nil, err
}
var resp DegooGetChildren5Data
if err := json.Unmarshal(data, &resp); err != nil {
return nil, err
}
allItems = append(allItems, resp.GetFileChildren5.Items...)
if resp.GetFileChildren5.NextToken == "" {
break
}
nextToken = resp.GetFileChildren5.NextToken
}
return allItems, nil
}
// getOverlay4 fetches metadata for a single item by ID.
func (d *Degoo) getOverlay4(ctx context.Context, id string) (DegooFileItem, error) {
const query = `query GetOverlay4($Token: String!, $ID: IDType!) { getOverlay4(Token: $Token, ID: $ID) { ID ParentID Name Category Size CreationTime LastModificationTime LastUploadTime URL FilePath IsInRecycleBin DeviceID MetadataID } }`
variables := map[string]interface{}{
"Token": d.AccessToken,
"ID": map[string]string{
"FileID": id,
},
}
data, err := d.apiCall(ctx, "GetOverlay4", query, variables)
if err != nil {
return DegooFileItem{}, err
}
var resp DegooGetOverlay4Data
if err := json.Unmarshal(data, &resp); err != nil {
return DegooFileItem{}, fmt.Errorf("failed to parse item metadata: %w", err)
}
return resp.GetOverlay4, nil
}
func (d *Degoo) getUserInfo(ctx context.Context) (DegooGetUserInfo3Data, error) {
const query = "query GetUserInfo3($Token: String!) { getUserInfo3(Token: $Token) { UsedQuota TotalQuota } }"
variables := map[string]interface{}{
"Token": d.AccessToken,
}
data, err := d.apiCall(ctx, "GetUserInfo3", query, variables)
var resp DegooGetUserInfo3Data
if err != nil {
return resp, err
}
if err = json.Unmarshal(data, &resp); err != nil {
return resp, fmt.Errorf("failed to parse user info: %w", err)
}
return resp, nil
}
-33
View File
@@ -1,33 +0,0 @@
package doubao
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
// Usually one of two
// driver.RootPath
driver.RootID
// define other
Cookie string `json:"cookie" type:"text"`
UploadThread string `json:"upload_thread" default:"3"`
DownloadApi string `json:"download_api" type:"select" options:"get_file_url,get_download_info" default:"get_file_url"`
LimitRate float64 `json:"limit_rate" type:"float" default:"2" help:"limit all api request rate ([limit]r/1s)"`
}
var config = driver.Config{
Name: "Doubao",
LocalSort: true,
DefaultRoot: "0",
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Doubao{
Addition: Addition{
LimitRate: 2,
},
}
})
}
-597
View File
@@ -1,597 +0,0 @@
package doubao_new
import (
"crypto/aes"
"crypto/cipher"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"math/big"
"net/url"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
"github.com/go-resty/resty/v2"
"github.com/google/uuid"
"golang.org/x/crypto/pbkdf2"
)
const (
defaultAuthRefreshAheadSeconds = int64(120)
defaultDpopRefreshAheadSeconds = int64(5)
)
type Clock interface {
Now() (int64, error)
}
type SystemClock struct{}
func (SystemClock) Now() (int64, error) { return time.Now().Unix(), nil }
type DPoPTokenInput struct {
KeyPair *ecdsa.PrivateKey
ExpiresIn int64 // 默认 15
JTI string
HTM string
HTU string
IAT int64
Nonce string
Clock Clock
}
type DPoPTokenOutput struct {
DPoPToken string `json:"dpopToken"`
ExpiredTime int64 `json:"expiredTime"`
ExpiresIn int64 `json:"expiresIn"`
}
type JWTPayload struct {
Exp int64 `json:"exp,omitempty"`
Iat int64 `json:"iat,omitempty"`
Nbf int64 `json:"nbf,omitempty"`
Jti string `json:"jti,omitempty"`
Htm string `json:"htm,omitempty"`
Htu string `json:"htu,omitempty"`
Nonce string `json:"nonce,omitempty"`
Sub string `json:"sub,omitempty"`
}
type jwkECPrivateKey struct {
Kty string `json:"kty"`
Crv string `json:"crv"`
X string `json:"x"`
Y string `json:"y"`
D string `json:"d"`
}
type dpopKeyPairEnvelope struct {
PrivateKey *jwkECPrivateKey `json:"privateKey"`
KeyPair *jwkECPrivateKey `json:"keyPair"`
JWK *jwkECPrivateKey `json:"jwk"`
}
type encryptedDpopKeyPair struct {
Data string `json:"data"`
Ciphertext string `json:"ciphertext"`
Encrypted string `json:"encrypted"`
Secret string `json:"secret"`
Password string `json:"password"`
Passphrase string `json:"passphrase"`
}
func GenerateDPoPToken(in DPoPTokenInput) (*DPoPTokenOutput, error) {
if in.KeyPair == nil {
return nil, errors.New("keyPair required")
}
if in.KeyPair.Curve != elliptic.P256() {
return nil, errors.New("ES256 requires P-256 key")
}
if in.Clock == nil {
in.Clock = SystemClock{}
}
if in.ExpiresIn <= 0 {
in.ExpiresIn = 15
}
now, err := in.Clock.Now()
if err != nil {
return nil, err
}
payload := map[string]any{
"jti": pickStr(in.JTI, uuid.NewString()),
"htm": pickStr(in.HTM, ""),
"htu": pickStr(in.HTU, ""),
"iat": pickI64(in.IAT, now),
"nonce": pickStr(in.Nonce, uuid.NewString()),
}
if in.ExpiresIn > 0 {
payload["exp"] = payload["iat"].(int64) + in.ExpiresIn
}
pub := in.KeyPair.PublicKey
header := map[string]any{
"typ": "dpop+jwt",
"alg": "ES256",
"jwk": map[string]string{
"kty": "EC",
"crv": "P-256",
"x": b64url(pad32(pub.X.Bytes())),
"y": b64url(pad32(pub.Y.Bytes())),
},
}
hb, err := json.Marshal(header)
if err != nil {
return nil, err
}
pb, err := json.Marshal(payload)
if err != nil {
return nil, err
}
hEnc := b64url(hb)
pEnc := b64url(pb)
signingInput := hEnc + "." + pEnc
sum := sha256.Sum256([]byte(signingInput))
r, s, err := ecdsa.Sign(rand.Reader, in.KeyPair, sum[:])
if err != nil {
return nil, err
}
sig := append(pad32(r.Bytes()), pad32(s.Bytes())...)
token := signingInput + "." + b64url(sig)
iat := payload["iat"].(int64)
return &DPoPTokenOutput{
DPoPToken: token,
ExpiredTime: iat + in.ExpiresIn,
ExpiresIn: in.ExpiresIn,
}, nil
}
func GenerateDPoPKeyPair() (*ecdsa.PrivateKey, error) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return nil, err
}
return validateP256Key(key)
}
func ParseJWTPayload(token string, out any) error {
token = strings.TrimSpace(trimTokenScheme(token))
parts := strings.Split(token, ".")
if len(parts) < 2 {
return fmt.Errorf("invalid JWT format")
}
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return fmt.Errorf("failed to decode JWT payload: %w", err)
}
if err := json.Unmarshal(payload, out); err != nil {
return fmt.Errorf("failed to parse JWT payload: %w", err)
}
return nil
}
func parseECPrivateKeyJWK(raw string) (*ecdsa.PrivateKey, error) {
var jwk jwkECPrivateKey
if err := json.Unmarshal([]byte(raw), &jwk); err != nil {
return nil, err
}
if jwk.D == "" || jwk.X == "" || jwk.Y == "" {
var env dpopKeyPairEnvelope
if err := json.Unmarshal([]byte(raw), &env); err != nil {
return nil, err
}
switch {
case env.PrivateKey != nil:
jwk = *env.PrivateKey
case env.KeyPair != nil:
jwk = *env.KeyPair
case env.JWK != nil:
jwk = *env.JWK
default:
return nil, errors.New("missing private key JWK")
}
}
if jwk.Kty != "" && jwk.Kty != "EC" {
return nil, errors.New("unsupported JWK kty")
}
if jwk.Crv != "" && jwk.Crv != "P-256" {
return nil, errors.New("unsupported JWK curve")
}
if jwk.D == "" || jwk.X == "" || jwk.Y == "" {
return nil, errors.New("incomplete JWK")
}
xBytes, err := base64.RawURLEncoding.DecodeString(jwk.X)
if err != nil {
return nil, fmt.Errorf("invalid jwk x: %w", err)
}
yBytes, err := base64.RawURLEncoding.DecodeString(jwk.Y)
if err != nil {
return nil, fmt.Errorf("invalid jwk y: %w", err)
}
dBytes, err := base64.RawURLEncoding.DecodeString(jwk.D)
if err != nil {
return nil, fmt.Errorf("invalid jwk d: %w", err)
}
key := &ecdsa.PrivateKey{
PublicKey: ecdsa.PublicKey{
Curve: elliptic.P256(),
X: new(big.Int).SetBytes(xBytes),
Y: new(big.Int).SetBytes(yBytes),
},
D: new(big.Int).SetBytes(dBytes),
}
return validateP256Key(key)
}
func parseEncryptedDPoPKeyPair(raw, secret string) (*ecdsa.PrivateKey, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil, errors.New("empty encrypted key pair")
}
var payload encryptedDpopKeyPair
ciphertext := raw
if strings.HasPrefix(raw, "{") {
if err := json.Unmarshal([]byte(raw), &payload); err != nil {
return nil, err
}
switch {
case strings.TrimSpace(payload.Data) != "":
ciphertext = strings.TrimSpace(payload.Data)
case strings.TrimSpace(payload.Ciphertext) != "":
ciphertext = strings.TrimSpace(payload.Ciphertext)
case strings.TrimSpace(payload.Encrypted) != "":
ciphertext = strings.TrimSpace(payload.Encrypted)
default:
return nil, errors.New("missing encrypted dpop payload")
}
}
decoded, err := decodeBase64Loose(ciphertext)
if err != nil {
return nil, err
}
if len(decoded) <= 12 {
return nil, errors.New("encrypted dpop payload too short")
}
plain, err := decryptDoubaoKeyPair(decoded, secret)
if err != nil {
return nil, fmt.Errorf("failed to decrypt with secret: %w", err)
}
return parseECPrivateKeyJWK(string(plain))
}
func decryptDoubaoKeyPair(ciphertext []byte, secret string) ([]byte, error) {
key := pbkdf2.Key([]byte(secret), []byte("fixed-salt"), 100000, 32, sha256.New)
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
aead, err := cipher.NewGCM(block)
if err != nil {
return nil, err
}
nonceSize := aead.NonceSize()
if len(ciphertext) <= nonceSize {
return nil, errors.New("ciphertext too short")
}
nonce := ciphertext[:nonceSize]
enc := ciphertext[nonceSize:]
return aead.Open(nil, nonce, enc, nil)
}
func decodeBase64Loose(raw string) ([]byte, error) {
raw = strings.TrimSpace(raw)
raw = strings.ReplaceAll(raw, "\n", "")
raw = strings.ReplaceAll(raw, "\r", "")
raw = strings.ReplaceAll(raw, "\t", "")
raw = strings.ReplaceAll(raw, " ", "")
encodings := []*base64.Encoding{
base64.StdEncoding,
base64.RawStdEncoding,
base64.URLEncoding,
base64.RawURLEncoding,
}
var lastErr error
for _, enc := range encodings {
decoded, err := enc.DecodeString(raw)
if err == nil {
return decoded, nil
}
lastErr = err
}
if lastErr == nil {
lastErr = errors.New("invalid base64")
}
return nil, lastErr
}
func validateP256Key(key *ecdsa.PrivateKey) (*ecdsa.PrivateKey, error) {
if key == nil {
return nil, errors.New("nil private key")
}
if key.Curve != elliptic.P256() {
return nil, errors.New("ES256 requires P-256 key")
}
if key.PublicKey.X == nil || key.PublicKey.Y == nil || key.D == nil {
return nil, errors.New("invalid private key")
}
if !key.Curve.IsOnCurve(key.PublicKey.X, key.PublicKey.Y) {
return nil, errors.New("public key is not on P-256 curve")
}
return key, nil
}
func trimTokenScheme(token string) string {
token = strings.TrimSpace(token)
if i := strings.IndexByte(token, ' '); i > 0 {
scheme := strings.ToLower(strings.TrimSpace(token[:i]))
if scheme == "bearer" || scheme == "dpop" {
return strings.TrimSpace(token[i+1:])
}
}
return token
}
func b64url(b []byte) string {
return base64.RawURLEncoding.EncodeToString(b)
}
func pad32(b []byte) []byte {
if len(b) >= 32 {
return b[len(b)-32:]
}
out := make([]byte, 32)
copy(out[32-len(b):], b)
return out
}
func pickStr(v, def string) string {
if v != "" {
return v
}
return def
}
func pickI64(v, def int64) int64 {
if v != 0 {
return v
}
return def
}
func (d *DoubaoNew) resolveAuthorization() string {
auth := trimTokenScheme(d.Authorization)
if auth == "" {
return ""
}
return "DPoP " + auth
}
func shouldRefreshJWT(token string) bool {
if token == "" {
return true
}
var payload JWTPayload
if err := ParseJWTPayload(token, &payload); err != nil {
return true
}
if payload.Exp <= 0 {
return false
}
return payload.Exp <= time.Now().Unix()+defaultAuthRefreshAheadSeconds
}
func (d *DoubaoNew) fetchBizAuth(dpop string, public bool) (string, error) {
var reqUrl string
client := base.RestyClient.Clone()
req := client.R()
req.SetHeader("accept", "application/json, text/javascript")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
req.SetHeader("content-type", "application/x-www-form-urlencoded")
if public {
reqUrl = DoubaoURL + "/passport/anonymity_user/biz_auth/"
} else {
reqUrl = DoubaoURL + "/passport/user/biz_auth/"
if d.Cookie != "" {
req.SetHeader("cookie", d.Cookie)
if csrf := strings.TrimSpace(cookie.GetStr(d.Cookie, "passport_csrf_token")); csrf != "" {
req.SetHeader("x-tt-passport-csrf-token", csrf)
}
}
if oldAuth := d.resolveAuthorization(); oldAuth != "" {
req.SetHeader("authorization", oldAuth)
}
}
if dpop != "" {
req.SetHeader("dpop", dpop)
}
values := url.Values{}
values.Set("client_id", d.AuthClientID)
values.Set("client_type", d.AuthClientType)
values.Set("scope", d.AuthScope)
values.Set("d_pop", dpop)
req.SetBody(values.Encode())
req.SetQueryParam("aid", d.AppID)
req.SetQueryParam("account_sdk_source", d.AuthSDKSource)
req.SetQueryParam("sdk_version", d.AuthSDKVersion)
res, err := req.Post(reqUrl)
if err != nil {
return "", err
}
var resp bizAuthResp
if err = json.Unmarshal(res.Body(), &resp); err != nil {
return "", err
}
if resp.Message != "success" || resp.Data.AccessToken == "" {
return "", fmt.Errorf("[doubao_new] %s: %s", resp.Message, resp.Data.Description)
}
return resp.Data.AccessToken, nil
}
func (d *DoubaoNew) refreshAuthorizationWithDPoP(dpop string) (string, error) {
token, err := d.fetchBizAuth(dpop, false)
if err == nil && token != "" {
return token, nil
}
if err == nil {
err = errors.New("biz auth refresh failed")
}
return "", err
}
func (d *DoubaoNew) resolveDpopForRequest(method, rawURL string) (string, error) {
if d.DPoPKeyPair != nil {
proof, err := GenerateDPoPToken(DPoPTokenInput{
KeyPair: d.DPoPKeyPair,
HTM: strings.ToUpper(strings.TrimSpace(method)),
HTU: normalizeDPoPURL(rawURL),
})
if err != nil {
return "", err
}
return proof.DPoPToken, nil
}
static := d.DPoP
if static == "" {
return "", nil
}
if !d.IgnoreJWTCheck {
if payload, err := parseDPoPPayload(static); err == nil && payload.Exp > 0 {
now := time.Now().Unix()
if payload.Exp <= now+defaultDpopRefreshAheadSeconds {
return "", errors.New("static dpop token expired or near expiry; configure dpop_key_pair for automatic refresh")
}
}
}
return static, nil
}
func (d *DoubaoNew) ensureAuthAdditons() bool {
return d.DPoPKeySecret != "" && d.AuthClientID != "" && d.AuthClientType != "" &&
d.AuthScope != "" && d.AuthSDKSource != "" && d.AuthSDKVersion != ""
}
func (d *DoubaoNew) resolveAuthorizationForRequest(method, rawURL string) (string, error) {
if !shouldRefreshJWT(d.Authorization) {
return d.resolveAuthorization(), nil
}
if d.DPoPKeyPair == nil || strings.TrimSpace(d.Cookie) == "" || !d.ensureAuthAdditons() {
return d.resolveAuthorization(), nil
}
d.authRefreshMu.Lock()
defer d.authRefreshMu.Unlock()
if !shouldRefreshJWT(d.Authorization) {
return d.resolveAuthorization(), nil
}
refreshDpop, err := d.resolveDpopForRequest(method, rawURL)
if err != nil || refreshDpop == "" {
return "", err
}
newToken, err := d.refreshAuthorizationWithDPoP(refreshDpop)
if err != nil {
if auth := d.resolveAuthorization(); auth != "" {
return auth, nil
}
return "", err
}
d.Authorization = trimTokenScheme(newToken)
return d.resolveAuthorization(), nil
}
func (d *DoubaoNew) resolveAuthorizationForPublic() (dpop string, auth string, err error) {
if d.DPoPPublic != "" && !shouldRefreshJWT(d.AuthorizationPublic) {
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
}
if !d.ensureAuthAdditons() {
return "", "", fmt.Errorf("[doubao_new] missing auth additions, please fill them all")
}
d.authRefreshPublicMu.Lock()
defer d.authRefreshPublicMu.Unlock()
if d.DPoPPublic != "" && !shouldRefreshJWT(d.AuthorizationPublic) {
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
}
// generate new public dpop
keypair, err := GenerateDPoPKeyPair()
if err != nil {
return "", "", err
}
proof, err := GenerateDPoPToken(DPoPTokenInput{
KeyPair: keypair,
})
d.DPoPPublic = proof.DPoPToken
// get authorization token
d.AuthorizationPublic, err = d.fetchBizAuth(proof.DPoPToken, true)
if err != nil {
return "", "", err
}
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
}
func (d *DoubaoNew) applyAuthHeaders(req *resty.Request, method, rawURL string) error {
auth, err := d.resolveAuthorizationForRequest(method, rawURL)
if err != nil {
return err
}
if auth != "" {
req.SetHeader("authorization", auth)
}
dpop, err := d.resolveDpopForRequest(method, rawURL)
if err != nil {
return err
}
if dpop != "" {
req.SetHeader("dpop", dpop)
}
return nil
}
func normalizeDPoPURL(rawURL string) string {
u, err := url.Parse(rawURL)
if err != nil {
return rawURL
}
u.Fragment = ""
return u.String()
}
func parseDPoPPayload(token string) (*JWTPayload, error) {
var payload JWTPayload
if err := ParseJWTPayload(token, &payload); err != nil {
return nil, err
}
return &payload, nil
}
-533
View File
@@ -1,533 +0,0 @@
package doubao_new
import (
"bytes"
"context"
"crypto/ecdsa"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"sort"
"strings"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
type DoubaoNew struct {
model.Storage
Addition
TtLogid string
// DPoP access token (Authorization header value, without DPoP prefix)
Authorization string
AuthorizationPublic string
// DPoP header value
DPoP string
DPoPPublic string
// DPoP key pair for generating DPoP
DPoPKeyPairStr string
DPoPKeyPair *ecdsa.PrivateKey
authRefreshMu sync.Mutex
authRefreshPublicMu sync.Mutex
}
func (d *DoubaoNew) Config() driver.Config {
return config
}
func (d *DoubaoNew) GetAddition() driver.Additional {
return &d.Addition
}
func (d *DoubaoNew) Init(ctx context.Context) error {
if cookieStr := strings.TrimSpace(d.Cookie); cookieStr != "" {
d.Cookie = cookieStr
auth := trimTokenScheme(cookie.GetStr(d.Cookie, "LARK_SUITE_ACCESS_TOKEN"))
if auth != "" {
d.Authorization = auth
}
dpop := strings.TrimSpace(cookie.GetStr(d.Cookie, "LARK_SUITE_DPOP"))
if dpop != "" {
d.DPoP = dpop
}
keypair := strings.TrimSpace(cookie.GetStr(d.Cookie, "feishu_dpop_keypair"))
if keypair != "" && d.DPoPKeySecret != "" {
d.DPoPKeyPairStr = keypair
d.DPoPKeyPair, _ = parseEncryptedDPoPKeyPair(keypair, d.DPoPKeySecret)
}
}
return nil
}
func (d *DoubaoNew) Drop(ctx context.Context) error {
if d.Authorization != "" {
d.Cookie = cookie.SetStr(d.Cookie, "LARK_SUITE_ACCESS_TOKEN", d.Authorization)
}
if d.DPoP != "" {
d.Cookie = cookie.SetStr(d.Cookie, "LARK_SUITE_DPOP", d.DPoP)
}
if d.DPoPKeyPairStr != "" {
d.Cookie = cookie.SetStr(d.Cookie, "feishu_dpop_keypair", d.DPoPKeyPairStr)
}
op.MustSaveDriverStorage(d)
return nil
}
func (d *DoubaoNew) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
nodes, err := d.listAllChildren(ctx, dir.GetID())
if err != nil {
return nil, err
}
objs := make([]model.Obj, 0, len(nodes))
for _, node := range nodes {
if node.NodeToken == "" || node.ObjToken == "" {
continue
}
size := parseSize(node.Extra.Size)
isFolder := node.Type == 0
if isFolder && node.NodeToken == dir.GetID() {
continue
}
obj := &Object{
Object: model.Object{
ID: node.NodeToken,
Path: dir.GetID(),
Name: node.Name,
Size: size,
Modified: time.Unix(node.EditTime, 0),
Ctime: time.Unix(node.CreateTime, 0),
IsFolder: isFolder,
},
ObjToken: node.ObjToken,
NodeType: node.NodeType,
ObjType: node.Type,
URL: node.URL,
}
objs = append(objs, obj)
}
return objs, nil
}
func (d *DoubaoNew) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
obj, ok := file.(*Object)
if !ok {
return nil, errors.New("unsupported object type")
}
if obj.IsFolder {
return nil, fmt.Errorf("link is directory")
}
var (
err error
auth, dpop string
)
if d.ShareLink {
err := d.createShare(ctx, obj)
if err != nil {
return nil, err
}
dpop, auth, err = d.resolveAuthorizationForPublic()
} else {
// TODO: append previewLink() with auth args to support ShareLink
if args.Type == "preview" || args.Type == "thumb" {
if link, err := d.previewLink(ctx, obj, args); err == nil {
return link, nil
}
}
auth = d.resolveAuthorization()
dpop, err = d.resolveDpopForRequest(http.MethodGet, DownloadBaseURL+"/space/api/box/stream/download/all/"+obj.ObjToken+"/")
}
if err != nil {
return nil, err
}
if auth == "" || dpop == "" {
return nil, errors.New("missing authorization or dpop")
}
if obj.ObjToken == "" {
return nil, errors.New("missing obj_token")
}
query := url.Values{}
query.Set("authorization", auth)
query.Set("dpop", dpop)
downloadURL := DownloadBaseURL + "/space/api/box/stream/download/all/" + obj.ObjToken + "/?" + query.Encode()
headers := http.Header{
"Referer": []string{DoubaoURL + "/"},
"User-Agent": []string{base.UserAgent},
}
return &model.Link{
URL: downloadURL,
Header: headers,
}, nil
}
func (d *DoubaoNew) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
node, err := d.createFolder(ctx, parentDir.GetID(), dirName)
if err != nil {
return nil, err
}
return &Object{
Object: model.Object{
ID: node.NodeToken,
Path: parentDir.GetID(),
Name: node.Name,
Size: parseSize(node.Extra.Size),
Modified: time.Unix(node.EditTime, 0),
Ctime: time.Unix(node.CreateTime, 0),
IsFolder: true,
},
ObjToken: node.ObjToken,
NodeType: node.NodeType,
ObjType: node.Type,
URL: node.URL,
}, nil
}
func (d *DoubaoNew) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
if srcObj == nil {
return nil, errors.New("nil source object")
}
if dstDir == nil {
return nil, errors.New("nil destination dir")
}
srcToken := srcObj.GetID()
if srcToken == "" {
if obj, ok := srcObj.(*Object); ok {
srcToken = obj.ObjToken
}
}
if srcToken == "" {
return nil, errors.New("missing source token")
}
if err := d.moveObj(ctx, srcToken, dstDir.GetID()); err != nil {
return nil, err
}
if obj, ok := srcObj.(*Object); ok {
clone := *obj
clone.Path = dstDir.GetID()
return &clone, nil
}
return srcObj, nil
}
func (d *DoubaoNew) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
if srcObj == nil {
return nil, errors.New("nil source object")
}
if srcObj.IsDir() {
if err := d.renameFolder(ctx, srcObj.GetID(), newName); err != nil {
return nil, err
}
} else {
fileToken := ""
if obj, ok := srcObj.(*Object); ok {
fileToken = obj.ObjToken
}
if fileToken == "" {
fileToken = srcObj.GetID()
}
if err := d.renameFile(ctx, fileToken, newName); err != nil {
return nil, err
}
}
if obj, ok := srcObj.(*Object); ok {
clone := *obj
clone.Name = newName
return &clone, nil
}
return srcObj, nil
}
func (d *DoubaoNew) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
// TODO copy obj, optional
return nil, errs.NotImplement
}
func (d *DoubaoNew) Remove(ctx context.Context, obj model.Obj) error {
if obj == nil {
return errors.New("nil object")
}
token := obj.GetID()
if token == "" {
if o, ok := obj.(*Object); ok {
token = o.ObjToken
}
}
if token == "" {
return errors.New("missing object token")
}
return d.removeObj(ctx, []string{token})
}
func (d *DoubaoNew) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
if file == nil {
return nil, errors.New("nil file")
}
if file.GetSize() <= 0 {
return nil, errors.New("invalid file size")
}
uploadPrep, err := d.prepareUpload(ctx, file.GetName(), file.GetSize(), dstDir.GetID())
if err != nil {
return nil, err
}
if uploadPrep.BlockSize <= 0 {
return nil, errors.New("invalid block size from prepare")
}
tmpFile, err := utils.CreateTempFile(file, file.GetSize())
if err != nil {
return nil, err
}
defer func() {
_ = tmpFile.Close()
_ = os.Remove(tmpFile.Name())
}()
blockSize := uploadPrep.BlockSize
totalSize := file.GetSize()
numBlocks := int((totalSize + blockSize - 1) / blockSize)
blocks := make([]UploadBlock, 0, numBlocks)
blockMeta := make(map[int]UploadBlock, numBlocks)
for seq := 0; seq < numBlocks; seq++ {
offset := int64(seq) * blockSize
length := blockSize
if remain := totalSize - offset; remain < length {
length = remain
}
buf := make([]byte, int(length))
n, err := tmpFile.ReadAt(buf, offset)
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
return nil, err
}
buf = buf[:n]
sum := sha256.Sum256(buf)
hash := base64.StdEncoding.EncodeToString(sum[:])
checksum := adler32String(buf)
block := UploadBlock{
Hash: hash,
Seq: seq,
Size: int64(n),
Checksum: checksum,
IsUploaded: true,
}
blocks = append(blocks, block)
blockMeta[seq] = block
}
needed, err := d.uploadBlocks(ctx, uploadPrep.UploadID, blocks, "explorer")
if err != nil {
return nil, err
}
if len(needed.NeededUploadBlocks) > 0 {
sort.Slice(needed.NeededUploadBlocks, func(i, j int) bool {
return needed.NeededUploadBlocks[i].Seq < needed.NeededUploadBlocks[j].Seq
})
const maxMergeBlockCount = 20
var (
groupSeqs []int
groupChecksums []string
groupSizes []int64
groupRealSize int64
groupExpectSum int64
groupBuf bytes.Buffer
uploadedBytes int64
)
flushGroup := func() error {
if len(groupSeqs) == 0 {
return nil
}
data := groupBuf.Bytes()
expectLen := groupExpectSum
if int64(len(data)) != expectLen {
return fmt.Errorf("[doubao_new] merge blocks invalid body len: got=%d expect=%d seqs=%v", len(data), expectLen, groupSeqs)
}
mergeResp, err := d.mergeUploadBlocks(ctx, uploadPrep.UploadID, groupSeqs, groupChecksums, groupSizes, blockSize, data)
if err != nil {
return err
}
if len(mergeResp.SuccessSeqList) != len(groupSeqs) {
return fmt.Errorf("[doubao_new] merge blocks incomplete: %v", mergeResp.SuccessSeqList)
}
success := make(map[int]bool, len(mergeResp.SuccessSeqList))
for _, seq := range mergeResp.SuccessSeqList {
success[seq] = true
}
for _, seq := range groupSeqs {
if !success[seq] {
return fmt.Errorf("[doubao_new] merge blocks missing seq %d", seq)
}
}
uploadedBytes += groupRealSize
groupSeqs = groupSeqs[:0]
groupChecksums = groupChecksums[:0]
groupSizes = groupSizes[:0]
groupRealSize = 0
groupExpectSum = 0
groupBuf.Reset()
if up != nil {
percent := float64(uploadedBytes) / float64(totalSize) * 100
up(percent)
}
return nil
}
for _, item := range needed.NeededUploadBlocks {
if _, ok := blockMeta[item.Seq]; !ok {
return nil, fmt.Errorf("[doubao_new] missing block meta for seq %d", item.Seq)
}
if item.Size <= 0 {
return nil, fmt.Errorf("[doubao_new] invalid block size from needed list: seq=%d size=%d", item.Seq, item.Size)
}
offset := int64(item.Seq) * blockSize
buf := make([]byte, int(item.Size))
n, err := tmpFile.ReadAt(buf, offset)
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
return nil, err
}
if n != len(buf) {
return nil, fmt.Errorf("[doubao_new] short read: seq=%d want=%d got=%d", item.Seq, len(buf), n)
}
buf = buf[:n]
realAdler := adler32String(buf)
if realAdler != item.Checksum {
return nil, fmt.Errorf("[doubao_new] block checksum mismatch: seq=%d offset=%d adler32=%s step2=%s", item.Seq, offset, realAdler, item.Checksum)
}
payloadStart := groupBuf.Len()
groupBuf.Write(buf)
payloadEnd := groupBuf.Len()
payloadAdler := adler32String(groupBuf.Bytes()[payloadStart:payloadEnd])
if payloadAdler != item.Checksum {
return nil, fmt.Errorf("[doubao_new] payload checksum mismatch: seq=%d start=%d end=%d adler32=%s step2=%s", item.Seq, payloadStart, payloadEnd, payloadAdler, item.Checksum)
}
groupSeqs = append(groupSeqs, item.Seq)
groupChecksums = append(groupChecksums, item.Checksum)
groupSizes = append(groupSizes, item.Size)
groupRealSize += int64(n)
groupExpectSum += item.Size
if len(groupSeqs) >= maxMergeBlockCount {
if err := flushGroup(); err != nil {
return nil, err
}
}
}
if err := flushGroup(); err != nil {
return nil, err
}
if up != nil {
up(100)
}
} else if up != nil {
up(100)
}
numBlocksFinish := uploadPrep.NumBlocks
if numBlocksFinish <= 0 {
numBlocksFinish = numBlocks
}
finish, err := d.finishUpload(ctx, uploadPrep.UploadID, numBlocksFinish, "explorer")
if err != nil {
return nil, err
}
nodeToken := finish.Extra.NodeToken
if nodeToken == "" {
nodeToken = finish.FileToken
}
now := time.Now()
return &Object{
Object: model.Object{
ID: nodeToken,
Path: dstDir.GetID(),
Name: file.GetName(),
Size: file.GetSize(),
Modified: now,
Ctime: now,
IsFolder: false,
},
ObjToken: finish.FileToken,
}, nil
}
func (d *DoubaoNew) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
data, err := d.getUserStorage(ctx)
if err != nil {
return nil, err
}
return &model.StorageDetails{
DiskUsage: model.DiskUsage{
TotalSpace: data.TotalSizeLimitBytes,
UsedSpace: data.UsedSizeBytes,
},
}, nil
}
func (d *DoubaoNew) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
switch args.Method {
case "doubao_preview", "preview":
obj, ok := args.Obj.(*Object)
if !ok {
return nil, errors.New("unsupported object type")
}
info, err := d.getFileInfo(ctx, obj.ObjToken)
if err != nil {
return nil, err
}
entry, ok := info.PreviewMeta.Data["22"]
if !ok || entry.Status != 0 {
return nil, errs.NotSupport
}
imgExt := ".webp"
pageNums := 1
if entry.Extra != "" {
var extra PreviewImageExtra
if err := json.Unmarshal([]byte(entry.Extra), &extra); err == nil {
if extra.ImgExt != "" {
imgExt = extra.ImgExt
}
if extra.PageNums > 0 {
pageNums = extra.PageNums
}
}
}
return base.Json{
"version": info.Version,
"img_ext": imgExt,
"page_nums": pageNums,
}, nil
default:
return nil, errs.NotSupport
}
}
var _ driver.Driver = (*DoubaoNew)(nil)
-39
View File
@@ -1,39 +0,0 @@
package doubao_new
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
// Usually one of two
driver.RootID
// define other
Cookie string `json:"cookie" required:"true" help:"Web Cookie"`
AppID string `json:"app_id" required:"true" default:"497858" help:"Doubao App ID"`
DPoPKeySecret string `json:"dpop_key_secret" help:"DPoP Key Secret for generating DPoP token"`
AuthClientID string `json:"auth_client_id" help:"Doubao Biz Auth Client ID"`
AuthClientType string `json:"auth_client_type" help:"Doubao Biz Auth Client Type"`
AuthScope string `json:"auth_scope" help:"Doubao Biz Auth Scope"`
AuthSDKSource string `json:"auth_sdk_source" help:"Doubao Biz Auth SDK Source"`
AuthSDKVersion string `json:"auth_sdk_version" help:"Doubao Biz Auth SDK Version"`
ShareLink bool `json:"share_link" help:"Whether to use share link for download"`
IgnoreJWTCheck bool `json:"ignore_jwt_check" help:"Whether to ignore JWT check to prevent time issue"`
}
var config = driver.Config{
Name: "DoubaoNew",
LocalSort: true,
DefaultRoot: "",
Alert: `danger|Do not use 302 if the storage is public accessible.
Otherwise, the download link may leak sensitive information such as access token or signature.
Others may use the leaked link to access all your files.`,
NoOverwriteUpload: false,
PreferProxy: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &DoubaoNew{}
})
}
-192
View File
@@ -1,192 +0,0 @@
package doubao_new
import "github.com/OpenListTeam/OpenList/v4/internal/model"
type BaseResp struct {
Code int `json:"code"`
Msg string `json:"msg,omitempty"`
Message string `json:"message,omitempty"`
}
type ListResp struct {
BaseResp
Data ListData `json:"data"`
}
type ListData struct {
HasMore bool `json:"has_more"`
LastLabel string `json:"last_label"`
NodeList []string `json:"node_list"`
Entities struct {
Nodes map[string]Node `json:"nodes"`
Users map[string]User `json:"users"`
} `json:"entities"`
}
type Node struct {
Token string `json:"token"`
NodeToken string `json:"node_token"`
ObjToken string `json:"obj_token"`
Name string `json:"name"`
Type int `json:"type"`
NodeType int `json:"node_type"`
OwnerID string `json:"owner_id"`
EditUID string `json:"edit_uid"`
CreateTime int64 `json:"create_time"`
EditTime int64 `json:"edit_time"`
URL string `json:"url"`
Extra struct {
Size string `json:"size"`
} `json:"extra"`
}
type User struct {
ID string `json:"id"`
Name string `json:"name"`
}
type Object struct {
model.Object
ObjToken string
NodeType int
ObjType int
URL string
}
type CreateFolderResp struct {
BaseResp
Data struct {
Entities struct {
Nodes map[string]Node `json:"nodes"`
} `json:"entities"`
NodeList []string `json:"node_list"`
} `json:"data"`
}
type FileInfoResp struct {
Code int `json:"code"`
Message string `json:"message"`
Data FileInfo `json:"data"`
}
type FileInfo struct {
Name string `json:"name"`
NumBlocks int `json:"num_blocks"`
Version string `json:"version"`
MimeType string `json:"mime_type"`
MountPoint string `json:"mount_point"`
PreviewMeta PreviewMeta `json:"preview_meta"`
}
type PreviewMeta struct {
Data map[string]PreviewMetaEntry `json:"data"`
}
type PreviewMetaEntry struct {
Status int `json:"status"`
Extra string `json:"extra"`
PreviewFileSize int64 `json:"preview_file_size"`
}
type PreviewImageExtra struct {
ImgExt string `json:"img_ext"`
PageNums int `json:"page_nums"`
}
type UserStorageResp struct {
BaseResp
Data UserStorageData `json:"data"`
}
type UserStorageData struct {
ShowSizeLimit bool `json:"show_size_limit"`
TotalSizeLimitBytes int64 `json:"total_size_limit_bytes"`
UsedSizeBytes int64 `json:"used_size_bytes"`
}
type UploadPrepareResp struct {
BaseResp
Data UploadPrepareData `json:"data"`
}
type UploadPrepareData struct {
BlockSize int64 `json:"block_size"`
NumBlocks int `json:"num_blocks"`
OptionBlockSize int64 `json:"option_block_size"`
DedupeSupport bool `json:"dedupe_support"`
UploadID string `json:"upload_id"`
}
type UploadBlock struct {
Hash string `json:"hash"`
Seq int `json:"seq"`
Size int64 `json:"size"`
Checksum string `json:"checksum"`
IsUploaded bool `json:"isUploaded"`
}
type UploadBlocksResp struct {
BaseResp
Data UploadBlocksData `json:"data"`
}
type UploadBlocksData struct {
NeededUploadBlocks []UploadBlockNeed `json:"needed_upload_blocks"`
}
type UploadBlockNeed struct {
Seq int `json:"seq"`
Size int64 `json:"size"`
Checksum string `json:"checksum"`
Hash string `json:"hash"`
}
type UploadMergeResp struct {
BaseResp
Data UploadMergeData `json:"data"`
}
type UploadMergeData struct {
SuccessSeqList []int `json:"success_seq_list"`
}
type UploadFinishResp struct {
BaseResp
Data UploadFinishData `json:"data"`
}
type UploadFinishData struct {
Version string `json:"version"`
DataVersion string `json:"data_version"`
Extra struct {
NodeToken string `json:"node_token"`
} `json:"extra"`
FileToken string `json:"file_token"`
}
type RemoveResp struct {
BaseResp
Data struct {
TaskID string `json:"task_id"`
} `json:"data"`
}
type TaskStatusResp struct {
BaseResp
Data TaskStatusData `json:"data"`
}
type TaskStatusData struct {
IsFinish bool `json:"is_finish"`
IsFail bool `json:"is_fail"`
}
type bizAuthResp struct {
Data struct {
AccessToken string `json:"access_token"`
AuthScheme string `json:"auth_scheme"`
ExpiresIn int64 `json:"expires_in"`
Description string `json:"description,omitempty"`
} `json:"data"`
Message string `json:"message"`
}
-283
View File
@@ -1,283 +0,0 @@
package doubao_new
import (
"bytes"
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strconv"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/go-resty/resty/v2"
)
func (d *DoubaoNew) prepareUpload(ctx context.Context, name string, size int64, mountNodeToken string) (UploadPrepareData, error) {
var resp UploadPrepareResp
_, err := d.request(ctx, "/space/api/box/upload/prepare/", http.MethodPost, func(req *resty.Request) {
values := url.Values{}
values.Set("shouldBypassScsDialog", "true")
values.Set("doubao_storage", "imagex_other")
values.Set("doubao_app_id", d.AppID)
req.SetQueryParamsFromValues(values)
req.SetHeader("Content-Type", "application/json")
req.SetHeader("x-command", "space.api.box.upload.prepare")
req.SetHeader("rpc-persist-doubao-pan", "true")
req.SetHeader("cache-control", "no-cache")
req.SetHeader("pragma", "no-cache")
body := base.Json{
"mount_point": "explorer",
"mount_node_token": "",
"name": name,
"size": size,
"size_checker": true,
}
if mountNodeToken != "" {
body["mount_node_token"] = mountNodeToken
}
req.SetBody(body)
}, &resp)
if err != nil {
return UploadPrepareData{}, err
}
return resp.Data, nil
}
func (d *DoubaoNew) uploadBlocks(ctx context.Context, uploadID string, blocks []UploadBlock, mountPoint string) (UploadBlocksData, error) {
if uploadID == "" {
return UploadBlocksData{}, fmt.Errorf("[doubao_new] upload blocks missing upload_id")
}
if mountPoint == "" {
mountPoint = "explorer"
}
var resp UploadBlocksResp
_, err := d.request(ctx, "/space/api/box/upload/blocks/", http.MethodPost, func(req *resty.Request) {
values := url.Values{}
values.Set("shouldBypassScsDialog", "true")
values.Set("doubao_storage", "imagex_other")
values.Set("doubao_app_id", d.AppID)
req.SetQueryParamsFromValues(values)
req.SetHeader("Content-Type", "application/json")
req.SetHeader("x-command", "space.api.box.upload.blocks")
req.SetHeader("rpc-persist-doubao-pan", "true")
req.SetHeader("cache-control", "no-cache")
req.SetHeader("pragma", "no-cache")
req.SetBody(base.Json{
"blocks": blocks,
"upload_id": uploadID,
"mount_point": mountPoint,
})
}, &resp)
if err != nil {
return UploadBlocksData{}, err
}
return resp.Data, nil
}
func (d *DoubaoNew) mergeUploadBlocks(ctx context.Context, uploadID string, seqList []int, checksumList []string, sizeList []int64, blockOriginSize int64, data []byte) (UploadMergeData, error) {
if uploadID == "" {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks missing upload_id")
}
if len(seqList) == 0 {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty seq list")
}
if len(checksumList) == 0 {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty checksum list")
}
if len(sizeList) != len(seqList) {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks size list mismatch")
}
if blockOriginSize <= 0 {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks invalid block origin size")
}
if len(data) == 0 {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty data")
}
seqHeader := joinIntComma(seqList)
checksumHeader := buildCommaHeader(checksumList)
client := base.NewRestyClient()
client.SetCookieJar(nil)
req := client.R()
req.SetContext(ctx)
req.SetHeader("accept", "application/json, text/plain, */*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
req.SetHeader("rpc-persist-doubao-pan", "true")
req.SetHeader("content-type", "application/octet-stream")
req.SetHeader("x-block-list-checksum", checksumHeader)
req.SetHeader("x-seq-list", seqHeader)
req.SetHeader("x-block-origin-size", strconv.FormatInt(blockOriginSize, 10))
req.SetHeader("x-command", "space.api.box.stream.upload.merge_block")
req.SetHeader("x-csrftoken", "")
reqID := ""
if buf := make([]byte, 16); true {
if _, err := rand.Read(buf); err == nil {
reqID = hex.EncodeToString(buf)
}
}
if reqID != "" {
req.SetHeader("x-request-id", reqID)
}
values := url.Values{}
values.Set("shouldBypassScsDialog", "true")
values.Set("upload_id", uploadID)
values.Set("mount_point", "explorer")
values.Set("doubao_storage", "imagex_other")
values.Set("doubao_app_id", d.AppID)
urlStr := DownloadBaseURL + "/space/api/box/stream/upload/merge_block/?" + values.Encode()
if err := d.applyAuthHeaders(req, http.MethodPost, urlStr); err != nil {
return UploadMergeData{}, err
}
req.Header.Del("cookie")
if req.Header.Get("x-command") == "" {
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks missing x-command header")
}
req.SetBody(data)
res, err := req.Execute(http.MethodPost, urlStr)
if err != nil {
return UploadMergeData{}, err
}
if v := res.Header().Get("X-Tt-Logid"); v != "" {
d.TtLogid = v
} else if v := res.Header().Get("x-tt-logid"); v != "" {
d.TtLogid = v
}
body := res.Body()
var resp UploadMergeResp
if err := json.Unmarshal(body, &resp); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return UploadMergeData{}, fmt.Errorf("%s", msg)
}
if resp.Code != 0 {
if res != nil && res.StatusCode() == http.StatusBadRequest && resp.Code == 2 {
success := make([]int, 0, len(seqList))
offset := 0
for i, seq := range seqList {
size := sizeList[i]
if size <= 0 {
return UploadMergeData{SuccessSeqList: success}, fmt.Errorf("[doubao_new] v3 fallback invalid size: seq=%d size=%d", seq, size)
}
if offset+int(size) > len(data) {
return UploadMergeData{SuccessSeqList: success}, fmt.Errorf("[doubao_new] v3 fallback payload out of range: seq=%d offset=%d size=%d total=%d", seq, offset, size, len(data))
}
payload := data[offset : offset+int(size)]
block := UploadBlockNeed{
Seq: seq,
Size: size,
Checksum: checksumList[i],
}
if err := d.uploadBlockV3(ctx, uploadID, block, payload); err != nil {
return UploadMergeData{SuccessSeqList: success}, err
}
success = append(success, seq)
offset += int(size)
}
return UploadMergeData{SuccessSeqList: success}, nil
}
errMsg := resp.Msg
if errMsg == "" {
errMsg = resp.Message
}
return UploadMergeData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
}
return resp.Data, nil
}
func (d *DoubaoNew) uploadBlockV3(ctx context.Context, uploadID string, block UploadBlockNeed, data []byte) error {
if uploadID == "" {
return fmt.Errorf("[doubao_new] upload v3 block missing upload_id")
}
if block.Seq < 0 {
return fmt.Errorf("[doubao_new] upload v3 block invalid seq")
}
if len(data) == 0 {
return fmt.Errorf("[doubao_new] upload v3 block empty data")
}
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "*/*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
req.SetHeader("rpc-persist-doubao-pan", "true")
req.SetHeader("x-block-seq", strconv.Itoa(block.Seq))
req.SetHeader("x-block-checksum", block.Checksum)
req.SetMultipartFormData(map[string]string{
"upload_id": uploadID,
"size": strconv.FormatInt(int64(len(data)), 10),
})
req.SetMultipartField("file", "blob", "application/octet-stream", bytes.NewReader(data))
values := url.Values{}
values.Set("shouldBypassScsDialog", "true")
values.Set("upload_id", uploadID)
values.Set("seq", strconv.Itoa(block.Seq))
values.Set("size", strconv.FormatInt(int64(len(data)), 10))
values.Set("checksum", block.Checksum)
values.Set("mount_point", "explorer")
values.Set("doubao_storage", "imagex_other")
values.Set("doubao_app_id", d.AppID)
urlStr := DownloadBaseURL + "/space/api/box/stream/upload/v3/block/?" + values.Encode()
if err := d.applyAuthHeaders(req, http.MethodPost, urlStr); err != nil {
return err
}
res, err := req.Execute(http.MethodPost, urlStr)
if err != nil {
return err
}
body := res.Body()
if err := decodeBaseResp(body, res); err != nil {
return err
}
return nil
}
func (d *DoubaoNew) finishUpload(ctx context.Context, uploadID string, numBlocks int, mountPoint string) (UploadFinishData, error) {
if uploadID == "" {
return UploadFinishData{}, fmt.Errorf("[doubao_new] finish upload missing upload_id")
}
if numBlocks <= 0 {
return UploadFinishData{}, fmt.Errorf("[doubao_new] finish upload invalid num_blocks")
}
if mountPoint == "" {
mountPoint = "explorer"
}
var resp UploadFinishResp
_, err := d.request(ctx, "/space/api/box/upload/finish/", http.MethodPost, func(req *resty.Request) {
values := url.Values{}
values.Set("shouldBypassScsDialog", "true")
values.Set("doubao_storage", "imagex_other")
values.Set("doubao_app_id", d.AppID)
req.SetQueryParamsFromValues(values)
req.SetHeader("Content-Type", "application/json")
req.SetHeader("x-command", "space.api.box.upload.finish")
req.SetHeader("rpc-persist-doubao-pan", "true")
req.SetHeader("cache-control", "no-cache")
req.SetHeader("pragma", "no-cache")
req.SetHeader("biz-scene", "file_upload")
req.SetHeader("biz-ua-type", "Web")
req.SetBody(base.Json{
"upload_id": uploadID,
"num_blocks": numBlocks,
"mount_point": mountPoint,
"push_open_history_record": 1,
})
}, &resp)
if err != nil {
return UploadFinishData{}, err
}
return resp.Data, nil
}
-713
View File
@@ -1,713 +0,0 @@
package doubao_new
import (
"context"
"encoding/json"
"errors"
"fmt"
"hash/adler32"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
"github.com/go-resty/resty/v2"
)
const (
BaseURL = "https://my.feishu.cn"
DownloadBaseURL = "https://internal-api-drive-stream.feishu.cn"
DoubaoURL = "https://www.doubao.com"
)
var defaultObjTypes = []string{"124", "0", "12", "30", "123", "22"}
func (d *DoubaoNew) request(ctx context.Context, path string, method string, callback base.ReqCallback, resp interface{}) ([]byte, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "*/*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, method, BaseURL+path); err != nil {
return nil, err
}
if callback != nil {
callback(req)
}
res, err := req.Execute(method, BaseURL+path)
if err != nil {
return nil, err
}
if res != nil {
if v := res.Header().Get("X-Tt-Logid"); v != "" {
d.TtLogid = v
} else if v := res.Header().Get("x-tt-logid"); v != "" {
d.TtLogid = v
}
}
body := res.Body()
var common BaseResp
if err = json.Unmarshal(body, &common); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return body, fmt.Errorf("%s", msg)
}
if common.Code != 0 {
errMsg := common.Msg
if errMsg == "" {
errMsg = common.Message
}
return body, fmt.Errorf("[doubao_new] API error (code: %d): %s", common.Code, errMsg)
}
if resp != nil {
if err = json.Unmarshal(body, resp); err != nil {
return body, err
}
}
return body, nil
}
func adler32String(data []byte) string {
sum := adler32.Checksum(data)
return strconv.FormatUint(uint64(sum), 10)
}
func buildCommaHeader(items []string) string {
return strings.Join(items, ",")
}
func joinIntComma(items []int) string {
if len(items) == 0 {
return ""
}
var sb strings.Builder
for i, v := range items {
if i > 0 {
sb.WriteByte(',')
}
sb.WriteString(strconv.Itoa(v))
}
return sb.String()
}
func previewList(items []string, n int) string {
if n <= 0 || len(items) == 0 {
return ""
}
if len(items) < n {
n = len(items)
}
return strings.Join(items[:n], ",")
}
func parseSize(size string) int64 {
if size == "" {
return 0
}
val, err := strconv.ParseInt(size, 10, 64)
if err != nil {
return 0
}
return val
}
func (d *DoubaoNew) listChildren(ctx context.Context, parentToken string, lastLabel string, length int) (ListData, error) {
var resp ListResp
_, err := d.request(ctx, "/space/api/explorer/doubao/children/list/", http.MethodGet, func(req *resty.Request) {
values := url.Values{}
for _, t := range defaultObjTypes {
values.Add("obj_type", t)
}
values.Set("length", strconv.Itoa(length))
values.Set("rank", "0")
values.Set("asc", "0")
values.Set("min_length", "40")
values.Set("thumbnail_width", "1028")
values.Set("thumbnail_height", "1028")
values.Set("thumbnail_policy", "4")
if parentToken != "" {
values.Set("token", parentToken)
}
if lastLabel != "" {
values.Set("last_label", lastLabel)
}
req.SetQueryParamsFromValues(values)
}, &resp)
if err != nil {
return ListData{}, err
}
return resp.Data, nil
}
func (d *DoubaoNew) listAllChildren(ctx context.Context, parentToken string) ([]Node, error) {
length := 50
nodes := make([]Node, 0, length)
lastLabel := ""
for range 100 {
data, err := d.listChildren(ctx, parentToken, lastLabel, length)
if err != nil {
return nil, err
}
if len(data.NodeList) > 0 {
for _, token := range data.NodeList {
node, ok := data.Entities.Nodes[token]
if !ok {
continue
}
nodes = append(nodes, node)
}
} else {
for _, node := range data.Entities.Nodes {
nodes = append(nodes, node)
}
}
if !data.HasMore || data.LastLabel == "" || data.LastLabel == lastLabel {
break
}
lastLabel = data.LastLabel
}
if len(nodes) == 0 {
return nil, nil
}
return nodes, nil
}
func (d *DoubaoNew) getFileInfo(ctx context.Context, fileToken string) (FileInfo, error) {
var resp FileInfoResp
_, err := d.request(ctx, "/space/api/box/file/info/", http.MethodPost, func(req *resty.Request) {
req.SetHeader("Content-Type", "application/json")
req.SetBody(base.Json{
"caller": "explorer",
"file_token": fileToken,
"mount_point": "explorer",
"option_params": []string{"preview_meta", "check_cipher"},
})
}, &resp)
if err != nil {
return FileInfo{}, err
}
return resp.Data, nil
}
func (d *DoubaoNew) previewLink(ctx context.Context, obj *Object, args model.LinkArgs) (*model.Link, error) {
auth := d.resolveAuthorization()
dpop, err := d.resolveDpopForRequest(http.MethodGet, fmt.Sprintf("%s/space/api/box/stream/download/preview_sub/%s", BaseURL, obj.ObjToken))
if auth == "" || dpop == "" {
return nil, errors.New("missing authorization or dpop")
}
if obj.ObjToken == "" {
return nil, errors.New("missing obj_token")
}
info, err := d.getFileInfo(ctx, obj.ObjToken)
if err != nil {
return nil, err
}
entry, ok := info.PreviewMeta.Data["22"]
if !ok || entry.Status != 0 {
return nil, errors.New("preview not available")
}
subID := ""
pageIndex := 0
if subID == "" {
imgExt := ".webp"
pageNums := 0
if entry.Extra != "" {
var extra PreviewImageExtra
if err := json.Unmarshal([]byte(entry.Extra), &extra); err == nil {
if extra.ImgExt != "" {
imgExt = extra.ImgExt
}
pageNums = extra.PageNums
}
}
if pageNums > 0 && pageIndex >= pageNums {
pageIndex = pageNums - 1
}
subID = fmt.Sprintf("img_%d%s", pageIndex, imgExt)
}
query := url.Values{}
query.Set("preview_type", "22")
query.Set("sub_id", subID)
if info.Version != "" {
query.Set("version", info.Version)
}
previewURL := fmt.Sprintf("%s/space/api/box/stream/download/preview_sub/%s?%s", BaseURL, obj.ObjToken, query.Encode())
headers := http.Header{
"Referer": []string{DoubaoURL + "/"},
"User-Agent": []string{base.UserAgent},
"Authorization": []string{auth},
"Dpop": []string{dpop},
}
return &model.Link{
URL: previewURL,
Header: headers,
}, nil
}
func (d *DoubaoNew) createShare(ctx context.Context, obj *Object) error {
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "application/json, text/plain, */*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/suite/permission/public/update.v5/"); err != nil {
return nil, nil, err
}
if csrfToken != "" {
req.SetHeader("x-csrftoken", csrfToken)
}
req.SetHeader("Content-Type", "application/json")
req.SetBody(base.Json{
"external_access_entity": 1,
"link_share_entity": 4,
"token": obj.ObjToken,
"type": obj.ObjType,
})
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/suite/permission/public/update.v5/")
if err != nil {
return nil, nil, err
}
return res, res.Body(), nil
}
res, body, err := doRequestWithCsrf(doRequest)
if err != nil {
return err
}
if err := decodeBaseResp(body, res); err != nil {
return err
}
return nil
}
func (d *DoubaoNew) createFolder(ctx context.Context, parentToken, name string) (Node, error) {
data := url.Values{}
data.Set("name", name)
data.Set("source", "0")
if parentToken != "" {
data.Set("parent_token", parentToken)
}
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "*/*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/create/folder/"); err != nil {
return nil, nil, err
}
if csrfToken != "" {
req.SetHeader("x-csrftoken", csrfToken)
}
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
req.SetBody(data.Encode())
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/create/folder/")
if err != nil {
return nil, nil, err
}
return res, res.Body(), nil
}
res, body, err := doRequestWithCsrf(doRequest)
if err != nil {
return Node{}, err
}
if err := decodeBaseResp(body, res); err != nil {
return Node{}, err
}
var resp CreateFolderResp
if err := json.Unmarshal(body, &resp); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return Node{}, fmt.Errorf("%s", msg)
}
var node Node
if len(resp.Data.NodeList) > 0 {
if n, ok := resp.Data.Entities.Nodes[resp.Data.NodeList[0]]; ok {
node = n
}
}
if node.Token == "" {
for _, n := range resp.Data.Entities.Nodes {
node = n
break
}
}
if node.Token == "" && node.ObjToken == "" && node.NodeToken == "" {
return Node{}, fmt.Errorf("[doubao_new] create folder failed: empty response")
}
if node.NodeToken == "" {
if node.Token != "" {
node.NodeToken = node.Token
} else if node.ObjToken != "" {
node.NodeToken = node.ObjToken
}
}
if node.ObjToken == "" && node.Token != "" {
node.ObjToken = node.Token
}
return node, nil
}
func (d *DoubaoNew) renameFolder(ctx context.Context, token, name string) error {
if token == "" {
return fmt.Errorf("[doubao_new] rename folder missing token")
}
data := url.Values{}
data.Set("token", token)
data.Set("name", name)
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "*/*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/rename/"); err != nil {
return nil, nil, err
}
if csrfToken != "" {
req.SetHeader("x-csrftoken", csrfToken)
}
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
req.SetBody(data.Encode())
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/rename/")
if err != nil {
return nil, nil, err
}
return res, res.Body(), nil
}
res, body, err := doRequestWithCsrf(doRequest)
if err != nil {
return err
}
return decodeBaseResp(body, res)
}
func isCsrfTokenError(body []byte, res *resty.Response) bool {
if len(body) == 0 {
return false
}
if strings.Contains(strings.ToLower(string(body)), "csrf token error") {
return true
}
if res != nil && res.StatusCode() == http.StatusForbidden {
return true
}
return false
}
func doRequestWithCsrf(doRequest func(csrfToken string) (*resty.Response, []byte, error)) (*resty.Response, []byte, error) {
res, body, err := doRequest("")
if err != nil {
return res, body, err
}
if isCsrfTokenError(body, res) {
csrfToken := extractCsrfTokenFromResponse(res)
if csrfToken != "" {
return doRequest(csrfToken)
}
}
return res, body, err
}
func extractCsrfTokenFromResponse(res *resty.Response) string {
if res == nil || res.Request == nil {
return ""
}
if res.Request.RawRequest != nil {
if csrf := cookie.GetStr(res.Request.RawRequest.Header.Get("Cookie"), "_csrf_token"); csrf != "" {
return csrf
}
}
if csrf := cookie.GetStr(res.Request.Header.Get("Cookie"), "_csrf_token"); csrf != "" {
return csrf
}
for _, c := range res.Cookies() {
if c.Name == "_csrf_token" {
return c.Value
}
}
return ""
}
func decodeBaseResp(body []byte, res *resty.Response) error {
var common BaseResp
if err := json.Unmarshal(body, &common); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return fmt.Errorf("%s", msg)
}
if common.Code != 0 {
errMsg := common.Msg
if errMsg == "" {
errMsg = common.Message
}
return fmt.Errorf("[doubao_new] API error (code: %d): %s", common.Code, errMsg)
}
return nil
}
func (d *DoubaoNew) renameFile(ctx context.Context, fileToken, name string) error {
if fileToken == "" {
return fmt.Errorf("[doubao_new] rename file missing file token")
}
_, err := d.request(ctx, "/space/api/box/file/update_info/", http.MethodPost, func(req *resty.Request) {
req.SetHeader("Content-Type", "application/json")
req.SetBody(base.Json{
"file_token": fileToken,
"name": name,
})
}, nil)
return err
}
func (d *DoubaoNew) moveObj(ctx context.Context, srcToken, destToken string) error {
if srcToken == "" {
return fmt.Errorf("[doubao_new] move missing src token")
}
data := url.Values{}
data.Set("src_token", srcToken)
if destToken != "" {
data.Set("dest_token", destToken)
}
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "*/*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/move/"); err != nil {
return nil, nil, err
}
if csrfToken != "" {
req.SetHeader("x-csrftoken", csrfToken)
}
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
req.SetBody(data.Encode())
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/move/")
if err != nil {
return nil, nil, err
}
return res, res.Body(), nil
}
res, body, err := doRequestWithCsrf(doRequest)
if err != nil {
return err
}
return decodeBaseResp(body, res)
}
func (d *DoubaoNew) removeObj(ctx context.Context, tokens []string) error {
if len(tokens) == 0 {
return fmt.Errorf("[doubao_new] remove missing tokens")
}
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "application/json, text/plain, */*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v3/remove/"); err != nil {
return nil, nil, err
}
if csrfToken != "" {
req.SetHeader("x-csrftoken", csrfToken)
}
req.SetHeader("Content-Type", "application/json")
req.SetBody(base.Json{
"tokens": tokens,
"apply": 1,
})
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v3/remove/")
if err != nil {
return nil, nil, err
}
return res, res.Body(), nil
}
res, body, err := doRequestWithCsrf(doRequest)
if err != nil {
return err
}
var resp RemoveResp
if err := json.Unmarshal(body, &resp); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return fmt.Errorf("%s", msg)
}
if resp.Code != 0 {
errMsg := resp.Msg
if errMsg == "" {
errMsg = resp.Message
}
return fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
}
if resp.Data.TaskID == "" {
return nil
}
return d.waitTask(ctx, resp.Data.TaskID)
}
func (d *DoubaoNew) getUserStorage(ctx context.Context) (UserStorageData, error) {
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "*/*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
req.SetHeader("agw-js-conv", "str")
req.SetHeader("content-type", "application/json")
if err := d.applyAuthHeaders(req, http.MethodPost, DoubaoURL+"/alice/aispace/facade/get_user_storage"); err != nil {
return UserStorageData{}, err
}
if d.Cookie != "" {
req.SetHeader("cookie", d.Cookie)
}
req.SetBody(base.Json{})
res, err := req.Execute(http.MethodPost, DoubaoURL+"/alice/aispace/facade/get_user_storage")
if err != nil {
return UserStorageData{}, err
}
body := res.Body()
var resp UserStorageResp
if err := json.Unmarshal(body, &resp); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return UserStorageData{}, fmt.Errorf("%s", msg)
}
if resp.Code != 0 {
errMsg := resp.Msg
if errMsg == "" {
errMsg = resp.Message
}
return UserStorageData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
}
return resp.Data, nil
}
func (d *DoubaoNew) waitTask(ctx context.Context, taskID string) error {
const (
taskPollInterval = time.Second
taskPollMaxAttempts = 120
)
var lastErr error
for attempt := 0; attempt < taskPollMaxAttempts; attempt++ {
if attempt > 0 {
if err := waitWithContext(ctx, taskPollInterval); err != nil {
return err
}
}
status, err := d.getTaskStatus(ctx, taskID)
if err != nil {
lastErr = err
continue
}
if status.IsFail {
return fmt.Errorf("[doubao_new] remove task failed: %s", taskID)
}
if status.IsFinish {
return nil
}
}
if lastErr != nil {
return lastErr
}
return fmt.Errorf("[doubao_new] remove task timed out: %s", taskID)
}
func (d *DoubaoNew) getTaskStatus(ctx context.Context, taskID string) (TaskStatusData, error) {
if taskID == "" {
return TaskStatusData{}, fmt.Errorf("[doubao_new] task status missing task_id")
}
req := base.RestyClient.R()
req.SetContext(ctx)
req.SetHeader("accept", "application/json, text/plain, */*")
req.SetHeader("origin", DoubaoURL)
req.SetHeader("referer", DoubaoURL+"/")
if err := d.applyAuthHeaders(req, http.MethodGet, BaseURL+"/space/api/explorer/v2/task/"); err != nil {
return TaskStatusData{}, err
}
req.SetQueryParam("task_id", taskID)
res, err := req.Execute(http.MethodGet, BaseURL+"/space/api/explorer/v2/task/")
if err != nil {
return TaskStatusData{}, err
}
body := res.Body()
var resp TaskStatusResp
if err := json.Unmarshal(body, &resp); err != nil {
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
res.Status(),
res.Header().Get("Content-Type"),
string(body),
err,
)
return TaskStatusData{}, fmt.Errorf("%s", msg)
}
if resp.Code != 0 {
errMsg := resp.Msg
if errMsg == "" {
errMsg = resp.Message
}
return TaskStatusData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
}
return resp.Data, nil
}
func waitWithContext(ctx context.Context, d time.Duration) error {
timer := time.NewTimer(d)
defer timer.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-timer.C:
return nil
}
}
-261
View File
@@ -1,261 +0,0 @@
package emby
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"path"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Emby struct {
model.Storage
Addition
client *http.Client
token string
userID string
}
func (d *Emby) Config() driver.Config {
return config
}
func (d *Emby) GetAddition() driver.Additional {
return &d.Addition
}
func (d *Emby) Init(ctx context.Context) error {
d.URL = strings.TrimRight(strings.TrimSpace(d.URL), "/")
if d.URL == "" {
return fmt.Errorf("url is required")
}
if strings.TrimSpace(d.RootFolderID) == "" {
d.RootFolderID = "1"
}
d.client = base.HttpClient
d.token = strings.TrimSpace(d.ApiKey)
d.userID = strings.TrimSpace(d.UserID)
if d.token != "" {
if d.userID == "" {
return fmt.Errorf("user_id is required when api_key is set")
}
op.MustSaveDriverStorage(d)
return nil
}
if strings.TrimSpace(d.Username) == "" || strings.TrimSpace(d.Password) == "" {
return fmt.Errorf("please provide api_key+user_id or username+password")
}
if err := d.login(ctx); err != nil {
return err
}
d.ApiKey = d.token
d.UserID = d.userID
op.MustSaveDriverStorage(d)
return nil
}
func (d *Emby) Drop(ctx context.Context) error {
return nil
}
func (d *Emby) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
parentID := strings.TrimSpace(d.RootFolderID)
if dir != nil && strings.TrimSpace(dir.GetID()) != "" {
parentID = strings.TrimSpace(dir.GetID())
}
items, err := d.getItems(ctx, parentID)
if err != nil {
return nil, err
}
parentPath := "/"
if dir != nil && strings.TrimSpace(dir.GetPath()) != "" {
parentPath = dir.GetPath()
}
objs := make([]model.Obj, 0, len(items.Items))
for _, it := range items.Items {
modified := time.Now()
if it.DateCreated != "" {
if t, parseErr := time.Parse(time.RFC3339Nano, it.DateCreated); parseErr == nil {
modified = t
}
}
name := strings.TrimSpace(it.Name)
id := strings.TrimSpace(it.ID)
displayName := name
if name != "" && id != "" {
if it.IsFolder {
displayName = fmt.Sprintf("%s (ID%s)", name, id)
} else {
ext := path.Ext(strings.TrimSpace(it.Path))
if ext == "" {
ext = path.Ext(name)
}
base := strings.TrimSpace(strings.TrimSuffix(name, ext))
episodeCode := ""
if m := episodeCodeRegexp.FindString(base); m != "" {
episodeCode = strings.ToUpper(m)
} else if it.ParentIndex > 0 && it.IndexNumber > 0 {
episodeCode = fmt.Sprintf("S%02dE%02d", it.ParentIndex, it.IndexNumber)
}
title := strings.TrimSpace(base)
if episodeCode != "" {
title = strings.TrimSpace(episodeCodeRegexp.ReplaceAllString(title, ""))
title = strings.TrimSpace(strings.Trim(title, "-_:[]() "))
}
series := strings.TrimSpace(it.SeriesName)
if series == "" && episodeCode != "" {
if idx := strings.Index(title, " - "); idx > 0 {
series = strings.TrimSpace(title[:idx])
title = strings.TrimSpace(title[idx+3:])
}
}
core := title
if series != "" {
if title == "" || strings.EqualFold(series, title) {
core = series
} else {
core = series + " " + title
}
}
if core == "" {
core = base
}
if episodeCode != "" {
core = fmt.Sprintf("%s - [%s]", core, episodeCode)
}
if ext == "" {
displayName = fmt.Sprintf("%s (ID%s)", core, id)
} else {
displayName = fmt.Sprintf("%s (ID%s)%s", core, id, ext)
}
}
}
obj := &model.Object{
ID: id,
Name: displayName,
Path: path.Join(parentPath, displayName),
Size: it.Size,
Modified: modified,
IsFolder: it.IsFolder,
}
if it.IsFolder {
obj.Size = 0
}
objs = append(objs, obj)
}
return objs, nil
}
func (d *Emby) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if file.IsDir() {
return nil, errs.NotFile
}
fileID := strings.TrimSpace(file.GetID())
if fileID == "" {
return nil, fmt.Errorf("invalid file id")
}
u, err := url.Parse(d.URL)
if err != nil {
return nil, err
}
linkMethod := strings.ToLower(strings.TrimSpace(d.LinkMethod))
useDownload := linkMethod == "download"
mediaSourceID := ""
mediaContainer := ""
if !useDownload {
detailURL, parseErr := url.Parse(d.URL + "/Users/" + d.userID + "/Items/" + fileID)
if parseErr == nil {
q := detailURL.Query()
q.Set("Fields", "MediaSources")
q.Set("api_key", d.token)
detailURL.RawQuery = q.Encode()
req, reqErr := http.NewRequestWithContext(ctx, http.MethodGet, detailURL.String(), nil)
if reqErr == nil {
resp, doErr := d.client.Do(req)
if doErr == nil {
func() {
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return
}
var detail itemDetailResp
if decodeErr := json.NewDecoder(resp.Body).Decode(&detail); decodeErr != nil || len(detail.MediaSources) == 0 {
return
}
for i := range detail.MediaSources {
if strings.TrimSpace(detail.MediaSources[i].ID) != "" && detail.MediaSources[i].SupportsDirectStream {
mediaSourceID = strings.TrimSpace(detail.MediaSources[i].ID)
mediaContainer = strings.TrimSpace(detail.MediaSources[i].Container)
return
}
}
for i := range detail.MediaSources {
if strings.TrimSpace(detail.MediaSources[i].ID) != "" {
mediaSourceID = strings.TrimSpace(detail.MediaSources[i].ID)
mediaContainer = strings.TrimSpace(detail.MediaSources[i].Container)
return
}
}
}()
}
}
}
}
if useDownload {
u.Path = path.Join(u.Path, "/Items", fileID, "Download")
} else {
if mediaContainer != "" {
u.Path = path.Join(u.Path, "/Videos", fileID, "stream."+mediaContainer)
} else {
u.Path = path.Join(u.Path, "/Videos", fileID, "stream")
}
}
q := u.Query()
q.Set("api_key", d.token)
if mediaSourceID != "" {
q.Set("MediaSourceId", mediaSourceID)
}
if !useDownload {
q.Set("Static", "true")
}
u.RawQuery = q.Encode()
return &model.Link{
URL: u.String(),
Header: http.Header{
"User-Agent": []string{base.UserAgent},
},
}, nil
}
var _ driver.Driver = (*Emby)(nil)
-30
View File
@@ -1,30 +0,0 @@
package emby
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootID
URL string `json:"url" required:"true"`
ApiKey string `json:"api_key"`
UserID string `json:"user_id"`
Username string `json:"username"`
Password string `json:"password"`
LinkMethod string `json:"link_method" type:"select" options:"stream,download" default:"stream"`
}
var config = driver.Config{
Name: "Emby",
LocalSort: true,
NoUpload: true,
DefaultRoot: "1",
CheckStatus: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Emby{}
})
}
-41
View File
@@ -1,41 +0,0 @@
package emby
type authReq struct {
Username string `json:"Username"`
Pw string `json:"Pw"`
}
type authResp struct {
AccessToken string `json:"AccessToken"`
User struct {
ID string `json:"Id"`
} `json:"User"`
}
type listResp struct {
Items []embyItem `json:"Items"`
TotalRecordCount int `json:"TotalRecordCount"`
}
type embyItem struct {
Name string `json:"Name"`
ID string `json:"Id"`
Type string `json:"Type"`
Path string `json:"Path"`
SeriesName string `json:"SeriesName"`
IndexNumber int `json:"IndexNumber"`
ParentIndex int `json:"ParentIndexNumber"`
IsFolder bool `json:"IsFolder"`
Size int64 `json:"Size"`
DateCreated string `json:"DateCreated"`
}
type itemDetailResp struct {
MediaSources []embyMediaSource `json:"MediaSources"`
}
type embyMediaSource struct {
ID string `json:"Id"`
Container string `json:"Container"`
SupportsDirectStream bool `json:"SupportsDirectStream"`
}
-91
View File
@@ -1,91 +0,0 @@
package emby
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"regexp"
"strings"
)
var episodeCodeRegexp = regexp.MustCompile(`(?i)\bS\d{1,2}E\d{1,2}\b`)
func (d *Emby) login(ctx context.Context) error {
payload, err := json.Marshal(authReq{
Username: d.Username,
Pw: d.Password,
})
if err != nil {
return err
}
endpoint := d.URL + "/Users/AuthenticateByName"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(payload))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-Emby-Authorization", `MediaBrowser Client="OpenList", Device="OpenList", DeviceId="openlist-emby", Version="1.0.0"`)
resp, err := d.client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf("emby auth failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(body)))
}
var data authResp
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return err
}
if strings.TrimSpace(data.AccessToken) == "" || strings.TrimSpace(data.User.ID) == "" {
return fmt.Errorf("emby auth response missing access token or user id")
}
d.token = data.AccessToken
d.userID = data.User.ID
return nil
}
func (d *Emby) getItems(ctx context.Context, parentID string) (*listResp, error) {
u, err := url.Parse(d.URL + "/Users/" + d.userID + "/Items")
if err != nil {
return nil, err
}
q := u.Query()
q.Set("ParentId", parentID)
q.Set("Recursive", "false")
q.Set("Fields", "Path,Size,DateCreated,SeriesName,IndexNumber,ParentIndexNumber")
q.Set("api_key", d.token)
u.RawQuery = q.Encode()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
resp, err := d.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("emby list failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(body)))
}
var data listResp
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return nil, err
}
return &data, nil
}
-31
View File
@@ -1,31 +0,0 @@
package google_drive
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootID
RefreshToken string `json:"refresh_token" required:"true"`
OrderBy string `json:"order_by" type:"string" help:"such as: folder,name,modifiedTime"`
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc"`
UseOnlineAPI bool `json:"use_online_api" default:"true"`
APIAddress string `json:"api_url_address" default:"https://api.oplist.org/googleui/renewapi"`
ClientID string `json:"client_id"`
ClientSecret string `json:"client_secret"`
ChunkSize int64 `json:"chunk_size" type:"number" default:"5" help:"chunk size while uploading (unit: MB)"`
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
}
var config = driver.Config{
Name: "GoogleDrive",
OnlyProxy: true,
DefaultRoot: "root",
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &GoogleDrive{}
})
}
-111
View File
@@ -1,111 +0,0 @@
package halalcloudopen
import (
"sync"
"time"
sdkUser "github.com/halalcloud/golang-sdk-lite/halalcloud/services/user"
)
var (
slicePostErrorRetryInterval = time.Second * 120
retryTimes = 5
)
type halalCommon struct {
// *AuthService // 登录信息
UserInfo *sdkUser.User // 用户信息
refreshTokenFunc func(token string) error
// serv *AuthService
configs sync.Map
}
func (m *halalCommon) GetAccessToken() (string, error) {
value, exists := m.configs.Load("access_token")
if !exists {
return "", nil // 如果不存在,返回空字符串
}
return value.(string), nil // 返回配置项的值
}
// GetRefreshToken implements ConfigStore.
func (m *halalCommon) GetRefreshToken() (string, error) {
value, exists := m.configs.Load("refresh_token")
if !exists {
return "", nil // 如果不存在,返回空字符串
}
return value.(string), nil // 返回配置项的值
}
// SetAccessToken implements ConfigStore.
func (m *halalCommon) SetAccessToken(token string) error {
m.configs.Store("access_token", token)
return nil
}
// SetRefreshToken implements ConfigStore.
func (m *halalCommon) SetRefreshToken(token string) error {
m.configs.Store("refresh_token", token)
if m.refreshTokenFunc != nil {
return m.refreshTokenFunc(token)
}
return nil
}
// SetToken implements ConfigStore.
func (m *halalCommon) SetToken(accessToken string, refreshToken string, expiresIn int64) error {
m.configs.Store("access_token", accessToken)
m.configs.Store("refresh_token", refreshToken)
m.configs.Store("expires_in", expiresIn)
if m.refreshTokenFunc != nil {
return m.refreshTokenFunc(refreshToken)
}
return nil
}
// ClearConfigs implements ConfigStore.
func (m *halalCommon) ClearConfigs() error {
m.configs = sync.Map{} // 清空map
return nil
}
// DeleteConfig implements ConfigStore.
func (m *halalCommon) DeleteConfig(key string) error {
_, exists := m.configs.Load(key)
if !exists {
return nil // 如果不存在,直接返回
}
m.configs.Delete(key) // 删除指定的配置项
return nil
}
// GetConfig implements ConfigStore.
func (m *halalCommon) GetConfig(key string) (string, error) {
value, exists := m.configs.Load(key)
if !exists {
return "", nil // 如果不存在,返回空字符串
}
return value.(string), nil // 返回配置项的值
}
// ListConfigs implements ConfigStore.
func (m *halalCommon) ListConfigs() (map[string]string, error) {
configs := make(map[string]string)
m.configs.Range(func(key, value interface{}) bool {
configs[key.(string)] = value.(string) // 将每个配置项添加到map中
return true // 继续遍历
})
return configs, nil // 返回所有配置项
}
// SetConfig implements ConfigStore.
func (m *halalCommon) SetConfig(key string, value string) error {
m.configs.Store(key, value) // 使用Store方法设置或更新配置项
return nil // 成功设置配置项后返回nil
}
func NewHalalCommon() *halalCommon {
return &halalCommon{
configs: sync.Map{},
}
}
-29
View File
@@ -1,29 +0,0 @@
package halalcloudopen
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
sdkClient "github.com/halalcloud/golang-sdk-lite/halalcloud/apiclient"
sdkUser "github.com/halalcloud/golang-sdk-lite/halalcloud/services/user"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
)
type HalalCloudOpen struct {
*halalCommon
model.Storage
Addition
sdkClient *sdkClient.Client
sdkUserFileService *sdkUserFile.UserFileService
sdkUserService *sdkUser.UserService
uploadThread int
}
func (d *HalalCloudOpen) Config() driver.Config {
return config
}
func (d *HalalCloudOpen) GetAddition() driver.Additional {
return &d.Addition
}
var _ driver.Driver = (*HalalCloudOpen)(nil)
-127
View File
@@ -1,127 +0,0 @@
package halalcloudopen
import (
"context"
"github.com/OpenListTeam/OpenList/v4/internal/model"
sdkModel "github.com/halalcloud/golang-sdk-lite/halalcloud/model"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
)
func (d *HalalCloudOpen) getFiles(ctx context.Context, dir model.Obj) ([]model.Obj, error) {
files := make([]model.Obj, 0)
limit := int64(100)
token := ""
for {
result, err := d.sdkUserFileService.List(ctx, &sdkUserFile.FileListRequest{
Parent: &sdkUserFile.File{Path: dir.GetPath()},
ListInfo: &sdkModel.ScanListRequest{
Limit: limit,
Token: token,
},
})
if err != nil {
return nil, err
}
for i := 0; len(result.Files) > i; i++ {
files = append(files, NewObjFile(result.Files[i]))
}
if result.ListInfo == nil || result.ListInfo.Token == "" {
break
}
token = result.ListInfo.Token
}
return files, nil
}
func (d *HalalCloudOpen) makeDir(ctx context.Context, dir model.Obj, name string) (model.Obj, error) {
_, err := d.sdkUserFileService.Create(ctx, &sdkUserFile.File{
Path: dir.GetPath(),
Name: name,
})
return nil, err
}
func (d *HalalCloudOpen) move(ctx context.Context, obj model.Obj, dir model.Obj) (model.Obj, error) {
oldDir := obj.GetPath()
newDir := dir.GetPath()
_, err := d.sdkUserFileService.Move(ctx, &sdkUserFile.BatchOperationRequest{
Source: []*sdkUserFile.File{
{
Path: oldDir,
},
},
Dest: &sdkUserFile.File{
Path: newDir,
},
})
return nil, err
}
func (d *HalalCloudOpen) rename(ctx context.Context, obj model.Obj, name string) (model.Obj, error) {
_, err := d.sdkUserFileService.Rename(ctx, &sdkUserFile.File{
Path: obj.GetPath(),
Name: name,
})
return nil, err
}
func (d *HalalCloudOpen) copy(ctx context.Context, obj model.Obj, dir model.Obj) (model.Obj, error) {
id := obj.GetID()
sourcePath := obj.GetPath()
if len(id) > 0 {
sourcePath = ""
}
destID := dir.GetID()
destPath := dir.GetPath()
if len(destID) > 0 {
destPath = ""
}
dest := &sdkUserFile.File{
Path: destPath,
Identity: destID,
}
_, err := d.sdkUserFileService.Copy(ctx, &sdkUserFile.BatchOperationRequest{
Source: []*sdkUserFile.File{
{
Path: sourcePath,
Identity: id,
},
},
Dest: dest,
})
return nil, err
}
func (d *HalalCloudOpen) remove(ctx context.Context, obj model.Obj) error {
id := obj.GetID()
_, err := d.sdkUserFileService.Delete(ctx, &sdkUserFile.BatchOperationRequest{
Source: []*sdkUserFile.File{
{
Identity: id,
Path: obj.GetPath(),
},
},
})
return err
}
func (d *HalalCloudOpen) details(ctx context.Context) (*model.StorageDetails, error) {
ret, err := d.sdkUserService.GetStatisticsAndQuota(ctx)
if err != nil {
return nil, err
}
return &model.StorageDetails{
DiskUsage: model.DiskUsage{
TotalSpace: ret.DiskStatisticsQuota.BytesQuota,
UsedSpace: ret.DiskStatisticsQuota.BytesUsed,
},
}, nil
}
-108
View File
@@ -1,108 +0,0 @@
package halalcloudopen
import (
"context"
"crypto/sha1"
"io"
"strconv"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
"github.com/rclone/rclone/lib/readers"
)
func (d *HalalCloudOpen) getLink(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
if args.Redirect {
// return nil, model.ErrUnsupported
fid := file.GetID()
fpath := file.GetPath()
if fid != "" {
fpath = ""
}
fi, err := d.sdkUserFileService.GetDirectDownloadAddress(ctx, &sdkUserFile.DirectDownloadRequest{
Identity: fid,
Path: fpath,
})
if err != nil {
return nil, err
}
expireAt := fi.ExpireAt
duration := time.Until(time.UnixMilli(expireAt))
return &model.Link{
URL: fi.DownloadAddress,
Expiration: &duration,
}, nil
}
result, err := d.sdkUserFileService.ParseFileSlice(ctx, &sdkUserFile.File{
Identity: file.GetID(),
Path: file.GetPath(),
})
if err != nil {
return nil, err
}
fileAddrs := []*sdkUserFile.SliceDownloadInfo{}
var addressDuration int64
nodesNumber := len(result.RawNodes)
nodesIndex := nodesNumber - 1
startIndex, endIndex := 0, nodesIndex
for nodesIndex >= 0 {
if nodesIndex >= 200 {
endIndex = 200
} else {
endIndex = nodesNumber
}
for ; endIndex <= nodesNumber; endIndex += 200 {
if endIndex == 0 {
endIndex = 1
}
sliceAddress, err := d.sdkUserFileService.GetSliceDownloadAddress(ctx, &sdkUserFile.SliceDownloadAddressRequest{
Identity: result.RawNodes[startIndex:endIndex],
Version: 1,
})
if err != nil {
return nil, err
}
addressDuration, _ = strconv.ParseInt(sliceAddress.ExpireAt, 10, 64)
fileAddrs = append(fileAddrs, sliceAddress.Addresses...)
startIndex = endIndex
nodesIndex -= 200
}
}
size, _ := strconv.ParseInt(result.FileSize, 10, 64)
chunks := getChunkSizes(result.Sizes)
resultRangeReader := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
length := httpRange.Length
if httpRange.Length < 0 || httpRange.Start+httpRange.Length >= size {
length = size - httpRange.Start
}
oo := &openObject{
ctx: ctx,
d: fileAddrs,
chunk: []byte{},
chunks: chunks,
skip: httpRange.Start,
sha: result.Sha1,
shaTemp: sha1.New(),
}
return readers.NewLimitedReadCloser(oo, length), nil
}
var duration time.Duration
if addressDuration != 0 {
duration = time.Until(time.UnixMilli(addressDuration))
} else {
duration = time.Until(time.Now().Add(time.Hour))
}
return &model.Link{
RangeReader: stream.RateLimitRangeReaderFunc(resultRangeReader),
Expiration: &duration,
}, nil
}
-50
View File
@@ -1,50 +0,0 @@
package halalcloudopen
import (
"context"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/halalcloud/golang-sdk-lite/halalcloud/apiclient"
sdkUser "github.com/halalcloud/golang-sdk-lite/halalcloud/services/user"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
)
func (d *HalalCloudOpen) Init(ctx context.Context) error {
if d.uploadThread < 1 || d.uploadThread > 32 {
d.uploadThread, d.UploadThread = 3, 3
}
if d.halalCommon == nil {
d.halalCommon = &halalCommon{
UserInfo: &sdkUser.User{},
refreshTokenFunc: func(token string) error {
d.Addition.RefreshToken = token
op.MustSaveDriverStorage(d)
return nil
},
}
}
if d.Addition.RefreshToken != "" {
d.halalCommon.SetRefreshToken(d.Addition.RefreshToken)
}
timeout := d.Addition.TimeOut
if timeout <= 0 {
timeout = 60
}
host := d.Addition.Host
if host == "" {
host = "openapi.2dland.cn"
}
client := apiclient.NewClient(nil, host, d.Addition.ClientID, d.Addition.ClientSecret, d.halalCommon, apiclient.WithTimeout(time.Second*time.Duration(timeout)))
d.sdkClient = client
d.sdkUserFileService = sdkUserFile.NewUserFileService(client)
d.sdkUserService = sdkUser.NewUserService(client)
userInfo, err := d.sdkUserService.Get(ctx, &sdkUser.User{})
if err != nil {
return err
}
d.halalCommon.UserInfo = userInfo
// 能够获取到用户信息,已经检查了 RefreshToken 的有效性,无需再次检查
return nil
}
@@ -1,48 +0,0 @@
package halalcloudopen
import (
"context"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
)
func (d *HalalCloudOpen) Drop(ctx context.Context) error {
return nil
}
func (d *HalalCloudOpen) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
return d.getFiles(ctx, dir)
}
func (d *HalalCloudOpen) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
return d.getLink(ctx, file, args)
}
func (d *HalalCloudOpen) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
return d.makeDir(ctx, parentDir, dirName)
}
func (d *HalalCloudOpen) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
return d.move(ctx, srcObj, dstDir)
}
func (d *HalalCloudOpen) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
return d.rename(ctx, srcObj, newName)
}
func (d *HalalCloudOpen) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
return d.copy(ctx, srcObj, dstDir)
}
func (d *HalalCloudOpen) Remove(ctx context.Context, obj model.Obj) error {
return d.remove(ctx, obj)
}
func (d *HalalCloudOpen) Put(ctx context.Context, dstDir model.Obj, stream model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
return d.put(ctx, dstDir, stream, up)
}
func (d *HalalCloudOpen) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
return d.details(ctx)
}
@@ -1,258 +0,0 @@
package halalcloudopen
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"path"
"strings"
"time"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
"github.com/ipfs/go-cid"
log "github.com/sirupsen/logrus"
)
func (d *HalalCloudOpen) put(ctx context.Context, dstDir model.Obj, fileStream model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
newPath := path.Join(dstDir.GetPath(), fileStream.GetName())
uploadTask, err := d.sdkUserFileService.CreateUploadTask(ctx, &sdkUserFile.File{
Path: newPath,
Size: fileStream.GetSize(),
})
if err != nil {
return nil, err
}
if uploadTask.Created {
return nil, nil
}
slicesList := make([]string, 0)
codec := uint64(0x55)
if uploadTask.BlockCodec > 0 {
codec = uint64(uploadTask.BlockCodec)
}
blockHashType := uploadTask.BlockHashType
mhType := uint64(0x12)
if blockHashType > 0 {
mhType = uint64(blockHashType)
}
prefix := cid.Prefix{
Codec: codec,
MhLength: -1,
MhType: mhType,
Version: 1,
}
blockSize := uploadTask.BlockSize
//
// Not sure whether FileStream supports concurrent read and write operations, so currently using single-threaded upload to ensure safety.
// read file
bufferSize := int(blockSize)
buffer := make([]byte, bufferSize)
offset := 0
teeReader := io.TeeReader(fileStream, driver.NewProgress(fileStream.GetSize(), up))
for {
n, err := teeReader.Read(buffer[offset:]) // 这里 len(buf[offset:]) <= 4MB
if n > 0 {
offset += n
if offset == int(blockSize) {
uploadCid, err := postFileSlice(ctx, buffer, uploadTask.Task, uploadTask.UploadAddress, prefix, retryTimes)
if err != nil {
return nil, err
}
slicesList = append(slicesList, uploadCid.String())
offset = 0
}
}
if err != nil {
if err == io.EOF {
if offset > 0 {
uploadCid, err := postFileSlice(ctx, buffer[:offset], uploadTask.Task, uploadTask.UploadAddress, prefix, retryTimes)
if err != nil {
return nil, err
}
slicesList = append(slicesList, uploadCid.String())
}
break
}
return nil, err
}
}
newFile, err := makeFile(ctx, slicesList, uploadTask.Task, uploadTask.UploadAddress, retryTimes)
if err != nil {
return nil, err
}
return NewObjFile(newFile), nil
}
func makeFile(ctx context.Context, fileSlice []string, taskID string, uploadAddress string, retry int) (*sdkUserFile.File, error) {
var lastError error = nil
for range retry {
newFile, err := doMakeFile(fileSlice, taskID, uploadAddress)
if err == nil {
return newFile, nil
}
if ctx.Err() != nil {
return nil, err
}
log.Errorf("make file slice failed, retrying... error: %s", err.Error())
if strings.Contains(err.Error(), "not found") {
return nil, err
}
lastError = err
time.Sleep(slicePostErrorRetryInterval)
}
return nil, fmt.Errorf("mk file slice failed after %d times, error: %s", retry, lastError.Error())
}
func doMakeFile(fileSlice []string, taskID string, uploadAddress string) (*sdkUserFile.File, error) {
accessUrl := uploadAddress + "/" + taskID
getTimeOut := time.Minute * 2
u, err := url.Parse(accessUrl)
if err != nil {
return nil, err
}
n, _ := json.Marshal(fileSlice)
httpRequest := http.Request{
Method: http.MethodPost,
URL: u,
Header: map[string][]string{
"Accept": {"application/json"},
"Content-Type": {"application/json"},
//"Content-Length": {strconv.Itoa(len(n))},
},
Body: io.NopCloser(bytes.NewReader(n)),
}
httpClient := http.Client{
Timeout: getTimeOut,
}
httpResponse, err := httpClient.Do(&httpRequest)
if err != nil {
return nil, err
}
defer httpResponse.Body.Close()
if httpResponse.StatusCode != http.StatusOK && httpResponse.StatusCode != http.StatusCreated {
b, _ := io.ReadAll(httpResponse.Body)
message := string(b)
log.Errorf("make file failed, status code: %d, message: %s", httpResponse.StatusCode, message)
return nil, fmt.Errorf("mk file slice failed, status code: %d, message: %s", httpResponse.StatusCode, message)
}
b, _ := io.ReadAll(httpResponse.Body)
var result *UploadedFile
err = json.Unmarshal(b, &result)
if err != nil {
log.Errorf("make file failed from response, status code: %d, message: %s", httpResponse.StatusCode, string(b))
return nil, err
}
return &sdkUserFile.File{
Identity: result.Identity,
Path: result.Path,
Size: result.Size,
ContentIdentity: result.ContentIdentity,
}, nil
}
func postFileSlice(ctx context.Context, fileSlice []byte, taskID string, uploadAddress string, preix cid.Prefix, retry int) (cid.Cid, error) {
var lastError error = nil
for range retry {
newCid, err := doPostFileSlice(fileSlice, taskID, uploadAddress, preix)
if err == nil {
return newCid, nil
}
if ctx.Err() != nil {
return cid.Undef, err
}
time.Sleep(slicePostErrorRetryInterval)
lastError = err
}
return cid.Undef, fmt.Errorf("upload file slice failed after %d times, error: %s", retry, lastError.Error())
}
func doPostFileSlice(fileSlice []byte, taskID string, uploadAddress string, preix cid.Prefix) (cid.Cid, error) {
// 1. sum file slice
newCid, err := preix.Sum(fileSlice)
if err != nil {
return cid.Undef, err
}
// 2. post file slice
sliceCidString := newCid.String()
// /{taskID}/{sliceID}
accessUrl := uploadAddress + "/" + taskID + "/" + sliceCidString
getTimeOut := time.Second * 30
// get {accessUrl} in {getTimeOut}
u, err := url.Parse(accessUrl)
if err != nil {
return cid.Undef, err
}
// header: accept: application/json
// header: content-type: application/octet-stream
// header: content-length: {fileSlice.length}
// header: x-content-cid: {sliceCidString}
// header: x-task-id: {taskID}
httpRequest := http.Request{
Method: http.MethodGet,
URL: u,
Header: map[string][]string{
"Accept": {"application/json"},
},
}
httpClient := http.Client{
Timeout: getTimeOut,
}
httpResponse, err := httpClient.Do(&httpRequest)
if err != nil {
log.Errorf("access %s failed, method: %s", accessUrl, http.MethodGet)
return cid.Undef, err
}
if httpResponse.StatusCode != http.StatusOK {
log.Errorf("access %s failed, method: %s, status code: %d", accessUrl, http.MethodGet, httpResponse.StatusCode)
return cid.Undef, fmt.Errorf("upload file slice failed, status code: %d", httpResponse.StatusCode)
}
var result bool
b, err := io.ReadAll(httpResponse.Body)
if err != nil {
return cid.Undef, err
}
err = json.Unmarshal(b, &result)
if err != nil {
return cid.Undef, err
}
if result {
return newCid, nil
}
httpRequest = http.Request{
Method: http.MethodPost,
URL: u,
Header: map[string][]string{
"Accept": {"application/json"},
"Content-Type": {"application/octet-stream"},
// "Content-Length": {strconv.Itoa(len(fileSlice))},
},
Body: io.NopCloser(bytes.NewReader(fileSlice)),
}
httpResponse, err = httpClient.Do(&httpRequest)
if err != nil {
return cid.Undef, err
}
defer httpResponse.Body.Close()
if httpResponse.StatusCode != http.StatusOK && httpResponse.StatusCode != http.StatusCreated {
b, _ := io.ReadAll(httpResponse.Body)
message := string(b)
log.Errorf("upload file slice failed, status code: %d, message: %s", httpResponse.StatusCode, message)
return cid.Undef, fmt.Errorf("upload file slice failed, status code: %d, message: %s", httpResponse.StatusCode, message)
}
//
return newCid, nil
}
-40
View File
@@ -1,40 +0,0 @@
package halalcloudopen
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
// Usually one of two
driver.RootPath
// define other
RefreshToken string `json:"refresh_token" required:"false" help:"If using a personal API approach, the RefreshToken is not required."`
UploadThread int `json:"upload_thread" type:"number" default:"3" help:"1 <= thread <= 32"`
ClientID string `json:"client_id" required:"true" default:""`
ClientSecret string `json:"client_secret" required:"true" default:""`
Host string `json:"host" required:"false" default:"openapi.2dland.cn"`
TimeOut int `json:"timeout" type:"number" default:"60" help:"timeout in seconds"`
}
var config = driver.Config{
Name: "HalalCloudOpen",
OnlyProxy: false,
DefaultRoot: "/",
NoLinkURL: false,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &HalalCloudOpen{}
})
}
type UploadedFile struct {
Identity string `json:"identity"`
UserIdentity string `json:"user_identity"`
Path string `json:"path"`
Size int64 `json:"size"`
ContentIdentity string `json:"content_identity"`
}
-60
View File
@@ -1,60 +0,0 @@
package halalcloudopen
import (
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
)
type ObjFile struct {
sdkFile *sdkUserFile.File
fileSize int64
modTime time.Time
createTime time.Time
}
func NewObjFile(f *sdkUserFile.File) model.Obj {
ofile := &ObjFile{sdkFile: f}
ofile.fileSize = f.Size
modTimeTs := f.UpdateTs
ofile.modTime = time.UnixMilli(modTimeTs)
createTimeTs := f.CreateTs
ofile.createTime = time.UnixMilli(createTimeTs)
return ofile
}
func (f *ObjFile) GetSize() int64 {
return f.fileSize
}
func (f *ObjFile) GetName() string {
return f.sdkFile.Name
}
func (f *ObjFile) ModTime() time.Time {
return f.modTime
}
func (f *ObjFile) IsDir() bool {
return f.sdkFile.Dir
}
func (f *ObjFile) GetHash() utils.HashInfo {
return utils.HashInfo{
// TODO: support more hash types
}
}
func (f *ObjFile) GetID() string {
return f.sdkFile.Identity
}
func (f *ObjFile) GetPath() string {
return f.sdkFile.Path
}
func (f *ObjFile) CreateTime() time.Time {
return f.createTime
}
-185
View File
@@ -1,185 +0,0 @@
package halalcloudopen
import (
"context"
"crypto/md5"
"encoding/hex"
"errors"
"fmt"
"hash"
"io"
"net/http"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
"github.com/ipfs/go-cid"
)
// get the next chunk
func (oo *openObject) getChunk(_ context.Context) (err error) {
if oo.id >= len(oo.chunks) {
return io.EOF
}
var chunk []byte
err = utils.Retry(3, time.Second, func() (err error) {
chunk, err = getRawFiles(oo.d[oo.id])
return err
})
if err != nil {
return err
}
oo.id++
oo.chunk = chunk
return nil
}
// Read reads up to len(p) bytes into p.
func (oo *openObject) Read(p []byte) (n int, err error) {
oo.mu.Lock()
defer oo.mu.Unlock()
if oo.closed {
return 0, fmt.Errorf("read on closed file")
}
// Skip data at the start if requested
for oo.skip > 0 {
//size := 1024 * 1024
_, size, err := oo.ChunkLocation(oo.id)
if err != nil {
return 0, err
}
if oo.skip < int64(size) {
break
}
oo.id++
oo.skip -= int64(size)
}
if len(oo.chunk) == 0 {
err = oo.getChunk(oo.ctx)
if err != nil {
return 0, err
}
if oo.skip > 0 {
oo.chunk = (oo.chunk)[oo.skip:]
oo.skip = 0
}
}
n = copy(p, oo.chunk)
oo.shaTemp.Write(p[:n])
oo.chunk = (oo.chunk)[n:]
return n, nil
}
// Close closed the file - MAC errors are reported here
func (oo *openObject) Close() (err error) {
oo.mu.Lock()
defer oo.mu.Unlock()
if oo.closed {
return nil
}
// 校验Sha1
if string(oo.shaTemp.Sum(nil)) != oo.sha {
return fmt.Errorf("failed to finish download: SHA mismatch")
}
oo.closed = true
return nil
}
func GetMD5Hash(text string) string {
tHash := md5.Sum([]byte(text))
return hex.EncodeToString(tHash[:])
}
type chunkSize struct {
position int64
size int
}
type openObject struct {
ctx context.Context
mu sync.Mutex
d []*sdkUserFile.SliceDownloadInfo
id int
skip int64
chunk []byte
chunks []chunkSize
closed bool
sha string
shaTemp hash.Hash
}
func getChunkSizes(sliceSize []*sdkUserFile.SliceSize) (chunks []chunkSize) {
chunks = make([]chunkSize, 0)
for _, s := range sliceSize {
// 对最后一个做特殊处理
endIndex := s.EndIndex
startIndex := s.StartIndex
if endIndex == 0 {
endIndex = startIndex
}
for j := startIndex; j <= endIndex; j++ {
size := s.Size
chunks = append(chunks, chunkSize{position: j, size: int(size)})
}
}
return chunks
}
func (oo *openObject) ChunkLocation(id int) (position int64, size int, err error) {
if id < 0 || id >= len(oo.chunks) {
return 0, 0, errors.New("invalid arguments")
}
return (oo.chunks)[id].position, (oo.chunks)[id].size, nil
}
func getRawFiles(addr *sdkUserFile.SliceDownloadInfo) ([]byte, error) {
if addr == nil {
return nil, errors.New("addr is nil")
}
client := http.Client{
Timeout: time.Duration(60 * time.Second), // Set timeout to 60 seconds
}
resp, err := client.Get(addr.DownloadAddress)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("bad status: %s, body: %s", resp.Status, body)
}
if addr.Encrypt > 0 {
cd := uint8(addr.Encrypt)
for idx := 0; idx < len(body); idx++ {
body[idx] = body[idx] ^ cd
}
}
storeType := addr.StoreType
if storeType != 10 {
sourceCid, err := cid.Decode(addr.Identity)
if err != nil {
return nil, err
}
checkCid, err := sourceCid.Prefix().Sum(body)
if err != nil {
return nil, err
}
if !checkCid.Equals(sourceCid) {
return nil, fmt.Errorf("bad cid: %s, body: %s", checkCid.String(), body)
}
}
return body, nil
}
-16
View File
@@ -1,16 +0,0 @@
//go:build !windows && !plan9 && !netbsd && !aix && !illumos && !solaris && !js
package local
import (
"os"
"path/filepath"
"syscall"
)
func copyNamedPipe(dstPath string, mode os.FileMode, dirMode os.FileMode) error {
if err := os.MkdirAll(filepath.Dir(dstPath), dirMode); err != nil {
return err
}
return syscall.Mkfifo(dstPath, uint32(mode))
}
-9
View File
@@ -1,9 +0,0 @@
//go:build windows || plan9 || netbsd || aix || illumos || solaris || js
package local
import "os"
func copyNamedPipe(_ string, _, _ os.FileMode) error {
return nil
}
-44
View File
@@ -1,44 +0,0 @@
package local
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/OpenListTeam/OpenList/v4/internal/model"
)
func TestRemoveDeletesThumbCache(t *testing.T) {
root := t.TempDir()
cacheDir := filepath.Join(root, "thumbs")
if err := os.Mkdir(cacheDir, 0o755); err != nil {
t.Fatal(err)
}
filePath := filepath.Join(root, "photo.jpg")
if err := os.WriteFile(filePath, []byte("image"), 0o666); err != nil {
t.Fatal(err)
}
driver := &Local{
Addition: Addition{
ThumbCacheFolder: cacheDir,
},
}
thumbPath := driver.thumbCachePath(filePath)
if err := os.WriteFile(thumbPath, []byte("thumb"), 0o666); err != nil {
t.Fatal(err)
}
err := driver.Remove(context.Background(), &model.Object{
Path: filePath,
Name: filepath.Base(filePath),
})
if err != nil {
t.Fatal(err)
}
if _, err = os.Stat(thumbPath); !os.IsNotExist(err) {
t.Fatalf("expected thumb cache to be removed, got %v", err)
}
}
-35
View File
@@ -1,35 +0,0 @@
//go:build !windows
package local
import (
"errors"
"io/fs"
"strings"
"syscall"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"golang.org/x/sys/unix"
)
func isHidden(f fs.FileInfo, _ string) bool {
return strings.HasPrefix(f.Name(), ".")
}
func getDiskUsage(path string) (model.DiskUsage, error) {
var stat syscall.Statfs_t
err := syscall.Statfs(path, &stat)
if err != nil {
return model.DiskUsage{}, err
}
total := int64(stat.Blocks) * int64(stat.Bsize)
free := int64(stat.Bfree) * int64(stat.Bsize)
return model.DiskUsage{
TotalSpace: total,
UsedSpace: total - free,
}, nil
}
func isCrossDeviceError(err error) bool {
return errors.Is(err, unix.EXDEV)
}
-55
View File
@@ -1,55 +0,0 @@
//go:build windows
package local
import (
"errors"
"io/fs"
"path/filepath"
"syscall"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"golang.org/x/sys/windows"
)
func isHidden(f fs.FileInfo, fullPath string) bool {
filePath := filepath.Join(fullPath, f.Name())
namePtr, err := syscall.UTF16PtrFromString(filePath)
if err != nil {
return false
}
attrs, err := syscall.GetFileAttributes(namePtr)
if err != nil {
return false
}
return attrs&syscall.FILE_ATTRIBUTE_HIDDEN != 0
}
func getDiskUsage(path string) (model.DiskUsage, error) {
abs, err := filepath.Abs(path)
if err != nil {
return model.DiskUsage{}, err
}
root := filepath.VolumeName(abs)
if len(root) != 2 || root[1] != ':' {
return model.DiskUsage{}, errors.New("cannot get disk label")
}
var freeBytes, totalBytes, totalFreeBytes uint64
err = windows.GetDiskFreeSpaceEx(
windows.StringToUTF16Ptr(root),
&freeBytes,
&totalBytes,
&totalFreeBytes,
)
if err != nil {
return model.DiskUsage{}, err
}
return model.DiskUsage{
TotalSpace: int64(totalBytes),
UsedSpace: int64(totalBytes - freeBytes),
}, nil
}
func isCrossDeviceError(err error) bool {
return errors.Is(err, windows.ERROR_NOT_SAME_DEVICE)
}
-435
View File
@@ -1,435 +0,0 @@
package mediafire
/*
Package mediafire
Author: Da3zKi7<da3zki7@duck.com>
Date: 2025-09-11
D@' 3z K!7 - The King Of Cracking
Modifications by ILoveScratch2<ilovescratch@foxmail.com>
Date: 2025-09-21
Date: 2025-09-26
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
*/
import (
"context"
"fmt"
"math/rand"
"net/http"
"strconv"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/cron"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"golang.org/x/time/rate"
)
type Mediafire struct {
model.Storage
Addition
cron *cron.Cron
actionToken string
limiter *rate.Limiter
appBase string
apiBase string
hostBase string
maxRetries int
secChUa string
secChUaPlatform string
userAgent string
}
func (d *Mediafire) Config() driver.Config {
return config
}
func (d *Mediafire) GetAddition() driver.Additional {
return &d.Addition
}
// Init initializes the MediaFire driver with session token and cookie validation
func (d *Mediafire) Init(ctx context.Context) error {
if d.Cookie == "" {
return fmt.Errorf("Init :: [MediaFire] {critical} missing Cookie")
}
// If SessionToken is empty, try to get it from cookie
if d.SessionToken == "" {
if _, err := d.getSessionToken(ctx); err != nil {
return fmt.Errorf("Init :: [MediaFire] {critical} failed to get session token from cookie: %w", err)
}
}
// Setup rate limiter if rate limit is configured
if d.LimitRate > 0 {
d.limiter = rate.NewLimiter(rate.Limit(d.LimitRate), 1)
}
// Validate and refresh session token if needed
if _, err := d.getSessionToken(ctx); err != nil {
d.renewToken(ctx)
// Avoids 10 mins token expiry (6- 9)
num := rand.Intn(4) + 6
d.cron = cron.NewCron(time.Minute * time.Duration(num))
d.cron.Do(func() {
// Crazy, but working way to refresh session token
d.renewToken(ctx)
})
}
return nil
}
// Drop cleans up driver resources
func (d *Mediafire) Drop(ctx context.Context) error {
// Clear cached resources
d.actionToken = ""
if d.cron != nil {
d.cron.Stop()
d.cron = nil
}
return nil
}
// List retrieves files and folders from the specified directory
func (d *Mediafire) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
files, err := d.getFiles(ctx, dir.GetID())
if err != nil {
return nil, err
}
return utils.SliceConvert(files, func(src File) (model.Obj, error) {
return d.fileToObj(src), nil
})
}
// Link generates a direct download link for the specified file
func (d *Mediafire) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
downloadUrl, err := d.getDirectDownloadLink(ctx, file.GetID())
if err != nil {
return nil, err
}
res, err := base.NoRedirectClient.R().SetDoNotParseResponse(true).SetContext(ctx).Head(downloadUrl)
if err != nil {
return nil, err
}
defer func() {
_ = res.RawBody().Close()
}()
if res.StatusCode() == 302 {
downloadUrl = res.Header().Get("location")
}
return &model.Link{
URL: downloadUrl,
Header: http.Header{
"Origin": []string{d.appBase},
"Referer": []string{d.appBase + "/"},
"sec-ch-ua": []string{d.secChUa},
"sec-ch-ua-platform": []string{d.secChUaPlatform},
"User-Agent": []string{d.userAgent},
},
}, nil
}
// MakeDir creates a new folder in the specified parent directory
func (d *Mediafire) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
data := map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"parent_key": parentDir.GetID(),
"foldername": dirName,
}
var resp MediafireFolderCreateResponse
_, err := d.postForm(ctx, "/folder/create.php", data, &resp)
if err != nil {
return nil, err
}
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
created, _ := time.Parse("2006-01-02T15:04:05Z", resp.Response.CreatedUTC)
return &model.Object{
ID: resp.Response.FolderKey,
Name: resp.Response.Name,
Size: 0,
Modified: created,
Ctime: created,
IsFolder: true,
}, nil
}
// Move relocates a file or folder to a different parent directory
func (d *Mediafire) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
var data map[string]string
var endpoint string
if srcObj.IsDir() {
endpoint = "/folder/move.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"folder_key_src": srcObj.GetID(),
"folder_key_dst": dstDir.GetID(),
}
} else {
endpoint = "/file/move.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"quick_key": srcObj.GetID(),
"folder_key": dstDir.GetID(),
}
}
var resp MediafireMoveResponse
_, err := d.postForm(ctx, endpoint, data, &resp)
if err != nil {
return nil, err
}
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
return srcObj, nil
}
// Rename changes the name of a file or folder
func (d *Mediafire) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
var data map[string]string
var endpoint string
if srcObj.IsDir() {
endpoint = "/folder/update.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"folder_key": srcObj.GetID(),
"foldername": newName,
}
} else {
endpoint = "/file/update.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"quick_key": srcObj.GetID(),
"filename": newName,
}
}
var resp MediafireRenameResponse
_, err := d.postForm(ctx, endpoint, data, &resp)
if err != nil {
return nil, err
}
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
return &model.Object{
ID: srcObj.GetID(),
Name: newName,
Size: srcObj.GetSize(),
Modified: srcObj.ModTime(),
Ctime: srcObj.CreateTime(),
IsFolder: srcObj.IsDir(),
}, nil
}
// Copy creates a duplicate of a file or folder in the specified destination directory
func (d *Mediafire) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
var data map[string]string
var endpoint string
if srcObj.IsDir() {
endpoint = "/folder/copy.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"folder_key_src": srcObj.GetID(),
"folder_key_dst": dstDir.GetID(),
}
} else {
endpoint = "/file/copy.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"quick_key": srcObj.GetID(),
"folder_key": dstDir.GetID(),
}
}
var resp MediafireCopyResponse
_, err := d.postForm(ctx, endpoint, data, &resp)
if err != nil {
return nil, err
}
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
var newID string
if srcObj.IsDir() {
if len(resp.Response.NewFolderKeys) > 0 {
newID = resp.Response.NewFolderKeys[0]
}
} else {
if len(resp.Response.NewQuickKeys) > 0 {
newID = resp.Response.NewQuickKeys[0]
}
}
return &model.Object{
ID: newID,
Name: srcObj.GetName(),
Size: srcObj.GetSize(),
Modified: srcObj.ModTime(),
Ctime: srcObj.CreateTime(),
IsFolder: srcObj.IsDir(),
}, nil
}
// Remove deletes a file or folder permanently
func (d *Mediafire) Remove(ctx context.Context, obj model.Obj) error {
var data map[string]string
var endpoint string
if obj.IsDir() {
endpoint = "/folder/delete.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"folder_key": obj.GetID(),
}
} else {
endpoint = "/file/delete.php"
data = map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"quick_key": obj.GetID(),
}
}
var resp MediafireRemoveResponse
_, err := d.postForm(ctx, endpoint, data, &resp)
if err != nil {
return err
}
return checkAPIResult(resp.Response.Result)
}
// Put uploads a file to the specified directory with support for resumable upload and quick upload
func (d *Mediafire) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
fileHash := file.GetHash().GetHash(utils.SHA256)
var err error
// Try to use existing hash first, cache only if necessary
if len(fileHash) != utils.SHA256.Width {
_, fileHash, err = stream.CacheFullAndHash(file, &up, utils.SHA256)
if err != nil {
return nil, err
}
}
checkResp, err := d.uploadCheck(ctx, file.GetName(), file.GetSize(), fileHash, dstDir.GetID())
if err != nil {
return nil, err
}
if checkResp.Response.HashExists == "yes" && checkResp.Response.InAccount == "yes" {
up(100.0)
existingFile, err := d.getExistingFileInfo(ctx, fileHash, file.GetName(), dstDir.GetID())
if err == nil && existingFile != nil {
// File exists, return existing file info
return &model.Object{
ID: existingFile.GetID(),
Name: file.GetName(),
Size: file.GetSize(),
}, nil
}
// If getExistingFileInfo fails, log and continue with normal upload
// This ensures upload doesn't fail due to search issues
}
var pollKey string
if checkResp.Response.ResumableUpload.AllUnitsReady != "yes" {
pollKey, err = d.uploadUnits(ctx, file, checkResp, file.GetName(), fileHash, dstDir.GetID(), up)
if err != nil {
return nil, err
}
} else {
pollKey = checkResp.Response.ResumableUpload.UploadKey
}
defer up(100.0)
pollResp, err := d.pollUpload(ctx, pollKey)
if err != nil {
return nil, err
}
return &model.Object{
ID: pollResp.Response.Doupload.QuickKey,
Name: file.GetName(),
Size: file.GetSize(),
}, nil
}
func (d *Mediafire) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
data := map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
}
var resp MediafireUserInfoResponse
_, err := d.postForm(ctx, "/user/get_info.php", data, &resp)
if err != nil {
return nil, err
}
used, err := strconv.ParseInt(resp.Response.UserInfo.UsedStorageSize, 10, 64)
if err != nil {
return nil, err
}
total, err := strconv.ParseInt(resp.Response.UserInfo.StorageLimit, 10, 64)
if err != nil {
return nil, err
}
return &model.StorageDetails{
DiskUsage: model.DiskUsage{
TotalSpace: total,
UsedSpace: used,
},
}, nil
}
var _ driver.Driver = (*Mediafire)(nil)
-60
View File
@@ -1,60 +0,0 @@
package mediafire
/*
Package mediafire
Author: Da3zKi7<da3zki7@duck.com>
Date: 2025-09-11
D@' 3z K!7 - The King Of Cracking
Modifications by ILoveScratch2<ilovescratch@foxmail.com>
Date: 2025-09-21
Date: 2025-09-26
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
*/
import (
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
//driver.RootID
SessionToken string `json:"session_token" required:"false" type:"string" help:"Optional for MediaFire API, can be auto-acquired from cookie"`
Cookie string `json:"cookie" required:"true" type:"string" help:"Required for MediaFire API authentication"`
OrderBy string `json:"order_by" type:"select" options:"name,time,size" default:"name"`
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc" default:"asc"`
ChunkSize int64 `json:"chunk_size" type:"number" default:"100"`
UploadThreads int `json:"upload_threads" type:"number" default:"3" help:"concurrent upload threads"`
LimitRate float64 `json:"limit_rate" type:"float" default:"2" help:"limit all api request rate ([limit]r/1s)"`
}
var config = driver.Config{
Name: "MediaFire",
LocalSort: false,
OnlyProxy: false,
NoCache: false,
NoUpload: false,
NeedMs: false,
DefaultRoot: "/",
CheckStatus: false,
Alert: "",
NoOverwriteUpload: true,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Mediafire{
appBase: "https://app.mediafire.com",
apiBase: "https://www.mediafire.com/api/1.5",
hostBase: "https://www.mediafire.com",
maxRetries: 3,
userAgent: base.UserAgent,
}
})
}
-246
View File
@@ -1,246 +0,0 @@
package mediafire
/*
Package mediafire
Author: Da3zKi7<da3zki7@duck.com>
Date: 2025-09-11
D@' 3z K!7 - The King Of Cracking
*/
type MediafireRenewTokenResponse struct {
Response struct {
Action string `json:"action"`
SessionToken string `json:"session_token"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafireResponse struct {
Response struct {
Action string `json:"action"`
FolderContent struct {
ChunkSize string `json:"chunk_size"`
ContentType string `json:"content_type"`
ChunkNumber string `json:"chunk_number"`
FolderKey string `json:"folderkey"`
Folders []MediafireFolder `json:"folders,omitempty"`
Files []MediafireFile `json:"files,omitempty"`
MoreChunks string `json:"more_chunks"`
} `json:"folder_content"`
Result string `json:"result"`
} `json:"response"`
}
type MediafireFolder struct {
FolderKey string `json:"folderkey"`
Name string `json:"name"`
Created string `json:"created"`
CreatedUTC string `json:"created_utc"`
}
type MediafireFile struct {
QuickKey string `json:"quickkey"`
Filename string `json:"filename"`
Size string `json:"size"`
Created string `json:"created"`
CreatedUTC string `json:"created_utc"`
MimeType string `json:"mimetype"`
}
type File struct {
ID string
Name string
Size int64
CreatedUTC string
IsFolder bool
}
type FolderContentResponse struct {
Folders []MediafireFolder
Files []MediafireFile
MoreChunks bool
}
type MediafireLinksResponse struct {
Response struct {
Action string `json:"action"`
Links []struct {
QuickKey string `json:"quickkey"`
View string `json:"view"`
NormalDownload string `json:"normal_download"`
OneTime struct {
Download string `json:"download"`
View string `json:"view"`
} `json:"one_time"`
} `json:"links"`
OneTimeKeyRequestCount string `json:"one_time_key_request_count"`
OneTimeKeyRequestMaxCount string `json:"one_time_key_request_max_count"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafireDirectDownloadResponse struct {
Response struct {
Action string `json:"action"`
Links []struct {
QuickKey string `json:"quickkey"`
DirectDownload string `json:"direct_download"`
} `json:"links"`
DirectDownloadFreeBandwidth string `json:"direct_download_free_bandwidth"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafireFolderCreateResponse struct {
Response struct {
Action string `json:"action"`
FolderKey string `json:"folder_key"`
UploadKey string `json:"upload_key"`
ParentFolderKey string `json:"parent_folderkey"`
Name string `json:"name"`
Description string `json:"description"`
Created string `json:"created"`
CreatedUTC string `json:"created_utc"`
Privacy string `json:"privacy"`
FileCount string `json:"file_count"`
FolderCount string `json:"folder_count"`
Revision string `json:"revision"`
DropboxEnabled string `json:"dropbox_enabled"`
Flag string `json:"flag"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
NewDeviceRevision int `json:"new_device_revision"`
} `json:"response"`
}
type MediafireMoveResponse struct {
Response struct {
Action string `json:"action"`
Asynchronous string `json:"asynchronous,omitempty"`
NewNames []string `json:"new_names"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
NewDeviceRevision int `json:"new_device_revision"`
} `json:"response"`
}
type MediafireRenameResponse struct {
Response struct {
Action string `json:"action"`
Asynchronous string `json:"asynchronous,omitempty"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
NewDeviceRevision int `json:"new_device_revision"`
} `json:"response"`
}
type MediafireCopyResponse struct {
Response struct {
Action string `json:"action"`
Asynchronous string `json:"asynchronous,omitempty"`
NewQuickKeys []string `json:"new_quickkeys,omitempty"`
NewFolderKeys []string `json:"new_folderkeys,omitempty"`
SkippedCount string `json:"skipped_count,omitempty"`
OtherCount string `json:"other_count,omitempty"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
NewDeviceRevision int `json:"new_device_revision"`
} `json:"response"`
}
type MediafireRemoveResponse struct {
Response struct {
Action string `json:"action"`
Asynchronous string `json:"asynchronous,omitempty"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
NewDeviceRevision int `json:"new_device_revision"`
} `json:"response"`
}
type MediafireCheckResponse struct {
Response struct {
Action string `json:"action"`
HashExists string `json:"hash_exists"`
InAccount string `json:"in_account"`
InFolder string `json:"in_folder"`
FileExists string `json:"file_exists"`
ResumableUpload struct {
AllUnitsReady string `json:"all_units_ready"`
NumberOfUnits string `json:"number_of_units"`
UnitSize string `json:"unit_size"`
Bitmap struct {
Count string `json:"count"`
Words []string `json:"words"`
} `json:"bitmap"`
UploadKey string `json:"upload_key"`
} `json:"resumable_upload"`
AvailableSpace string `json:"available_space"`
UsedStorageSize string `json:"used_storage_size"`
StorageLimit string `json:"storage_limit"`
StorageLimitExceeded string `json:"storage_limit_exceeded"`
UploadURL struct {
Simple string `json:"simple"`
SimpleFallback string `json:"simple_fallback"`
Resumable string `json:"resumable"`
ResumableFallback string `json:"resumable_fallback"`
} `json:"upload_url"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafireActionTokenResponse struct {
Response struct {
Action string `json:"action"`
ActionToken string `json:"action_token"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafirePollResponse struct {
Response struct {
Action string `json:"action"`
Doupload struct {
Result string `json:"result"`
Status string `json:"status"`
Description string `json:"description"`
QuickKey string `json:"quickkey"`
Hash string `json:"hash"`
Filename string `json:"filename"`
Size string `json:"size"`
Created string `json:"created"`
CreatedUTC string `json:"created_utc"`
Revision string `json:"revision"`
} `json:"doupload"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafireFileSearchResponse struct {
Response struct {
Action string `json:"action"`
FileInfo []File `json:"file_info"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
type MediafireUserInfoResponse struct {
Response struct {
Action string `json:"action"`
UserInfo struct {
Email string `json:"string"`
DisplayName string `json:"display_name"`
UsedStorageSize string `json:"used_storage_size"`
StorageLimit string `json:"storage_limit"`
} `json:"user_info"`
Result string `json:"result"`
CurrentAPIVersion string `json:"current_api_version"`
} `json:"response"`
}
-734
View File
@@ -1,734 +0,0 @@
package mediafire
/*
Package mediafire
Author: Da3zKi7<da3zki7@duck.com>
Date: 2025-09-11
D@' 3z K!7 - The King Of Cracking
Modifications by ILoveScratch2<ilovescratch@foxmail.com>
Date: 2025-09-21
Date: 2025-09-26
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
*/
import (
"compress/gzip"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/op"
"github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/avast/retry-go"
"github.com/go-resty/resty/v2"
)
// checkAPIResult validates MediaFire API response result and returns error if not successful
func checkAPIResult(result string) error {
if result != "Success" {
return fmt.Errorf("MediaFire API error: %s", result)
}
return nil
}
// getSessionToken retrieves and validates session token from MediaFire
func (d *Mediafire) getSessionToken(ctx context.Context) (string, error) {
if d.limiter != nil {
if err := d.limiter.Wait(ctx); err != nil {
return "", fmt.Errorf("rate limit wait failed: %w", err)
}
}
tokenURL := d.hostBase + "/application/get_session_token.php"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, tokenURL, nil)
if err != nil {
return "", err
}
req.Header.Set("Accept", "*/*")
req.Header.Set("Accept-Encoding", "gzip")
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
req.Header.Set("Content-Length", "0")
req.Header.Set("Cookie", d.Cookie)
req.Header.Set("DNT", "1")
req.Header.Set("Origin", d.hostBase)
req.Header.Set("Priority", "u=1, i")
req.Header.Set("Referer", (d.hostBase + "/"))
req.Header.Set("Sec-Ch-Ua", d.secChUa)
req.Header.Set("Sec-Ch-Ua-Mobile", "?0")
req.Header.Set("Sec-Ch-Ua-Platform", d.secChUaPlatform)
req.Header.Set("Sec-Fetch-Dest", "empty")
req.Header.Set("Sec-Fetch-Mode", "cors")
req.Header.Set("Sec-Fetch-Site", "same-site")
req.Header.Set("User-Agent", d.userAgent)
// req.Header.Set("Connection", "keep-alive")
resp, err := base.HttpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
var body []byte
// Handle gzip decompression if needed
if resp.Header.Get("Content-Encoding") == "gzip" {
gzipReader, err := gzip.NewReader(resp.Body)
if err != nil {
return "", fmt.Errorf("failed to create gzip reader: %w", err)
}
defer gzipReader.Close()
body, _ = io.ReadAll(gzipReader)
} else {
body, err = io.ReadAll(resp.Body)
}
if err != nil {
return "", err
}
// fmt.Printf("getSessionToken :: Raw response: %s\n", string(body))
// fmt.Printf("getSessionToken :: Parsed response: %+v\n", resp)
var tokenResp struct {
Response struct {
SessionToken string `json:"session_token"`
} `json:"response"`
}
if resp.StatusCode == 200 {
if err := json.Unmarshal(body, &tokenResp); err != nil {
return "", err
}
if tokenResp.Response.SessionToken == "" {
return "", fmt.Errorf("empty session token received")
}
cookieMap := make(map[string]string)
for _, cookie := range resp.Cookies() {
cookieMap[cookie.Name] = cookie.Value
}
if len(cookieMap) > 0 {
var cookies []string
for name, value := range cookieMap {
cookies = append(cookies, fmt.Sprintf("%s=%s", name, value))
}
d.Cookie = strings.Join(cookies, "; ")
op.MustSaveDriverStorage(d)
// fmt.Printf("getSessionToken :: Captured cookies: %s\n", d.Cookie)
}
} else {
return "", fmt.Errorf("getSessionToken :: failed to get session token, status code: %d", resp.StatusCode)
}
d.SessionToken = tokenResp.Response.SessionToken
// fmt.Printf("Init :: Obtain Session Token %v", d.SessionToken)
op.MustSaveDriverStorage(d)
return d.SessionToken, nil
}
// renewToken refreshes the current session token when expired
func (d *Mediafire) renewToken(ctx context.Context) error {
query := map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
}
var resp MediafireRenewTokenResponse
_, err := d.postForm(ctx, "/user/renew_session_token.php", query, &resp)
if err != nil {
return fmt.Errorf("failed to renew token: %w", err)
}
// fmt.Printf("getInfo :: Raw response: %s\n", string(body))
// fmt.Printf("getInfo :: Parsed response: %+v\n", resp)
if resp.Response.Result != "Success" {
return fmt.Errorf("MediaFire token renewal failed: %s", resp.Response.Result)
}
d.SessionToken = resp.Response.SessionToken
// fmt.Printf("Init :: Renew Session Token: %s", resp.Response.Result)
op.MustSaveDriverStorage(d)
return nil
}
func (d *Mediafire) getFiles(ctx context.Context, folderKey string) ([]File, error) {
// Pre-allocate slice with reasonable capacity to reduce memory allocations
files := make([]File, 0, d.ChunkSize*2) // Estimate: ChunkSize for files + folders
hasMore := true
chunkNumber := 1
for hasMore {
resp, err := d.getFolderContent(ctx, folderKey, chunkNumber)
if err != nil {
return nil, err
}
// Process folders and files in single loop to improve cache locality
totalItems := len(resp.Folders) + len(resp.Files)
if cap(files)-len(files) < totalItems {
// Grow slice if needed
newFiles := make([]File, len(files), len(files)+totalItems+int(d.ChunkSize))
copy(newFiles, files)
files = newFiles
}
for _, folder := range resp.Folders {
files = append(files, File{
ID: folder.FolderKey,
Name: folder.Name,
Size: 0,
CreatedUTC: folder.CreatedUTC,
IsFolder: true,
})
}
for _, file := range resp.Files {
size, _ := strconv.ParseInt(file.Size, 10, 64)
files = append(files, File{
ID: file.QuickKey,
Name: file.Filename,
Size: size,
CreatedUTC: file.CreatedUTC,
IsFolder: false,
})
}
hasMore = resp.MoreChunks
chunkNumber++
}
return files, nil
}
func (d *Mediafire) getFolderContent(ctx context.Context, folderKey string, chunkNumber int) (*FolderContentResponse, error) {
foldersResp, err := d.getFolderContentByType(ctx, folderKey, "folders", chunkNumber)
if err != nil {
return nil, err
}
filesResp, err := d.getFolderContentByType(ctx, folderKey, "files", chunkNumber)
if err != nil {
return nil, err
}
return &FolderContentResponse{
Folders: foldersResp.Response.FolderContent.Folders,
Files: filesResp.Response.FolderContent.Files,
MoreChunks: foldersResp.Response.FolderContent.MoreChunks == "yes" || filesResp.Response.FolderContent.MoreChunks == "yes",
}, nil
}
func (d *Mediafire) getFolderContentByType(ctx context.Context, folderKey, contentType string, chunkNumber int) (*MediafireResponse, error) {
data := map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"folder_key": folderKey,
"content_type": contentType,
"chunk": strconv.Itoa(chunkNumber),
"chunk_size": strconv.FormatInt(d.ChunkSize, 10),
"details": "yes",
"order_direction": d.OrderDirection,
"order_by": d.OrderBy,
"filter": "",
}
var resp MediafireResponse
_, err := d.postForm(ctx, "/folder/get_content.php", data, &resp)
if err != nil {
return nil, err
}
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
return &resp, nil
}
// fileToObj converts MediaFire file data to model.ObjThumb with thumbnail support
func (d *Mediafire) fileToObj(f File) *model.ObjThumb {
created, _ := time.Parse("2006-01-02T15:04:05Z", f.CreatedUTC)
var thumbnailURL string
if !f.IsFolder && f.ID != "" {
thumbnailURL = d.hostBase + "/convkey/acaa/" + f.ID + "3g.jpg"
}
return &model.ObjThumb{
Object: model.Object{
ID: f.ID,
// Path: "",
Name: f.Name,
Size: f.Size,
Modified: created,
Ctime: created,
IsFolder: f.IsFolder,
},
Thumbnail: model.Thumbnail{
Thumbnail: thumbnailURL,
},
}
}
func (d *Mediafire) setCommonHeaders(req *resty.Request) {
req.SetHeaders(map[string]string{
"Cookie": d.Cookie,
"User-Agent": d.userAgent,
"Origin": d.appBase,
"Referer": d.appBase + "/",
})
}
// apiRequest performs HTTP request to MediaFire API with rate limiting and common headers
func (d *Mediafire) apiRequest(ctx context.Context, method, endpoint string, queryParams, formData map[string]string, resp interface{}) ([]byte, error) {
if d.limiter != nil {
if err := d.limiter.Wait(ctx); err != nil {
return nil, fmt.Errorf("rate limit wait failed: %w", err)
}
}
req := base.RestyClient.R()
req.SetContext(ctx)
d.setCommonHeaders(req)
// Set query parameters for GET requests
if queryParams != nil {
req.SetQueryParams(queryParams)
}
// Set form data for POST requests
if formData != nil {
req.SetFormData(formData)
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
}
// Set response object if provided
if resp != nil {
req.SetResult(resp)
}
var res *resty.Response
var err error
// Execute request based on method
switch method {
case "GET":
res, err = req.Get(d.apiBase + endpoint)
case "POST":
res, err = req.Post(d.apiBase + endpoint)
default:
return nil, fmt.Errorf("unsupported HTTP method: %s", method)
}
if err != nil {
return nil, err
}
return res.Body(), nil
}
func (d *Mediafire) getForm(ctx context.Context, endpoint string, query map[string]string, resp interface{}) ([]byte, error) {
return d.apiRequest(ctx, "GET", endpoint, query, nil, resp)
}
func (d *Mediafire) postForm(ctx context.Context, endpoint string, data map[string]string, resp interface{}) ([]byte, error) {
return d.apiRequest(ctx, "POST", endpoint, nil, data, resp)
}
func (d *Mediafire) getDirectDownloadLink(ctx context.Context, fileID string) (string, error) {
data := map[string]string{
"session_token": d.SessionToken,
"quick_key": fileID,
"link_type": "direct_download",
"response_format": "json",
}
var resp MediafireDirectDownloadResponse
_, err := d.getForm(ctx, "/file/get_links.php", data, &resp)
if err != nil {
return "", err
}
if err := checkAPIResult(resp.Response.Result); err != nil {
return "", err
}
if len(resp.Response.Links) == 0 {
return "", fmt.Errorf("no download links found")
}
return resp.Response.Links[0].DirectDownload, nil
}
func (d *Mediafire) uploadCheck(ctx context.Context, filename string, filesize int64, filehash, folderKey string) (*MediafireCheckResponse, error) {
actionToken, err := d.getActionToken(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get action token: %w", err)
}
query := map[string]string{
"session_token": actionToken, /* d.SessionToken */
"filename": filename,
"size": strconv.FormatInt(filesize, 10),
"hash": filehash,
"folder_key": folderKey,
"resumable": "yes",
"response_format": "json",
}
var resp MediafireCheckResponse
_, err = d.postForm(ctx, "/upload/check.php", query, &resp)
if err != nil {
return nil, err
}
// fmt.Printf("uploadCheck :: Raw response: %s\n", string(body))
// fmt.Printf("uploadCheck :: Parsed response: %+v\n", resp)
// fmt.Printf("uploadCheck :: ResumableUpload section: %+v\n", resp.Response.ResumableUpload)
// fmt.Printf("uploadCheck :: Upload key specifically: '%s'\n", resp.Response.ResumableUpload.UploadKey)
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
return &resp, nil
}
func (d *Mediafire) uploadUnits(ctx context.Context, file model.FileStreamer, checkResp *MediafireCheckResponse, filename, fileHash, folderKey string, up driver.UpdateProgress) (string, error) {
unitSize, _ := strconv.ParseInt(checkResp.Response.ResumableUpload.UnitSize, 10, 64)
numUnits, _ := strconv.Atoi(checkResp.Response.ResumableUpload.NumberOfUnits)
uploadKey := checkResp.Response.ResumableUpload.UploadKey
stringWords := checkResp.Response.ResumableUpload.Bitmap.Words
intWords := make([]int, 0, len(stringWords))
for _, word := range stringWords {
if intWord, err := strconv.Atoi(word); err == nil {
intWords = append(intWords, intWord)
}
}
// Intelligent buffer sizing for large files
bufferSize := int(unitSize)
fileSize := file.GetSize()
// Split in chunks
if fileSize > d.ChunkSize*1024*1024 {
// Large, use ChunkSize (default = 100MB)
bufferSize = min(int(fileSize), int(d.ChunkSize)*1024*1024)
} else if fileSize > 10*1024*1024 {
// Medium, use full file size for concurrent access
bufferSize = int(fileSize)
}
// Create stream section reader for efficient chunking
ss, err := stream.NewStreamSectionReader(file, bufferSize, &up)
if err != nil {
return "", err
}
// Cal minimal parallel upload threads, allows MediaFire resumable upload to rule it over custom value
// If file is big, likely will respect d.UploadThreads instead of MediaFire's suggestion i.e. 5 threads
thread := min(numUnits, d.UploadThreads)
// Create ordered group for sequential upload processing with retry logic
threadG, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, thread,
retry.Attempts(3),
retry.Delay(time.Second),
retry.DelayType(retry.BackOffDelay))
var finalUploadKey string
var keyMutex sync.Mutex
fileSize = file.GetSize()
for unitID := range numUnits {
if utils.IsCanceled(uploadCtx) {
break
}
start := int64(unitID) * unitSize
size := unitSize
if start+size > fileSize {
size = fileSize - start
}
var reader io.ReadSeeker
var unitHash string
// Use lifecycle pattern for proper resource management
threadG.GoWithLifecycle(errgroup.Lifecycle{
Before: func(ctx context.Context) (err error) {
// Skip already uploaded units
if d.isUnitUploaded(intWords, unitID) {
return ss.DiscardSection(start, size)
}
reader, err = ss.GetSectionReader(start, size)
return
},
Do: func(ctx context.Context) (err error) {
if reader == nil {
return nil // Skip if reader is not initialized (already uploaded)
}
reader.Seek(0, io.SeekStart)
if unitHash == "" {
var err error
unitHash, err = utils.HashReader(utils.SHA256, reader)
if err != nil {
return err
}
reader.Seek(0, io.SeekStart)
}
// Perform upload
actionToken, err := d.getActionToken(ctx)
if err != nil {
return err
}
if d.limiter != nil {
if err := d.limiter.Wait(ctx); err != nil {
return fmt.Errorf("rate limit wait failed: %w", err)
}
}
url := d.apiBase + "/upload/resumable.php"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, driver.NewLimitedUploadStream(ctx, reader))
if err != nil {
return err
}
q := req.URL.Query()
q.Add("folder_key", folderKey)
q.Add("response_format", "json")
q.Add("session_token", actionToken)
q.Add("key", uploadKey)
req.URL.RawQuery = q.Encode()
req.Header.Set("x-filehash", fileHash)
req.Header.Set("x-filesize", strconv.FormatInt(fileSize, 10))
req.Header.Set("x-unit-id", strconv.Itoa(unitID))
req.Header.Set("x-unit-size", strconv.FormatInt(size, 10))
req.Header.Set("x-unit-hash", unitHash)
req.Header.Set("x-filename", filename)
req.Header.Set("Content-Type", "application/octet-stream")
req.ContentLength = size
/* fmt.Printf("Debug resumable upload request:\n")
fmt.Printf(" URL: %s\n", req.URL.String())
fmt.Printf(" Headers: %+v\n", req.Header)
fmt.Printf(" Unit ID: %d\n", unitID)
fmt.Printf(" Unit Size: %d\n", len(unitData))
fmt.Printf(" Upload Key: %s\n", uploadKey)
fmt.Printf(" Action Token: %s\n", actionToken) */
res, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
body, err := io.ReadAll(res.Body)
if err != nil {
return fmt.Errorf("failed to read response body: %v", err)
}
// fmt.Printf("MediaFire resumable upload response (status %d): %s\n", res.StatusCode, string(body))
var uploadResp struct {
Response struct {
Doupload struct {
Key string `json:"key"`
} `json:"doupload"`
Result string `json:"result"`
} `json:"response"`
}
if err := json.Unmarshal(body, &uploadResp); err != nil {
return fmt.Errorf("failed to parse response: %v", err)
}
if res.StatusCode != 200 {
return fmt.Errorf("resumable upload failed with status %d", res.StatusCode)
}
// Thread-safe update of final upload key
keyMutex.Lock()
finalUploadKey = uploadResp.Response.Doupload.Key
keyMutex.Unlock()
up(float64(threadG.Success()+1) * 100 / float64(numUnits+1))
return nil
},
After: func(err error) {
if reader != nil {
// Cleanup resources
ss.FreeSectionReader(reader)
}
},
})
}
if err := threadG.Wait(); err != nil {
return "", err
}
return finalUploadKey, nil
}
/*func (d *Mediafire) uploadSingleUnit(ctx context.Context, file model.FileStreamer, unitID int, unitSize int64, fileHash, filename, uploadKey, folderKey string, fileSize int64) (string, error) {
start := int64(unitID) * unitSize
size := unitSize
if start+size > fileSize {
size = fileSize - start
}
unitData := make([]byte, size)
_, err := file.Read(unitData)
if err != nil {
return "", err
}
return d.resumableUpload(ctx, folderKey, uploadKey, unitData, unitID, fileHash, filename, fileSize)
}*/
func (d *Mediafire) getActionToken(ctx context.Context) (string, error) {
if d.actionToken != "" {
return d.actionToken, nil
}
data := map[string]string{
"type": "upload",
"lifespan": "1440",
"response_format": "json",
"session_token": d.SessionToken,
}
var resp MediafireActionTokenResponse
_, err := d.postForm(ctx, "/user/get_action_token.php", data, &resp)
if err != nil {
return "", err
}
if resp.Response.Result != "Success" {
return "", fmt.Errorf("MediaFire action token failed: %s", resp.Response.Result)
}
return resp.Response.ActionToken, nil
}
func (d *Mediafire) pollUpload(ctx context.Context, key string) (*MediafirePollResponse, error) {
actionToken, err := d.getActionToken(ctx)
if err != nil {
return nil, fmt.Errorf("failed to get action token: %w", err)
}
// fmt.Printf("Debug Key: %+v\n", key)
query := map[string]string{
"key": key,
"response_format": "json",
"session_token": actionToken, /* d.SessionToken */
}
var resp MediafirePollResponse
_, err = d.postForm(ctx, "/upload/poll_upload.php", query, &resp)
if err != nil {
return nil, err
}
// fmt.Printf("pollUpload :: Raw response: %s\n", string(body))
// fmt.Printf("pollUpload :: Parsed response: %+v\n", resp)
// fmt.Printf("pollUpload :: Debug Result: %+v\n", resp.Response.Result)
if err := checkAPIResult(resp.Response.Result); err != nil {
return nil, err
}
return &resp, nil
}
func (d *Mediafire) isUnitUploaded(words []int, unitID int) bool {
wordIndex := unitID / 16
bitIndex := unitID % 16
if wordIndex >= len(words) {
return false
}
return (words[wordIndex]>>bitIndex)&1 == 1
}
func (d *Mediafire) getExistingFileInfo(ctx context.Context, fileHash, filename, folderKey string) (*model.ObjThumb, error) {
// First try to find by hash directly (most efficient)
if fileInfo, err := d.getFileByHash(ctx, fileHash); err == nil && fileInfo != nil {
return fileInfo, nil
}
// If hash search fails, search in the target folder
// This is a fallback method in case the file exists but hash search doesn't work
files, err := d.getFiles(ctx, folderKey)
if err != nil {
return nil, err
}
for _, file := range files {
if file.Name == filename && !file.IsFolder {
return d.fileToObj(file), nil
}
}
return nil, fmt.Errorf("existing file not found")
}
func (d *Mediafire) getFileByHash(ctx context.Context, hash string) (*model.ObjThumb, error) {
query := map[string]string{
"session_token": d.SessionToken,
"response_format": "json",
"hash": hash,
}
var resp MediafireFileSearchResponse
_, err := d.postForm(ctx, "/file/get_info.php", query, &resp)
if err != nil {
return nil, err
}
if resp.Response.Result != "Success" {
return nil, fmt.Errorf("MediaFire file search failed: %s", resp.Response.Result)
}
if len(resp.Response.FileInfo) == 0 {
return nil, fmt.Errorf("file not found by hash")
}
file := resp.Response.FileInfo[0]
return d.fileToObj(file), nil
}
-35
View File
@@ -1,35 +0,0 @@
package onedrive
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
Region string `json:"region" type:"select" required:"true" options:"global,cn,us,de" default:"global"`
IsSharepoint bool `json:"is_sharepoint"`
UseOnlineAPI bool `json:"use_online_api" default:"true"`
APIAddress string `json:"api_url_address" default:"https://api.oplist.org/onedrive/renewapi"`
ClientID string `json:"client_id"`
ClientSecret string `json:"client_secret"`
RedirectUri string `json:"redirect_uri" required:"true" default:"https://api.oplist.org/onedrive/callback"`
RefreshToken string `json:"refresh_token" required:"true"`
SiteId string `json:"site_id"`
ChunkSize int64 `json:"chunk_size" type:"number" default:"5"`
CustomHost string `json:"custom_host" help:"Custom host for onedrive download link"`
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
EnableDirectUpload bool `json:"enable_direct_upload" default:"false" help:"Enable direct upload from client to OneDrive"`
}
var config = driver.Config{
Name: "Onedrive",
LocalSort: true,
DefaultRoot: "/",
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &Onedrive{}
})
}
-31
View File
@@ -1,31 +0,0 @@
package onedrive_app
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
Region string `json:"region" type:"select" required:"true" options:"global,cn,us,de" default:"global"`
ClientID string `json:"client_id" required:"true"`
ClientSecret string `json:"client_secret" required:"true"`
TenantID string `json:"tenant_id"`
Email string `json:"email"`
ChunkSize int64 `json:"chunk_size" type:"number" default:"5"`
CustomHost string `json:"custom_host" help:"Custom host for onedrive download link"`
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
EnableDirectUpload bool `json:"enable_direct_upload" default:"false" help:"Enable direct upload from client to OneDrive"`
}
var config = driver.Config{
Name: "OnedriveAPP",
LocalSort: true,
DefaultRoot: "/",
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &OnedriveAPP{}
})
}
-803
View File
@@ -1,803 +0,0 @@
package onedrive_sharelink
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
stdpath "path"
"regexp"
"strings"
"sync"
"time"
"github.com/OpenListTeam/OpenList/v4/drivers/base"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/internal/net"
streamPkg "github.com/OpenListTeam/OpenList/v4/internal/stream"
"github.com/OpenListTeam/OpenList/v4/pkg/cron"
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
"github.com/OpenListTeam/OpenList/v4/pkg/singleflight"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/avast/retry-go"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
const (
headerTTL = 25 * time.Minute
driveTokenTTL = 20 * time.Minute
directLinkTTL = 20 * time.Minute
simpleUploadLimit = 250 * 1024 * 1024
uploadSessionChunk = 10 * 1024 * 1024
)
type OnedriveSharelink struct {
model.Storage
cron *cron.Cron
Addition
headerMu sync.RWMutex
sg singleflight.Group[http.Header]
}
func (d *OnedriveSharelink) Config() driver.Config {
return config
}
func (d *OnedriveSharelink) GetAddition() driver.Additional {
return &d.Addition
}
func (d *OnedriveSharelink) Init(ctx context.Context) error {
// Initialize error variable
var err error
// If there is "-my" in the URL, it is NOT a SharePoint link
d.IsSharepoint = !strings.Contains(d.ShareLinkURL, "-my")
// Initialize cron job to run every hour
d.cron = cron.NewCron(time.Hour * 1)
d.cron.Do(func() {
var err error
h, err := d.getHeaders(ctx)
if err != nil {
log.Errorf("%+v", err)
return
}
d.storeHeaders(h)
})
// Get initial headers
h, err := d.getHeaders(ctx)
if err != nil {
return err
}
d.storeHeaders(h)
return nil
}
func (d *OnedriveSharelink) Drop(ctx context.Context) error {
return nil
}
func (d *OnedriveSharelink) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
files, err := d.getFiles(ctx, dir.GetPath())
if err != nil {
return nil, err
}
folderSizes, err := d.driveChildrenFolderSizes(ctx, dir.GetPath())
if err != nil {
log.Warnf("onedrive_sharelink: failed to get folder sizes for %s: %+v", dir.GetPath(), err)
}
// Convert the slice of files to the required model.Obj format
return utils.SliceConvert(files, func(src Item) (model.Obj, error) {
obj := fileToObj(src)
if size, ok := folderSizes[obj.GetName()]; ok {
obj.Size = size
}
obj.Path = stdpath.Join(dir.GetPath(), obj.GetName())
return obj, nil
})
}
func (d *OnedriveSharelink) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
// Get the unique ID of the file
uniqueId := file.GetID()
// Cut the first char and the last char
uniqueId = uniqueId[1 : len(uniqueId)-1]
url := d.downloadLinkPrefix + uniqueId
header, err := d.getValidHeaders(ctx)
if err != nil {
return nil, err
}
if args.Redirect {
directURL, err := d.resolveDirectDownloadURL(ctx, file, url, header)
if err != nil {
return nil, err
}
expiration := directLinkTTL
return &model.Link{
URL: directURL,
Expiration: &expiration,
}, nil
}
return &model.Link{
URL: url,
Header: header,
RangeReader: rangeReaderFunc(func(ctx context.Context, hr http_range.Range) (io.ReadCloser, error) {
return d.rangeReadWithRefresh(ctx, url, hr)
}),
}, nil
}
func (d *OnedriveSharelink) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
token, err := d.getValidDriveAccessToken(ctx)
if err != nil {
return err
}
apiURL := injectAccessToken(d.drivePathAPIURL(parentDir.GetPath())+"/children", token)
body := map[string]any{
"name": dirName,
"folder": map[string]any{},
"@microsoft.graph.conflictBehavior": "fail",
}
resp, err := d.doJSON(ctx, http.MethodPost, apiURL, body)
if err != nil {
return err
}
defer resp.Body.Close()
switch resp.StatusCode {
case http.StatusOK, http.StatusCreated:
return nil
case http.StatusConflict:
return errs.ObjectAlreadyExists
default:
data, _ := io.ReadAll(resp.Body)
return fmt.Errorf("failed to create folder, status code: %d, body: %s", resp.StatusCode, string(data))
}
}
func (d *OnedriveSharelink) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
// TODO move obj, optional
return errs.NotImplement
}
func (d *OnedriveSharelink) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
// TODO rename obj, optional
return errs.NotImplement
}
func (d *OnedriveSharelink) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
// TODO copy obj, optional
return errs.NotImplement
}
func (d *OnedriveSharelink) Remove(ctx context.Context, obj model.Obj) error {
// TODO remove obj, optional
return errs.NotImplement
}
func (d *OnedriveSharelink) Put(ctx context.Context, dstDir model.Obj, stream model.FileStreamer, up driver.UpdateProgress) error {
info, err := d.createUploadInfo(ctx, stdpath.Join(dstDir.GetPath(), stream.GetName()), stream.GetSize())
if err != nil {
return err
}
if info.ChunkSize == 0 {
return d.uploadContent(ctx, info.UploadURL, stream, up)
}
return d.uploadToSession(ctx, info.UploadURL, stream, up)
}
func (d *OnedriveSharelink) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
if d.DisableDiskUsage {
return nil, errs.NotImplement
}
size, err := d.driveItemSize(ctx, "/")
if err != nil {
return nil, err
}
return &model.StorageDetails{
DiskUsage: model.DiskUsage{
TotalSpace: size,
UsedSpace: size,
},
}, nil
}
func (d *OnedriveSharelink) driveItemSize(ctx context.Context, path string) (int64, error) {
token, err := d.getValidDriveAccessToken(ctx)
if err != nil {
return 0, err
}
apiURL := injectAccessToken(d.drivePathAPIURL(path), token)
resp, err := d.doJSON(ctx, http.MethodGet, apiURL, nil)
if err != nil {
return 0, err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
data, _ := io.ReadAll(resp.Body)
return 0, fmt.Errorf("failed to get folder details, status code: %d, body: %s", resp.StatusCode, string(data))
}
var item struct {
Size int64 `json:"size"`
}
if err := json.NewDecoder(resp.Body).Decode(&item); err != nil {
return 0, err
}
return item.Size, nil
}
func (d *OnedriveSharelink) driveChildrenFolderSizes(ctx context.Context, path string) (map[string]int64, error) {
token, err := d.getValidDriveAccessToken(ctx)
if err != nil {
return nil, err
}
rawURL := d.drivePathAPIURL(path) + "/children?$select=name,size,folder"
sizes := make(map[string]int64)
for rawURL != "" {
resp, err := d.doJSON(ctx, http.MethodGet, injectAccessToken(rawURL, token), nil)
if err != nil {
return nil, err
}
var data struct {
Value []struct {
Name string `json:"name"`
Size int64 `json:"size"`
Folder any `json:"folder"`
} `json:"value"`
NextLink string `json:"@odata.nextLink"`
}
func() {
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(resp.Body)
err = fmt.Errorf("failed to get children details, status code: %d, body: %s", resp.StatusCode, string(body))
return
}
err = json.NewDecoder(resp.Body).Decode(&data)
}()
if err != nil {
return nil, err
}
for _, item := range data.Value {
if item.Folder != nil {
sizes[item.Name] = item.Size
}
}
rawURL = data.NextLink
}
return sizes, nil
}
func (d *OnedriveSharelink) GetDirectUploadTools() []string {
if !d.EnableDirectUpload {
return nil
}
return []string{"HttpDirect"}
}
func (d *OnedriveSharelink) GetDirectUploadInfo(ctx context.Context, tool string, dstDir model.Obj, fileName string, fileSize int64) (any, error) {
if !d.EnableDirectUpload {
return nil, errs.NotImplement
}
if tool != "HttpDirect" {
return nil, errs.NotImplement
}
return d.createUploadInfo(ctx, stdpath.Join(dstDir.GetPath(), fileName), fileSize)
}
func (d *OnedriveSharelink) createUploadInfo(ctx context.Context, path string, fileSize int64) (*model.HttpDirectUploadInfo, error) {
token, err := d.getValidDriveAccessToken(ctx)
if err != nil {
return nil, err
}
if fileSize >= 0 && fileSize <= simpleUploadLimit {
return &model.HttpDirectUploadInfo{
UploadURL: injectAccessToken(d.drivePathAPIURL(path)+"/content", token),
Method: http.MethodPut,
}, nil
}
apiURL := injectAccessToken(d.drivePathAPIURL(path)+"/createUploadSession", token)
body := map[string]any{
"item": map[string]any{
"@microsoft.graph.conflictBehavior": "rename",
},
}
resp, err := d.doJSON(ctx, http.MethodPost, apiURL, body)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
data, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("failed to create upload session, status code: %d, body: %s", resp.StatusCode, string(data))
}
var data uploadSessionResp
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return nil, err
}
if data.UploadURL == "" {
return nil, fmt.Errorf("failed to get upload URL from response")
}
return &model.HttpDirectUploadInfo{
UploadURL: data.UploadURL,
ChunkSize: uploadSessionChunk,
Method: http.MethodPut,
}, nil
}
func (d *OnedriveSharelink) uploadContent(ctx context.Context, uploadURL string, file model.FileStreamer, up driver.UpdateProgress) error {
if up == nil {
up = func(float64) {}
}
reader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
Reader: &driver.SimpleReaderWithSize{
Reader: file,
Size: file.GetSize(),
},
UpdateProgress: up,
})
req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadURL, reader)
if err != nil {
return err
}
req.ContentLength = file.GetSize()
resp, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusCreated {
data, _ := io.ReadAll(resp.Body)
return fmt.Errorf("failed to upload content, status code: %d, body: %s", resp.StatusCode, string(data))
}
return nil
}
func (d *OnedriveSharelink) uploadToSession(ctx context.Context, uploadURL string, file model.FileStreamer, up driver.UpdateProgress) error {
if up == nil {
up = func(float64) {}
}
if file.GetSize() <= 0 {
return d.uploadSessionChunk(ctx, uploadURL, file, 0, 0, file.GetSize())
}
ss, err := streamPkg.NewStreamSectionReader(file, uploadSessionChunk, &up)
if err != nil {
return err
}
var finish int64
for finish < file.GetSize() {
if utils.IsCanceled(ctx) {
return ctx.Err()
}
left := file.GetSize() - finish
byteSize := min(left, int64(uploadSessionChunk))
rd, err := ss.GetSectionReader(finish, byteSize)
if err != nil {
return err
}
err = retry.Do(
func() error {
if _, err := rd.Seek(0, io.SeekStart); err != nil {
return err
}
return d.uploadSessionChunk(ctx, uploadURL, rd, finish, byteSize, file.GetSize())
},
retry.Context(ctx),
retry.Attempts(3),
retry.DelayType(retry.BackOffDelay),
retry.Delay(time.Second),
)
ss.FreeSectionReader(rd)
if err != nil {
return err
}
finish += byteSize
up(float64(finish) * 100 / float64(file.GetSize()))
}
return nil
}
func (d *OnedriveSharelink) uploadSessionChunk(ctx context.Context, uploadURL string, reader io.Reader, start, size, total int64) error {
req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadURL, driver.NewLimitedUploadStream(ctx, reader))
if err != nil {
return err
}
req.ContentLength = size
if total > 0 {
req.Header.Set("Content-Range", fmt.Sprintf("bytes %d-%d/%d", start, start+size-1, total))
}
resp, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
switch {
case resp.StatusCode >= 500 && resp.StatusCode <= 504:
return fmt.Errorf("server error: %d", resp.StatusCode)
case resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusCreated && resp.StatusCode != http.StatusAccepted:
data, _ := io.ReadAll(resp.Body)
return errors.New(string(data))
default:
return nil
}
}
func (d *OnedriveSharelink) drivePathAPIURL(path string) string {
drivePath := stdpath.Join(d.driveRootPath, path)
drivePath = utils.FixAndCleanPath(drivePath)
if drivePath == "/" {
return d.DriveURL + "/root"
}
return fmt.Sprintf("%s/root:%s:", d.DriveURL, utils.EncodePath(drivePath, true))
}
func injectAccessToken(rawURL, token string) string {
if token == "" {
return rawURL
}
u, err := url.Parse(rawURL)
if err != nil {
return rawURL
}
values := u.Query()
if strings.HasPrefix(token, "access_token=") {
values.Set("access_token", strings.TrimPrefix(token, "access_token="))
} else {
values.Set("access_token", token)
}
u.RawQuery = values.Encode()
return u.String()
}
func (d *OnedriveSharelink) doJSON(ctx context.Context, method, rawURL string, body any) (*http.Response, error) {
var reader io.Reader
if body != nil {
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
reader = bytes.NewReader(data)
}
req, err := http.NewRequestWithContext(ctx, method, rawURL, reader)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json;odata.metadata=minimal")
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
return base.HttpClient.Do(req)
}
func (d *OnedriveSharelink) getValidDriveAccessToken(ctx context.Context) (string, error) {
d.headerMu.RLock()
token := d.DriveAccessToken
expired := time.Since(time.Unix(d.DriveTokenTime, 0)) > driveTokenTTL
d.headerMu.RUnlock()
if token != "" && !expired {
return token, nil
}
if err := d.refreshDriveContext(ctx); err != nil {
d.headerMu.RLock()
token = d.DriveAccessToken
d.headerMu.RUnlock()
if token != "" {
log.Warnf("onedrive_sharelink: use cached drive access token after refresh failure: %+v", err)
return token, nil
}
return "", err
}
d.headerMu.RLock()
defer d.headerMu.RUnlock()
return d.DriveAccessToken, nil
}
func (d *OnedriveSharelink) refreshDriveContext(ctx context.Context) error {
header, err := d.getValidHeaders(ctx)
if err != nil {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, d.ShareLinkURL, nil)
if err != nil {
return err
}
req.Header = cloneHeader(header)
resp, err := NewNoRedirectCLient().Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
redirectURL := resp.Header.Get("Location")
if redirectURL == "" && resp.Request != nil && resp.Request.URL != nil {
redirectURL = resp.Request.URL.String()
}
if redirectURL == "" {
return fmt.Errorf("share link did not return redirect URL")
}
return d.refreshDriveContextFromRedirect(ctx, redirectURL, header)
}
func (d *OnedriveSharelink) refreshDriveContextFromRedirect(ctx context.Context, redirectURL string, header http.Header) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, redirectURL, nil)
if err != nil {
return err
}
req.Header = cloneHeader(header)
resp, err := base.HttpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("onedrive page request failed, status code: %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return err
}
ctxInfo, err := parsePageContext(body)
if err != nil {
return err
}
rootPath, err := driveRootPathFromRedirect(redirectURL, ctxInfo.ListURL)
if err != nil {
return err
}
d.headerMu.Lock()
d.DriveURL = ctxInfo.DriveInfo.DriveURL
d.DriveAccessToken = ctxInfo.DriveInfo.DriveAccessToken
d.DriveTokenTime = time.Now().Unix()
d.driveRootPath = rootPath
d.headerMu.Unlock()
return nil
}
func driveRootPathFromRedirect(redirectURL, listURL string) (string, error) {
u, err := url.Parse(redirectURL)
if err != nil {
return "", err
}
id := u.Query().Get("id")
if id == "" {
return "/", nil
}
if listURL == "" {
return "/", nil
}
if id == listURL {
return "/", nil
}
prefix := strings.TrimRight(listURL, "/") + "/"
if strings.HasPrefix(id, prefix) {
return utils.FixAndCleanPath(strings.TrimPrefix(id, strings.TrimRight(listURL, "/"))), nil
}
return "/", nil
}
var pageContextRE = regexp.MustCompile(`(?s)var _spPageContextInfo=(\{.*?\});_spPageContextInfo`)
func parsePageContext(body []byte) (*pageContextInfo, error) {
match := pageContextRE.FindSubmatch(body)
if len(match) < 2 {
return nil, fmt.Errorf("failed to find _spPageContextInfo")
}
var info pageContextInfo
if err := json.Unmarshal(match[1], &info); err != nil {
return nil, err
}
if info.DriveInfo.DriveURL == "" {
return nil, fmt.Errorf("failed to get drive URL from page context")
}
if info.DriveInfo.DriveAccessToken == "" {
return nil, fmt.Errorf("failed to get drive access token from page context")
}
return &info, nil
}
//func (d *OnedriveSharelink) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
// return nil, errs.NotSupport
//}
var _ driver.Driver = (*OnedriveSharelink)(nil)
// rangeReadWithRefresh tries once with current headers, and if the response
// looks invalid (error status or html login page), it refreshes headers and retries.
func (d *OnedriveSharelink) rangeReadWithRefresh(ctx context.Context, url string, hr http_range.Range) (io.ReadCloser, error) {
tryOnce := func(header http.Header) (io.ReadCloser, error) {
h := cloneHeader(header)
if h == nil {
h = http.Header{}
}
h = http_range.ApplyRangeToHttpHeader(hr, h)
resp, err := net.RequestHttp(ctx, http.MethodGet, h, url)
if err != nil {
return nil, err
}
ct := strings.ToLower(resp.Header.Get("Content-Type"))
if strings.Contains(ct, "text/html") {
_ = resp.Body.Close()
return nil, fmt.Errorf("unexpected html response")
}
return resp.Body, nil
}
header, err := d.getValidHeaders(ctx)
if err != nil {
return nil, err
}
if body, err := tryOnce(header); err == nil {
return body, nil
}
// refresh and retry once
header, err = d.refreshHeaders(ctx)
if err != nil {
return nil, err
}
return tryOnce(header)
}
type rangeReaderFunc func(ctx context.Context, hr http_range.Range) (io.ReadCloser, error)
func (f rangeReaderFunc) RangeRead(ctx context.Context, hr http_range.Range) (io.ReadCloser, error) {
return f(ctx, hr)
}
func cloneHeader(header http.Header) http.Header {
if header == nil {
return nil
}
return header.Clone()
}
func (d *OnedriveSharelink) resolveDirectDownloadURL(ctx context.Context, file model.Obj, rawURL string, header http.Header) (string, error) {
var errs []error
if obj, ok := unwrapObject(file); ok {
if obj.SPItemURL != "" {
directURL, err := d.resolveSPItemDownloadURL(ctx, obj.SPItemURL, header)
if err == nil {
return directURL, nil
}
errs = append(errs, err)
}
if obj.ContentDownloadURL != "" {
return obj.ContentDownloadURL, nil
}
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
if err != nil {
return "", err
}
req.Header = cloneHeader(header)
if req.Header == nil {
req.Header = http.Header{}
}
resp, err := NewNoRedirectCLient().Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
location := resp.Header.Get("Location")
if location == "" {
errs = append(errs, fmt.Errorf("download.aspx returned no redirect location, status code: %d", resp.StatusCode))
return "", fmt.Errorf("onedrive_sharelink: direct download URL unavailable: %v", errs)
}
u, err := req.URL.Parse(location)
if err != nil {
return "", err
}
return u.String(), nil
}
type spItemDownloadResp struct {
ContentDownloadURL string `json:"@content.downloadUrl"`
}
func unwrapObject(obj model.Obj) (*Object, bool) {
for {
switch o := obj.(type) {
case *Object:
return o, true
case model.ObjUnwrap:
obj = o.Unwrap()
default:
return nil, false
}
}
}
func (d *OnedriveSharelink) resolveSPItemDownloadURL(ctx context.Context, spItemURL string, header http.Header) (string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, spItemURL, nil)
if err != nil {
return "", err
}
req.Header = cloneHeader(header)
if req.Header == nil {
req.Header = http.Header{}
}
req.Header.Set("Accept", "application/json;odata.metadata=minimal")
resp, err := NewNoRedirectCLient().Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return "", fmt.Errorf("sp item metadata request failed, status code: %d", resp.StatusCode)
}
var data spItemDownloadResp
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
return "", err
}
if data.ContentDownloadURL == "" {
return "", fmt.Errorf("sp item metadata response missing @content.downloadUrl")
}
return data.ContentDownloadURL, nil
}
func (d *OnedriveSharelink) headerSnapshot() http.Header {
d.headerMu.RLock()
defer d.headerMu.RUnlock()
return cloneHeader(d.Headers)
}
func (d *OnedriveSharelink) storeHeaders(header http.Header) {
if header == nil {
return
}
d.headerMu.Lock()
d.Headers = header
d.HeaderTime = time.Now().Unix()
d.headerMu.Unlock()
}
func (d *OnedriveSharelink) headersExpired() bool {
d.headerMu.RLock()
defer d.headerMu.RUnlock()
return time.Since(time.Unix(d.HeaderTime, 0)) > headerTTL
}
func (d *OnedriveSharelink) refreshHeaders(ctx context.Context) (http.Header, error) {
header, err, _ := d.sg.Do("refresh", func() (http.Header, error) {
h, e := d.getHeaders(ctx)
if e != nil {
return nil, e
}
d.storeHeaders(h)
return h, nil
})
return header, err
}
func (d *OnedriveSharelink) getValidHeaders(ctx context.Context) (http.Header, error) {
if h := d.headerSnapshot(); h != nil && !d.headersExpired() {
return h, nil
}
h, err := d.refreshHeaders(ctx)
if err != nil {
if h2 := d.headerSnapshot(); h2 != nil {
log.Warnf("onedrive_sharelink: use cached headers after refresh failure: %+v", err)
return h2, nil
}
return nil, err
}
return h, nil
}
-33
View File
@@ -1,33 +0,0 @@
package openlist
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
Address string `json:"url" required:"true"`
MetaPassword string `json:"meta_password"`
Username string `json:"username"`
Password string `json:"password"`
Token string `json:"token"`
PassIPToUpsteam bool `json:"pass_ip_to_upsteam" default:"true"`
PassUAToUpsteam bool `json:"pass_ua_to_upsteam" default:"true"`
ForwardArchiveReq bool `json:"forward_archive_requests" default:"true"`
PassRefreshFlagToUpsteam bool `json:"pass_refresh_flag_to_upsteam" default:"false"`
}
var config = driver.Config{
Name: "OpenList",
LocalSort: true,
DefaultRoot: "/",
ProxyRangeOption: true,
LinkCacheMode: driver.LinkCacheAuto,
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &OpenList{}
})
}
-182
View File
@@ -1,182 +0,0 @@
package openlist_share
import (
"context"
"fmt"
"net/http"
"net/url"
stdpath "path"
"strings"
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/errs"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
"github.com/OpenListTeam/OpenList/v4/server/common"
"github.com/go-resty/resty/v2"
)
type OpenListShare struct {
model.Storage
Addition
serverArchivePreview bool
}
func (d *OpenListShare) Config() driver.Config {
return config
}
func (d *OpenListShare) GetAddition() driver.Additional {
return &d.Addition
}
func (d *OpenListShare) Init(ctx context.Context) error {
d.Addition.Address = strings.TrimSuffix(d.Addition.Address, "/")
var settings common.Resp[map[string]string]
_, _, err := d.request("/public/settings", http.MethodGet, func(req *resty.Request) {
req.SetResult(&settings)
})
if err != nil {
return err
}
d.serverArchivePreview = settings.Data["share_archive_preview"] == "true"
return nil
}
func (d *OpenListShare) Drop(ctx context.Context) error {
return nil
}
func (d *OpenListShare) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
var resp common.Resp[FsListResp]
_, _, err := d.request("/fs/list", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ListReq{
PageReq: model.PageReq{
Page: 1,
PerPage: 0,
},
Path: stdpath.Join(fmt.Sprintf("/@s/%s", d.ShareId), dir.GetPath()),
Password: d.Pwd,
Refresh: false,
})
})
if err != nil {
return nil, err
}
var files []model.Obj
for _, f := range resp.Data.Content {
file := model.ObjThumb{
Object: model.Object{
Name: f.Name,
Path: stdpath.Join(dir.GetPath(), f.Name),
Modified: f.Modified,
Ctime: f.Created,
Size: f.Size,
IsFolder: f.IsDir,
HashInfo: utils.FromString(f.HashInfo),
},
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
}
files = append(files, &file)
}
return files, nil
}
func (d *OpenListShare) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
path := utils.FixAndCleanPath(stdpath.Join(d.ShareId, file.GetPath()))
u := fmt.Sprintf("%s/sd%s?pwd=%s", d.Address, path, d.Pwd)
return &model.Link{URL: u}, nil
}
func (d *OpenListShare) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
if !d.serverArchivePreview || !d.ForwardArchiveReq {
return nil, errs.NotImplement
}
var resp common.Resp[ArchiveMetaResp]
_, code, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ArchiveMetaReq{
ArchivePass: args.Password,
Path: stdpath.Join(fmt.Sprintf("/@s/%s", d.ShareId), obj.GetPath()),
Password: d.Pwd,
Refresh: false,
})
})
if code == 202 {
return nil, errs.WrongArchivePassword
}
if err != nil {
return nil, err
}
var tree []model.ObjTree
if resp.Data.Content != nil {
tree = make([]model.ObjTree, 0, len(resp.Data.Content))
for _, content := range resp.Data.Content {
tree = append(tree, &content)
}
}
return &model.ArchiveMetaInfo{
Comment: resp.Data.Comment,
Encrypted: resp.Data.Encrypted,
Tree: tree,
}, nil
}
func (d *OpenListShare) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
if !d.serverArchivePreview || !d.ForwardArchiveReq {
return nil, errs.NotImplement
}
var resp common.Resp[ArchiveListResp]
_, code, err := d.request("/fs/archive/list", http.MethodPost, func(req *resty.Request) {
req.SetResult(&resp).SetBody(ArchiveListReq{
ArchiveMetaReq: ArchiveMetaReq{
ArchivePass: args.Password,
Path: stdpath.Join(fmt.Sprintf("/@s/%s", d.ShareId), obj.GetPath()),
Password: d.Pwd,
Refresh: false,
},
PageReq: model.PageReq{
Page: 1,
PerPage: 0,
},
InnerPath: args.InnerPath,
})
})
if code == 202 {
return nil, errs.WrongArchivePassword
}
if err != nil {
return nil, err
}
var files []model.Obj
for _, f := range resp.Data.Content {
file := model.ObjThumb{
Object: model.Object{
Name: f.Name,
Modified: f.Modified,
Ctime: f.Created,
Size: f.Size,
IsFolder: f.IsDir,
HashInfo: utils.FromString(f.HashInfo),
},
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
}
files = append(files, &file)
}
return files, nil
}
func (d *OpenListShare) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
if !d.serverArchivePreview || !d.ForwardArchiveReq {
return nil, errs.NotSupport
}
path := utils.FixAndCleanPath(stdpath.Join(d.ShareId, obj.GetPath()))
u := fmt.Sprintf("%s/sad%s?pwd=%s&inner=%s&pass=%s",
d.Address,
path,
d.Pwd,
utils.EncodePath(args.InnerPath, true),
url.QueryEscape(args.Password))
return &model.Link{URL: u}, nil
}
var _ driver.Driver = (*OpenListShare)(nil)
-27
View File
@@ -1,27 +0,0 @@
package openlist_share
import (
"github.com/OpenListTeam/OpenList/v4/internal/driver"
"github.com/OpenListTeam/OpenList/v4/internal/op"
)
type Addition struct {
driver.RootPath
Address string `json:"url" required:"true"`
ShareId string `json:"sid" required:"true"`
Pwd string `json:"pwd"`
ForwardArchiveReq bool `json:"forward_archive_requests" default:"true"`
}
var config = driver.Config{
Name: "OpenListShare",
LocalSort: true,
NoUpload: true,
DefaultRoot: "/",
}
func init() {
op.RegisterDriver(func() driver.Driver {
return &OpenListShare{}
})
}
-111
View File
@@ -1,111 +0,0 @@
package openlist_share
import (
"time"
"github.com/OpenListTeam/OpenList/v4/internal/model"
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
)
type ListReq struct {
model.PageReq
Path string `json:"path" form:"path"`
Password string `json:"password" form:"password"`
Refresh bool `json:"refresh"`
}
type ObjResp struct {
Name string `json:"name"`
Size int64 `json:"size"`
IsDir bool `json:"is_dir"`
Modified time.Time `json:"modified"`
Created time.Time `json:"created"`
Sign string `json:"sign"`
Thumb string `json:"thumb"`
Type int `json:"type"`
HashInfo string `json:"hashinfo"`
}
type FsListResp struct {
Content []ObjResp `json:"content"`
Total int64 `json:"total"`
Readme string `json:"readme"`
Write bool `json:"write"`
Provider string `json:"provider"`
}
type ArchiveMetaReq struct {
ArchivePass string `json:"archive_pass"`
Password string `json:"password"`
Path string `json:"path"`
Refresh bool `json:"refresh"`
}
type TreeResp struct {
ObjResp
Children []TreeResp `json:"children"`
hashCache *utils.HashInfo
}
func (t *TreeResp) GetSize() int64 {
return t.Size
}
func (t *TreeResp) GetName() string {
return t.Name
}
func (t *TreeResp) ModTime() time.Time {
return t.Modified
}
func (t *TreeResp) CreateTime() time.Time {
return t.Created
}
func (t *TreeResp) IsDir() bool {
return t.ObjResp.IsDir
}
func (t *TreeResp) GetHash() utils.HashInfo {
return utils.FromString(t.HashInfo)
}
func (t *TreeResp) GetID() string {
return ""
}
func (t *TreeResp) GetPath() string {
return ""
}
func (t *TreeResp) GetChildren() []model.ObjTree {
ret := make([]model.ObjTree, 0, len(t.Children))
for _, child := range t.Children {
ret = append(ret, &child)
}
return ret
}
func (t *TreeResp) Thumb() string {
return t.ObjResp.Thumb
}
type ArchiveMetaResp struct {
Comment string `json:"comment"`
Encrypted bool `json:"encrypted"`
Content []TreeResp `json:"content"`
RawURL string `json:"raw_url"`
Sign string `json:"sign"`
}
type ArchiveListReq struct {
model.PageReq
ArchiveMetaReq
InnerPath string `json:"inner_path"`
}
type ArchiveListResp struct {
Content []ObjResp `json:"content"`
Total int64 `json:"total"`
}

Some files were not shown because too many files have changed in this diff Show More