mirror of
https://github.com/OpenListTeam/OpenList.git
synced 2026-10-10 21:13:10 +08:00
Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 pikachuim
|
55d3827dee | ||
|
pikachuim
|
1fbc9427df | ||
|
pikachuim
|
bb3d139a47 | ||
|
pikachuim
|
d227ab85d6 | ||
|
pikachuim
|
5342ae96d0 | ||
|
pikachuim
|
273e15a050 | ||
|
pikachuim
|
13aad2c2fa | ||
|
j2rong4cn
|
368dc65a6e | ||
|
j2rong4cn
|
8b4b6ba970 | ||
|
j2rong4cn
|
4d28e838ce | ||
|
j2rong4cn
|
3930d4789a |
@@ -1,114 +0,0 @@
|
||||
<!--
|
||||
PR title / PR 标题:
|
||||
- Use Conventional Commits: `type(scope): summary`
|
||||
- Allowed types: `feat`, `docs`, `fix`, `style`, `refactor`, `chore`
|
||||
- Scope is required by the current PR title check.
|
||||
- For breaking changes, add `!`: `feat(driver)!: change auth flow`
|
||||
-->
|
||||
|
||||
## Summary / 摘要
|
||||
|
||||
<!--
|
||||
Briefly describe what changed and why.
|
||||
简要说明改了什么，以及为什么需要改。
|
||||
-->
|
||||
|
||||
<!--
|
||||
- List user-visible behavior changes.
|
||||
- List important implementation changes.
|
||||
- Mention config, storage, API, or compatibility changes if any.
|
||||
|
||||
- 列出用户可感知的行为变化。
|
||||
- 列出重要实现变化。
|
||||
- 如涉及配置、存储、API 或兼容性变化，请明确说明。
|
||||
-->
|
||||
|
||||
- [ ] This PR has breaking changes.
|
||||
/ 此 PR 包含破坏性变更。
|
||||
- [ ] This PR changes public API, config, storage format, or migration behavior.
|
||||
/ 此 PR 修改了公开 API、配置、存储格式或迁移行为。
|
||||
- [ ] This PR requires corresponding changes in related repositories.
|
||||
/ 此 PR 需要关联仓库同步修改。
|
||||
|
||||
Related repository PRs / 关联仓库 PR:
|
||||
|
||||
- OpenList-Frontend:
|
||||
- OpenList-Docs:
|
||||
|
||||
## Related Issues / 关联 Issue
|
||||
|
||||
<!--
|
||||
Use `Closes #123`, `Fixes #123`, or `Relates to #123`.
|
||||
Remove this section if not applicable.
|
||||
使用 `Closes #123`、`Fixes #123` 或 `Relates to #123`。
|
||||
不适用时请删除本节。
|
||||
-->
|
||||
|
||||
## Testing / 测试
|
||||
|
||||
<!--
|
||||
Describe commands, platforms, and manual checks.
|
||||
If not tested, explain why.
|
||||
|
||||
说明执行过的命令、测试平台和手动验证。
|
||||
如果未测试，请说明原因。
|
||||
-->
|
||||
|
||||
- [ ] `go test ./...`
|
||||
- [ ] Manual test / 手动测试:
|
||||
|
||||
## Checklist / 检查清单
|
||||
|
||||
- [ ] I have read [CONTRIBUTING](https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md).
|
||||
/ 我已阅读 [CONTRIBUTING](https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md)。
|
||||
- [ ] I confirm this contribution follows the repository license, contribution policy, and code of conduct.
|
||||
/ 我确认此贡献符合仓库许可证、贡献规范和行为准则。
|
||||
- [ ] I have formatted the changed code with `gofmt`, `go fmt`, or `prettier` where applicable.
|
||||
/ 我已按适用情况使用 `gofmt`、`go fmt` 或 `prettier` 格式化变更代码。
|
||||
- [ ] I have requested review from relevant maintainers or code owners where applicable.
|
||||
/ 我已在适用情况下请求相关维护者或代码所有者审查。
|
||||
|
||||
## AI Disclosure / AI 使用声明
|
||||
|
||||
<!--
|
||||
Please disclose any substantial AI assistance used in this PR.
|
||||
Minor AI assistance, such as typo fixes, autocomplete, formatting suggestions,
|
||||
or wording polish, does not need to be disclosed.
|
||||
Remove this section if not applicable.
|
||||
|
||||
请披露此 PR 中使用的重要 AI 辅助内容。
|
||||
轻微 AI 辅助，例如拼写修正、自动补全、格式建议或文字润色，无需披露。
|
||||
如不适用，请删除本节。
|
||||
|
||||
Deliberate non-disclosure may be treated as a trust and compliance issue.
|
||||
|
||||
故意隐瞒 AI 使用情况可能被视为信任与合规问题。
|
||||
-->
|
||||
|
||||
- [ ] This PR includes AI-assisted content.
|
||||
/ 此 PR 包含 AI 辅助内容。
|
||||
|
||||
Tools used / 使用工具:
|
||||
|
||||
- [ ] ChatGPT
|
||||
- [ ] Codex
|
||||
- [ ] GitHub Copilot
|
||||
- [ ] Claude
|
||||
- [ ] Gemini
|
||||
- [ ] Other (please specify) / 其他（请注明）:
|
||||
|
||||
Usage scope / 使用范围:
|
||||
|
||||
- [ ] Code generation / 代码生成
|
||||
- [ ] Refactoring / 重构
|
||||
- [ ] Documentation / 文档
|
||||
- [ ] Tests / 测试
|
||||
- [ ] Translation / 翻译
|
||||
- [ ] Review assistance / 审查辅助
|
||||
|
||||
- [ ] I have reviewed and validated all AI-assisted content included in this PR.
|
||||
/ 我已审核并验证此 PR 中的所有 AI 辅助内容。
|
||||
- [ ] I have ensured that all AI-assisted commits include `Co-Authored-By` attribution.
|
||||
/ 我已确保所有 AI 辅助提交都包含 `Co-Authored-By` 归属信息。
|
||||
- [ ] I can reproduce all AI-assisted content included in this PR without any AI tools.
|
||||
/ 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。
|
||||
@@ -1,215 +0,0 @@
|
||||
name: Container Registry Cleanup
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
package:
|
||||
description: 'The name of the package to clean up'
|
||||
required: true
|
||||
default: 'openlist-git'
|
||||
owner:
|
||||
description: 'The owner of the package (user or organization)'
|
||||
required: false
|
||||
default: 'openlistteam'
|
||||
older_than:
|
||||
description: 'Delete untagged images older than this many days'
|
||||
required: false
|
||||
default: '-1'
|
||||
untagged_timestamp_tolerance:
|
||||
description: 'Tolerance in milliseconds for untagged images close to tagged ones'
|
||||
required: false
|
||||
# We do this because multi-arch docker containers will be pushed as separate untagged images
|
||||
# to the container registry and GitHub cannot recognize them as part of a tagged image.
|
||||
# Setting this option large enough prevents accidental deletion of one of these images.
|
||||
# Here we set it to 10 seconds
|
||||
default: '10000'
|
||||
token:
|
||||
description: 'GitHub token with permissions to delete packages (leave empty to use default GITHUB_TOKEN)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
jobs:
|
||||
cleanup:
|
||||
runs-on: ubuntu-slim
|
||||
|
||||
steps:
|
||||
- name: Check inputs and tools
|
||||
id: setup
|
||||
shell: bash
|
||||
run: |
|
||||
PACKAGE="${{ github.event.inputs.package || 'openlist-git' }}"
|
||||
OLDER="${{ github.event.inputs.older_than || '-1' }}"
|
||||
TOLERANCE_VAL="${{ github.event.inputs.untagged_timestamp_tolerance || '10000' }}"
|
||||
TOKEN_INPUT="${{ github.event.inputs.token || '' }}"
|
||||
OWNER="${{ github.event.inputs.owner || 'openlistteam' }}"
|
||||
|
||||
echo "package_name=$PACKAGE" >> "$GITHUB_OUTPUT"
|
||||
echo "older_than=$OLDER" >> "$GITHUB_OUTPUT"
|
||||
echo "tolerance=$TOLERANCE_VAL" >> "$GITHUB_OUTPUT"
|
||||
echo "owner=$OWNER" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if [[ -z "$TOKEN_INPUT" ]]; then
|
||||
echo "::notice:: No token provided, using GITHUB_TOKEN instead."
|
||||
echo "token=${{ secrets.GITHUB_TOKEN }}" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "::add-mask::$TOKEN_INPUT"
|
||||
echo "token=$TOKEN_INPUT" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
if [[ -z "$PACKAGE" || -z "$OWNER" || ( -z "$TOKEN_INPUT" && -z "${{ secrets.GITHUB_TOKEN }}" ) ]]; then
|
||||
echo "::error:: Missing required inputs (package, token, or owner context)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
command -v curl >/dev/null 2>&1 || { echo "::error::curl is required but not installed."; exit 1; }
|
||||
command -v jq >/dev/null 2>&1 || { echo "::error::jq is required but not installed."; exit 1; }
|
||||
|
||||
- name: Determine owner type (user or organization)
|
||||
id: owner_type
|
||||
shell: bash
|
||||
run: |
|
||||
OWNER="${{ steps.setup.outputs.owner }}"
|
||||
TOKEN="${{ steps.setup.outputs.token }}"
|
||||
|
||||
TYPE=$(curl -s -H "Authorization: token $TOKEN" \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
"https://api.github.com/users/${OWNER}" | jq -r '.type')
|
||||
|
||||
if [[ "$TYPE" == "Organization" ]]; then
|
||||
OWNER_PATH="orgs"
|
||||
else
|
||||
OWNER_PATH="users"
|
||||
fi
|
||||
|
||||
echo "owner_path=$OWNER_PATH" >> "$GITHUB_OUTPUT"
|
||||
echo "Detected owner type: $TYPE (using /${OWNER_PATH}/)"
|
||||
|
||||
- name: Access all versions from GitHub
|
||||
id: fetch
|
||||
shell: bash
|
||||
run: |
|
||||
PACKAGE_NAME="${{ steps.setup.outputs.package_name }}"
|
||||
OWNER="${{ steps.setup.outputs.owner }}"
|
||||
TOKEN="${{ steps.setup.outputs.token }}"
|
||||
OWNER_PATH="${{ steps.owner_type.outputs.owner_path }}"
|
||||
|
||||
BASE_URL="https://api.github.com/${OWNER_PATH}/${OWNER}/packages/container/${PACKAGE_NAME}/versions"
|
||||
URL="${BASE_URL}?per_page=100"
|
||||
TEMP_FILE="/tmp/all_versions.ndjson"
|
||||
|
||||
> "$TEMP_FILE"
|
||||
PAGE=1
|
||||
while [[ -n "$URL" && $PAGE -le 10 ]]; do
|
||||
echo "Fetching: $URL"
|
||||
|
||||
curl -s -H "Authorization: token $TOKEN" \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
-D headers.txt \
|
||||
"$URL" > page.json
|
||||
|
||||
jq -c '.[]' page.json >> "$TEMP_FILE"
|
||||
|
||||
URL=$(grep -i '^link:' headers.txt | grep -o '<[^>]*>;\s*rel="next"' | sed 's/<\(.*\)>.*/\1/' || true)
|
||||
PAGE=$((PAGE + 1))
|
||||
done
|
||||
rm -f headers.txt page.json
|
||||
|
||||
echo "temp_file=$TEMP_FILE" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Filter untagged versions
|
||||
id: filter
|
||||
shell: bash
|
||||
run: |
|
||||
OLDER_THAN="${{ steps.setup.outputs.older_than }}"
|
||||
TOLERANCE="${{ steps.setup.outputs.tolerance }}"
|
||||
TEMP_FILE="${{ steps.fetch.outputs.temp_file }}"
|
||||
|
||||
TO_DELETE=$(jq -c -s \
|
||||
--arg older_than "$OLDER_THAN" \
|
||||
--arg tolerance "$TOLERANCE" \
|
||||
'
|
||||
def abs: if . < 0 then -. else . end;
|
||||
def to_ms: sub("\\.[0-9]+Z$"; "Z") | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime * 1000;
|
||||
|
||||
. as $all |
|
||||
($older_than | tonumber) as $older_than_val |
|
||||
($tolerance | tonumber) as $tolerance_val |
|
||||
now as $now |
|
||||
(if $older_than_val > 0 then ($now - ($older_than_val * 86400)) * 1000 else 0 end) as $cutoff_ms |
|
||||
|
||||
[ $all[] | select((.metadata.container.tags // []) | length > 0) | .created_at | to_ms ] as $tagged_times |
|
||||
|
||||
$all[] |
|
||||
select((.metadata.container.tags // []) | length == 0) |
|
||||
. as $item |
|
||||
(.created_at | to_ms) as $created_ms |
|
||||
|
||||
(if $older_than_val > 0 then $created_ms < $cutoff_ms else true end) as $pass_age |
|
||||
|
||||
(if $tolerance_val <= 0 then true
|
||||
else [ $tagged_times[] | select(($created_ms - .) | abs < $tolerance_val) ] | length == 0
|
||||
end) as $pass_tolerance |
|
||||
|
||||
select($pass_age and $pass_tolerance) |
|
||||
{id, name}
|
||||
' "$TEMP_FILE")
|
||||
|
||||
rm -f "$TEMP_FILE"
|
||||
|
||||
if [[ -z "$TO_DELETE" ]]; then
|
||||
DELETE_COUNT=0
|
||||
else
|
||||
DELETE_COUNT=$(echo "$TO_DELETE" | wc -l | tr -d ' ')
|
||||
fi
|
||||
|
||||
echo "Found $DELETE_COUNT untagged images no longer necessary"
|
||||
|
||||
{
|
||||
echo 'to_delete<<EOF'
|
||||
echo "$TO_DELETE"
|
||||
echo 'EOF'
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "delete_count=$DELETE_COUNT" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Delete untagged versions
|
||||
id: delete
|
||||
shell: bash
|
||||
run: |
|
||||
PACKAGE_NAME="${{ steps.setup.outputs.package_name }}"
|
||||
OWNER="${{ steps.setup.outputs.owner }}"
|
||||
TOKEN="${{ steps.setup.outputs.token }}"
|
||||
OWNER_PATH="${{ steps.owner_type.outputs.owner_path }}"
|
||||
DELETE_COUNT="${{ steps.filter.outputs.delete_count }}"
|
||||
TO_DELETE='${{ steps.filter.outputs.to_delete }}'
|
||||
|
||||
if [[ "$DELETE_COUNT" -gt 0 && -n "$TO_DELETE" ]]; then
|
||||
while IFS= read -r item; do
|
||||
[[ -z "$item" ]] && continue
|
||||
|
||||
# $item 是完整的单行 JSON，如 {"id":123,"name":"sha256:..."}
|
||||
ID=$(echo "$item" | jq -r '.id')
|
||||
NAME=$(echo "$item" | jq -r '.name')
|
||||
|
||||
DELETE_URL="https://api.github.com/${OWNER_PATH}/${OWNER}/packages/container/${PACKAGE_NAME}/versions/${ID}"
|
||||
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
|
||||
-H "Authorization: token $TOKEN" \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
"$DELETE_URL")
|
||||
|
||||
if [[ "$HTTP_CODE" == "204" ]]; then
|
||||
echo "Deleted untagged container image '${NAME}'"
|
||||
else
|
||||
echo "::warning:: Failed to delete ${NAME} (ID: ${ID}). HTTP Status: ${HTTP_CODE}"
|
||||
fi
|
||||
done <<< "$TO_DELETE"
|
||||
else
|
||||
echo "No images to delete."
|
||||
fi
|
||||
|
||||
echo "Cleanup completed."
|
||||
@@ -1,75 +0,0 @@
|
||||
name: Issue Auto Reply
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened]
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
|
||||
jobs:
|
||||
auto-reply:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check issue for unchecked tasks and reply
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
|
||||
with:
|
||||
script: |
|
||||
if (context.payload.issue.title.startsWith('[Announcements]')) return;
|
||||
const titleNotEdited = /(请修改标题|Please modify the title)/i.test(context.payload.issue.title);
|
||||
const issueBody = context.payload.issue.body || "";
|
||||
const aiSection = issueBody.match(/^### (AI生成内容|AI Generated Content)\r?\n([\s\S]*?)(?=^### |$(?![\s\S]))/m);
|
||||
const aiOptionsPattern = aiSection?.[1] === 'AI生成内容'
|
||||
? /^\s*- \[([ xX])\] 我使用了AI工具生成此内容\r?\n- \[([ xX])\] 我没有使用AI工具生成此内容\s*$/
|
||||
: /^\s*- \[([ xX])\] I used AI tools to generate this content\r?\n- \[([ xX])\] I did not use AI tools to generate this content\s*$/;
|
||||
const aiOptions = (aiSection?.[2] || '').match(aiOptionsPattern);
|
||||
const validAiDisclosure = aiOptions !== null && (aiOptions[1] !== ' ') !== (aiOptions[2] !== ' ');
|
||||
const aiModelSection = issueBody.match(/^### (AI模型是|AI model used)\r?\n([\s\S]*?)(?=^### |$(?![\s\S]))/m);
|
||||
const aiModel = (aiModelSection?.[2] || '').trim();
|
||||
const missingAiModel = validAiDisclosure && aiOptions[1] !== ' ' && (aiModel === '' || aiModel === '_No response_');
|
||||
const confirmNotRead = /- \[[xX]\] (?:我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody);
|
||||
const closeIssue = titleNotEdited || confirmNotRead || !validAiDisclosure || missingAiModel;
|
||||
let comment;
|
||||
if (titleNotEdited) {
|
||||
comment = `⚠️ 请修改标题以更好地描述您的问题或需求，并删除示例提示。当前 Issue 将被自动关闭并锁定。如需继续提交，请创建新的 Issue。
|
||||
⚠️ Please modify the title to better describe your issue or request, and remove the example prompt. This issue will be automatically closed and locked. If you wish to proceed, please create a new issue.
|
||||
`;
|
||||
} else if (confirmNotRead || !validAiDisclosure || missingAiModel) {
|
||||
comment = `⚠️ 你的 Issue 不符合提交规则。请先阅读相关规范后再重新提交。当前 Issue 将被自动关闭并锁定。如需继续提交，请确认已了解规则后创建新的 Issue。
|
||||
⚠️ Your issue does not comply with the submission rules. Please read the guidelines before submitting again. This issue will be automatically closed and locked. If you wish to proceed, please confirm that you have reviewed the rules before creating a new issue.
|
||||
`;
|
||||
} else if (/- \[ \] (?!我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody.replace(aiSection[0], ''))) {
|
||||
comment = `感谢您联系OpenList。我们会尽快回复您。
|
||||
Thanks for contacting OpenList. We will reply to you as soon as possible.
|
||||
|
||||
由于您提出的 Issue 中包含部分未确认的项目，为了更好地管理项目，在人工审核后可能会直接关闭此问题。
|
||||
如果您能确认并补充相关未确认项目的信息，欢迎随时重新提交。我们会及时关注并处理。感谢您的理解与支持！
|
||||
Since your issue contains some unchecked tasks, it may be closed after manual review.
|
||||
If you can confirm and provide information for the unchecked tasks, feel free to resubmit.
|
||||
We will pay attention and handle it in a timely manner.
|
||||
|
||||
感谢您的理解与支持！
|
||||
Thank you for your understanding and support!
|
||||
`;
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
|
||||
await github.rest.issues.createComment({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: comment
|
||||
});
|
||||
if (closeIssue) {
|
||||
await github.rest.issues.update({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
state: 'closed',
|
||||
state_reason: 'not_planned',
|
||||
labels: ['invalid']
|
||||
});
|
||||
await github.rest.issues.lock({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number
|
||||
});
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
name: PR Title Check
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [opened]
|
||||
|
||||
permissions:
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
pr-title-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check PR title for required prefix and comment
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
|
||||
with:
|
||||
script: |
|
||||
const title = context.payload.pull_request.title;
|
||||
if (/^(feat|docs|fix|style|refactor|chore)\(.+?\)!?: /i.test(title)) return;
|
||||
const comment = `⚠️ PR 标题需以 \`feat(): \`, \`docs(): \`, \`fix(): \`, \`style(): \`, \`refactor(): \`, \`chore(): \` 其中之一开头，例如：\`feat(component): 新增功能\`。
|
||||
⚠️ The PR title must start with \`feat(): \`, \`docs(): \`, \`fix(): \`, \`style(): \`, or \`refactor(): \`, \`chore(): \`. For example: \`feat(component): add new feature\`.
|
||||
|
||||
如果跨多个组件，请使用主要组件作为前缀，并在标题中枚举、描述中说明。
|
||||
If it spans multiple components, use the main component as the prefix and enumerate in the title, describe in the body.
|
||||
|
||||
如果是破坏性变更，请在类型后添加 \`!\`，例如 \`feat(component)!: 破坏性变更\`。
|
||||
For breaking changes, add \`!\` after the type, e.g., \`feat(component)!: breaking change\`.
|
||||
|
||||
`;
|
||||
await github.rest.issues.createComment({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: comment
|
||||
});
|
||||
@@ -1,38 +0,0 @@
|
||||
name: Sync to Gitee
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
sync:
|
||||
runs-on: ubuntu-latest
|
||||
name: Sync GitHub to Gitee
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup SSH
|
||||
run: |
|
||||
mkdir -p ~/.ssh
|
||||
echo "${{ secrets.GITEE_SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
|
||||
chmod 600 ~/.ssh/id_rsa
|
||||
ssh-keyscan gitee.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Create single commit and push
|
||||
run: |
|
||||
git config user.name "GitHub Actions"
|
||||
git config user.email "actions@github.com"
|
||||
|
||||
# Create a new branch
|
||||
git checkout --orphan new-main
|
||||
git add .
|
||||
git commit -m "Sync from GitHub: $(date)"
|
||||
|
||||
# Add Gitee remote and force push
|
||||
git remote add gitee ${{ vars.GITEE_REPO_URL }}
|
||||
git push --force gitee new-main:main
|
||||
@@ -1,53 +0,0 @@
|
||||
name: Trigger OpenWRT Update
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Release tag to trigger update for'
|
||||
required: true
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
trigger-makefile-update:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Trigger Makefile hash update
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9
|
||||
with:
|
||||
github-token: ${{ secrets.EXTERNAL_REPO_TOKEN_LUCI_APP_OPENLIST }}
|
||||
script: |
|
||||
const repository = process.env.REPOSITORY;
|
||||
const [owner, repo] = repository.split('/');
|
||||
|
||||
await github.rest.repos.createDispatchEvent({
|
||||
owner: owner,
|
||||
repo: repo,
|
||||
event_type: 'update-hashes',
|
||||
client_payload: {
|
||||
source_repository: process.env.SOURCE_REPOSITORY,
|
||||
release_tag: process.env.RELEASE_TAG,
|
||||
release_name: process.env.RELEASE_TAG,
|
||||
release_url: process.env.RELEASE_URL,
|
||||
triggered_by: process.env.ACTOR,
|
||||
trigger_reason: process.env.EVENT_NAME
|
||||
}
|
||||
});
|
||||
env:
|
||||
REPOSITORY: ${{ vars.HOOK_REPO || 'OpenListTeam/OpenList-OpenWRT' }}
|
||||
SOURCE_REPOSITORY: ${{ github.repository }}
|
||||
RELEASE_TAG: ${{ inputs.tag || github.ref_name }}
|
||||
RELEASE_URL: ${{ github.server_url }}/${{ github.repository }}/releases/tag/${{ inputs.tag || github.ref_name }}
|
||||
ACTOR: ${{ github.actor }}
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
|
||||
- name: Log trigger information
|
||||
run: |
|
||||
echo "🚀 Successfully triggered Makefile hash update"
|
||||
echo "📦 Target repository: OpenListTeam/luci-app-openlist"
|
||||
echo "🏷️ Tag: ${{ inputs.tag || github.ref_name }}"
|
||||
echo "👤 Triggered by: ${{ github.actor }}"
|
||||
echo "📅 Trigger time: $(date -u '+%Y-%m-%d %H:%M:%S UTC')"
|
||||
+2
-4
@@ -22,7 +22,6 @@ output/
|
||||
# vendor/
|
||||
/bin/*
|
||||
*.json
|
||||
!renovate.json
|
||||
/build
|
||||
/data/
|
||||
/tmp/
|
||||
@@ -30,7 +29,6 @@ output/
|
||||
/lang/
|
||||
/daemon/
|
||||
/public/dist/*
|
||||
!/public/dist/README.md
|
||||
/!public/dist/README.md
|
||||
|
||||
.VSCodeCounter
|
||||
*.syso
|
||||
.VSCodeCounter
|
||||
@@ -1,60 +0,0 @@
|
||||
# Agent Instructions
|
||||
|
||||
## Issues
|
||||
|
||||
Before creating an issue, review the available issue templates in the `.github` directory.
|
||||
|
||||
When drafting the issue:
|
||||
|
||||
- Use the most appropriate template.
|
||||
- Follow the template structure.
|
||||
- Fill in all required sections.
|
||||
- Remove sections that the template explicitly marks as optional or not applicable.
|
||||
- Do not invent reproduction steps, logs, screenshots, or expected behavior.
|
||||
|
||||
## Pull Requests
|
||||
|
||||
Before creating a pull request, read `.github/PULL_REQUEST_TEMPLATE.md`.
|
||||
|
||||
When drafting the pull request:
|
||||
|
||||
- Follow the template structure.
|
||||
- Use the title format required by the template.
|
||||
- Fill in or remove each section according to the template guidance.
|
||||
- Include testing details, or explain why testing was not run.
|
||||
- Do not invent testing results.
|
||||
- Do not claim validation, verification, or review steps that were not actually performed.
|
||||
|
||||
## Automated Contributions
|
||||
|
||||
Fully automated contributions are not considered equivalent to normal community participation.
|
||||
|
||||
A contribution may be considered fully automated if it is submitted through an automated agent, or if the submitting account participates in project discussions through an automated agent, without meaningful human review or intervention.
|
||||
|
||||
When making this determination, maintainers may consider the overall behavior of the account, including but not limited to disclosed agent usage, interaction patterns, response characteristics, and other available evidence. No single factor is determinative.
|
||||
|
||||
Maintainers reserve the right to accept, reject, modify, or reimplement any contribution independently of any action taken against the submitting account. Acceptance of a contribution does not imply acceptance of the submitting account or its contribution method. If an account is determined to be primarily operated through automated processes, we may need to restrict its future participation in contributions until that determination is rescinded.
|
||||
|
||||
## Git Commits
|
||||
|
||||
When creating commits, follow the repository `git-commit` skill rules:
|
||||
|
||||
- Use Conventional Commits title format: `type(scope): subject`.
|
||||
- Allowed types: `feat`, `fix`, `refactor`, `perf`, `docs`, `style`, `test`, `build`, `ci`, `chore`, `revert`.
|
||||
- Use a meaningful scope based on the main module, package, or feature.
|
||||
- Write the subject in imperative mood and describe the actual change.
|
||||
- Use a concise Markdown list in the commit body, with each item describing one key change.
|
||||
- Do not invent changes that are not present in the diff.
|
||||
- Do not describe behavior, refactors, fixes, or tests that are not reflected in the commit.
|
||||
|
||||
Include at most one `Co-authored-by` trailer that matches the AI assistant actually used to produce the change.
|
||||
|
||||
Examples:
|
||||
|
||||
- `Co-authored-by: Codex <267193182+codex@users.noreply.github.com>`
|
||||
- `Co-authored-by: GitHub Copilot <copilot@github.com>`
|
||||
- `Co-authored-by: Claude <81847+claude@users.noreply.github.com>`
|
||||
|
||||
If you are not one of the listed assistants, do not add a `Co-authored-by` trailer.
|
||||
|
||||
Instead, ask the human collaborator to provide the exact `Co-authored-by` trailer to use. Do not invent, infer, or generate one yourself.
|
||||
-110
@@ -1,110 +0,0 @@
|
||||
# Contributing
|
||||
|
||||
## Setup your machine
|
||||
|
||||
`OpenList` is written in [Go](https://golang.org/) and [SolidJS](https://www.solidjs.com/).
|
||||
|
||||
Prerequisites:
|
||||
|
||||
- [git](https://git-scm.com)
|
||||
- [Go](https://golang.org/doc/install) version declared in [`go.mod`](./go.mod)
|
||||
- [gcc](https://gcc.gnu.org/)
|
||||
- [nodejs](https://nodejs.org/)
|
||||
|
||||
## Cloning a fork
|
||||
|
||||
Fork and clone `OpenList` and `OpenList-Frontend` anywhere:
|
||||
|
||||
```shell
|
||||
git clone https://github.com/<your-username>/OpenList.git
|
||||
git clone --recurse-submodules https://github.com/<your-username>/OpenList-Frontend.git
|
||||
```
|
||||
|
||||
## Creating a branch
|
||||
|
||||
Create a new branch from the `main` branch, with an appropriate name.
|
||||
|
||||
```shell
|
||||
git checkout -b <branch-name>
|
||||
```
|
||||
|
||||
## Preview your change
|
||||
|
||||
### backend
|
||||
|
||||
```shell
|
||||
go run main.go
|
||||
```
|
||||
|
||||
### frontend
|
||||
|
||||
```shell
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
## Add a new driver
|
||||
|
||||
Copy `drivers/template` folder and rename it, and follow the comments in it.
|
||||
|
||||
## Community and policies
|
||||
|
||||
By contributing, you agree to follow the repository's code of conduct and license terms.
|
||||
|
||||
- Code of conduct: [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md)
|
||||
- License: [LICENSE](./LICENSE)
|
||||
- Security issues: please report privately according to [SECURITY.md](./SECURITY.md)
|
||||
|
||||
If your contribution includes substantial AI-assisted content, disclose the tools used and the scope of assistance in the pull request.
|
||||
|
||||
## Create a commit
|
||||
|
||||
Commit messages should be well formatted, and to make that "standardized".
|
||||
|
||||
Submit your pull request. For PR titles, follow [Conventional Commits](https://www.conventionalcommits.org).
|
||||
|
||||
<https://github.com/OpenListTeam/OpenList/issues/376>
|
||||
|
||||
It's suggested to sign your commits. See: [How to sign commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
|
||||
|
||||
## Submit a pull request
|
||||
|
||||
Please make sure your code has been formatted with `go fmt` or [prettier](https://prettier.io/) before submitting.
|
||||
|
||||
Push your branch to your `openlist` fork and open a pull request against the `main` branch.
|
||||
|
||||
## Merge your pull request
|
||||
|
||||
Your pull request will be merged after review. Please wait for the maintainer to merge your pull request after review.
|
||||
|
||||
At least 1 approving review is required by reviewers with write access. You can also request a review from maintainers.
|
||||
|
||||
## Delete your branch
|
||||
|
||||
(Optional) After your pull request is merged, you can delete your branch.
|
||||
|
||||
## AI Disclosure
|
||||
|
||||
If your pull request includes substantial AI-assisted content, disclose it in the PR description.
|
||||
|
||||
The pull request description must follow the repository's pull request template.
|
||||
|
||||
Fully automated contributions are not considered equivalent to normal community participation.
|
||||
|
||||
A contribution may be considered fully automated if it is submitted through an automated agent, or if the submitting account participates in project discussions through an automated agent, without meaningful human review or intervention.
|
||||
|
||||
When making this determination, maintainers may consider the overall behavior of the account, including but not limited to disclosed agent usage, interaction patterns, response characteristics, and other available evidence. No single factor is determinative.
|
||||
|
||||
Maintainers reserve the right to accept, reject, modify, or reimplement any contribution independently of any action taken against the submitting account. Acceptance of a contribution does not imply acceptance of the submitting account or its contribution method. If an account is determined to be primarily operated through automated processes, we may need to restrict its future participation in contributions until that determination is rescinded.
|
||||
|
||||
Please include:
|
||||
|
||||
- Tools used, such as ChatGPT, GitHub Copilot, Claude, Cursor, or other AI tools.
|
||||
- Usage scope, such as code generation, refactoring, documentation, tests, translation, or review assistance.
|
||||
- Confirmation that you have reviewed and validated all AI-assisted content before submission.
|
||||
- Confirmation that the submitted content complies with this repository's license and contribution policies.
|
||||
|
||||
Minor AI assistance, such as typo fixes, autocomplete, formatting suggestions, or wording polish, does not need to be disclosed.
|
||||
|
||||
---
|
||||
|
||||
Thank you for your contribution! Let's make OpenList better together!
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList هو تفرّع من AList يتميز بالمرونة والحوكمة طويلة الأمد ويقوده المجتمع، وقد بُني للدفاع عن البرمجيات مفتوحة المصدر ضد الهجمات القائمة على الثقة.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | العربية
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## إخلاء المسؤولية
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## الميزات
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group, Share)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## الوثائق
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## العرض التجريبي
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## النقاش
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## الراعي
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## المتبرعون
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## الترخيص
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## تواصل معنا
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## المساهمون
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList ist ein widerstandsfähiger, langfristig verwalteter und Community-getriebener Fork von AList, entwickelt zum Schutz von Open Source vor vertrauensbasierten Angriffen.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | Deutsch | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Haftungsausschluss
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Funktionen
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group, Share)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Dokumentation
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Demo
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Diskussion
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Sponsor
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Spender
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Lizenz
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Kontakt
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Mitwirkende
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList es un fork de AList resiliente, con gobernanza a largo plazo e impulsado por la comunidad, creado para defender el código abierto frente a ataques basados en la confianza.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | Español | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Aviso legal
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Características
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group, Share)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Documentación
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Demo
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Discusión
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Patrocinador
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Donantes
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Licencia
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Contacto
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Colaboradores
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList est un fork d’AList résilient, gouverné sur le long terme et porté par la communauté, conçu pour protéger l’open source contre les attaques fondées sur la confiance.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | Français | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Avertissement
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Fonctionnalités
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group, Share)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Documentation
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Démo
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Discussion
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Sponsor
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Donateurs
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Licence
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Nous contacter
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Contributeurs
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList는 신뢰 기반 공격으로부터 오픈 소스를 보호하기 위해 만들어진, 탄력적이고 장기 거버넌스를 지향하는 커뮤니티 주도 AList 포크입니다.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | 한국어 | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## 면책 조항
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## 기능
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group, Share)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## 문서
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## 데모
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## 토론
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## 후원
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## 후원자
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## 라이선스
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## 문의
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## 기여자
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList — устойчивый, управляемый сообществом форк AList с долгосрочным управлением, созданный для защиты open source от атак, основанных на доверии.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | Русский | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Отказ от ответственности
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Возможности
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group, Share)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Документация
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Демо
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Обсуждение
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Спонсор
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Доноры
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Лицензия
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Связаться с нами
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Участники
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
-89
@@ -1,89 +0,0 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
Only the latest stable release receives security patches. We strongly recommend always keeping OpenList up to date.
|
||||
|
||||
| Version | Supported |
|
||||
| -------------------- | ------------------ |
|
||||
| Latest stable (v4.x) | :white_check_mark: |
|
||||
| Older versions | :x: |
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
**Please do NOT report security vulnerabilities through public GitHub Issues.**
|
||||
|
||||
If you discover a security vulnerability in OpenList, please report it responsibly by using one of the following channels:
|
||||
|
||||
- **GitHub Private Security Advisory** (preferred): [Submit here](https://github.com/OpenListTeam/OpenList/security/advisories/new)
|
||||
- **Telegram**: Contact a maintainer privately via [@OpenListTeam](https://t.me/OpenListTeam)
|
||||
|
||||
When reporting, please include as much of the following as possible:
|
||||
|
||||
- A description of the vulnerability and its potential impact
|
||||
- The affected version(s)
|
||||
- Step-by-step instructions to reproduce the issue
|
||||
- Any proof-of-concept code or screenshots (if applicable)
|
||||
- Suggested mitigation or fix (optional but appreciated)
|
||||
|
||||
## Security Best Practices for Users
|
||||
|
||||
To keep your OpenList instance secure:
|
||||
|
||||
- Always update to the latest release.
|
||||
- Use a strong, unique admin password and change it after first login.
|
||||
- Enable HTTPS (TLS) for your deployment — do **not** expose OpenList over plain HTTP on the public internet.
|
||||
- Limit exposed ports using a reverse proxy (e.g., Nginx, Caddy).
|
||||
- Set up access controls and avoid enabling guest access unless necessary.
|
||||
- Regularly review mounted storage permissions and revoke unused API tokens.
|
||||
- When using Docker, avoid running the container as root if possible.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
We sincerely thank all security researchers and community members who responsibly disclose vulnerabilities and help make OpenList safer for everyone.
|
||||
|
||||
---
|
||||
|
||||
# 安全政策
|
||||
|
||||
## 支持的版本
|
||||
|
||||
我们仅对最新稳定版本提供安全补丁。强烈建议始终保持 OpenList 为最新版本。
|
||||
|
||||
| 版本 | 是否支持 |
|
||||
| ------------------ | ------------------ |
|
||||
| 最新稳定版（v4.x） | :white_check_mark: |
|
||||
| 旧版本 | :x: |
|
||||
|
||||
## 报告漏洞
|
||||
|
||||
**请勿通过公开的 GitHub Issues 报告安全漏洞。**
|
||||
|
||||
如果您在 OpenList 中发现安全漏洞，请通过以下渠道之一负责任地进行报告：
|
||||
|
||||
- **GitHub 私密安全公告**（推荐）：[点击提交](https://github.com/OpenListTeam/OpenList/security/advisories/new)
|
||||
- **Telegram**：通过 [@OpenListTeam](https://t.me/OpenListTeam) 私信联系维护者
|
||||
|
||||
报告时，请尽量提供以下信息：
|
||||
|
||||
- 漏洞描述及其潜在影响
|
||||
- 受影响的版本
|
||||
- 复现问题的详细步骤
|
||||
- 概念验证代码或截图（如有）
|
||||
- 建议的缓解措施或修复方案（可选，但非常欢迎）
|
||||
|
||||
## 用户安全最佳实践
|
||||
|
||||
为保障您的 OpenList 实例安全：
|
||||
|
||||
- 始终更新至最新版本。
|
||||
- 使用强且唯一的管理员密码，并在首次登录后立即修改。
|
||||
- 为您的部署启用 HTTPS（TLS）—— **请勿**在公网上以明文 HTTP 方式暴露 OpenList。
|
||||
- 使用反向代理（如 Nginx、Caddy）限制对外暴露的端口。
|
||||
- 配置访问控制，非必要情况下不要开启访客访问。
|
||||
- 定期检查已挂载存储的权限，并撤销未使用的 API 令牌。
|
||||
- 使用 Docker 部署时，尽可能避免以 root 用户运行容器。
|
||||
|
||||
## 致谢
|
||||
|
||||
我们衷心感谢所有负责任地披露漏洞、帮助 OpenList 变得更加安全的安全研究人员和社区成员。
|
||||
@@ -0,0 +1,11 @@
|
||||
version: v1
|
||||
plugins:
|
||||
- plugin: buf.build/protocolbuffers/go:v1.36.7
|
||||
out: .
|
||||
opt:
|
||||
- paths=source_relative
|
||||
- plugin: buf.build/grpc/go:v1.5.1
|
||||
out: .
|
||||
opt:
|
||||
- paths=source_relative
|
||||
- require_unimplemented_servers=false
|
||||
+29
-30
@@ -1,43 +1,42 @@
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"context"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
|
||||
"github.com/OpenListTeam/OpenList/v5/internal/bootstrap"
|
||||
"github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func Init() {
|
||||
bootstrap.Init()
|
||||
func Init(ctx context.Context) {
|
||||
if flags.Dev {
|
||||
flags.Debug = true
|
||||
}
|
||||
initLogrus()
|
||||
bootstrap.InitConfig()
|
||||
bootstrap.InitDriverPlugins()
|
||||
}
|
||||
|
||||
func Release() {
|
||||
bootstrap.Release()
|
||||
|
||||
}
|
||||
|
||||
var pid = -1
|
||||
var pidFile string
|
||||
|
||||
func initDaemon() {
|
||||
ex, err := os.Executable()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
exPath := filepath.Dir(ex)
|
||||
_ = os.MkdirAll(filepath.Join(exPath, "daemon"), 0700)
|
||||
pidFile = filepath.Join(exPath, "daemon/pid")
|
||||
if utils.Exists(pidFile) {
|
||||
bytes, err := os.ReadFile(pidFile)
|
||||
if err != nil {
|
||||
log.Fatal("failed to read pid file", err)
|
||||
}
|
||||
id, err := strconv.Atoi(string(bytes))
|
||||
if err != nil {
|
||||
log.Fatal("failed to parse pid data", err)
|
||||
}
|
||||
pid = id
|
||||
func initLog(l *logrus.Logger) {
|
||||
if flags.Debug {
|
||||
l.SetLevel(logrus.DebugLevel)
|
||||
l.SetReportCaller(true)
|
||||
} else {
|
||||
l.SetLevel(logrus.InfoLevel)
|
||||
l.SetReportCaller(false)
|
||||
}
|
||||
}
|
||||
func initLogrus() {
|
||||
formatter := logrus.TextFormatter{
|
||||
ForceColors: true,
|
||||
EnvironmentOverrideColors: true,
|
||||
TimestampFormat: "2006-01-02 15:04:05",
|
||||
FullTimestamp: true,
|
||||
}
|
||||
logrus.SetFormatter(&formatter)
|
||||
initLog(logrus.StandardLogger())
|
||||
}
|
||||
|
||||
+31
-2
@@ -1,11 +1,40 @@
|
||||
package flags
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var (
|
||||
DataDir string
|
||||
ConfigPath string
|
||||
ConfigFile string
|
||||
Debug bool
|
||||
NoPrefix bool
|
||||
Dev bool
|
||||
ForceBinDir bool
|
||||
LogStd bool
|
||||
|
||||
pwd string
|
||||
)
|
||||
|
||||
// Program working directory
|
||||
func PWD() string {
|
||||
if pwd != "" {
|
||||
return pwd
|
||||
}
|
||||
if ForceBinDir {
|
||||
ex, err := os.Executable()
|
||||
if err != nil {
|
||||
logrus.Fatal(err)
|
||||
}
|
||||
pwd = filepath.Dir(ex)
|
||||
return pwd
|
||||
}
|
||||
d, err := os.Getwd()
|
||||
if err != nil {
|
||||
logrus.Fatal(err)
|
||||
}
|
||||
pwd = d
|
||||
return d
|
||||
}
|
||||
|
||||
+4
-8
@@ -4,10 +4,7 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/cmd/flags"
|
||||
_ "github.com/OpenListTeam/OpenList/v4/drivers"
|
||||
_ "github.com/OpenListTeam/OpenList/v4/internal/archive"
|
||||
_ "github.com/OpenListTeam/OpenList/v4/internal/offline_download"
|
||||
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
@@ -27,11 +24,10 @@ func Execute() {
|
||||
}
|
||||
|
||||
func init() {
|
||||
RootCmd.PersistentFlags().StringVar(&flags.DataDir, "data", "data", "data directory (relative paths are resolved against the current working directory)")
|
||||
RootCmd.PersistentFlags().StringVar(&flags.ConfigPath, "config", "", "path to config.json (relative to current working directory; defaults to [data directory]/config.json, where [data directory] is set by --data)")
|
||||
RootCmd.PersistentFlags().StringVarP(&flags.ConfigFile, "config", "c", "data/config.json", "config file")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.Debug, "debug", false, "start with debug mode")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.NoPrefix, "no-prefix", false, "disable env prefix")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.Dev, "dev", false, "start with dev mode")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.ForceBinDir, "force-bin-dir", false, "Force to use the directory where the binary file is located as data directory")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.LogStd, "log-std", false, "Force to log to std")
|
||||
RootCmd.PersistentFlags().BoolVarP(&flags.ForceBinDir, "force-bin-dir", "f", false, "force to use the directory where the binary file is located as data directory")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.LogStd, "log-std", false, "force to log to std")
|
||||
}
|
||||
|
||||
+125
-19
@@ -1,13 +1,26 @@
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
|
||||
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
|
||||
"github.com/OpenListTeam/OpenList/v5/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v5/server"
|
||||
"github.com/gin-gonic/gin"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"github.com/spf13/cobra"
|
||||
"golang.org/x/net/http2"
|
||||
"golang.org/x/net/http2/h2c"
|
||||
)
|
||||
|
||||
// ServerCmd represents the server command
|
||||
@@ -16,34 +29,127 @@ var ServerCmd = &cobra.Command{
|
||||
Short: "Start the server at the specified address",
|
||||
Long: `Start the server at the specified address
|
||||
the address is defined in config file`,
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
bootstrap.Init()
|
||||
defer bootstrap.Release()
|
||||
bootstrap.Start()
|
||||
// Wait for interrupt signal to gracefully shutdown the server with
|
||||
// a timeout of 1 second.
|
||||
Run: func(_ *cobra.Command, args []string) {
|
||||
serverCtx, serverCancel := context.WithCancel(context.Background())
|
||||
defer serverCancel()
|
||||
Init(serverCtx)
|
||||
|
||||
if !flags.Debug {
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
}
|
||||
r := gin.New()
|
||||
r.Use(gin.LoggerWithWriter(log.StandardLogger().Out))
|
||||
r.Use(gin.RecoveryWithWriter(log.StandardLogger().Out))
|
||||
server.Init(r)
|
||||
|
||||
var httpHandler http.Handler = r
|
||||
if conf.Conf.Scheme.EnableH2c {
|
||||
httpHandler = h2c.NewHandler(r, &http2.Server{})
|
||||
}
|
||||
var httpSrv, httpsSrv, unixSrv *http.Server
|
||||
if conf.Conf.Scheme.HttpPort > 0 {
|
||||
httpBase := fmt.Sprintf("%s:%d", conf.Conf.Scheme.Address, conf.Conf.Scheme.HttpPort)
|
||||
log.Infoln("start HTTP server", "@", httpBase)
|
||||
httpSrv = &http.Server{Addr: httpBase, Handler: httpHandler}
|
||||
go func() {
|
||||
err := httpSrv.ListenAndServe()
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Errorln("start HTTP server", ":", err)
|
||||
serverCancel()
|
||||
}
|
||||
}()
|
||||
}
|
||||
if conf.Conf.Scheme.HttpsPort > 0 {
|
||||
httpsBase := fmt.Sprintf("%s:%d", conf.Conf.Scheme.Address, conf.Conf.Scheme.HttpsPort)
|
||||
log.Infoln("start HTTPS server", "@", httpsBase)
|
||||
httpsSrv = &http.Server{Addr: httpsBase, Handler: r}
|
||||
go func() {
|
||||
err := httpsSrv.ListenAndServeTLS(conf.Conf.Scheme.CertFile, conf.Conf.Scheme.KeyFile)
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Errorln("start HTTPS server", ":", err)
|
||||
serverCancel()
|
||||
}
|
||||
}()
|
||||
}
|
||||
if conf.Conf.Scheme.UnixFile != "" {
|
||||
log.Infoln("start Unix server", "@", conf.Conf.Scheme.UnixFile)
|
||||
unixSrv = &http.Server{Handler: httpHandler}
|
||||
go func() {
|
||||
listener, err := net.Listen("unix", conf.Conf.Scheme.UnixFile)
|
||||
if err != nil {
|
||||
log.Errorln("start Unix server", ":", err)
|
||||
serverCancel()
|
||||
return
|
||||
}
|
||||
|
||||
mode, err := strconv.ParseUint(conf.Conf.Scheme.UnixFilePerm, 8, 32)
|
||||
if err != nil {
|
||||
log.Errorln("parse unix_file_perm", ":", err)
|
||||
} else {
|
||||
err = os.Chmod(conf.Conf.Scheme.UnixFile, os.FileMode(mode))
|
||||
if err != nil {
|
||||
log.Errorln("chmod socket file", ":", err)
|
||||
}
|
||||
}
|
||||
|
||||
err = unixSrv.Serve(listener)
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Errorln("start Unix server", ":", err)
|
||||
serverCancel()
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
quit := make(chan os.Signal, 1)
|
||||
// kill (no param) default send syscanll.SIGTERM
|
||||
// kill -2 is syscall.SIGINT
|
||||
// kill -9 is syscall. SIGKILL but can"t be catch, so don't need add it
|
||||
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
|
||||
<-quit
|
||||
bootstrap.Shutdown(1 * time.Second)
|
||||
select {
|
||||
case <-quit:
|
||||
case <-serverCtx.Done():
|
||||
}
|
||||
|
||||
log.Println("shutdown server...")
|
||||
Release()
|
||||
|
||||
quitCtx, quitCancel := context.WithTimeout(context.Background(), time.Second)
|
||||
defer quitCancel()
|
||||
var wg sync.WaitGroup
|
||||
if httpSrv != nil {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := httpSrv.Shutdown(quitCtx); err != nil {
|
||||
log.Errorln("shutdown HTTP server", ":", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
if httpsSrv != nil {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := httpsSrv.Shutdown(quitCtx); err != nil {
|
||||
log.Errorln("shutdown HTTPS server", ":", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
if unixSrv != nil {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := unixSrv.Shutdown(quitCtx); err != nil {
|
||||
log.Errorln("shutdown Unix server", ":", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
log.Println("server exit")
|
||||
},
|
||||
}
|
||||
|
||||
func init() {
|
||||
RootCmd.AddCommand(ServerCmd)
|
||||
|
||||
// Here you will define your flags and configuration settings.
|
||||
|
||||
// Cobra supports Persistent Flags which will work for this command
|
||||
// and all subcommands, e.g.:
|
||||
// serverCmd.PersistentFlags().String("foo", "", "A help for foo")
|
||||
|
||||
// Cobra supports local flags which will only run when this command
|
||||
// is called directly, e.g.:
|
||||
// serverCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
|
||||
}
|
||||
|
||||
// OutOpenListInit 暴露用于外部启动server的函数
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
package _115
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
driver115 "github.com/SheltonZhu/115driver/pkg/driver"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var (
|
||||
md5Salt = "Qclm8MGWUv59TnrR0XPg"
|
||||
appVer = "35.6.0.3"
|
||||
)
|
||||
|
||||
func (d *Pan115) getAppVersion() (string, error) {
|
||||
result := VersionResp{}
|
||||
res, err := base.RestyClient.R().Get(driver115.ApiGetVersion)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
err = utils.Json.Unmarshal(res.Body(), &result)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(result.Error) > 0 {
|
||||
return "", errors.New(result.Error)
|
||||
}
|
||||
return result.Data.Win.Version, nil
|
||||
}
|
||||
|
||||
func (d *Pan115) getAppVer() string {
|
||||
ver, err := d.getAppVersion()
|
||||
if err != nil {
|
||||
log.Warnf("[115] get app version failed: %v", err)
|
||||
return appVer
|
||||
}
|
||||
if len(ver) > 0 {
|
||||
return ver
|
||||
}
|
||||
return appVer
|
||||
}
|
||||
|
||||
func (d *Pan115) initAppVer() {
|
||||
appVer = d.getAppVer()
|
||||
log.Debugf("use app version: %v", appVer)
|
||||
}
|
||||
|
||||
type VersionResp struct {
|
||||
Error string `json:"error,omitempty"`
|
||||
Data Versions `json:"data"`
|
||||
}
|
||||
|
||||
type Versions struct {
|
||||
Win Version `json:"win"`
|
||||
}
|
||||
|
||||
type Version struct {
|
||||
Version string `json:"version_code"`
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
package _115_open
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
func ParseInt64(v json.Number) (int64, error) {
|
||||
i, err := v.Int64()
|
||||
if err == nil {
|
||||
return i, nil
|
||||
}
|
||||
f, e1 := v.Float64()
|
||||
if e1 == nil {
|
||||
return int64(f), nil
|
||||
}
|
||||
return int64(0), err
|
||||
}
|
||||
|
||||
func parseTime(value string) int64 {
|
||||
timestamp, _ := strconv.ParseInt(value, 10, 64)
|
||||
return timestamp
|
||||
}
|
||||
@@ -1,129 +0,0 @@
|
||||
package _123_open
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
var (
|
||||
AccessToken = "https://open-api.123pan.com/api/v1/access_token"
|
||||
)
|
||||
|
||||
func expiresInToExpiredAt(expiresIn int64) (time.Time, error) {
|
||||
if expiresIn <= 0 {
|
||||
return time.Time{}, errors.New("invalid expires_in from official API")
|
||||
}
|
||||
return time.Now().UTC().Add(time.Duration(expiresIn) * time.Second), nil
|
||||
}
|
||||
|
||||
type tokenManager struct {
|
||||
// accessToken string
|
||||
expiredAt time.Time
|
||||
mu sync.Mutex
|
||||
blockRefresh bool
|
||||
}
|
||||
|
||||
func (d *Open123) getAccessToken(forceRefresh bool) (string, error) {
|
||||
tm := d.tm
|
||||
tm.mu.Lock()
|
||||
defer tm.mu.Unlock()
|
||||
if tm.blockRefresh {
|
||||
return "", errors.New("Authentication expired")
|
||||
}
|
||||
if !forceRefresh && d.AccessToken != "" && time.Now().Before(tm.expiredAt.Add(-5*time.Minute)) {
|
||||
return d.AccessToken, nil
|
||||
}
|
||||
if err := d.flushAccessToken(); err != nil {
|
||||
// token expired and failed to refresh, block further refresh attempts
|
||||
tm.blockRefresh = true
|
||||
return "", err
|
||||
}
|
||||
return d.AccessToken, nil
|
||||
}
|
||||
|
||||
func (d *Open123) flushAccessToken() error {
|
||||
// Official app renewapi response contains access_token, refresh_token and expires_in.
|
||||
if d.UseOnlineAPI && d.RefreshToken != "" && len(d.APIAddress) > 0 {
|
||||
var resp RefreshTokenResp
|
||||
_, err := base.RestyClient.R().
|
||||
SetResult(&resp).
|
||||
SetQueryParams(map[string]string{
|
||||
"refresh_ui": d.RefreshToken,
|
||||
"server_use": "true",
|
||||
"driver_txt": "123cloud_oa",
|
||||
}).
|
||||
Get(d.APIAddress)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if resp.AccessToken == "" || resp.RefreshToken == "" {
|
||||
errMessage := resp.ErrorDescription
|
||||
if errMessage == "" {
|
||||
errMessage = resp.Text
|
||||
}
|
||||
if errMessage == "" {
|
||||
errMessage = resp.Message
|
||||
}
|
||||
if errMessage == "" {
|
||||
errMessage = resp.Error
|
||||
}
|
||||
if errMessage != "" {
|
||||
return fmt.Errorf("failed to refresh token: %s", errMessage)
|
||||
}
|
||||
return fmt.Errorf("empty access_token or refresh_token returned from official API")
|
||||
}
|
||||
expiredAt, err := expiresInToExpiredAt(resp.ExpiresIn)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
d.AccessToken = resp.AccessToken
|
||||
d.RefreshToken = resp.RefreshToken
|
||||
d.tm.expiredAt = expiredAt
|
||||
op.MustSaveDriverStorage(d)
|
||||
d.tm.blockRefresh = false
|
||||
return nil
|
||||
}
|
||||
|
||||
// Developer API response contains code/message/data(accessToken, expiredAt).
|
||||
if d.ClientID != "" && d.ClientSecret != "" {
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeaders(map[string]string{
|
||||
"platform": "open_platform",
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
var resp AccessTokenResp
|
||||
req.SetBody(base.Json{
|
||||
"clientID": d.ClientID,
|
||||
"clientSecret": d.ClientSecret,
|
||||
})
|
||||
req.SetResult(&resp)
|
||||
_, err := req.Execute(http.MethodPost, AccessToken)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
return fmt.Errorf("get access token failed: %s", resp.Message)
|
||||
}
|
||||
if resp.Data.AccessToken == "" || resp.Data.ExpiredAt == "" {
|
||||
return errors.New("invalid token payload from developer API")
|
||||
}
|
||||
expiredAt, err := time.Parse(time.RFC3339, resp.Data.ExpiredAt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse expire time failed: %w", err)
|
||||
}
|
||||
d.AccessToken = resp.Data.AccessToken
|
||||
d.tm.expiredAt = expiredAt.UTC()
|
||||
op.MustSaveDriverStorage(d)
|
||||
d.tm.blockRefresh = false
|
||||
return nil
|
||||
}
|
||||
return errors.New("no valid authentication method available")
|
||||
}
|
||||
@@ -1,278 +0,0 @@
|
||||
package _123_open
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/md5"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/go-resty/resty/v2"
|
||||
"github.com/google/uuid"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var ( // 不同情况下获取的AccessTokenQPS限制不同 如下模块化易于拓展
|
||||
Api = "https://open-api.123pan.com"
|
||||
|
||||
UserInfo = InitApiInfo(Api+"/api/v1/user/info", 1)
|
||||
FileList = InitApiInfo(Api+"/api/v2/file/list", 3)
|
||||
DownloadInfo = InitApiInfo(Api+"/api/v1/file/download_info", 5)
|
||||
DirectLink = InitApiInfo(Api+"/api/v1/direct-link/url", 5)
|
||||
Mkdir = InitApiInfo(Api+"/upload/v1/file/mkdir", 2)
|
||||
Move = InitApiInfo(Api+"/api/v1/file/move", 1)
|
||||
Rename = InitApiInfo(Api+"/api/v1/file/name", 1)
|
||||
Trash = InitApiInfo(Api+"/api/v1/file/trash", 2)
|
||||
UploadCreate = InitApiInfo(Api+"/upload/v2/file/create", 2)
|
||||
UploadComplete = InitApiInfo(Api+"/upload/v2/file/upload_complete", 0)
|
||||
UploadSHA1Reuse = InitApiInfo(Api+"/upload/v2/file/sha1_reuse", 2)
|
||||
|
||||
OfflineDownload = InitApiInfo(Api+"/api/v1/offline/download", 1)
|
||||
OfflineDownloadProcess = InitApiInfo(Api+"/api/v1/offline/download/process", 5)
|
||||
)
|
||||
|
||||
func (d *Open123) Request(apiInfo *ApiInfo, method string, callback base.ReqCallback, resp interface{}) ([]byte, error) {
|
||||
for {
|
||||
token, err := d.getAccessToken(false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeaders(map[string]string{
|
||||
"authorization": "Bearer " + token,
|
||||
"platform": "open_platform",
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
if resp != nil {
|
||||
req.SetResult(resp)
|
||||
}
|
||||
|
||||
log.Debugf("API: %s, QPS: %d, NowLen: %d", apiInfo.url, apiInfo.qps, apiInfo.NowLen())
|
||||
|
||||
apiInfo.Require()
|
||||
defer apiInfo.Release()
|
||||
res, err := req.Execute(method, apiInfo.url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body := res.Body()
|
||||
|
||||
// 解析为通用响应
|
||||
var baseResp BaseResp
|
||||
if err = json.Unmarshal(body, &baseResp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if baseResp.Code == 0 {
|
||||
return body, nil
|
||||
} else if baseResp.Code == 401 {
|
||||
// 强制刷新Token, 有小概率会 race condition 导致多次刷新Token，但不影响正确运行
|
||||
if _, err := d.getAccessToken(true); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else if baseResp.Code == 429 {
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
log.Warningf("API: %s, QPS: %d, 请求太频繁，对应API提示过多请减小QPS", apiInfo.url, apiInfo.qps)
|
||||
} else {
|
||||
return nil, errors.New(baseResp.Message)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Open123) SignURL(originURL, privateKey string, uid uint64, validDuration time.Duration) (newURL string, err error) {
|
||||
// 生成Unix时间戳
|
||||
ts := time.Now().Add(validDuration).Unix()
|
||||
|
||||
// 生成随机数（建议使用UUID，不能包含中划线（-））
|
||||
rand := strings.ReplaceAll(uuid.New().String(), "-", "")
|
||||
|
||||
// 解析URL
|
||||
objURL, err := url.Parse(originURL)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// 待签名字符串，格式：path-timestamp-rand-uid-privateKey
|
||||
unsignedStr := fmt.Sprintf("%s-%d-%s-%d-%s", objURL.Path, ts, rand, uid, privateKey)
|
||||
md5Hash := md5.Sum([]byte(unsignedStr))
|
||||
// 生成鉴权参数，格式：timestamp-rand-uid-md5hash
|
||||
authKey := fmt.Sprintf("%d-%s-%d-%x", ts, rand, uid, md5Hash)
|
||||
|
||||
// 添加鉴权参数到URL查询参数
|
||||
v := objURL.Query()
|
||||
v.Add("auth_key", authKey)
|
||||
objURL.RawQuery = v.Encode()
|
||||
|
||||
return objURL.String(), nil
|
||||
}
|
||||
|
||||
func (d *Open123) getUserInfo(ctx context.Context) (*UserInfoResp, error) {
|
||||
var resp UserInfoResp
|
||||
|
||||
if _, err := d.Request(UserInfo, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getUID(ctx context.Context) (uint64, error) {
|
||||
if d.UID != 0 {
|
||||
return d.UID, nil
|
||||
}
|
||||
resp, err := d.getUserInfo(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
d.UID = resp.Data.UID
|
||||
return resp.Data.UID, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getFiles(parentFileId int64, limit int, lastFileId int64) (*FileListResp, error) {
|
||||
var resp FileListResp
|
||||
|
||||
_, err := d.Request(FileList, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(
|
||||
map[string]string{
|
||||
"parentFileId": strconv.FormatInt(parentFileId, 10),
|
||||
"limit": strconv.Itoa(limit),
|
||||
"lastFileId": strconv.FormatInt(lastFileId, 10),
|
||||
"trashed": "false",
|
||||
"searchMode": "",
|
||||
"searchData": "",
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getDownloadInfo(fileId int64) (*DownloadInfoResp, error) {
|
||||
var resp DownloadInfoResp
|
||||
|
||||
_, err := d.Request(DownloadInfo, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"fileId": strconv.FormatInt(fileId, 10),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getDirectLink(fileId int64) (*DirectLinkResp, error) {
|
||||
var resp DirectLinkResp
|
||||
|
||||
_, err := d.Request(DirectLink, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"fileID": strconv.FormatInt(fileId, 10),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) mkdir(parentID int64, name string) error {
|
||||
_, err := d.Request(Mkdir, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"parentID": strconv.FormatInt(parentID, 10),
|
||||
"name": name,
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) move(fileID, toParentFileID int64) error {
|
||||
_, err := d.Request(Move, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"fileIDs": []int64{fileID},
|
||||
"toParentFileID": toParentFileID,
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) rename(fileId int64, fileName string) error {
|
||||
_, err := d.Request(Rename, http.MethodPut, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"fileId": fileId,
|
||||
"fileName": fileName,
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) trash(fileId int64) error {
|
||||
_, err := d.Request(Trash, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"fileIDs": []int64{fileId},
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) createOfflineDownloadTask(ctx context.Context, url string, dirID, callback string) (taskID int, err error) {
|
||||
body := base.Json{
|
||||
"url": url,
|
||||
"dirID": dirID,
|
||||
}
|
||||
if len(callback) > 0 {
|
||||
body["callBackUrl"] = callback
|
||||
}
|
||||
var resp OfflineDownloadResp
|
||||
_, err = d.Request(OfflineDownload, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(body)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return resp.Data.TaskID, nil
|
||||
}
|
||||
|
||||
func (d *Open123) queryOfflineDownloadStatus(ctx context.Context, taskID int) (process float64, status int, err error) {
|
||||
var resp OfflineDownloadProcessResp
|
||||
_, err = d.Request(OfflineDownloadProcess, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"taskID": strconv.Itoa(taskID),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return .0, 0, err
|
||||
}
|
||||
return resp.Data.Process, resp.Data.Status, nil
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
package _139
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
//Account string `json:"account" required:"true"`
|
||||
Authorization string `json:"authorization" type:"text" help:"Authorization can be used alone. If empty, use username + password; mail_cookies is optional and will be established/updated automatically. Existing mail_cookies can also be used alone for fast login."`
|
||||
Username string `json:"username" help:"Use together with password when Authorization is empty. mail_cookies may be left empty on the first login."`
|
||||
Password string `json:"password" secret:"true" help:"Use together with username when Authorization is empty. mail_cookies may be left empty on the first login."`
|
||||
SmsCode string `json:"sms_code" secret:"true" help:"Fill this only after OpenList reports that a 139 Mail SMS verification code was sent, then save the storage again."`
|
||||
MailCookies string `json:"mail_cookies" type:"text" help:"Optional cookies from mail.10086.cn. Leave empty for a first username/password login; cookies created or updated by password/SMS login are persisted and reused as device context. Existing cookies may also be used alone for fast login."`
|
||||
driver.RootID
|
||||
Type string `json:"type" type:"select" options:"personal_new,family,group,personal,share" default:"personal_new"`
|
||||
LinkID string `json:"link_id" type:"text" help:"Multiple shares are separated by commas or new lines. Use link_id#password for password-protected shares."`
|
||||
CloudID string `json:"cloud_id"`
|
||||
UserDomainID string `json:"user_domain_id" help:"ud_id in Cookie, fill in to show disk usage"`
|
||||
CustomUploadPartSize int64 `json:"custom_upload_part_size" type:"number" default:"0" help:"0 for auto"`
|
||||
ReportRealSize bool `json:"report_real_size" type:"bool" default:"true" help:"Enable to report the real file size during upload"`
|
||||
UseLargeThumbnail bool `json:"use_large_thumbnail" type:"bool" default:"false" help:"Enable to use large thumbnail for images"`
|
||||
UseOldStreamUpload bool `json:"use_old_stream_upload" type:"bool" default:"false" help:"Enable to use old stream upload method (not support rapid upload)"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "139Yun",
|
||||
LocalSort: true,
|
||||
ProxyRangeOption: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
d := &Yun139{}
|
||||
d.ProxyRange = true
|
||||
return d
|
||||
})
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
package _139
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestShareEntriesAndRefEncoding(t *testing.T) {
|
||||
d := &Yun139{Addition: Addition{
|
||||
Type: MetaShare,
|
||||
LinkID: "share-a,share-b,share-c#pass",
|
||||
}}
|
||||
|
||||
entries := d.shareEntries()
|
||||
if len(entries) != 3 {
|
||||
t.Fatalf("expected 3 share entries, got %d", len(entries))
|
||||
}
|
||||
if entries[2].LinkID != "share-c" || entries[2].Password != "pass" {
|
||||
t.Fatalf("unexpected password share entry: %+v", entries[2])
|
||||
}
|
||||
|
||||
refs := []shareRef{
|
||||
{LinkID: entries[0].LinkID, Password: entries[0].Password, NodeID: "root-a"},
|
||||
{LinkID: entries[2].LinkID, Password: entries[2].Password, NodeID: "root-b"},
|
||||
}
|
||||
encoded := encodeShareRefs(refs)
|
||||
decoded, ok := decodeShareRefs(encoded)
|
||||
if !ok || len(decoded) != len(refs) {
|
||||
t.Fatalf("failed to decode merged share refs: %q", encoded)
|
||||
}
|
||||
for i := range refs {
|
||||
if decoded[i] != refs[i] {
|
||||
t.Fatalf("unexpected decoded ref at %d: %+v", i, decoded[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
-2089
File diff suppressed because it is too large
Load Diff
@@ -1,474 +0,0 @@
|
||||
package _139
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
func useRetryingTestClient(t *testing.T) {
|
||||
t.Helper()
|
||||
oldClient := base.RestyClient
|
||||
base.RestyClient = resty.New().SetRetryCount(3)
|
||||
t.Cleanup(func() {
|
||||
base.RestyClient = oldClient
|
||||
})
|
||||
}
|
||||
|
||||
func TestMergeMailCookiesPreservesExistingOrderAndAppendsNewNames(t *testing.T) {
|
||||
got := mergeMailCookieHeader("z=zv; behaviorid=b; Os_SSo_Sid=old", []*http.Cookie{
|
||||
{Name: "RMKEY", Value: "rm"},
|
||||
{Name: "Os_SSo_Sid", Value: "sid"},
|
||||
{Name: "a", Value: "av"},
|
||||
})
|
||||
want := "z=zv;behaviorid=b;Os_SSo_Sid=sid;RMKEY=rm;a=av"
|
||||
if got != want {
|
||||
t.Fatalf("mergeMailCookieHeader() = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractFastLoginCookies(t *testing.T) {
|
||||
sid, rmkey := extractFastLoginCookies("RMKEY=rm; Os_SSo_Sid=sid")
|
||||
if sid != "sid" || rmkey != "rm" {
|
||||
t.Fatalf("extractFastLoginCookies() = %q, %q; want sid, rm", sid, rmkey)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCredentialState(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
d Yun139
|
||||
want credentialState
|
||||
err bool
|
||||
}{
|
||||
{
|
||||
name: "authorization",
|
||||
d: Yun139{Addition: Addition{Authorization: " auth "}},
|
||||
want: credentialStateAuthorization,
|
||||
},
|
||||
{
|
||||
name: "full login",
|
||||
d: Yun139{Addition: Addition{
|
||||
MailCookies: "RMKEY=rm; Os_SSo_Sid=sid",
|
||||
Username: "user",
|
||||
Password: "password",
|
||||
}},
|
||||
want: credentialStateFullLogin,
|
||||
},
|
||||
{
|
||||
name: "full login without initial cookies",
|
||||
d: Yun139{Addition: Addition{
|
||||
Username: "user",
|
||||
Password: "password",
|
||||
}},
|
||||
want: credentialStateFullLogin,
|
||||
},
|
||||
{
|
||||
name: "cookies only",
|
||||
d: Yun139{Addition: Addition{MailCookies: "RMKEY=rm; Os_SSo_Sid=sid"}},
|
||||
want: credentialStateCookiesOnly,
|
||||
},
|
||||
{
|
||||
name: "partial password login",
|
||||
d: Yun139{Addition: Addition{Username: "user"}},
|
||||
err: true,
|
||||
},
|
||||
{
|
||||
name: "missing credentials",
|
||||
d: Yun139{},
|
||||
err: true,
|
||||
},
|
||||
{
|
||||
name: "invalid cookie",
|
||||
d: Yun139{Addition: Addition{MailCookies: "invalid-cookie"}},
|
||||
err: true,
|
||||
},
|
||||
{
|
||||
name: "authorization with basic prefix",
|
||||
d: Yun139{Addition: Addition{Authorization: "Basic abc"}},
|
||||
err: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := tt.d.credentialState()
|
||||
if tt.err {
|
||||
if err == nil {
|
||||
t.Fatal("credentialState() expected error")
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("credentialState() unexpected error: %v", err)
|
||||
}
|
||||
if got != tt.want {
|
||||
t.Fatalf("credentialState() = %v, want %v", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntegrationLoginObtainsAuthorization(t *testing.T) {
|
||||
if os.Getenv("OPENLIST_139_INTEGRATION") != "1" {
|
||||
t.Skip("set OPENLIST_139_INTEGRATION=1 to run live 139Yun login checks")
|
||||
}
|
||||
base.RestyClient = resty.New().
|
||||
SetHeader("user-agent", base.UserAgent).
|
||||
SetRetryCount(3).
|
||||
SetRetryResetReaders(true).
|
||||
SetTimeout(30 * time.Second)
|
||||
|
||||
username := os.Getenv("OPENLIST_139_USERNAME")
|
||||
password := os.Getenv("OPENLIST_139_PASSWORD")
|
||||
mailCookies := os.Getenv("OPENLIST_139_MAIL_COOKIES")
|
||||
authorization := strings.TrimSpace(os.Getenv("OPENLIST_139_AUTHORIZATION"))
|
||||
|
||||
if authorization != "" {
|
||||
t.Run("authorization", func(t *testing.T) {
|
||||
d := Yun139{Addition: Addition{Authorization: authorization}}
|
||||
state, err := d.credentialState()
|
||||
if err != nil {
|
||||
t.Fatalf("credentialState() unexpected error: %v", err)
|
||||
}
|
||||
if state != credentialStateAuthorization {
|
||||
t.Fatalf("credentialState() = %v, want authorization", state)
|
||||
}
|
||||
if d.Authorization == "" || strings.HasPrefix(strings.ToLower(d.Authorization), "basic ") {
|
||||
t.Fatal("authorization should be present without Basic prefix")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
if mailCookies == "" {
|
||||
t.Fatal("OPENLIST_139_MAIL_COOKIES is required")
|
||||
}
|
||||
|
||||
runFastLogin := func(mailCookies string) error {
|
||||
d := Yun139{Addition: Addition{MailCookies: mailCookies}}
|
||||
state, err := d.credentialState()
|
||||
if err != nil {
|
||||
return fmt.Errorf("credentialState() error: %w", err)
|
||||
}
|
||||
if state != credentialStateCookiesOnly {
|
||||
return fmt.Errorf("credentialState() = %v, want cookies only", state)
|
||||
}
|
||||
sid, rmkey := extractFastLoginCookies(d.MailCookies)
|
||||
if sid == "" || rmkey == "" {
|
||||
return fmt.Errorf("mail cookies are missing Os_SSo_Sid or RMKEY")
|
||||
}
|
||||
token, err := d.step2_get_single_token(sid)
|
||||
if err != nil {
|
||||
return fmt.Errorf("step2_get_single_token() error: %w", err)
|
||||
}
|
||||
auth, err := d.step3_third_party_login(token)
|
||||
if err != nil {
|
||||
return fmt.Errorf("step3_third_party_login() error: %w", err)
|
||||
}
|
||||
if auth == "" {
|
||||
return fmt.Errorf("authorization is empty after fast login")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
if username == "" || password == "" {
|
||||
t.Fatal("OPENLIST_139_USERNAME and OPENLIST_139_PASSWORD are required for password fallback")
|
||||
}
|
||||
|
||||
var refreshedMailCookies string
|
||||
var generatedAuthorization string
|
||||
t.Run("password login fallback", func(t *testing.T) {
|
||||
d := Yun139{Addition: Addition{
|
||||
MailCookies: mailCookies,
|
||||
Username: username,
|
||||
Password: password,
|
||||
}}
|
||||
state, err := d.credentialState()
|
||||
if err != nil {
|
||||
t.Fatalf("credentialState() unexpected error: %v", err)
|
||||
}
|
||||
if state != credentialStateFullLogin {
|
||||
t.Fatalf("credentialState() = %v, want full login", state)
|
||||
}
|
||||
passId, err := d.step1_password_login()
|
||||
if err != nil {
|
||||
t.Fatalf("step1_password_login() error: %v", err)
|
||||
}
|
||||
token, err := d.step2_get_single_token(passId)
|
||||
if err != nil {
|
||||
t.Fatalf("step2_get_single_token() error: %v", err)
|
||||
}
|
||||
auth, err := d.step3_third_party_login(token)
|
||||
if err != nil {
|
||||
t.Fatalf("step3_third_party_login() error: %v", err)
|
||||
}
|
||||
d.Authorization = auth
|
||||
if auth == "" || d.Authorization == "" {
|
||||
t.Fatal("authorization is empty after password login")
|
||||
}
|
||||
generatedAuthorization = auth
|
||||
refreshedMailCookies = d.MailCookies
|
||||
})
|
||||
|
||||
t.Run("authorization generated by password login", func(t *testing.T) {
|
||||
if generatedAuthorization == "" {
|
||||
t.Fatal("password login did not generate authorization")
|
||||
}
|
||||
d := Yun139{Addition: Addition{Authorization: generatedAuthorization}}
|
||||
state, err := d.credentialState()
|
||||
if err != nil {
|
||||
t.Fatalf("credentialState() unexpected error: %v", err)
|
||||
}
|
||||
if state != credentialStateAuthorization {
|
||||
t.Fatalf("credentialState() = %v, want authorization", state)
|
||||
}
|
||||
if strings.HasPrefix(strings.ToLower(d.Authorization), "basic ") {
|
||||
t.Fatal("authorization should not include Basic prefix")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("mail cookies fast login from input", func(t *testing.T) {
|
||||
sid, rmkey := extractFastLoginCookies(mailCookies)
|
||||
if sid == "" || rmkey == "" {
|
||||
t.Skip("input mail cookies are missing Os_SSo_Sid or RMKEY")
|
||||
}
|
||||
if err := runFastLogin(mailCookies); err != nil {
|
||||
t.Skipf("input mail cookies are stale or unusable: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("mail cookies fast login after password login", func(t *testing.T) {
|
||||
if refreshedMailCookies == "" {
|
||||
t.Fatal("password login did not refresh mail cookies")
|
||||
}
|
||||
if err := runFastLogin(refreshedMailCookies); err != nil {
|
||||
t.Fatalf("refreshed mail cookies fast login failed: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestInvalidAuthorizationDoesNotUseCookieFastLogin(t *testing.T) {
|
||||
d := Yun139{Addition: Addition{
|
||||
Authorization: "not-base64",
|
||||
MailCookies: "Os_SSo_Sid=sid; RMKEY=rmkey",
|
||||
}}
|
||||
|
||||
err := d.refreshToken()
|
||||
if err == nil || !strings.Contains(err.Error(), "password login failed") {
|
||||
t.Fatalf("refreshToken() error = %v, want password login fallback error", err)
|
||||
}
|
||||
if d.Authorization != "not-base64" {
|
||||
t.Fatalf("Authorization = %q, want original invalid value retained", d.Authorization)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSMSSceneForRisk(t *testing.T) {
|
||||
tests := map[string]int{
|
||||
"S025": 1,
|
||||
"S035": 1,
|
||||
"PML401010062": 2,
|
||||
"MW0016": 4,
|
||||
}
|
||||
for riskCode, want := range tests {
|
||||
got, ok := smsSceneForRisk(riskCode)
|
||||
if !ok || got != want {
|
||||
t.Fatalf("smsSceneForRisk(%q) = %d, %t; want %d, true", riskCode, got, ok, want)
|
||||
}
|
||||
}
|
||||
if _, ok := smsSceneForRisk("PICTURE_ONLY"); ok {
|
||||
t.Fatal("smsSceneForRisk() accepted unsupported risk code")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSendSMSVerificationCode(t *testing.T) {
|
||||
useRetryingTestClient(t)
|
||||
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, err := io.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
t.Fatalf("read request body: %v", err)
|
||||
}
|
||||
if r.URL.Query().Get("func") != "login:sendSmsCodeByScene" {
|
||||
t.Errorf("func = %q", r.URL.Query().Get("func"))
|
||||
}
|
||||
if !strings.Contains(string(body), `<string name="scene">1</string>`) {
|
||||
t.Errorf("request body = %s", body)
|
||||
}
|
||||
if !strings.Contains(r.Header.Get("Cookie"), "device=fingerprint") {
|
||||
t.Errorf("Cookie = %q", r.Header.Get("Cookie"))
|
||||
}
|
||||
http.SetCookie(w, &http.Cookie{Name: "challenge", Value: "sms"})
|
||||
_, _ = io.WriteString(w, `{"code":"S_OK"}`)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
oldURL := mailSMSURL
|
||||
mailSMSURL = server.URL
|
||||
defer func() { mailSMSURL = oldURL }()
|
||||
|
||||
d := Yun139{Addition: Addition{Username: "18800000000", MailCookies: "device=fingerprint"}}
|
||||
if err := d.sendSMSVerificationCode("S025"); err != nil {
|
||||
t.Fatalf("sendSMSVerificationCode() error = %v", err)
|
||||
}
|
||||
if !strings.Contains(d.MailCookies, "challenge=sms") {
|
||||
t.Fatalf("MailCookies = %q", d.MailCookies)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSendSMSVerificationCodeStopsAtPictureChallenge(t *testing.T) {
|
||||
useRetryingTestClient(t)
|
||||
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
_, _ = io.WriteString(w, `{"code":"PML401010021"}`)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
oldURL := mailSMSURL
|
||||
mailSMSURL = server.URL
|
||||
defer func() { mailSMSURL = oldURL }()
|
||||
|
||||
d := Yun139{Addition: Addition{Username: "18800000000"}}
|
||||
err := d.sendSMSVerificationCode("S025")
|
||||
if err == nil || !strings.Contains(err.Error(), "requires picture verification") {
|
||||
t.Fatalf("sendSMSVerificationCode() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVerifySMSCode(t *testing.T) {
|
||||
useRetryingTestClient(t)
|
||||
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, err := io.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
t.Fatalf("read request body: %v", err)
|
||||
}
|
||||
if r.URL.Query().Get("func") != "/login/inlogin.action" {
|
||||
t.Errorf("func = %q", r.URL.Query().Get("func"))
|
||||
}
|
||||
wantHash := sha1Hash("fetion.com.cn:123456")
|
||||
if !strings.Contains(string(body), `<string name="loginPassword">`+wantHash+`</string>`) {
|
||||
t.Errorf("request body does not contain SMS code hash: %s", body)
|
||||
}
|
||||
http.SetCookie(w, &http.Cookie{Name: "RMKEY", Value: "new-rmkey"})
|
||||
_, _ = io.WriteString(w, `{"code":"S_OK","var":{"loginSuccessUrl":"https://mail.10086.cn/?sid=sms-sid"}}`)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
oldURL := mailSMSURL
|
||||
mailSMSURL = server.URL
|
||||
defer func() { mailSMSURL = oldURL }()
|
||||
|
||||
d := Yun139{Addition: Addition{
|
||||
Username: "18800000000",
|
||||
SmsCode: "123456",
|
||||
MailCookies: "challenge=sms",
|
||||
}}
|
||||
sid, err := d.verifySMSCode("S025")
|
||||
if err != nil {
|
||||
t.Fatalf("verifySMSCode() error = %v", err)
|
||||
}
|
||||
if sid != "sms-sid" {
|
||||
t.Fatalf("sid = %q", sid)
|
||||
}
|
||||
if d.SmsCode != "" {
|
||||
t.Fatalf("SmsCode = %q, want cleared", d.SmsCode)
|
||||
}
|
||||
if !strings.Contains(d.MailCookies, "RMKEY=new-rmkey") {
|
||||
t.Fatalf("MailCookies = %q", d.MailCookies)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPasswordLoginWithoutInitialCookiesStopsAtRedirectAndPersistsCookies(t *testing.T) {
|
||||
useRetryingTestClient(t)
|
||||
|
||||
var loginRequests, redirectRequests int
|
||||
var server *httptest.Server
|
||||
server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/Login/Login.ashx":
|
||||
loginRequests++
|
||||
http.SetCookie(w, &http.Cookie{Name: "RMKEY", Value: "new-rm"})
|
||||
http.SetCookie(w, &http.Cookie{Name: "Os_SSo_Sid", Value: "new-sid"})
|
||||
w.Header().Set("Location", server.URL+"/appmail?sid=single-sid")
|
||||
w.WriteHeader(http.StatusFound)
|
||||
case "/appmail":
|
||||
redirectRequests++
|
||||
w.WriteHeader(http.StatusOK)
|
||||
default:
|
||||
http.NotFound(w, r)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
oldURL := mailPasswordURL
|
||||
mailPasswordURL = server.URL + "/Login/Login.ashx"
|
||||
defer func() { mailPasswordURL = oldURL }()
|
||||
|
||||
d := Yun139{Addition: Addition{Username: "18800000000", Password: "password"}}
|
||||
sid, err := d.step1_password_login()
|
||||
if err != nil || sid != "single-sid" {
|
||||
t.Fatalf("sid=%q err=%v", sid, err)
|
||||
}
|
||||
if loginRequests != 1 || redirectRequests != 0 {
|
||||
t.Fatalf("login/redirect requests=%d/%d, want 1/0", loginRequests, redirectRequests)
|
||||
}
|
||||
if !strings.Contains(d.MailCookies, "RMKEY=new-rm") || !strings.Contains(d.MailCookies, "Os_SSo_Sid=new-sid") {
|
||||
t.Fatalf("response cookies were not persisted: %q", d.MailCookies)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPasswordLoginReusesRawRefreshedCookies(t *testing.T) {
|
||||
useRetryingTestClient(t)
|
||||
|
||||
var n int
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
n++
|
||||
cookie := r.Header.Get("Cookie")
|
||||
if !strings.Contains(cookie, "JSESSIONID=stale") || !strings.Contains(cookie, "behaviorid=device") {
|
||||
t.Errorf("login %d lost raw cookie context: %q", n, cookie)
|
||||
}
|
||||
if n == 1 {
|
||||
http.SetCookie(w, &http.Cookie{Name: "RMKEY", Value: "rm-1"})
|
||||
http.SetCookie(w, &http.Cookie{Name: "Os_SSo_Sid", Value: "sid-1"})
|
||||
w.Header().Set("Location", "https://mail.10086.cn/?sid=first")
|
||||
} else {
|
||||
if !strings.Contains(cookie, "RMKEY=rm-1") || !strings.Contains(cookie, "Os_SSo_Sid=sid-1") {
|
||||
t.Errorf("second login did not reuse first refreshed cookies: %q", cookie)
|
||||
}
|
||||
http.SetCookie(w, &http.Cookie{Name: "RMKEY", Value: "rm-2"})
|
||||
http.SetCookie(w, &http.Cookie{Name: "Os_SSo_Sid", Value: "sid-2"})
|
||||
w.Header().Set("Location", "https://mail.10086.cn/?sid=second")
|
||||
}
|
||||
w.WriteHeader(http.StatusFound)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
oldURL := mailPasswordURL
|
||||
mailPasswordURL = server.URL
|
||||
defer func() { mailPasswordURL = oldURL }()
|
||||
|
||||
d := Yun139{Addition: Addition{Username: "18800000000", Password: "password", MailCookies: "Os_SSo_Sid=old; RMKEY=old; JSESSIONID=stale; behaviorid=device"}}
|
||||
if sid, err := d.step1_password_login(); err != nil || sid != "first" {
|
||||
t.Fatalf("first login sid=%q err=%v", sid, err)
|
||||
}
|
||||
refreshed := d.MailCookies
|
||||
if !strings.Contains(refreshed, "RMKEY=rm-1") || !strings.Contains(refreshed, "Os_SSo_Sid=sid-1") {
|
||||
t.Fatalf("first login did not persist refreshed cookies: %q", refreshed)
|
||||
}
|
||||
d.Authorization = ""
|
||||
d.MailCookies = refreshed
|
||||
if sid, err := d.step1_password_login(); err != nil || sid != "second" {
|
||||
t.Fatalf("second login sid=%q err=%v", sid, err)
|
||||
}
|
||||
if n != 2 {
|
||||
t.Fatalf("login count=%d, want 2", n)
|
||||
}
|
||||
}
|
||||
@@ -1,149 +0,0 @@
|
||||
package _189
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
|
||||
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
|
||||
// 计算 sliceMD5
|
||||
sliceMD5 := fileMD5
|
||||
if len(sliceMD5s) > 1 {
|
||||
joined := strings.Join(sliceMD5s, "\n")
|
||||
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
|
||||
}
|
||||
|
||||
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
|
||||
t.Info.PieceLength = sliceSize
|
||||
t.SetPieces(pieceHashes)
|
||||
t.SetCASInfo(&torrent.CASInfo{
|
||||
FileMD5: fileMD5,
|
||||
SliceMD5: sliceMD5,
|
||||
SliceMD5s: sliceMD5s,
|
||||
SliceSize: sliceSize,
|
||||
Cloud: "189",
|
||||
})
|
||||
|
||||
return t.Encode()
|
||||
}
|
||||
|
||||
// RapidUploadFromTorrent 从 torrent 文件中提取 CAS 信息进行秒传
|
||||
func (d *Cloud189) RapidUploadFromTorrent(ctx context.Context, dstDir model.Obj, torrentData []byte) error {
|
||||
// 解析 torrent
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
// 检查是否包含 CAS 扩展信息
|
||||
if !t.HasCASInfo() {
|
||||
return fmt.Errorf("torrent 不包含 CAS 扩展信息，无法秒传")
|
||||
}
|
||||
|
||||
cas := t.CAS
|
||||
fileName := t.Info.Name
|
||||
fileSize := t.GetTotalSize()
|
||||
|
||||
// 获取 sessionKey
|
||||
sessionKey, err := d.getSessionKey()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.sessionKey = sessionKey
|
||||
|
||||
// 初始化上传
|
||||
res, err := d.uploadRequest("/person/initMultiUpload", map[string]string{
|
||||
"parentFolderId": dstDir.GetID(),
|
||||
"fileName": encode(fileName),
|
||||
"fileSize": fmt.Sprint(fileSize),
|
||||
"sliceSize": fmt.Sprint(cas.SliceSize),
|
||||
"lazyCheck": "1",
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("初始化上传失败: %w", err)
|
||||
}
|
||||
|
||||
uploadFileId := utils.Json.Get(res, "data", "uploadFileId").ToString()
|
||||
|
||||
// 提交上传（使用 CAS 信息秒传）
|
||||
_, err = d.uploadRequest("/person/commitMultiUploadFile", map[string]string{
|
||||
"uploadFileId": uploadFileId,
|
||||
"fileMd5": cas.FileMD5,
|
||||
"sliceMd5": cas.SliceMD5,
|
||||
"lazyCheck": "1",
|
||||
"opertype": "3",
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("秒传提交失败: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
|
||||
func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64, sliceSize int64) ([]byte, error) {
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = torrent.DefaultPieceSize
|
||||
}
|
||||
|
||||
hw := torrent.NewHashWriter(sliceSize, sliceSize)
|
||||
|
||||
buf := make([]byte, 32*1024)
|
||||
for {
|
||||
n, err := reader.Read(buf)
|
||||
if n > 0 {
|
||||
hw.Write(buf[:n])
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
hw.Finish()
|
||||
|
||||
fileMD5 := hw.GetFileMD5()
|
||||
sliceMD5s := hw.GetSliceMD5s()
|
||||
pieceHashes := hw.GetPieceHashes()
|
||||
|
||||
return GenerateTorrent(fileName, fileSize, fileMD5, sliceMD5s, sliceSize, pieceHashes)
|
||||
}
|
||||
|
||||
// ComputePieceSHA1 计算单个分片的 SHA-1 哈希
|
||||
func ComputePieceSHA1(data []byte) []byte {
|
||||
h := sha1.Sum(data)
|
||||
return h[:]
|
||||
}
|
||||
|
||||
// ExtractCASFromTorrent 从 torrent 数据中提取 CAS 信息
|
||||
func ExtractCASFromTorrent(torrentData []byte) (*torrent.CASInfo, string, int64, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, "", 0, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
if !t.HasCASInfo() {
|
||||
return nil, "", 0, fmt.Errorf("torrent 不包含 CAS 扩展信息")
|
||||
}
|
||||
|
||||
return t.CAS, t.Info.Name, t.GetTotalSize(), nil
|
||||
}
|
||||
|
||||
// GetInfoHashHex 获取 torrent 的 info_hash（十六进制字符串）
|
||||
func GetInfoHashHex(torrentData []byte) (string, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(t.InfoHash), nil
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
package _189_tv
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/xml"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestTimeUnmarshal(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
want time.Time
|
||||
}{
|
||||
{"numeric date", `"2026-08-11 10:37:18"`, time.Date(2026, 8, 11, 10, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"legacy month date", `"Aug 11, 2026 10:37:18 PM"`, time.Date(2026, 8, 11, 22, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"new format with tz", `"Aug 11, 2026, 10:37:18 PM +08"`, time.Date(2026, 8, 11, 22, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"new format no tz", `"Aug 11, 2026, 10:37:18 PM"`, time.Date(2026, 8, 11, 22, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"narrow no-break space (U+202F)", "\"Aug 12, 2026, 12:35:41\u202fAM +08\"", time.Date(2026, 8, 12, 0, 35, 41, 0, time.FixedZone("", 8*3600))},
|
||||
{"no-break space (U+00A0)", "\"Aug 12, 2026, 12:35:41\u00a0AM +08\"", time.Date(2026, 8, 12, 0, 35, 41, 0, time.FixedZone("", 8*3600))},
|
||||
{"JSON escaped narrow space", `"Sep 4, 2026, 10:59:33\u202fPM +08"`, time.Date(2026, 9, 4, 22, 59, 33, 0, time.FixedZone("", 8*3600))},
|
||||
{"JSON escaped space without timezone", `"Sep 4, 2026, 12:59:33\u00a0AM"`, time.Date(2026, 9, 4, 0, 59, 33, 0, time.FixedZone("", 8*3600))},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var tm Time
|
||||
if err := json.Unmarshal([]byte(tt.input), &tm); err != nil {
|
||||
t.Fatalf("Unmarshal(%s) error: %v", tt.input, err)
|
||||
}
|
||||
if !tt.want.Equal(time.Time(tm)) {
|
||||
t.Fatalf("Unmarshal(%s) = %v, want %v", tt.input, time.Time(tm), tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeUnmarshalRejectsInvalid(t *testing.T) {
|
||||
var tm Time
|
||||
if err := tm.Unmarshal([]byte("Aug 12, 2026, 25:32:44 AM")); err == nil {
|
||||
t.Fatal("Unmarshal accepted an invalid time")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeUnmarshalJSONRejectsInvalid(t *testing.T) {
|
||||
for _, input := range []string{`"invalid"`, `123`, `null`, `"Sep 4, 2026, 10:59:33\uZZZZPM +08"`} {
|
||||
t.Run(input, func(t *testing.T) {
|
||||
var tm Time
|
||||
if err := tm.UnmarshalJSON([]byte(input)); err == nil {
|
||||
t.Fatalf("UnmarshalJSON(%s) accepted invalid input", input)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeUnmarshalXML(t *testing.T) {
|
||||
var tm Time
|
||||
if err := xml.Unmarshal([]byte(`<time>Sep 4, 2026, 10:59:33 PM +08</time>`), &tm); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := time.Date(2026, 9, 4, 22, 59, 33, 0, time.FixedZone("", 8*3600))
|
||||
if !want.Equal(time.Time(tm)) {
|
||||
t.Fatalf("UnmarshalXML = %v, want %v", time.Time(tm), want)
|
||||
}
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
package _189pc
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"encoding/xml"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestTimeUnmarshal(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
want time.Time
|
||||
}{
|
||||
{"numeric date", `"2026-08-11 10:37:18"`, time.Date(2026, 8, 11, 10, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"legacy month date", `"Aug 11, 2026 10:37:18 PM"`, time.Date(2026, 8, 11, 22, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"new format with tz", `"Aug 11, 2026, 10:37:18 PM +08"`, time.Date(2026, 8, 11, 22, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"new format no tz", `"Aug 11, 2026, 10:37:18 PM"`, time.Date(2026, 8, 11, 22, 37, 18, 0, time.FixedZone("", 8*3600))},
|
||||
{"narrow no-break space (U+202F)", "\"Aug 12, 2026, 12:35:41\u202fAM +08\"", time.Date(2026, 8, 12, 0, 35, 41, 0, time.FixedZone("", 8*3600))},
|
||||
{"no-break space (U+00A0)", "\"Aug 12, 2026, 12:35:41\u00a0AM +08\"", time.Date(2026, 8, 12, 0, 35, 41, 0, time.FixedZone("", 8*3600))},
|
||||
{"JSON escaped narrow space", `"Sep 4, 2026, 10:59:33\u202fPM +08"`, time.Date(2026, 9, 4, 22, 59, 33, 0, time.FixedZone("", 8*3600))},
|
||||
{"JSON escaped space without timezone", `"Sep 4, 2026, 12:59:33\u00a0AM"`, time.Date(2026, 9, 4, 0, 59, 33, 0, time.FixedZone("", 8*3600))},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var tm Time
|
||||
if err := json.Unmarshal([]byte(tt.input), &tm); err != nil {
|
||||
t.Fatalf("Unmarshal(%s) error: %v", tt.input, err)
|
||||
}
|
||||
if !tt.want.Equal(time.Time(tm)) {
|
||||
t.Fatalf("Unmarshal(%s) = %v, want %v", tt.input, time.Time(tm), tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeUnmarshalRejectsInvalid(t *testing.T) {
|
||||
var tm Time
|
||||
if err := tm.Unmarshal([]byte("Aug 12, 2026, 25:32:44 AM")); err == nil {
|
||||
t.Fatal("Unmarshal accepted an invalid time")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeUnmarshalJSONRejectsInvalid(t *testing.T) {
|
||||
for _, input := range []string{`"invalid"`, `123`, `null`, `"Sep 4, 2026, 10:59:33\uZZZZPM +08"`} {
|
||||
t.Run(input, func(t *testing.T) {
|
||||
var tm Time
|
||||
if err := tm.UnmarshalJSON([]byte(input)); err == nil {
|
||||
t.Fatalf("UnmarshalJSON(%s) accepted invalid input", input)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeUnmarshalXML(t *testing.T) {
|
||||
var tm Time
|
||||
if err := xml.Unmarshal([]byte(`<time>Sep 4, 2026, 10:59:33 PM +08</time>`), &tm); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := time.Date(2026, 9, 4, 22, 59, 33, 0, time.FixedZone("", 8*3600))
|
||||
if !want.Equal(time.Time(tm)) {
|
||||
t.Fatalf("UnmarshalXML = %v, want %v", time.Time(tm), want)
|
||||
}
|
||||
}
|
||||
@@ -1,43 +0,0 @@
|
||||
package _189pc
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
LoginType string `json:"login_type" type:"select" options:"password,qrcode" default:"password" required:"true"`
|
||||
Username string `json:"username" help:"Not needed when an access token or refresh token is provided"`
|
||||
Password string `json:"password" help:"Not needed when an access token or refresh token is provided"`
|
||||
VCode string `json:"validate_code"`
|
||||
SmsCode string `json:"sms_code" help:"SMS code for the second device verification, fill it in and save again when login asks for it"`
|
||||
AccessToken string `json:"access_token" required:"false"`
|
||||
RefreshToken string `json:"refresh_token" help:"To switch accounts, please clear this field"`
|
||||
DeviceID string `json:"device_id" help:"DEVICEID cookie issued after the second device verification, keep it to avoid verifying again"`
|
||||
ClientSn string `json:"client_sn" help:"Device serial number captured from the official client, leave it empty if you do not have one"`
|
||||
JgOpenId string `json:"jg_open_id" help:"Optional push id reported by the official client"`
|
||||
UserFinger string `json:"user_finger" help:"Device fingerprint sent with login requests, generated and kept automatically when empty"`
|
||||
driver.RootID
|
||||
OrderBy string `json:"order_by" type:"select" options:"filename,filesize,lastOpTime" default:"filename"`
|
||||
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc" default:"asc"`
|
||||
Type string `json:"type" type:"select" options:"personal,family" default:"personal"`
|
||||
FamilyID string `json:"family_id"`
|
||||
UploadMethod string `json:"upload_method" type:"select" options:"stream,rapid,old" default:"stream"`
|
||||
UploadThread string `json:"upload_thread" default:"3" help:"1<=thread<=32"`
|
||||
FamilyTransfer bool `json:"family_transfer"`
|
||||
RapidUpload bool `json:"rapid_upload"`
|
||||
NoUseOcr bool `json:"no_use_ocr"`
|
||||
GenerateTorrent bool `json:"generate_torrent" help:"Generate torrent file with CAS extension after upload"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "189CloudPC",
|
||||
DefaultRoot: "-11",
|
||||
CheckStatus: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Cloud189PC{}
|
||||
})
|
||||
}
|
||||
@@ -1,449 +0,0 @@
|
||||
package _189pc
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-resty/resty/v2"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
|
||||
// fileMD5: 整文件 MD5（大写十六进制）
|
||||
// sliceMD5s: 每个分片的 MD5 列表（大写十六进制）
|
||||
// sliceSize: 分片大小
|
||||
// pieceHashes: SHA-1 piece hashes 拼接（每 20 字节一个）
|
||||
// fileName: 文件名
|
||||
// fileSize: 文件大小
|
||||
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
|
||||
// 计算 sliceMD5
|
||||
sliceMD5 := fileMD5
|
||||
if len(sliceMD5s) > 1 {
|
||||
joined := strings.Join(sliceMD5s, "\n")
|
||||
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
|
||||
}
|
||||
|
||||
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
|
||||
t.Info.PieceLength = sliceSize
|
||||
t.SetPieces(pieceHashes)
|
||||
t.SetCASInfo(&torrent.CASInfo{
|
||||
FileMD5: fileMD5,
|
||||
SliceMD5: sliceMD5,
|
||||
SliceMD5s: sliceMD5s,
|
||||
SliceSize: sliceSize,
|
||||
Cloud: "189",
|
||||
})
|
||||
|
||||
return t.Encode()
|
||||
}
|
||||
|
||||
// RapidUploadFromTorrent 从 torrent 文件中提取 CAS 信息进行秒传
|
||||
// 返回值：上传成功的文件对象、错误
|
||||
func (y *Cloud189PC) RapidUploadFromTorrent(ctx context.Context, dstDir model.Obj, torrentData []byte, overwrite bool) (model.Obj, error) {
|
||||
isFamily := y.isFamily()
|
||||
|
||||
// 解析 torrent
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
// 检查是否包含 CAS 扩展信息
|
||||
if !t.HasCASInfo() {
|
||||
return nil, fmt.Errorf("torrent 不包含 CAS 扩展信息，无法秒传")
|
||||
}
|
||||
|
||||
cas := t.CAS
|
||||
fileName := t.Info.Name
|
||||
fileSize := t.GetTotalSize()
|
||||
|
||||
// 统一 MD5 为大写（与正常上传保持一致，天翼云盘要求大写）
|
||||
fileMD5Upper := strings.ToUpper(cas.FileMD5)
|
||||
|
||||
// 优先使用 torrent 中嵌入的分片大小，与生成时保持一致
|
||||
sliceSize := cas.SliceSize
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = partSize(fileSize)
|
||||
}
|
||||
|
||||
// 计算 sliceMd5（与上传时一致的算法）
|
||||
// 优先使用 torrent 中已有的 SliceMD5；仅当有多分片列表时才重新计算
|
||||
sliceMd5Hex := strings.ToUpper(cas.SliceMD5)
|
||||
if sliceMd5Hex == "" {
|
||||
sliceMd5Hex = fileMD5Upper
|
||||
}
|
||||
if len(cas.SliceMD5s) > 1 {
|
||||
// 分片 MD5 也需要统一大写后再拼接计算
|
||||
upperSliceMD5s := make([]string, len(cas.SliceMD5s))
|
||||
for i, s := range cas.SliceMD5s {
|
||||
upperSliceMD5s[i] = strings.ToUpper(s)
|
||||
}
|
||||
sliceMd5Hex = strings.ToUpper(utils.GetMD5EncodeStr(strings.Join(upperSliceMD5s, "\n")))
|
||||
}
|
||||
|
||||
// 使用与 Web 端一致的三步秒传流程
|
||||
fullUrl := "https://upload.cloud.189.cn"
|
||||
if isFamily {
|
||||
fullUrl += "/family"
|
||||
} else {
|
||||
fullUrl += "/person"
|
||||
}
|
||||
|
||||
// Step 1: initMultiUpload（不传 fileMd5/sliceMd5，只传 lazyCheck）
|
||||
initParams := Params{
|
||||
"parentFolderId": dstDir.GetID(),
|
||||
"fileName": url.QueryEscape(fileName),
|
||||
"fileSize": fmt.Sprint(fileSize),
|
||||
"sliceSize": fmt.Sprint(sliceSize),
|
||||
"lazyCheck": "1",
|
||||
}
|
||||
if isFamily {
|
||||
initParams.Set("familyId", y.FamilyID)
|
||||
}
|
||||
|
||||
var uploadInfo InitMultiUploadResp
|
||||
_, err = y.request(fullUrl+"/initMultiUpload", "GET", func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, initParams, &uploadInfo, isFamily)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initMultiUpload 失败: %w", err)
|
||||
}
|
||||
|
||||
uploadFileId := uploadInfo.Data.UploadFileID
|
||||
|
||||
// Step 2: checkTransSecond（用 fileMd5 + sliceMd5 + uploadFileId 检查秒传）
|
||||
checkParams := Params{
|
||||
"fileMd5": fileMD5Upper,
|
||||
"sliceMd5": sliceMd5Hex,
|
||||
"uploadFileId": uploadFileId,
|
||||
}
|
||||
|
||||
var checkResp struct {
|
||||
Data struct {
|
||||
FileDataExists int `json:"fileDataExists"`
|
||||
} `json:"data"`
|
||||
}
|
||||
_, err = y.request(fullUrl+"/checkTransSecond", "GET", func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, checkParams, &checkResp, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Errorf("[RapidUpload] checkTransSecond 失败: uploadFileId=%s, err=%v", uploadFileId, err)
|
||||
return nil, fmt.Errorf("秒传检查失败: %w", err)
|
||||
}
|
||||
|
||||
if checkResp.Data.FileDataExists != 1 {
|
||||
return nil, fmt.Errorf("秒传失败：云端不存在该文件（fileMD5=%s, sliceMD5=%s, size=%d）", fileMD5Upper, sliceMd5Hex, fileSize)
|
||||
}
|
||||
|
||||
// Step 3: commitMultiUploadFile（传 fileMd5 + sliceMd5）
|
||||
|
||||
var resp CommitMultiUploadFileResp
|
||||
commitParams := Params{
|
||||
"uploadFileId": uploadFileId,
|
||||
"fileMd5": fileMD5Upper,
|
||||
"sliceMd5": sliceMd5Hex,
|
||||
"lazyCheck": "1",
|
||||
"opertype": IF(overwrite, "3", "1"),
|
||||
}
|
||||
|
||||
_, err = y.request(fullUrl+"/commitMultiUploadFile", "GET", func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, commitParams, &resp, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Errorf("[RapidUpload] commitMultiUploadFile 失败: uploadFileId=%s, err=%v", uploadFileId, err)
|
||||
return nil, fmt.Errorf("提交上传失败: %w", err)
|
||||
}
|
||||
|
||||
// 秒传成功后，将 torrent 文件上传到目标目录（异步，不影响秒传结果）
|
||||
if y.Addition.GenerateTorrent {
|
||||
capturedDstDir := dstDir
|
||||
capturedIsFamily := isFamily
|
||||
go func() {
|
||||
torrentName := fileName + ".cas.torrent"
|
||||
infoHash, _ := GetInfoHashHex(torrentData)
|
||||
utils.Log.Infof("秒传成功，上传 torrent: %s (info_hash: %s, size: %d bytes)",
|
||||
torrentName, infoHash, len(torrentData))
|
||||
|
||||
torrentFileStream := &stream.FileStream{
|
||||
Ctx: context.Background(),
|
||||
Obj: &model.Object{
|
||||
Name: torrentName,
|
||||
Size: int64(len(torrentData)),
|
||||
IsFolder: false,
|
||||
},
|
||||
Reader: bytes.NewReader(torrentData),
|
||||
Mimetype: "application/x-bittorrent",
|
||||
}
|
||||
_, uploadErr := y.fastUpload(context.Background(), capturedDstDir, torrentFileStream, func(p float64) {}, capturedIsFamily, false, false)
|
||||
if uploadErr != nil {
|
||||
utils.Log.Warnf("上传 torrent 文件失败: %v", uploadErr)
|
||||
} else {
|
||||
utils.Log.Infof("torrent 文件已上传: %s", torrentName)
|
||||
op.Cache.DeleteDirectory(y, capturedDstDir.GetPath())
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
return resp.toFile(), nil
|
||||
}
|
||||
|
||||
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
|
||||
// 适用于：已有文件需要生成 torrent 的场景（如下载完成后生成）
|
||||
func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64, sliceSize int64) ([]byte, error) {
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = torrent.DefaultPieceSize
|
||||
}
|
||||
|
||||
hw := torrent.NewHashWriter(sliceSize, sliceSize)
|
||||
|
||||
buf := make([]byte, 32*1024)
|
||||
for {
|
||||
n, err := reader.Read(buf)
|
||||
if n > 0 {
|
||||
hw.Write(buf[:n])
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
hw.Finish()
|
||||
|
||||
fileMD5 := hw.GetFileMD5()
|
||||
sliceMD5s := hw.GetSliceMD5s()
|
||||
pieceHashes := hw.GetPieceHashes()
|
||||
|
||||
return GenerateTorrent(fileName, fileSize, fileMD5, sliceMD5s, sliceSize, pieceHashes)
|
||||
}
|
||||
|
||||
// ComputePieceSHA1 计算单个分片的 SHA-1 哈希
|
||||
func ComputePieceSHA1(data []byte) []byte {
|
||||
h := sha1.Sum(data)
|
||||
return h[:]
|
||||
}
|
||||
|
||||
// ExtractCASFromTorrent 从 torrent 数据中提取 CAS 信息
|
||||
// 返回：CAS 信息、文件名、文件大小、错误
|
||||
func ExtractCASFromTorrent(torrentData []byte) (*torrent.CASInfo, string, int64, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, "", 0, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
if !t.HasCASInfo() {
|
||||
return nil, "", 0, fmt.Errorf("torrent 不包含 CAS 扩展信息")
|
||||
}
|
||||
|
||||
return t.CAS, t.Info.Name, t.GetTotalSize(), nil
|
||||
}
|
||||
|
||||
// InjectCASIntoTorrent 向已有的 torrent 文件注入 CAS 扩展信息
|
||||
// 用于：下载完成后，计算了 MD5 信息，写回到 torrent 中
|
||||
func InjectCASIntoTorrent(torrentData []byte, fileMD5 string, sliceMD5s []string, sliceSize int64) ([]byte, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
// 计算 sliceMD5
|
||||
sliceMD5 := fileMD5
|
||||
if len(sliceMD5s) > 1 {
|
||||
joined := strings.Join(sliceMD5s, "\n")
|
||||
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
|
||||
}
|
||||
|
||||
// 注入 CAS 信息
|
||||
t.SetCASInfo(&torrent.CASInfo{
|
||||
FileMD5: fileMD5,
|
||||
SliceMD5: sliceMD5,
|
||||
SliceMD5s: sliceMD5s,
|
||||
SliceSize: sliceSize,
|
||||
Cloud: "189",
|
||||
})
|
||||
|
||||
// 同时更新 info 中的 md5sum 字段
|
||||
if t.Info.MD5Sum == "" {
|
||||
t.Info.MD5Sum = fileMD5
|
||||
}
|
||||
|
||||
return t.Encode()
|
||||
}
|
||||
|
||||
// GetInfoHashHex 获取 torrent 的 info_hash（十六进制字符串）
|
||||
func GetInfoHashHex(torrentData []byte) (string, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(t.InfoHash), nil
|
||||
}
|
||||
|
||||
func isCASTorrentFile(fileName string) bool {
|
||||
return strings.HasSuffix(fileName, ".cas.torrent")
|
||||
}
|
||||
|
||||
// ComputeSliceMD5sFromReader 从 reader 中计算每个 10MB 分片的 MD5
|
||||
// 返回：整文件 MD5、分片 MD5 列表
|
||||
func ComputeSliceMD5sFromReader(reader io.Reader, sliceSize int64) (string, []string, error) {
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = torrent.DefaultPieceSize
|
||||
}
|
||||
|
||||
fileMD5Hash := utils.MD5.NewFunc()
|
||||
sliceMD5s := make([]string, 0)
|
||||
|
||||
buf := make([]byte, sliceSize)
|
||||
for {
|
||||
n, err := io.ReadFull(reader, buf)
|
||||
if n > 0 {
|
||||
chunk := buf[:n]
|
||||
fileMD5Hash.Write(chunk)
|
||||
// 计算该分片的 MD5
|
||||
sliceMD5 := strings.ToUpper(utils.HashData(utils.MD5, chunk))
|
||||
sliceMD5s = append(sliceMD5s, sliceMD5)
|
||||
}
|
||||
if err == io.EOF || err == io.ErrUnexpectedEOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
}
|
||||
|
||||
fileMD5Hex := strings.ToUpper(hex.EncodeToString(fileMD5Hash.Sum(nil)))
|
||||
return fileMD5Hex, sliceMD5s, nil
|
||||
}
|
||||
|
||||
// torrentFollowCopy 跟随复制 torrent 文件（异步，不影响主操作）
|
||||
// srcFolderId: 源文件所在目录 ID
|
||||
// srcFileName: 源文件名
|
||||
// dstDir: 目标目录
|
||||
func (y *Cloud189PC) torrentFollowCopy(srcFolderId string, srcFileName string, dstDir model.Obj) {
|
||||
if !y.Addition.GenerateTorrent {
|
||||
return
|
||||
}
|
||||
if srcFolderId == "" {
|
||||
return
|
||||
}
|
||||
torrentName := srcFileName + ".cas.torrent"
|
||||
isFamily := y.isFamily()
|
||||
|
||||
go func() {
|
||||
torrentFile, err := y.findFileByName(context.Background(), torrentName, srcFolderId, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Debugf("查找 torrent 文件失败（跟随复制）: %v", err)
|
||||
return
|
||||
}
|
||||
// 复制 torrent 文件到目标目录
|
||||
resp, copyErr := y.CreateBatchTask("COPY", IF(isFamily, y.FamilyID, ""), dstDir.GetID(),
|
||||
map[string]string{"targetFileName": dstDir.GetName()},
|
||||
BatchTaskInfo{
|
||||
FileId: torrentFile.GetID(),
|
||||
FileName: torrentFile.GetName(),
|
||||
IsFolder: 0,
|
||||
})
|
||||
if copyErr != nil {
|
||||
utils.Log.Warnf("跟随复制 torrent 文件失败: %v", copyErr)
|
||||
return
|
||||
}
|
||||
if err = y.WaitBatchTask("COPY", resp.TaskID, time.Second); err != nil {
|
||||
utils.Log.Warnf("等待跟随复制 torrent 文件失败: %v", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// torrentFollowMove 跟随移动 torrent 文件（异步，不影响主操作）
|
||||
// srcFolderId: 源文件所在目录 ID
|
||||
// srcFileName: 源文件名
|
||||
// dstDir: 目标目录
|
||||
func (y *Cloud189PC) torrentFollowMove(srcFolderId string, srcFileName string, dstDir model.Obj) {
|
||||
if !y.Addition.GenerateTorrent {
|
||||
return
|
||||
}
|
||||
if srcFolderId == "" {
|
||||
return
|
||||
}
|
||||
torrentName := srcFileName + ".cas.torrent"
|
||||
isFamily := y.isFamily()
|
||||
|
||||
go func() {
|
||||
torrentFile, err := y.findFileByName(context.Background(), torrentName, srcFolderId, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Debugf("查找 torrent 文件失败（跟随移动）: %v", err)
|
||||
return
|
||||
}
|
||||
// 移动 torrent 文件到目标目录
|
||||
resp, moveErr := y.CreateBatchTask("MOVE", IF(isFamily, y.FamilyID, ""), dstDir.GetID(),
|
||||
map[string]string{"targetFileName": dstDir.GetName()},
|
||||
BatchTaskInfo{
|
||||
FileId: torrentFile.GetID(),
|
||||
FileName: torrentFile.GetName(),
|
||||
IsFolder: 0,
|
||||
})
|
||||
if moveErr != nil {
|
||||
utils.Log.Warnf("跟随移动 torrent 文件失败: %v", moveErr)
|
||||
return
|
||||
}
|
||||
if err = y.WaitBatchTask("MOVE", resp.TaskID, time.Millisecond*400); err != nil {
|
||||
utils.Log.Warnf("等待跟随移动 torrent 文件失败: %v", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// torrentFollowRename 跟随重命名 torrent 文件（异步，不影响主操作）
|
||||
// folderId: 文件所在目录 ID
|
||||
// oldFileName: 原文件名
|
||||
// newFileName: 新文件名
|
||||
func (y *Cloud189PC) torrentFollowRename(folderId string, oldFileName string, newFileName string) {
|
||||
if !y.Addition.GenerateTorrent {
|
||||
return
|
||||
}
|
||||
if folderId == "" {
|
||||
return
|
||||
}
|
||||
oldTorrentName := oldFileName + ".cas.torrent"
|
||||
newTorrentName := newFileName + ".cas.torrent"
|
||||
isFamily := y.isFamily()
|
||||
|
||||
go func() {
|
||||
torrentFile, err := y.findFileByName(context.Background(), oldTorrentName, folderId, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Debugf("查找 torrent 文件失败（跟随重命名）: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// 重命名 torrent 文件
|
||||
queryParam := make(map[string]string)
|
||||
fullUrl := API_URL
|
||||
method := "POST"
|
||||
if isFamily {
|
||||
fullUrl += "/family/file"
|
||||
method = "GET"
|
||||
queryParam["familyId"] = y.FamilyID
|
||||
}
|
||||
fullUrl += "/renameFile.action"
|
||||
queryParam["fileId"] = torrentFile.GetID()
|
||||
queryParam["destFileName"] = newTorrentName
|
||||
|
||||
_, renameErr := y.request(fullUrl, method, func(req *resty.Request) {
|
||||
req.SetContext(context.Background()).SetQueryParams(queryParam)
|
||||
}, nil, &RenameResp{}, isFamily)
|
||||
if renameErr != nil {
|
||||
utils.Log.Warnf("跟随重命名 torrent 文件失败: %v", renameErr)
|
||||
}
|
||||
}()
|
||||
}
|
||||
@@ -1,578 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/rand"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/fs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/sign"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
)
|
||||
|
||||
type Alias struct {
|
||||
model.Storage
|
||||
Addition
|
||||
rootOrder []string
|
||||
pathMap map[string][]string
|
||||
root model.Obj
|
||||
}
|
||||
|
||||
func (d *Alias) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Alias) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Alias) Init(ctx context.Context) error {
|
||||
paths := strings.Split(d.Paths, "\n")
|
||||
d.rootOrder = make([]string, 0, len(paths))
|
||||
d.pathMap = make(map[string][]string)
|
||||
for _, path := range paths {
|
||||
path = strings.TrimSpace(path)
|
||||
if path == "" {
|
||||
continue
|
||||
}
|
||||
k, v := getPair(path)
|
||||
temp, ok := d.pathMap[k]
|
||||
if !ok {
|
||||
d.rootOrder = append(d.rootOrder, k)
|
||||
}
|
||||
d.pathMap[k] = append(temp, v)
|
||||
}
|
||||
|
||||
switch len(d.rootOrder) {
|
||||
case 0:
|
||||
return errors.New("paths is required")
|
||||
case 1:
|
||||
paths := d.pathMap[d.rootOrder[0]]
|
||||
roots := make(BalancedObjs, 0, len(paths))
|
||||
roots = append(roots, &model.Object{
|
||||
Name: "root",
|
||||
Path: paths[0],
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Locked,
|
||||
})
|
||||
for _, path := range paths[1:] {
|
||||
roots = append(roots, &model.Object{
|
||||
Path: path,
|
||||
})
|
||||
}
|
||||
d.root = roots
|
||||
default:
|
||||
d.root = &model.Object{
|
||||
Name: "root",
|
||||
Path: "/",
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.ReadOnly,
|
||||
}
|
||||
}
|
||||
|
||||
if !utils.SliceContains(ValidReadConflictPolicy, d.ReadConflictPolicy) {
|
||||
d.ReadConflictPolicy = FirstRWP
|
||||
}
|
||||
if !utils.SliceContains(ValidWriteConflictPolicy, d.WriteConflictPolicy) {
|
||||
d.WriteConflictPolicy = DisabledWP
|
||||
}
|
||||
if !utils.SliceContains(ValidPutConflictPolicy, d.PutConflictPolicy) {
|
||||
d.PutConflictPolicy = DisabledWP
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Alias) Drop(ctx context.Context) error {
|
||||
d.rootOrder = nil
|
||||
d.pathMap = nil
|
||||
d.root = nil
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Alias) GetRoot(ctx context.Context) (model.Obj, error) {
|
||||
if d.root == nil {
|
||||
return nil, errs.StorageNotInit
|
||||
}
|
||||
return d.root, nil
|
||||
}
|
||||
|
||||
// 通过op.Get调用的话，path一定是子路径(/开头)
|
||||
func (d *Alias) Get(ctx context.Context, path string) (model.Obj, error) {
|
||||
roots, sub := d.getRootsAndPath(path)
|
||||
if len(roots) == 0 {
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
for idx, root := range roots {
|
||||
rawPath := stdpath.Join(root, sub)
|
||||
obj, err := fs.Get(ctx, rawPath, &fs.GetArgs{NoLog: true})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
mask := model.GetObjMask(obj) &^ model.Temp
|
||||
if sub == "" {
|
||||
// 根目录
|
||||
mask |= model.Locked | model.Virtual
|
||||
}
|
||||
ret := model.Object{
|
||||
Path: rawPath,
|
||||
Name: obj.GetName(),
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: obj.IsDir(),
|
||||
HashInfo: obj.GetHash(),
|
||||
Mask: mask,
|
||||
}
|
||||
obj = &ret
|
||||
if d.ProviderPassThrough && !obj.IsDir() {
|
||||
if storage, err := fs.GetStorage(rawPath, &fs.GetStoragesArgs{}); err == nil {
|
||||
obj = &model.ObjectProvider{
|
||||
Object: ret,
|
||||
Provider: model.Provider{
|
||||
Provider: storage.Config().Name,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
roots = roots[idx+1:]
|
||||
var objs BalancedObjs
|
||||
if idx > 0 {
|
||||
objs = make(BalancedObjs, 0, len(roots)+2)
|
||||
} else {
|
||||
objs = make(BalancedObjs, 0, len(roots)+1)
|
||||
}
|
||||
objs = append(objs, obj)
|
||||
if idx > 0 {
|
||||
objs = append(objs, nil)
|
||||
}
|
||||
for _, d := range roots {
|
||||
objs = append(objs, &tempObj{model.Object{
|
||||
Path: stdpath.Join(d, sub),
|
||||
}})
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
|
||||
func (d *Alias) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
dirs, ok := dir.(BalancedObjs)
|
||||
if !ok {
|
||||
return d.listRoot(ctx, args.WithStorageDetails && d.DetailsPassThrough, args.Refresh), nil
|
||||
}
|
||||
|
||||
// 因为alias是NoCache且Get方法不会返回NotSupport或NotImplement错误
|
||||
// 所以这里对象不会传回到alias，也就不需要返回BalancedObjs了
|
||||
objMap := make(map[string]model.Obj)
|
||||
for _, dir := range dirs {
|
||||
if dir == nil {
|
||||
continue
|
||||
}
|
||||
dirPath := dir.GetPath()
|
||||
tmp, err := fs.List(ctx, dirPath, &fs.ListArgs{
|
||||
NoLog: true,
|
||||
Refresh: args.Refresh,
|
||||
WithStorageDetails: args.WithStorageDetails && d.DetailsPassThrough,
|
||||
})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for _, obj := range tmp {
|
||||
name := obj.GetName()
|
||||
if _, exists := objMap[name]; exists {
|
||||
continue
|
||||
}
|
||||
mask := model.GetObjMask(obj) &^ model.Temp
|
||||
objRes := model.Object{
|
||||
Name: name,
|
||||
Path: stdpath.Join(dirPath, name),
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: obj.IsDir(),
|
||||
Mask: mask,
|
||||
}
|
||||
var objRet model.Obj
|
||||
if thumb, ok := model.GetThumb(obj); ok {
|
||||
objRet = &model.ObjThumb{
|
||||
Object: objRes,
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumb,
|
||||
},
|
||||
}
|
||||
} else {
|
||||
objRet = &objRes
|
||||
}
|
||||
if details, ok := model.GetStorageDetails(obj); ok {
|
||||
objRet = &model.ObjStorageDetails{
|
||||
Obj: objRet,
|
||||
StorageDetails: details,
|
||||
}
|
||||
}
|
||||
objMap[name] = objRet
|
||||
}
|
||||
}
|
||||
objs := make([]model.Obj, 0, len(objMap))
|
||||
for _, obj := range objMap {
|
||||
objs = append(objs, obj)
|
||||
}
|
||||
if d.OrderBy == "" {
|
||||
sort := getAllSort(dirs)
|
||||
if sort.OrderBy != "" {
|
||||
model.SortFiles(objs, sort.OrderBy, sort.OrderDirection)
|
||||
}
|
||||
if d.ExtractFolder == "" && sort.ExtractFolder != "" {
|
||||
model.ExtractFolder(objs, sort.ExtractFolder)
|
||||
}
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *Alias) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if d.ReadConflictPolicy == AllRWP && !args.Redirect {
|
||||
files, err := d.getAllObjs(ctx, file, getWriteAndPutFilterFunc(AllRWP))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
linkClosers := make([]io.Closer, 0, len(files))
|
||||
rrf := make([]model.RangeReaderIF, 0, len(files))
|
||||
requireReference := false
|
||||
for _, f := range files {
|
||||
link, fi, err := d.link(ctx, f.GetPath(), args)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if fi.GetSize() != files.GetSize() {
|
||||
_ = link.Close()
|
||||
continue
|
||||
}
|
||||
l := &model.Link{
|
||||
URL: link.URL,
|
||||
Header: link.Header,
|
||||
RangeReader: link.RangeReader,
|
||||
Concurrency: link.Concurrency,
|
||||
PartSize: link.PartSize,
|
||||
ContentLength: link.ContentLength,
|
||||
}
|
||||
if d.DownloadConcurrency > 0 {
|
||||
l.Concurrency = d.DownloadConcurrency
|
||||
}
|
||||
if d.DownloadPartSize > 0 {
|
||||
l.PartSize = d.DownloadPartSize * utils.KB
|
||||
}
|
||||
if l.ContentLength == 0 {
|
||||
l.ContentLength = fi.GetSize()
|
||||
}
|
||||
rr, err := stream.GetRangeReaderFromLink(l.ContentLength, l)
|
||||
if err != nil {
|
||||
_ = link.Close()
|
||||
continue
|
||||
}
|
||||
linkClosers = append(linkClosers, link)
|
||||
if link.RequireReference {
|
||||
requireReference = true
|
||||
}
|
||||
rrf = append(rrf, rr)
|
||||
}
|
||||
rr := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
|
||||
return rrf[rand.Intn(len(rrf))].RangeRead(ctx, httpRange)
|
||||
}
|
||||
return &model.Link{
|
||||
RangeReader: stream.RangeReaderFunc(rr),
|
||||
SyncClosers: utils.NewSyncClosers(linkClosers...),
|
||||
RequireReference: requireReference,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var link *model.Link
|
||||
var fi model.Obj
|
||||
var err error
|
||||
files := file.(BalancedObjs)
|
||||
if d.ReadConflictPolicy == RandomBalancedRP || d.ReadConflictPolicy == AllRWP {
|
||||
rand.Shuffle(len(files), func(i, j int) {
|
||||
files[i], files[j] = files[j], files[i]
|
||||
})
|
||||
}
|
||||
for _, f := range files {
|
||||
if f == nil {
|
||||
continue
|
||||
}
|
||||
link, fi, err = d.link(ctx, f.GetPath(), args)
|
||||
if err == nil {
|
||||
if link == nil {
|
||||
// 重定向且需要通过代理
|
||||
return &model.Link{
|
||||
URL: fmt.Sprintf("%s/p%s?sign=%s",
|
||||
common.GetApiUrl(ctx),
|
||||
utils.EncodePath(f.GetPath(), true),
|
||||
sign.Sign(f.GetPath())),
|
||||
}, nil
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resultLink := link.Clone() // 复制一份，避免修改到原始link
|
||||
if args.Redirect {
|
||||
return resultLink, nil
|
||||
}
|
||||
if d.DownloadConcurrency > 0 {
|
||||
resultLink.Concurrency = d.DownloadConcurrency
|
||||
}
|
||||
if d.DownloadPartSize > 0 {
|
||||
resultLink.PartSize = d.DownloadPartSize * utils.KB
|
||||
}
|
||||
if resultLink.ContentLength == 0 {
|
||||
resultLink.ContentLength = fi.GetSize()
|
||||
}
|
||||
return resultLink, nil
|
||||
}
|
||||
|
||||
func (d *Alias) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
|
||||
// Other 不应负载均衡，这是因为前端是否调用 /fs/other 的判断条件是返回的 provider 的值
|
||||
// 而 ProviderPassThrough 开启时，返回的 provider 固定为第一个 obj 的后端驱动
|
||||
storage, actualPath, err := op.GetStorageAndActualPath(args.Obj.GetPath())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return op.Other(ctx, storage, model.FsOtherArgs{
|
||||
Path: actualPath,
|
||||
Method: args.Method,
|
||||
Data: args.Data,
|
||||
})
|
||||
}
|
||||
|
||||
func (d *Alias) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
objs, err := d.getWriteObjs(ctx, parentDir)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.MakeDir(ctx, stdpath.Join(obj.GetPath(), dirName)))
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
srcs, dsts, err := d.getMoveObjs(ctx, srcObj, dstDir)
|
||||
if err == nil {
|
||||
for i, dst := range dsts {
|
||||
src := srcs[i]
|
||||
_, e := fs.Move(ctx, src.GetPath(), dst.GetPath())
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
srcs = srcs[len(dsts):]
|
||||
for _, src := range srcs {
|
||||
e := fs.Remove(ctx, src.GetPath())
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
objs, err := d.getWriteObjs(ctx, srcObj)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.Rename(ctx, obj.GetPath(), newName))
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
srcs, dsts, err := d.getCopyObjs(ctx, srcObj, dstDir)
|
||||
if err == nil {
|
||||
for i, src := range srcs {
|
||||
dst := dsts[i]
|
||||
_, e := fs.Copy(ctx, src.GetPath(), dst.GetPath())
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Remove(ctx context.Context, obj model.Obj) error {
|
||||
objs, err := d.getWriteObjs(ctx, obj)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.Remove(ctx, obj.GetPath()))
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Put(ctx context.Context, dstDir model.Obj, s model.FileStreamer, up driver.UpdateProgress) error {
|
||||
objs, err := d.getPutObjs(ctx, dstDir)
|
||||
if err == nil {
|
||||
if len(objs) == 1 {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(objs.GetPath())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return op.Put(ctx, storage, reqActualPath, &stream.FileStream{
|
||||
Obj: s,
|
||||
Mimetype: s.GetMimetype(),
|
||||
Reader: s,
|
||||
}, up)
|
||||
} else {
|
||||
file, err := s.CacheFullAndWriter(nil, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
count := float64(len(objs) + 1)
|
||||
up(100 / count)
|
||||
for i, obj := range objs {
|
||||
err = errors.Join(err, fs.PutDirectly(ctx, obj.GetPath(), &stream.FileStream{
|
||||
Obj: s,
|
||||
Mimetype: s.GetMimetype(),
|
||||
Reader: file,
|
||||
}))
|
||||
up(float64(i+2) / float64(count) * 100)
|
||||
_, e := file.Seek(0, io.SeekStart)
|
||||
if e != nil {
|
||||
return errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) PutURL(ctx context.Context, dstDir model.Obj, name, url string) error {
|
||||
objs, err := d.getPutObjs(ctx, dstDir)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.PutURL(ctx, obj.GetPath(), name, url))
|
||||
}
|
||||
return err
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
reqPath := d.getBalancedPath(ctx, obj)
|
||||
if reqPath == "" {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
meta, err := d.getArchiveMeta(ctx, reqPath, args)
|
||||
if err == nil {
|
||||
return meta, nil
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
reqPath := d.getBalancedPath(ctx, obj)
|
||||
if reqPath == "" {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
l, err := d.listArchive(ctx, reqPath, args)
|
||||
if err == nil {
|
||||
return l, nil
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
// alias的两个驱动，一个支持驱动提取，一个不支持，如何兼容？
|
||||
// 如果访问的是不支持驱动提取的驱动内的压缩文件，GetArchiveMeta就会返回errs.NotImplement，提取URL前缀就会是/ae，Extract就不会被调用
|
||||
// 如果访问的是支持驱动提取的驱动内的压缩文件，GetArchiveMeta就会返回有效值，提取URL前缀就会是/ad，Extract就会被调用
|
||||
reqPath := d.getBalancedPath(ctx, obj)
|
||||
if reqPath == "" {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
link, err := d.extract(ctx, reqPath, args)
|
||||
if err != nil {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if link == nil {
|
||||
return &model.Link{
|
||||
URL: fmt.Sprintf("%s/ap%s?inner=%s&pass=%s&sign=%s",
|
||||
common.GetApiUrl(ctx),
|
||||
utils.EncodePath(reqPath, true),
|
||||
utils.EncodePath(args.InnerPath, true),
|
||||
url.QueryEscape(args.Password),
|
||||
sign.SignArchive(reqPath)),
|
||||
}, nil
|
||||
}
|
||||
return link.Clone(), nil
|
||||
}
|
||||
|
||||
func (d *Alias) ArchiveDecompress(ctx context.Context, srcObj, dstDir model.Obj, args model.ArchiveDecompressArgs) error {
|
||||
srcs, dsts, err := d.getCopyObjs(ctx, srcObj, dstDir)
|
||||
if err == nil {
|
||||
for i, src := range srcs {
|
||||
dst := dsts[i]
|
||||
_, e := fs.ArchiveDecompress(ctx, src.GetPath(), dst.GetPath(), args)
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
if !d.DetailsPassThrough {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if len(d.rootOrder) != 1 {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
backends := d.pathMap[d.rootOrder[0]]
|
||||
var storage driver.Driver
|
||||
for _, backend := range backends {
|
||||
s, err := fs.GetStorage(backend, &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if storage == nil {
|
||||
storage = s
|
||||
} else if storage.GetStorage().MountPath != s.GetStorage().MountPath {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
}
|
||||
if storage == nil { // should never access
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
return op.GetStorageDetails(ctx, storage)
|
||||
}
|
||||
|
||||
func (d *Alias) ResolveLinkCacheMode(path string) driver.LinkCacheMode {
|
||||
roots, sub := d.getRootsAndPath(path)
|
||||
if len(roots) == 0 {
|
||||
return 0
|
||||
}
|
||||
for _, root := range roots {
|
||||
storage, actualPath, err := op.GetStorageAndActualPath(stdpath.Join(root, sub))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if storage.Config().CheckStatus && storage.GetStorage().Status != op.WORK {
|
||||
continue
|
||||
}
|
||||
mode := storage.Config().LinkCacheMode
|
||||
if mode == -1 {
|
||||
return storage.(driver.LinkCacheModeResolver).ResolveLinkCacheMode(actualPath)
|
||||
} else {
|
||||
return mode
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Alias)(nil)
|
||||
@@ -1,34 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
Paths string `json:"paths" required:"true" type:"text"`
|
||||
ReadConflictPolicy string `json:"read_conflict_policy" type:"select" options:"first,random,all" default:"first"`
|
||||
WriteConflictPolicy string `json:"write_conflict_policy" type:"select" options:"disabled,first,deterministic,deterministic_or_all,all,all_strict" default:"disabled" help:"How the driver handles identical backend paths when renaming, removing, or making directories."`
|
||||
PutConflictPolicy string `json:"put_conflict_policy" type:"select" options:"disabled,first,deterministic,deterministic_or_all,all,all_strict,random,quota,quota_strict" default:"disabled" help:"How the driver handles identical backend paths when uploading, copying, moving, or decompressing."`
|
||||
FileConsistencyCheck bool `json:"file_consistency_check" type:"bool" default:"false"`
|
||||
DownloadConcurrency int `json:"download_concurrency" default:"0" required:"false" type:"number" help:"Need to enable proxy"`
|
||||
DownloadPartSize int `json:"download_part_size" default:"0" type:"number" required:"false" help:"Need to enable proxy. Unit: KB"`
|
||||
ProviderPassThrough bool `json:"provider_pass_through" type:"bool" default:"false"`
|
||||
DetailsPassThrough bool `json:"details_pass_through" type:"bool" default:"false"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Alias",
|
||||
LocalSort: true,
|
||||
NoCache: true,
|
||||
NoUpload: false,
|
||||
DefaultRoot: "/",
|
||||
ProxyRangeOption: true,
|
||||
LinkCacheMode: driver.LinkCacheAuto,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Alias{}
|
||||
})
|
||||
}
|
||||
@@ -1,78 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/pkg/errors"
|
||||
)
|
||||
|
||||
const (
|
||||
DisabledWP = "disabled"
|
||||
FirstRWP = "first"
|
||||
DeterministicWP = "deterministic"
|
||||
DeterministicOrAllWP = "deterministic_or_all"
|
||||
AllRWP = "all"
|
||||
AllStrictWP = "all_strict"
|
||||
RandomBalancedRP = "random"
|
||||
BalancedByQuotaP = "quota"
|
||||
BalancedByQuotaStrictP = "quota_strict"
|
||||
)
|
||||
|
||||
var (
|
||||
ValidReadConflictPolicy = []string{FirstRWP, RandomBalancedRP, AllRWP}
|
||||
ValidWriteConflictPolicy = []string{DisabledWP, FirstRWP, DeterministicWP, DeterministicOrAllWP, AllRWP,
|
||||
AllStrictWP}
|
||||
ValidPutConflictPolicy = []string{DisabledWP, FirstRWP, DeterministicWP, DeterministicOrAllWP, AllRWP,
|
||||
AllStrictWP, RandomBalancedRP, BalancedByQuotaP, BalancedByQuotaStrictP}
|
||||
)
|
||||
|
||||
var (
|
||||
ErrPathConflict = errors.New("path conflict")
|
||||
ErrSamePathLeak = errors.New("leak some of same-name dirs")
|
||||
ErrNoEnoughSpace = errors.New("none of same-name dirs has enough space")
|
||||
ErrNotEnoughSrcObjs = errors.New("cannot move fewer objs to more paths, please try copying")
|
||||
)
|
||||
|
||||
type BalancedObjs []model.Obj
|
||||
|
||||
func (b BalancedObjs) GetSize() int64 {
|
||||
return b[0].GetSize()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) ModTime() time.Time {
|
||||
return b[0].ModTime()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) CreateTime() time.Time {
|
||||
return b[0].CreateTime()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) IsDir() bool {
|
||||
return b[0].IsDir()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetHash() utils.HashInfo {
|
||||
return b[0].GetHash()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetName() string {
|
||||
return b[0].GetName()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetPath() string {
|
||||
return b[0].GetPath()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetID() string {
|
||||
return b[0].GetID()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) Unwrap() model.Obj {
|
||||
return b[0]
|
||||
}
|
||||
|
||||
var _ model.Obj = (BalancedObjs)(nil)
|
||||
|
||||
type tempObj struct{ model.Object }
|
||||
@@ -1,532 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"context"
|
||||
"math/rand"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/fs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/pkg/errors"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
type detailWithIndex struct {
|
||||
idx int
|
||||
val *model.StorageDetails
|
||||
}
|
||||
|
||||
func (d *Alias) listRoot(ctx context.Context, withDetails, refresh bool) []model.Obj {
|
||||
var objs []model.Obj
|
||||
detailsChan := make(chan detailWithIndex, len(d.pathMap))
|
||||
workerCount := 0
|
||||
for _, k := range d.rootOrder {
|
||||
obj := &model.Object{
|
||||
Name: k,
|
||||
Path: "/" + k,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Locked | model.Virtual,
|
||||
}
|
||||
idx := len(objs)
|
||||
objs = append(objs, obj)
|
||||
v := d.pathMap[k]
|
||||
if !withDetails || len(v) != 1 {
|
||||
continue
|
||||
}
|
||||
remoteDriver, err := op.GetStorageByMountPath(v[0])
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
obj.Modified = remoteDriver.GetStorage().Modified
|
||||
_, ok := remoteDriver.(driver.WithDetails)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
objs[idx] = &model.ObjStorageDetails{
|
||||
Obj: objs[idx],
|
||||
StorageDetails: nil,
|
||||
}
|
||||
workerCount++
|
||||
go func(dri driver.Driver, i int) {
|
||||
details, e := op.GetStorageDetails(ctx, dri, refresh)
|
||||
if e != nil {
|
||||
if !errors.Is(e, errs.NotImplement) && !errors.Is(e, errs.StorageNotInit) {
|
||||
log.Errorf("failed get %s storage details: %+v", dri.GetStorage().MountPath, e)
|
||||
}
|
||||
}
|
||||
detailsChan <- detailWithIndex{idx: i, val: details}
|
||||
}(remoteDriver, idx)
|
||||
}
|
||||
for workerCount > 0 {
|
||||
select {
|
||||
case r := <-detailsChan:
|
||||
objs[r.idx].(*model.ObjStorageDetails).StorageDetails = r.val
|
||||
workerCount--
|
||||
case <-time.After(time.Second):
|
||||
workerCount = 0
|
||||
}
|
||||
}
|
||||
return objs
|
||||
}
|
||||
|
||||
// do others that not defined in Driver interface
|
||||
func getPair(path string) (string, string) {
|
||||
if name, path, ok := strings.Cut(path, ":"); ok && !strings.Contains(name, "/") {
|
||||
return name, path
|
||||
}
|
||||
return stdpath.Base(path), path
|
||||
}
|
||||
|
||||
func (d *Alias) getRootsAndPath(path string) (roots []string, sub string) {
|
||||
if len(d.rootOrder) == 1 {
|
||||
return d.pathMap[d.rootOrder[0]], path
|
||||
}
|
||||
path = strings.TrimPrefix(path, "/")
|
||||
before, after, ok := strings.Cut(path, "/")
|
||||
if !ok {
|
||||
return d.pathMap[path], ""
|
||||
}
|
||||
return d.pathMap[before], after
|
||||
}
|
||||
|
||||
func (d *Alias) link(ctx context.Context, reqPath string, args model.LinkArgs) (*model.Link, model.Obj, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if args.Redirect && common.ShouldProxy(storage, stdpath.Base(reqPath)) {
|
||||
return nil, nil, nil
|
||||
}
|
||||
return op.Link(ctx, storage, reqActualPath, args)
|
||||
}
|
||||
|
||||
func isConsistent(a, b model.Obj) bool {
|
||||
if a.GetSize() != b.GetSize() {
|
||||
return false
|
||||
}
|
||||
for ht, v := range a.GetHash().All() {
|
||||
ah := b.GetHash().GetHash(ht)
|
||||
if ah != "" && ah != v {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (d *Alias) getAllObjs(ctx context.Context, bObj model.Obj, ifContinue func(err error) (bool, error)) (BalancedObjs, error) {
|
||||
objs := bObj.(BalancedObjs)
|
||||
length := 0
|
||||
for _, o := range objs {
|
||||
var err error
|
||||
var obj model.Obj
|
||||
temp, isTemp := o.(*tempObj)
|
||||
if isTemp {
|
||||
obj, err = fs.Get(ctx, o.GetPath(), &fs.GetArgs{NoLog: true})
|
||||
if err == nil {
|
||||
if !bObj.IsDir() {
|
||||
if obj.IsDir() {
|
||||
err = errs.NotFile
|
||||
} else if d.FileConsistencyCheck && !isConsistent(bObj, obj) {
|
||||
err = errs.ObjectNotFound
|
||||
}
|
||||
} else if !obj.IsDir() {
|
||||
err = errs.NotFolder
|
||||
}
|
||||
}
|
||||
} else if o == nil {
|
||||
err = errs.ObjectNotFound
|
||||
}
|
||||
|
||||
cont, err := ifContinue(err)
|
||||
if err != nil {
|
||||
if cont {
|
||||
continue
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if isTemp {
|
||||
objRes := temp.Object
|
||||
// objRes.Name = obj.GetName()
|
||||
// objRes.Size = obj.GetSize()
|
||||
// objRes.Modified = obj.ModTime()
|
||||
// objRes.HashInfo = obj.GetHash()
|
||||
objs[length] = &objRes
|
||||
} else {
|
||||
objs[length] = o
|
||||
}
|
||||
length++
|
||||
if !cont {
|
||||
break
|
||||
}
|
||||
}
|
||||
if length == 0 {
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
return objs[:length], nil
|
||||
}
|
||||
|
||||
func (d *Alias) getBalancedPath(ctx context.Context, file model.Obj) string {
|
||||
if d.ReadConflictPolicy == FirstRWP {
|
||||
return file.GetPath()
|
||||
}
|
||||
files := file.(BalancedObjs)
|
||||
if rand.Intn(len(files)) == 0 {
|
||||
return file.GetPath()
|
||||
}
|
||||
files, _ = d.getAllObjs(ctx, file, getWriteAndPutFilterFunc(AllRWP))
|
||||
return files[rand.Intn(len(files))].GetPath()
|
||||
}
|
||||
|
||||
func getWriteAndPutFilterFunc(policy string) func(error) (bool, error) {
|
||||
if policy == AllRWP {
|
||||
return func(err error) (bool, error) {
|
||||
return true, err
|
||||
}
|
||||
}
|
||||
all := true
|
||||
l := 0
|
||||
return func(err error) (bool, error) {
|
||||
if err != nil {
|
||||
switch policy {
|
||||
case AllStrictWP:
|
||||
return false, ErrSamePathLeak
|
||||
case DeterministicOrAllWP:
|
||||
if l >= 2 {
|
||||
return false, ErrSamePathLeak
|
||||
}
|
||||
}
|
||||
all = false
|
||||
} else {
|
||||
switch policy {
|
||||
case FirstRWP:
|
||||
return false, nil
|
||||
case DeterministicWP:
|
||||
if l > 0 {
|
||||
return false, ErrPathConflict
|
||||
}
|
||||
case DeterministicOrAllWP:
|
||||
if l > 0 && !all {
|
||||
return false, ErrSamePathLeak
|
||||
}
|
||||
}
|
||||
l += 1
|
||||
}
|
||||
return true, err
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Alias) getWriteObjs(ctx context.Context, obj model.Obj) (BalancedObjs, error) {
|
||||
if d.WriteConflictPolicy == DisabledWP {
|
||||
return nil, errs.PermissionDenied
|
||||
}
|
||||
return d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.WriteConflictPolicy))
|
||||
}
|
||||
|
||||
func (d *Alias) getPutObjs(ctx context.Context, obj model.Obj) (BalancedObjs, error) {
|
||||
if d.PutConflictPolicy == DisabledWP {
|
||||
return nil, errs.PermissionDenied
|
||||
}
|
||||
objs, err := d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.PutConflictPolicy))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
strict := false
|
||||
switch d.PutConflictPolicy {
|
||||
case RandomBalancedRP:
|
||||
ri := rand.Intn(len(objs))
|
||||
return objs[ri : ri+1], nil
|
||||
case BalancedByQuotaStrictP:
|
||||
strict = true
|
||||
fallthrough
|
||||
case BalancedByQuotaP:
|
||||
objs, ok := getRandomObjByQuotaBalanced(ctx, objs, strict, obj.GetSize())
|
||||
if !ok {
|
||||
return nil, ErrNoEnoughSpace
|
||||
}
|
||||
return objs, nil
|
||||
default:
|
||||
return objs, nil
|
||||
}
|
||||
}
|
||||
|
||||
func getRandomObjByQuotaBalanced(ctx context.Context, reqPath BalancedObjs, strict bool, objSize int64) (BalancedObjs, bool) {
|
||||
// Get all space
|
||||
details := make([]*model.StorageDetails, len(reqPath))
|
||||
detailsChan := make(chan detailWithIndex, len(reqPath))
|
||||
workerCount := 0
|
||||
for i, p := range reqPath {
|
||||
s, err := fs.GetStorage(p.GetPath(), &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if _, ok := s.(driver.WithDetails); !ok {
|
||||
continue
|
||||
}
|
||||
workerCount++
|
||||
go func(dri driver.Driver, i int) {
|
||||
d, e := op.GetStorageDetails(ctx, dri)
|
||||
if e != nil {
|
||||
if !errors.Is(e, errs.NotImplement) && !errors.Is(e, errs.StorageNotInit) {
|
||||
log.Errorf("failed get %s storage details: %+v", dri.GetStorage().MountPath, e)
|
||||
}
|
||||
}
|
||||
detailsChan <- detailWithIndex{idx: i, val: d}
|
||||
}(s, i)
|
||||
}
|
||||
for workerCount > 0 {
|
||||
select {
|
||||
case r := <-detailsChan:
|
||||
details[r.idx] = r.val
|
||||
workerCount--
|
||||
case <-time.After(time.Second):
|
||||
workerCount = 0
|
||||
}
|
||||
}
|
||||
|
||||
// Try select one that has space info
|
||||
selected, ok := selectRandom(details, func(d *model.StorageDetails) uint64 {
|
||||
if d == nil || d.FreeSpace() < objSize {
|
||||
return 0
|
||||
}
|
||||
return uint64(d.FreeSpace())
|
||||
})
|
||||
if !ok {
|
||||
if strict {
|
||||
return nil, false
|
||||
} else {
|
||||
// No strict mode, return any of non-details ones
|
||||
noDetails := make([]int, 0, len(details))
|
||||
for i, d := range details {
|
||||
if d == nil {
|
||||
noDetails = append(noDetails, i)
|
||||
}
|
||||
}
|
||||
if len(noDetails) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
selected = noDetails[rand.Intn(len(noDetails))]
|
||||
}
|
||||
}
|
||||
return reqPath[selected : selected+1], true
|
||||
}
|
||||
|
||||
func selectRandom[Item any](arr []Item, getWeight func(Item) uint64) (int, bool) {
|
||||
var totalWeight uint64 = 0
|
||||
for _, i := range arr {
|
||||
totalWeight += getWeight(i)
|
||||
}
|
||||
if totalWeight == 0 {
|
||||
return 0, false
|
||||
}
|
||||
r := rand.Uint64() % totalWeight
|
||||
for i, item := range arr {
|
||||
w := getWeight(item)
|
||||
if r < w {
|
||||
return i, true
|
||||
}
|
||||
r -= w
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
func (d *Alias) getCopyObjs(ctx context.Context, srcObj, dstDir model.Obj) (BalancedObjs, BalancedObjs, error) {
|
||||
if d.PutConflictPolicy == DisabledWP {
|
||||
return nil, nil, errs.PermissionDenied
|
||||
}
|
||||
dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
dstStorageMap := make(map[string][]model.Obj)
|
||||
allocatingDst := make(map[model.Obj]struct{})
|
||||
for _, o := range dstObjs {
|
||||
storage, e := fs.GetStorage(o.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
return nil, nil, errors.WithMessagef(e, "cannot copy to virtual path [%s]", o.GetPath())
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
dstStorageMap[mp] = append(dstStorageMap[mp], o)
|
||||
allocatingDst[o] = struct{}{}
|
||||
}
|
||||
tmpSrcObjs, err := d.getAllObjs(ctx, srcObj, getWriteAndPutFilterFunc(AllRWP))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
srcObjs := make(BalancedObjs, 0, len(dstObjs))
|
||||
for _, src := range tmpSrcObjs {
|
||||
storage, e := fs.GetStorage(src.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
continue
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
if tmp, ok := dstStorageMap[mp]; ok {
|
||||
for _, dst := range tmp {
|
||||
dstObjs[len(srcObjs)] = dst
|
||||
srcObjs = append(srcObjs, src)
|
||||
delete(allocatingDst, dst)
|
||||
}
|
||||
delete(dstStorageMap, mp)
|
||||
}
|
||||
}
|
||||
dstObjs = dstObjs[:len(srcObjs)]
|
||||
for dst := range allocatingDst {
|
||||
src := tmpSrcObjs[0]
|
||||
if d.ReadConflictPolicy == RandomBalancedRP || d.ReadConflictPolicy == AllRWP {
|
||||
src = tmpSrcObjs[rand.Intn(len(tmpSrcObjs))]
|
||||
}
|
||||
srcObjs = append(srcObjs, src)
|
||||
dstObjs = append(dstObjs, dst)
|
||||
}
|
||||
return srcObjs, dstObjs, nil
|
||||
}
|
||||
|
||||
func (d *Alias) getMoveObjs(ctx context.Context, srcObj, dstDir model.Obj) (BalancedObjs, BalancedObjs, error) {
|
||||
if d.PutConflictPolicy == DisabledWP {
|
||||
return nil, nil, errs.PermissionDenied
|
||||
}
|
||||
dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
tmpSrcObjs, err := d.getAllObjs(ctx, srcObj, getWriteAndPutFilterFunc(AllRWP))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if len(tmpSrcObjs) < len(dstObjs) {
|
||||
return nil, nil, ErrNotEnoughSrcObjs
|
||||
}
|
||||
dstStorageMap := make(map[string][]model.Obj)
|
||||
allocatingDst := make(map[model.Obj]struct{})
|
||||
for _, o := range dstObjs {
|
||||
storage, e := fs.GetStorage(o.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
return nil, nil, errors.WithMessagef(e, "cannot move to virtual path [%s]", o.GetPath())
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
dstStorageMap[mp] = append(dstStorageMap[mp], o)
|
||||
allocatingDst[o] = struct{}{}
|
||||
}
|
||||
srcObjs := make(BalancedObjs, 0, len(tmpSrcObjs))
|
||||
restSrcObjs := make(BalancedObjs, 0, len(tmpSrcObjs)-len(dstObjs))
|
||||
for _, src := range tmpSrcObjs {
|
||||
storage, e := fs.GetStorage(src.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
continue
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
if tmp, ok := dstStorageMap[mp]; ok {
|
||||
dst := tmp[0]
|
||||
if len(tmp) == 1 {
|
||||
delete(dstStorageMap, mp)
|
||||
} else {
|
||||
dstStorageMap[mp] = tmp[1:]
|
||||
}
|
||||
dstObjs[len(srcObjs)] = dst
|
||||
srcObjs = append(srcObjs, src)
|
||||
delete(allocatingDst, dst)
|
||||
} else {
|
||||
restSrcObjs = append(restSrcObjs, src)
|
||||
}
|
||||
}
|
||||
dstObjs = dstObjs[:len(srcObjs)]
|
||||
// len(restSrcObjs) >= len(allocatingDst)
|
||||
srcObjs = append(srcObjs, restSrcObjs...)
|
||||
for dst := range allocatingDst {
|
||||
dstObjs = append(dstObjs, dst)
|
||||
}
|
||||
return srcObjs, dstObjs, nil
|
||||
}
|
||||
|
||||
func (d *Alias) getArchiveMeta(ctx context.Context, reqPath string, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := storage.(driver.ArchiveReader); ok {
|
||||
return op.GetArchiveMeta(ctx, storage, reqActualPath, model.ArchiveMetaArgs{
|
||||
ArchiveArgs: args,
|
||||
Refresh: true,
|
||||
})
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) listArchive(ctx context.Context, reqPath string, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := storage.(driver.ArchiveReader); ok {
|
||||
return op.ListArchive(ctx, storage, reqActualPath, model.ArchiveListArgs{
|
||||
ArchiveInnerArgs: args,
|
||||
Refresh: true,
|
||||
})
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) extract(ctx context.Context, reqPath string, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := storage.(driver.ArchiveReader); !ok {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if args.Redirect && common.ShouldProxy(storage, stdpath.Base(reqPath)) {
|
||||
_, err := fs.Get(ctx, reqPath, &fs.GetArgs{NoLog: true})
|
||||
if err == nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
link, _, err := op.DriverExtract(ctx, storage, reqActualPath, args)
|
||||
return link, err
|
||||
}
|
||||
|
||||
func getAllSort(dirs []model.Obj) model.Sort {
|
||||
ret := model.Sort{}
|
||||
noSort := false
|
||||
noExtractFolder := false
|
||||
for _, dir := range dirs {
|
||||
if dir == nil {
|
||||
continue
|
||||
}
|
||||
storage, err := fs.GetStorage(dir.GetPath(), &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if !noSort && storage.GetStorage().OrderBy != "" {
|
||||
if ret.OrderBy == "" {
|
||||
ret.OrderBy = storage.GetStorage().OrderBy
|
||||
ret.OrderDirection = storage.GetStorage().OrderDirection
|
||||
if ret.OrderDirection == "" {
|
||||
ret.OrderDirection = "asc"
|
||||
}
|
||||
} else if ret.OrderBy != storage.GetStorage().OrderBy || ret.OrderDirection != storage.GetStorage().OrderDirection {
|
||||
ret.OrderBy = ""
|
||||
ret.OrderDirection = ""
|
||||
noSort = true
|
||||
}
|
||||
}
|
||||
if !noExtractFolder && storage.GetStorage().ExtractFolder != "" {
|
||||
if ret.ExtractFolder == "" {
|
||||
ret.ExtractFolder = storage.GetStorage().ExtractFolder
|
||||
} else if ret.ExtractFolder != storage.GetStorage().ExtractFolder {
|
||||
ret.ExtractFolder = ""
|
||||
noExtractFolder = true
|
||||
}
|
||||
}
|
||||
if noSort && noExtractFolder {
|
||||
break
|
||||
}
|
||||
}
|
||||
return ret
|
||||
}
|
||||
@@ -1,147 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type AliDoc struct {
|
||||
model.Storage
|
||||
Addition
|
||||
|
||||
client *resty.Client
|
||||
}
|
||||
|
||||
func (d *AliDoc) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *AliDoc) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *AliDoc) Init(ctx context.Context) error {
|
||||
d.Cookie = strings.TrimSpace(d.Cookie)
|
||||
d.RootFolderID = strings.TrimSpace(d.RootFolderID)
|
||||
if d.Cookie == "" {
|
||||
return fmt.Errorf("cookie is empty")
|
||||
}
|
||||
if d.RootFolderID == "" {
|
||||
return fmt.Errorf("root folder id is empty")
|
||||
}
|
||||
d.client = newClient()
|
||||
if err := d.checkCookie(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Drop(ctx context.Context) error {
|
||||
d.client = nil
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
items, err := d.list(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
objs := make([]model.Obj, 0, len(items))
|
||||
for _, item := range items {
|
||||
if strings.TrimSpace(item.DentryUUID) == "" || strings.TrimSpace(item.Name) == "" {
|
||||
continue
|
||||
}
|
||||
objs = append(objs, toObj(item))
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if file.IsDir() {
|
||||
return nil, fmt.Errorf("alidoc does not support directory links")
|
||||
}
|
||||
resp, err := d.download(ctx, file.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
url, err := firstDownloadURL(resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Link{
|
||||
URL: url,
|
||||
Header: http.Header{
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
"Referer": []string{apiBase + "/"},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
err := d.post(ctx, "/box/api/v2/dentry/createfolder", map[string]string{
|
||||
"dentryType": "folder",
|
||||
"name": dirName,
|
||||
"parentDentryUuid": parentDir.GetID(),
|
||||
"conflictHandleStrategy": "auto_rename",
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
err := d.post(ctx, "/box/api/v2/dentry/move", map[string]interface{}{
|
||||
"targetParentDentryUuid": dstDir.GetID(),
|
||||
"sourceDentryUuid": srcObj.GetID(),
|
||||
"operateFrom": 1,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
|
||||
err := d.post(ctx, "/box/api/v2/dentry/rename", map[string]string{
|
||||
"dentryUuid": srcObj.GetID(),
|
||||
"name": newName,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
srcObj.(*model.Object).Name = newName
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
return d.post(ctx, "/box/api/v2/dentry/copy", map[string]interface{}{
|
||||
"sourceDentryUuid": srcObj.GetID(),
|
||||
"targetParentDentryUuid": dstDir.GetID(),
|
||||
"operateFrom": 1,
|
||||
"onlyCopyMeta": false,
|
||||
})
|
||||
}
|
||||
|
||||
func (d *AliDoc) Remove(ctx context.Context, obj model.Obj) error {
|
||||
return d.post(ctx, "/box/api/v1/dentry/recycle", map[string]string{
|
||||
"dentryUuid": obj.GetID(),
|
||||
})
|
||||
}
|
||||
|
||||
var (
|
||||
_ driver.Driver = (*AliDoc)(nil)
|
||||
_ driver.MkdirResult = (*AliDoc)(nil)
|
||||
_ driver.MoveResult = (*AliDoc)(nil)
|
||||
_ driver.RenameResult = (*AliDoc)(nil)
|
||||
_ driver.Copy = (*AliDoc)(nil)
|
||||
_ driver.Remove = (*AliDoc)(nil)
|
||||
)
|
||||
@@ -1,23 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
Cookie string `json:"cookie" type:"text" required:"true" help:"钉钉文档网页 Cookie"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "AliDoc",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &AliDoc{}
|
||||
})
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
type apiResp struct {
|
||||
Status int `json:"status"`
|
||||
IsSuccess bool `json:"isSuccess"`
|
||||
Message string `json:"message"`
|
||||
Msg string `json:"msg"`
|
||||
}
|
||||
|
||||
func (r apiResp) ErrMessage() string {
|
||||
if r.Message != "" {
|
||||
return r.Message
|
||||
}
|
||||
if r.Msg != "" {
|
||||
return r.Msg
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type listResp struct {
|
||||
apiResp
|
||||
Data listData `json:"data"`
|
||||
}
|
||||
|
||||
type listData struct {
|
||||
Children []dentry `json:"children"`
|
||||
}
|
||||
|
||||
type dentry struct {
|
||||
DentryType string `json:"dentryType"`
|
||||
DentryUUID string `json:"dentryUuid"`
|
||||
ParentDentryUUID string `json:"parentDentryUuid"`
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
FileSize int64 `json:"fileSize"`
|
||||
CreatedTime int64 `json:"createdTime"`
|
||||
UpdatedTime int64 `json:"updatedTime"`
|
||||
ContentType string `json:"contentType"`
|
||||
Extension string `json:"extension"`
|
||||
DentryStatistic struct {
|
||||
ChildrenCount int `json:"childrenCount"`
|
||||
} `json:"dentryStatistic"`
|
||||
URL struct {
|
||||
PCChildAppPreviewURL string `json:"pcChildAppPreviewUrl"`
|
||||
PCChildAppURL string `json:"pcChildAppUrl"`
|
||||
} `json:"url"`
|
||||
}
|
||||
|
||||
type downloadResp struct {
|
||||
apiResp
|
||||
Data downloadData `json:"data"`
|
||||
}
|
||||
|
||||
type downloadData struct {
|
||||
OSSURLPreSignatureInfo struct {
|
||||
PreSignURLs []string `json:"preSignUrls"`
|
||||
} `json:"ossUrlPreSignatureInfo"`
|
||||
}
|
||||
|
||||
type uploadInfoResp struct {
|
||||
apiResp
|
||||
Data uploadInfoData `json:"data"`
|
||||
}
|
||||
|
||||
type uploadInfoData struct {
|
||||
CurrentTimestamp int64 `json:"currentTimestamp"`
|
||||
FileUploadProtocolConfig uploadProtocolConfig `json:"fileUploadProtocolConfig"`
|
||||
STSSignatureInfo uploadSTSSignatureInfo `json:"stsSignatureInfo"`
|
||||
UploadKey string `json:"uploadKey"`
|
||||
UploadType string `json:"uploadType"`
|
||||
}
|
||||
|
||||
type uploadProtocolConfig struct {
|
||||
MinPartSize int64 `json:"minPartSize"`
|
||||
}
|
||||
|
||||
type uploadSTSSignatureInfo struct {
|
||||
AccelerateCname string `json:"accelerateCname"`
|
||||
AccessKeyID string `json:"accessKeyId"`
|
||||
AccessKeySecret string `json:"accessKeySecret"`
|
||||
AccessToken string `json:"accessToken"`
|
||||
AccessTokenExpiration int64 `json:"accessTokenExpiration"`
|
||||
Bucket string `json:"bucket"`
|
||||
Cname string `json:"cname"`
|
||||
EndPoint string `json:"endPoint"`
|
||||
ObjectKey string `json:"objectKey"`
|
||||
}
|
||||
@@ -1,263 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
netutil "github.com/OpenListTeam/OpenList/v4/internal/net"
|
||||
streamPkg "github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/aliyun/aliyun-oss-go-sdk/oss"
|
||||
"github.com/avast/retry-go"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultAliDocMultipartThreshold = 16 * 1024 * 1024
|
||||
defaultAliDocPartSize = 100 * 1024
|
||||
maxAliDocMultipartParts = 10000
|
||||
)
|
||||
|
||||
func (d *AliDoc) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
size := file.GetSize()
|
||||
useMultipart := size > defaultAliDocMultipartThreshold
|
||||
info, err := d.getUploadInfo(ctx, dstDir.GetID(), file.GetName(), size, useMultipart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if size > 0 {
|
||||
partSize := calcAliDocPartSize(size, info.Data.FileUploadProtocolConfig.MinPartSize)
|
||||
if size > partSize && !useMultipart {
|
||||
useMultipart = true
|
||||
info, err = d.getUploadInfo(ctx, dstDir.GetID(), file.GetName(), size, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if useMultipart {
|
||||
err = d.multipartUpload(ctx, file, size, info, up)
|
||||
} else {
|
||||
err = d.singleUpload(ctx, file, size, info, up)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := d.commitUpload(ctx, dstDir.GetID(), file.GetName(), size, info.Data.UploadKey); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) getUploadInfo(ctx context.Context, parentDentryUUID, name string, fileSize int64, multipart bool) (uploadInfoResp, error) {
|
||||
var result uploadInfoResp
|
||||
body := map[string]interface{}{
|
||||
"uploadType": "STS_SIGNATURE",
|
||||
"supportUploadTypes": []string{"STS_SIGNATURE", "HTTP_TO_CENTER"},
|
||||
"parentDentryUuid": parentDentryUUID,
|
||||
"fileSize": fileSize,
|
||||
"name": name,
|
||||
"multipart": multipart,
|
||||
}
|
||||
resp, err := d.request(ctx).
|
||||
SetBody(body).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Post(apiBase + "/box/api/v2/file/uploadinfo")
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := checkResp(resp, result.apiResp); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if strings.TrimSpace(result.Data.STSSignatureInfo.Bucket) == "" {
|
||||
return result, fmt.Errorf("empty upload bucket")
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) commitUpload(ctx context.Context, parentDentryUUID, name string, fileSize int64, uploadKey string) error {
|
||||
uploadKey = strings.TrimSpace(uploadKey)
|
||||
if uploadKey == "" {
|
||||
return fmt.Errorf("empty upload key")
|
||||
}
|
||||
|
||||
var result apiResp
|
||||
body := map[string]interface{}{
|
||||
"parentDentryUuid": parentDentryUUID,
|
||||
"uploadKey": uploadKey,
|
||||
"fileSize": fileSize,
|
||||
"name": name,
|
||||
"toPrevDentryUuid": nil,
|
||||
"toNextDentryUuid": nil,
|
||||
"batchId": uuid.NewString(),
|
||||
"batchUploadType": 1,
|
||||
"batchParentDentryUuid": parentDentryUUID,
|
||||
}
|
||||
resp, err := d.request(ctx).
|
||||
SetBody(body).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Post(apiBase + "/box/api/v2/file/commit")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkResp(resp, result)
|
||||
}
|
||||
|
||||
func calcAliDocPartSize(fileSize, minPartSize int64) int64 {
|
||||
partSize := minPartSize
|
||||
if partSize <= 0 {
|
||||
partSize = defaultAliDocPartSize
|
||||
}
|
||||
if fileSize <= 0 {
|
||||
return partSize
|
||||
}
|
||||
minRequired := int64(math.Ceil(float64(fileSize) / maxAliDocMultipartParts))
|
||||
if minRequired > partSize {
|
||||
partSize = minRequired
|
||||
}
|
||||
return partSize
|
||||
}
|
||||
|
||||
func (d *AliDoc) singleUpload(ctx context.Context, src model.FileStreamer, size int64, info uploadInfoResp, up driver.UpdateProgress) error {
|
||||
bucket, objectKey, err := d.newOSSBucket(info)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = bucket.PutObject(
|
||||
objectKey,
|
||||
driver.NewLimitedUploadStream(ctx, io.TeeReader(src, driver.NewProgress(size, up))),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
up(100)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) multipartUpload(ctx context.Context, src model.FileStreamer, size int64, info uploadInfoResp, up driver.UpdateProgress) error {
|
||||
bucket, objectKey, err := d.newOSSBucket(info)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
imur, err := bucket.InitiateMultipartUpload(objectKey, oss.Sequential())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
partSize := calcAliDocPartSize(size, info.Data.FileUploadProtocolConfig.MinPartSize)
|
||||
partNum := int((size + partSize - 1) / partSize)
|
||||
parts := make([]oss.UploadPart, 0, partNum)
|
||||
ss, err := streamPkg.NewStreamSectionReader(src, int(partSize), &up)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var offset int64
|
||||
for partNumber := 1; partNumber <= partNum; partNumber++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
length := partSize
|
||||
if remain := size - offset; remain < length {
|
||||
length = remain
|
||||
}
|
||||
|
||||
reader, err := ss.GetSectionReader(offset, length)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var part oss.UploadPart
|
||||
var uploadErr error
|
||||
err = retry.Do(func() error {
|
||||
if _, err := reader.Seek(0, io.SeekStart); err != nil {
|
||||
return err
|
||||
}
|
||||
part, uploadErr = bucket.UploadPart(
|
||||
imur,
|
||||
driver.NewLimitedUploadStream(ctx, reader),
|
||||
length,
|
||||
partNumber,
|
||||
)
|
||||
return uploadErr
|
||||
},
|
||||
retry.Context(ctx),
|
||||
retry.Attempts(3),
|
||||
retry.DelayType(retry.BackOffDelay),
|
||||
retry.Delay(time.Second),
|
||||
)
|
||||
ss.FreeSectionReader(reader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
parts = append(parts, part)
|
||||
up(100 * float64(len(parts)) / float64(partNum+1))
|
||||
offset += length
|
||||
}
|
||||
|
||||
_, err = bucket.CompleteMultipartUpload(imur, parts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
up(100)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) newOSSBucket(info uploadInfoResp) (*oss.Bucket, string, error) {
|
||||
sts := info.Data.STSSignatureInfo
|
||||
objectKey := strings.TrimSpace(sts.ObjectKey)
|
||||
if objectKey == "" {
|
||||
objectKey = strings.TrimSpace(info.Data.UploadKey)
|
||||
}
|
||||
if objectKey == "" {
|
||||
return nil, "", fmt.Errorf("empty upload object key")
|
||||
}
|
||||
|
||||
endpoint, useCname := pickAliDocOSSEndpoint(sts)
|
||||
if endpoint == "" {
|
||||
return nil, "", fmt.Errorf("empty upload endpoint")
|
||||
}
|
||||
if !strings.HasPrefix(endpoint, "http://") && !strings.HasPrefix(endpoint, "https://") {
|
||||
endpoint = "https://" + endpoint
|
||||
}
|
||||
|
||||
options := []oss.ClientOption{oss.SecurityToken(sts.AccessToken)}
|
||||
if useCname {
|
||||
options = append(options, oss.UseCname(true))
|
||||
}
|
||||
client, err := netutil.NewOSSClient(
|
||||
endpoint,
|
||||
sts.AccessKeyID,
|
||||
sts.AccessKeySecret,
|
||||
options...,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
bucket, err := client.Bucket(sts.Bucket)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
return bucket, objectKey, nil
|
||||
}
|
||||
|
||||
func pickAliDocOSSEndpoint(sts uploadSTSSignatureInfo) (endpoint string, useCname bool) {
|
||||
if endpoint = strings.TrimSpace(sts.EndPoint); endpoint != "" {
|
||||
return endpoint, false
|
||||
}
|
||||
if endpoint = strings.TrimSpace(sts.Cname); endpoint != "" {
|
||||
return endpoint, true
|
||||
}
|
||||
if endpoint = strings.TrimSpace(sts.AccelerateCname); endpoint != "" {
|
||||
return endpoint, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
@@ -1,135 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
const apiBase = "https://alidocs.dingtalk.com"
|
||||
|
||||
func (d *AliDoc) request(ctx context.Context) *resty.Request {
|
||||
return d.client.R().
|
||||
SetContext(ctx).
|
||||
SetHeader("Cookie", d.Cookie).
|
||||
SetHeader("Accept", "application/json, text/plain, */*").
|
||||
SetHeader("Referer", apiBase+"/").
|
||||
SetHeader("Origin", apiBase)
|
||||
}
|
||||
|
||||
func msToTime(v int64) time.Time {
|
||||
if v <= 0 {
|
||||
return time.Time{}
|
||||
}
|
||||
return time.UnixMilli(v)
|
||||
}
|
||||
|
||||
func checkResp(resp *resty.Response, result apiResp) error {
|
||||
if resp != nil && resp.IsError() {
|
||||
if msg := result.ErrMessage(); msg != "" {
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
return fmt.Errorf("http error: %d", resp.StatusCode())
|
||||
}
|
||||
if !result.IsSuccess || result.Status != 200 {
|
||||
msg := result.ErrMessage()
|
||||
if msg == "" {
|
||||
msg = "request failed"
|
||||
}
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func toObj(item dentry) model.Obj {
|
||||
return &model.Object{
|
||||
ID: item.DentryUUID,
|
||||
Name: item.Name,
|
||||
Size: item.FileSize,
|
||||
Modified: msToTime(item.UpdatedTime),
|
||||
Ctime: msToTime(item.CreatedTime),
|
||||
IsFolder: item.DentryType == "folder",
|
||||
}
|
||||
}
|
||||
|
||||
func firstDownloadURL(resp downloadResp) (string, error) {
|
||||
if len(resp.Data.OSSURLPreSignatureInfo.PreSignURLs) == 0 {
|
||||
return "", fmt.Errorf("empty download url")
|
||||
}
|
||||
return resp.Data.OSSURLPreSignatureInfo.PreSignURLs[0], nil
|
||||
}
|
||||
|
||||
func newClient() *resty.Client {
|
||||
client := base.NewRestyClient()
|
||||
client.SetHeader("User-Agent", base.UserAgent)
|
||||
return client
|
||||
}
|
||||
|
||||
func (d *AliDoc) post(ctx context.Context, path string, body interface{}) error {
|
||||
var result apiResp
|
||||
resp, err := d.request(ctx).
|
||||
SetBody(body).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Post(apiBase + path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkResp(resp, result)
|
||||
}
|
||||
|
||||
func (d *AliDoc) checkCookie(ctx context.Context) error {
|
||||
var result apiResp
|
||||
resp, err := d.request(ctx).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Get(apiBase + "/portal/api/v1/mine/info")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkResp(resp, result)
|
||||
}
|
||||
|
||||
func (d *AliDoc) list(ctx context.Context, dentryUUID string) ([]dentry, error) {
|
||||
var result listResp
|
||||
resp, err := d.request(ctx).
|
||||
SetQueryParam("dentryUuid", dentryUUID).
|
||||
SetQueryParam("withParentAncestors", "true").
|
||||
SetQueryParam("orderType", "SORT_KEY").
|
||||
SetQueryParam("sortType", "desc").
|
||||
SetQueryParam("listDentrySource", "2").
|
||||
SetQueryParam("pageSize", "1000").
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Get(apiBase + "/box/api/v2/dentry/list")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := checkResp(resp, result.apiResp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return result.Data.Children, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) download(ctx context.Context, dentryUUID string) (downloadResp, error) {
|
||||
var result downloadResp
|
||||
resp, err := d.request(ctx).
|
||||
SetQueryParam("dentryUuid", dentryUUID).
|
||||
SetQueryParam("version", "1").
|
||||
SetQueryParam("supportDownloadTypes", "URL_PRE_SIGNATURE,HTTP_TO_CENTER").
|
||||
SetQueryParam("downloadType", "URL_PRE_SIGNATURE").
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Get(apiBase + "/box/api/v2/file/download")
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := checkResp(resp, result.apiResp); err != nil {
|
||||
return result, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
@@ -1,380 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
type AListV3 struct {
|
||||
model.Storage
|
||||
Addition
|
||||
}
|
||||
|
||||
func (d *AListV3) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *AListV3) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *AListV3) Init(ctx context.Context) error {
|
||||
d.Addition.Address = strings.TrimSuffix(d.Addition.Address, "/")
|
||||
var resp common.Resp[MeResp]
|
||||
_, _, err := d.request("/me", http.MethodGet, func(req *resty.Request) {
|
||||
req.SetResult(&resp)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// if the username is not empty and the username is not the same as the current username, then login again
|
||||
if d.Username != resp.Data.Username {
|
||||
err = d.login()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// re-get the user info
|
||||
_, _, err = d.request("/me", http.MethodGet, func(req *resty.Request) {
|
||||
req.SetResult(&resp)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if utils.SliceContains(resp.Data.Role, model.GUEST) {
|
||||
u := d.Address + "/api/public/settings"
|
||||
res, err := base.RestyClient.R().Get(u)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allowMounted := utils.Json.Get(res.Body(), "data", conf.AllowMounted).ToString() == "true"
|
||||
if !allowMounted {
|
||||
return fmt.Errorf("the site does not allow mounted")
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AListV3) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
var resp common.Resp[FsListResp]
|
||||
_, _, err := d.request("/fs/list", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ListReq{
|
||||
PageReq: model.PageReq{
|
||||
Page: 1,
|
||||
PerPage: 0,
|
||||
},
|
||||
Path: dir.GetPath(),
|
||||
Password: d.MetaPassword,
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []model.Obj
|
||||
for _, f := range resp.Data.Content {
|
||||
file := model.ObjThumb{
|
||||
Object: model.Object{
|
||||
Name: f.Name,
|
||||
Path: path.Join(dir.GetPath(), f.Name),
|
||||
Modified: f.Modified,
|
||||
Ctime: f.Created,
|
||||
Size: f.Size,
|
||||
IsFolder: f.IsDir,
|
||||
HashInfo: utils.FromString(f.HashInfo),
|
||||
},
|
||||
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
|
||||
}
|
||||
files = append(files, &file)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
var resp common.Resp[FsGetResp]
|
||||
headers := map[string]string{
|
||||
"User-Agent": base.UserAgent,
|
||||
}
|
||||
// if PassUAToUpsteam is true, then pass the user-agent to the upstream
|
||||
if d.PassUAToUpsteam {
|
||||
userAgent := args.Header.Get("user-agent")
|
||||
if userAgent != "" {
|
||||
headers["User-Agent"] = userAgent
|
||||
}
|
||||
}
|
||||
// if PassIPToUpsteam is true, then pass the ip address to the upstream
|
||||
if d.PassIPToUpsteam {
|
||||
ip := args.IP
|
||||
if ip != "" {
|
||||
headers["X-Forwarded-For"] = ip
|
||||
headers["X-Real-Ip"] = ip
|
||||
}
|
||||
}
|
||||
_, _, err := d.request("/fs/get", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(FsGetReq{
|
||||
Path: file.GetPath(),
|
||||
Password: d.MetaPassword,
|
||||
}).SetHeaders(headers)
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Link{
|
||||
URL: resp.Data.RawURL,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
_, _, err := d.request("/fs/mkdir", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(MkdirOrLinkReq{
|
||||
Path: path.Join(parentDir.GetPath(), dirName),
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
_, _, err := d.request("/fs/move", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(MoveCopyReq{
|
||||
SrcDir: path.Dir(srcObj.GetPath()),
|
||||
DstDir: dstDir.GetPath(),
|
||||
Names: []string{srcObj.GetName()},
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
_, _, err := d.request("/fs/rename", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(RenameReq{
|
||||
Path: srcObj.GetPath(),
|
||||
Name: newName,
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
_, _, err := d.request("/fs/copy", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(MoveCopyReq{
|
||||
SrcDir: path.Dir(srcObj.GetPath()),
|
||||
DstDir: dstDir.GetPath(),
|
||||
Names: []string{srcObj.GetName()},
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Remove(ctx context.Context, obj model.Obj) error {
|
||||
_, _, err := d.request("/fs/remove", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(RemoveReq{
|
||||
Dir: path.Dir(obj.GetPath()),
|
||||
Names: []string{obj.GetName()},
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Put(ctx context.Context, dstDir model.Obj, s model.FileStreamer, up driver.UpdateProgress) error {
|
||||
reader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: s,
|
||||
UpdateProgress: up,
|
||||
})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, d.Address+"/api/fs/put", reader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", d.Token)
|
||||
req.Header.Set("File-Path", path.Join(dstDir.GetPath(), s.GetName()))
|
||||
req.Header.Set("Password", d.MetaPassword)
|
||||
if md5 := s.GetHash().GetHash(utils.MD5); len(md5) > 0 {
|
||||
req.Header.Set("X-File-Md5", md5)
|
||||
}
|
||||
if sha1 := s.GetHash().GetHash(utils.SHA1); len(sha1) > 0 {
|
||||
req.Header.Set("X-File-Sha1", sha1)
|
||||
}
|
||||
if sha256 := s.GetHash().GetHash(utils.SHA256); len(sha256) > 0 {
|
||||
req.Header.Set("X-File-Sha256", sha256)
|
||||
}
|
||||
|
||||
req.ContentLength = s.GetSize()
|
||||
// client := base.NewHttpClient()
|
||||
// client.Timeout = time.Hour * 6
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
bytes, err := io.ReadAll(res.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
log.Debugf("[openlist] response body: %s", string(bytes))
|
||||
if res.StatusCode >= 400 {
|
||||
return fmt.Errorf("request failed, status: %s", res.Status)
|
||||
}
|
||||
code := utils.Json.Get(bytes, "code").ToInt()
|
||||
if code != 200 {
|
||||
if code == 401 || code == 403 {
|
||||
err = d.login()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("request failed,code: %d, message: %s", code, utils.Json.Get(bytes, "message").ToString())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AListV3) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
if !d.ForwardArchiveReq {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
var resp common.Resp[ArchiveMetaResp]
|
||||
_, code, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Password: d.MetaPassword,
|
||||
Path: obj.GetPath(),
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if code == 202 {
|
||||
return nil, errs.WrongArchivePassword
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tree []model.ObjTree
|
||||
if resp.Data.Content != nil {
|
||||
tree = make([]model.ObjTree, 0, len(resp.Data.Content))
|
||||
for _, content := range resp.Data.Content {
|
||||
tree = append(tree, &content)
|
||||
}
|
||||
}
|
||||
return &model.ArchiveMetaInfo{
|
||||
Comment: resp.Data.Comment,
|
||||
Encrypted: resp.Data.Encrypted,
|
||||
Tree: tree,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
if !d.ForwardArchiveReq {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
var resp common.Resp[ArchiveListResp]
|
||||
_, code, err := d.request("/fs/archive/list", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveListReq{
|
||||
ArchiveMetaReq: ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Password: d.MetaPassword,
|
||||
Path: obj.GetPath(),
|
||||
Refresh: false,
|
||||
},
|
||||
PageReq: model.PageReq{
|
||||
Page: 1,
|
||||
PerPage: 0,
|
||||
},
|
||||
InnerPath: args.InnerPath,
|
||||
})
|
||||
})
|
||||
if code == 202 {
|
||||
return nil, errs.WrongArchivePassword
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []model.Obj
|
||||
for _, f := range resp.Data.Content {
|
||||
file := model.ObjThumb{
|
||||
Object: model.Object{
|
||||
Name: f.Name,
|
||||
Modified: f.Modified,
|
||||
Ctime: f.Created,
|
||||
Size: f.Size,
|
||||
IsFolder: f.IsDir,
|
||||
HashInfo: utils.FromString(f.HashInfo),
|
||||
},
|
||||
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
|
||||
}
|
||||
files = append(files, &file)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
if !d.ForwardArchiveReq {
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
var resp common.Resp[ArchiveMetaResp]
|
||||
_, _, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Password: d.MetaPassword,
|
||||
Path: obj.GetPath(),
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Link{
|
||||
URL: fmt.Sprintf("%s?inner=%s&pass=%s&sign=%s",
|
||||
resp.Data.RawURL,
|
||||
utils.EncodePath(args.InnerPath, true),
|
||||
url.QueryEscape(args.Password),
|
||||
resp.Data.Sign),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) ArchiveDecompress(ctx context.Context, srcObj, dstDir model.Obj, args model.ArchiveDecompressArgs) error {
|
||||
if !d.ForwardArchiveReq {
|
||||
return errs.NotImplement
|
||||
}
|
||||
dir, name := path.Split(srcObj.GetPath())
|
||||
_, _, err := d.request("/fs/archive/decompress", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(DecompressReq{
|
||||
ArchivePass: args.Password,
|
||||
CacheFull: args.CacheFull,
|
||||
DstDir: dstDir.GetPath(),
|
||||
InnerPath: args.InnerPath,
|
||||
Name: []string{name},
|
||||
PutIntoNewDir: args.PutIntoNewDir,
|
||||
SrcDir: dir,
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) ResolveLinkCacheMode(_ string) driver.LinkCacheMode {
|
||||
var mode driver.LinkCacheMode
|
||||
if d.PassIPToUpsteam {
|
||||
mode |= driver.LinkCacheIP
|
||||
}
|
||||
if d.PassUAToUpsteam {
|
||||
mode |= driver.LinkCacheUA
|
||||
}
|
||||
return mode
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*AListV3)(nil)
|
||||
@@ -1,65 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
// TestListSetsChildPaths descends two levels the way op.Get does, feeding an
|
||||
// object from one listing back into List as dir. Without a Path on that object
|
||||
// the driver asks upstream for "", which a real server answers with its own
|
||||
// root -- hence the fake upstream's fallback, and the endless self-similar tree.
|
||||
func TestListSetsChildPaths(t *testing.T) {
|
||||
tree := map[string][]ObjResp{
|
||||
"/": {{Name: "root-marker", IsDir: true}},
|
||||
"/drive": {{Name: "concerts", IsDir: true}},
|
||||
"/drive/concerts": {{Name: "show.mkv"}},
|
||||
}
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
var req ListReq
|
||||
_ = json.NewDecoder(r.Body).Decode(&req)
|
||||
content, ok := tree[req.Path]
|
||||
if !ok {
|
||||
content = tree["/"]
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json") // resty only unmarshals JSON
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{
|
||||
"code": 200, "message": "success",
|
||||
"data": map[string]any{"content": content, "total": len(content)},
|
||||
})
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
|
||||
// conf.Conf is nil outside a booted server, so base.InitClient() is unusable.
|
||||
prev := base.RestyClient
|
||||
base.RestyClient = resty.New().SetTimeout(5 * time.Second)
|
||||
t.Cleanup(func() { base.RestyClient = prev })
|
||||
|
||||
d := &AListV3{Addition: Addition{
|
||||
RootPath: driver.RootPath{RootFolderPath: "/drive"},
|
||||
Address: srv.URL,
|
||||
}}
|
||||
dir := model.Obj(&model.Object{Path: "/drive", IsFolder: true})
|
||||
for _, want := range []string{"/drive/concerts", "/drive/concerts/show.mkv"} {
|
||||
objs, err := d.List(context.Background(), dir, model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("List(%q): %v", dir.GetPath(), err)
|
||||
}
|
||||
if len(objs) != 1 {
|
||||
t.Fatalf("List(%q) returned %d objects, want 1", dir.GetPath(), len(objs))
|
||||
}
|
||||
if got := objs[0].GetPath(); got != want {
|
||||
t.Fatalf("child of %q has path %q, want %q", dir.GetPath(), got, want)
|
||||
}
|
||||
dir = objs[0]
|
||||
}
|
||||
}
|
||||
@@ -1,182 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type ListReq struct {
|
||||
model.PageReq
|
||||
Path string `json:"path" form:"path"`
|
||||
Password string `json:"password" form:"password"`
|
||||
Refresh bool `json:"refresh"`
|
||||
}
|
||||
|
||||
type ObjResp struct {
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
IsDir bool `json:"is_dir"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Created time.Time `json:"created"`
|
||||
Sign string `json:"sign"`
|
||||
Thumb string `json:"thumb"`
|
||||
Type int `json:"type"`
|
||||
HashInfo string `json:"hashinfo"`
|
||||
}
|
||||
|
||||
type FsListResp struct {
|
||||
Content []ObjResp `json:"content"`
|
||||
Total int64 `json:"total"`
|
||||
Readme string `json:"readme"`
|
||||
Write bool `json:"write"`
|
||||
Provider string `json:"provider"`
|
||||
}
|
||||
|
||||
type FsGetReq struct {
|
||||
Path string `json:"path" form:"path"`
|
||||
Password string `json:"password" form:"password"`
|
||||
}
|
||||
|
||||
type FsGetResp struct {
|
||||
ObjResp
|
||||
RawURL string `json:"raw_url"`
|
||||
Readme string `json:"readme"`
|
||||
Provider string `json:"provider"`
|
||||
Related []ObjResp `json:"related"`
|
||||
}
|
||||
|
||||
type MkdirOrLinkReq struct {
|
||||
Path string `json:"path" form:"path"`
|
||||
}
|
||||
|
||||
type MoveCopyReq struct {
|
||||
SrcDir string `json:"src_dir"`
|
||||
DstDir string `json:"dst_dir"`
|
||||
Names []string `json:"names"`
|
||||
}
|
||||
|
||||
type RenameReq struct {
|
||||
Path string `json:"path"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type RemoveReq struct {
|
||||
Dir string `json:"dir"`
|
||||
Names []string `json:"names"`
|
||||
}
|
||||
|
||||
type LoginResp struct {
|
||||
Token string `json:"token"`
|
||||
}
|
||||
|
||||
type MeResp struct {
|
||||
Id int `json:"id"`
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
BasePath string `json:"base_path"`
|
||||
Role IntSlice `json:"role"`
|
||||
Disabled bool `json:"disabled"`
|
||||
Permission int `json:"permission"`
|
||||
SsoId string `json:"sso_id"`
|
||||
Otp bool `json:"otp"`
|
||||
}
|
||||
|
||||
type IntSlice []int
|
||||
|
||||
func (s *IntSlice) UnmarshalJSON(b []byte) error {
|
||||
var i int
|
||||
if json.Unmarshal(b, &i) == nil {
|
||||
*s = []int{i}
|
||||
return nil
|
||||
}
|
||||
return json.Unmarshal(b, (*[]int)(s))
|
||||
}
|
||||
|
||||
type ArchiveMetaReq struct {
|
||||
ArchivePass string `json:"archive_pass"`
|
||||
Password string `json:"password"`
|
||||
Path string `json:"path"`
|
||||
Refresh bool `json:"refresh"`
|
||||
}
|
||||
|
||||
type TreeResp struct {
|
||||
ObjResp
|
||||
Children []TreeResp `json:"children"`
|
||||
hashCache *utils.HashInfo
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetSize() int64 {
|
||||
return t.Size
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetName() string {
|
||||
return t.Name
|
||||
}
|
||||
|
||||
func (t *TreeResp) ModTime() time.Time {
|
||||
return t.Modified
|
||||
}
|
||||
|
||||
func (t *TreeResp) CreateTime() time.Time {
|
||||
return t.Created
|
||||
}
|
||||
|
||||
func (t *TreeResp) IsDir() bool {
|
||||
return t.ObjResp.IsDir
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetHash() utils.HashInfo {
|
||||
return utils.FromString(t.HashInfo)
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetID() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetPath() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetChildren() []model.ObjTree {
|
||||
ret := make([]model.ObjTree, 0, len(t.Children))
|
||||
for _, child := range t.Children {
|
||||
ret = append(ret, &child)
|
||||
}
|
||||
return ret
|
||||
}
|
||||
|
||||
func (t *TreeResp) Thumb() string {
|
||||
return t.ObjResp.Thumb
|
||||
}
|
||||
|
||||
type ArchiveMetaResp struct {
|
||||
Comment string `json:"comment"`
|
||||
Encrypted bool `json:"encrypted"`
|
||||
Content []TreeResp `json:"content"`
|
||||
RawURL string `json:"raw_url"`
|
||||
Sign string `json:"sign"`
|
||||
}
|
||||
|
||||
type ArchiveListReq struct {
|
||||
model.PageReq
|
||||
ArchiveMetaReq
|
||||
InnerPath string `json:"inner_path"`
|
||||
}
|
||||
|
||||
type ArchiveListResp struct {
|
||||
Content []ObjResp `json:"content"`
|
||||
Total int64 `json:"total"`
|
||||
}
|
||||
|
||||
type DecompressReq struct {
|
||||
ArchivePass string `json:"archive_pass"`
|
||||
CacheFull bool `json:"cache_full"`
|
||||
DstDir string `json:"dst_dir"`
|
||||
InnerPath string `json:"inner_path"`
|
||||
Name []string `json:"name"`
|
||||
PutIntoNewDir bool `json:"put_into_new_dir"`
|
||||
SrcDir string `json:"src_dir"`
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func (d *AListV3) login() error {
|
||||
if d.Username == "" {
|
||||
return nil
|
||||
}
|
||||
var resp common.Resp[LoginResp]
|
||||
_, _, err := d.request("/auth/login", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(base.Json{
|
||||
"username": d.Username,
|
||||
"password": d.Password,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.Token = resp.Data.Token
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AListV3) request(api, method string, callback base.ReqCallback, retry ...bool) ([]byte, int, error) {
|
||||
url := d.Address + "/api" + api
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeader("Authorization", d.Token)
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
res, err := req.Execute(method, url)
|
||||
if err != nil {
|
||||
code := 0
|
||||
if res != nil {
|
||||
code = res.StatusCode()
|
||||
}
|
||||
return nil, code, err
|
||||
}
|
||||
log.Debugf("[openlist] response body: %s", res.String())
|
||||
if res.StatusCode() >= 400 {
|
||||
return nil, res.StatusCode(), fmt.Errorf("request failed, status: %s", res.Status())
|
||||
}
|
||||
code := utils.Json.Get(res.Body(), "code").ToInt()
|
||||
if code != 200 {
|
||||
if (code == 401 || code == 403) && !utils.IsBool(retry...) {
|
||||
err = d.login()
|
||||
if err != nil {
|
||||
return nil, code, err
|
||||
}
|
||||
return d.request(api, method, callback, true)
|
||||
}
|
||||
return nil, code, fmt.Errorf("request failed,code: %d, message: %s", code, utils.Json.Get(res.Body(), "message").ToString())
|
||||
}
|
||||
return res.Body(), 200, nil
|
||||
}
|
||||
@@ -1,295 +0,0 @@
|
||||
package aliyundrive_open
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/rand/v2"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
anet "github.com/OpenListTeam/OpenList/v4/internal/net"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultCallbackConcurrency = 1
|
||||
callbackAcquireTimeout = time.Second
|
||||
callbackRequestAttempts = 3
|
||||
callbackRetryBaseDelay = 200 * time.Millisecond
|
||||
callbackErrorBodyLimit = 64 << 10
|
||||
)
|
||||
|
||||
var callbackLimiters = struct {
|
||||
sync.Mutex
|
||||
byUser map[string]*callbackLimiter
|
||||
}{byUser: make(map[string]*callbackLimiter)}
|
||||
|
||||
type callbackLimiter struct {
|
||||
userID string
|
||||
mu sync.Mutex
|
||||
active int
|
||||
nextID uint64
|
||||
registrations map[uint64]int
|
||||
changed chan struct{}
|
||||
}
|
||||
|
||||
type callbackRegistration struct {
|
||||
limiter *callbackLimiter
|
||||
id uint64
|
||||
once sync.Once
|
||||
}
|
||||
|
||||
type callbackPermit struct {
|
||||
limiter *callbackLimiter
|
||||
once sync.Once
|
||||
}
|
||||
|
||||
func normalizeCallbackConcurrency(limit int) int {
|
||||
if limit <= 0 {
|
||||
return defaultCallbackConcurrency
|
||||
}
|
||||
return limit
|
||||
}
|
||||
|
||||
func registerCallbackLimiter(userID string, limit int) *callbackRegistration {
|
||||
callbackLimiters.Lock()
|
||||
defer callbackLimiters.Unlock()
|
||||
|
||||
limiter := callbackLimiters.byUser[userID]
|
||||
if limiter == nil {
|
||||
limiter = &callbackLimiter{
|
||||
userID: userID,
|
||||
registrations: make(map[uint64]int),
|
||||
changed: make(chan struct{}),
|
||||
}
|
||||
callbackLimiters.byUser[userID] = limiter
|
||||
}
|
||||
limiter.mu.Lock()
|
||||
limiter.nextID++
|
||||
id := limiter.nextID
|
||||
limiter.registrations[id] = normalizeCallbackConcurrency(limit)
|
||||
limiter.signalLocked()
|
||||
limiter.mu.Unlock()
|
||||
return &callbackRegistration{limiter: limiter, id: id}
|
||||
}
|
||||
|
||||
func (r *callbackRegistration) unregister() {
|
||||
if r == nil || r.limiter == nil {
|
||||
return
|
||||
}
|
||||
r.once.Do(func() {
|
||||
callbackLimiters.Lock()
|
||||
defer callbackLimiters.Unlock()
|
||||
r.limiter.mu.Lock()
|
||||
delete(r.limiter.registrations, r.id)
|
||||
r.limiter.signalLocked()
|
||||
if len(r.limiter.registrations) == 0 && r.limiter.active == 0 {
|
||||
delete(callbackLimiters.byUser, r.limiter.userID)
|
||||
}
|
||||
r.limiter.mu.Unlock()
|
||||
})
|
||||
}
|
||||
|
||||
func (r *callbackRegistration) acquire(ctx context.Context) (*callbackPermit, error) {
|
||||
if r == nil || r.limiter == nil {
|
||||
return nil, errs.NewErr(errs.TemporaryCapacity, "callback limiter is unavailable")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
waitCtx, cancel := context.WithTimeout(ctx, callbackAcquireTimeout)
|
||||
defer cancel()
|
||||
for {
|
||||
r.limiter.mu.Lock()
|
||||
if r.limiter.active < r.limiter.limitLocked() {
|
||||
r.limiter.active++
|
||||
r.limiter.mu.Unlock()
|
||||
return &callbackPermit{limiter: r.limiter}, nil
|
||||
}
|
||||
changed := r.limiter.changed
|
||||
r.limiter.mu.Unlock()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
case <-waitCtx.Done():
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, errs.NewErr(errs.TemporaryCapacity, "timed out waiting for callback admission")
|
||||
case <-changed:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (l *callbackLimiter) limitLocked() int {
|
||||
limit := 0
|
||||
for _, registered := range l.registrations {
|
||||
if limit == 0 || registered < limit {
|
||||
limit = registered
|
||||
}
|
||||
}
|
||||
return limit
|
||||
}
|
||||
|
||||
func (l *callbackLimiter) signalLocked() {
|
||||
close(l.changed)
|
||||
l.changed = make(chan struct{})
|
||||
}
|
||||
|
||||
func (p *callbackPermit) release() {
|
||||
if p == nil || p.limiter == nil {
|
||||
return
|
||||
}
|
||||
p.once.Do(func() {
|
||||
callbackLimiters.Lock()
|
||||
defer callbackLimiters.Unlock()
|
||||
p.limiter.mu.Lock()
|
||||
p.limiter.active--
|
||||
p.limiter.signalLocked()
|
||||
if len(p.limiter.registrations) == 0 && p.limiter.active == 0 {
|
||||
delete(callbackLimiters.byUser, p.limiter.userID)
|
||||
}
|
||||
p.limiter.mu.Unlock()
|
||||
})
|
||||
}
|
||||
|
||||
func (d *AliyundriveOpen) callbackRegistration() *callbackRegistration {
|
||||
if d.callback != nil {
|
||||
return d.callback
|
||||
}
|
||||
if d.ref != nil {
|
||||
return d.ref.callbackRegistration()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliyundriveOpen) callbackRangeReader(url string, size int64) stream.RangeReaderFunc {
|
||||
return func(ctx context.Context, requested http_range.Range) (io.ReadCloser, error) {
|
||||
if requested.Length < 0 || requested.Start+requested.Length > size {
|
||||
requested.Length = size - requested.Start
|
||||
}
|
||||
for attempt := 0; attempt < callbackRequestAttempts; attempt++ {
|
||||
permit, err := d.callbackRegistration().acquire(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body, retry, err := openCallbackRange(ctx, url, size, requested)
|
||||
if !retry && err == nil {
|
||||
return newCallbackBody(ctx, body, permit.release), nil
|
||||
}
|
||||
permit.release()
|
||||
if !retry {
|
||||
return nil, err
|
||||
}
|
||||
if attempt+1 == callbackRequestAttempts {
|
||||
return nil, errs.NewErr(errs.TemporaryCapacity, "Aliyun callback concurrency limit rejected %d attempts", callbackRequestAttempts)
|
||||
}
|
||||
delay := callbackRetryBaseDelay << attempt
|
||||
delay += time.Duration(rand.Int64N(int64(delay / 2)))
|
||||
timer := time.NewTimer(delay)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
timer.Stop()
|
||||
return nil, ctx.Err()
|
||||
case <-timer.C:
|
||||
}
|
||||
}
|
||||
return nil, errs.NewErr(errs.TemporaryCapacity, "callback attempts exhausted")
|
||||
}
|
||||
}
|
||||
|
||||
func openCallbackRange(ctx context.Context, url string, size int64, requested http_range.Range) (io.ReadCloser, bool, error) {
|
||||
requestHeader, _ := ctx.Value(conf.RequestHeaderKey).(http.Header)
|
||||
header := anet.ProcessHeader(requestHeader, nil)
|
||||
header = http_range.ApplyRangeToHttpHeader(requested, header)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("create Aliyun callback request: %w", err)
|
||||
}
|
||||
req.Header = header
|
||||
response, err := anet.HttpClient().Do(req)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("Aliyun callback request failed: %w", err)
|
||||
}
|
||||
if response.StatusCode >= http.StatusBadRequest {
|
||||
defer response.Body.Close()
|
||||
body, readErr := io.ReadAll(io.LimitReader(response.Body, callbackErrorBodyLimit))
|
||||
if readErr != nil {
|
||||
return nil, false, fmt.Errorf("read Aliyun callback error response: %w", readErr)
|
||||
}
|
||||
if isCallbackCapacityRejection(response.StatusCode, body) {
|
||||
return nil, true, nil
|
||||
}
|
||||
message := strings.ReplaceAll(strings.TrimSpace(string(body)), url, "<redacted>")
|
||||
return nil, false, fmt.Errorf("Aliyun callback request failed: %w; response: %s", anet.HttpStatusCodeError(response.StatusCode), message)
|
||||
}
|
||||
if requested.Start == 0 && requested.Length == size || response.StatusCode == http.StatusPartialContent || callbackContentRangeStartsAt(response.Header, requested.Start) {
|
||||
return response.Body, false, nil
|
||||
}
|
||||
if response.StatusCode == http.StatusOK {
|
||||
body, rangeErr := anet.GetRangedHttpReader(response.Body, requested.Start, requested.Length)
|
||||
if rangeErr != nil {
|
||||
response.Body.Close()
|
||||
return nil, false, rangeErr
|
||||
}
|
||||
return body, false, nil
|
||||
}
|
||||
return response.Body, false, nil
|
||||
}
|
||||
|
||||
func isCallbackCapacityRejection(status int, body []byte) bool {
|
||||
return status == http.StatusForbidden &&
|
||||
strings.Contains(string(body), "RequestDeniedByCallback") &&
|
||||
strings.Contains(string(body), "ExceedMaxConcurrency")
|
||||
}
|
||||
|
||||
func callbackContentRangeStartsAt(header http.Header, offset int64) bool {
|
||||
start, _, err := http_range.ParseContentRange(header.Get("Content-Range"))
|
||||
return err == nil && start == offset
|
||||
}
|
||||
|
||||
type callbackBody struct {
|
||||
body io.ReadCloser
|
||||
release func()
|
||||
once sync.Once
|
||||
mu sync.Mutex
|
||||
stop func() bool
|
||||
}
|
||||
|
||||
func newCallbackBody(ctx context.Context, body io.ReadCloser, release func()) *callbackBody {
|
||||
b := &callbackBody{body: body, release: release}
|
||||
stop := context.AfterFunc(ctx, func() { _ = b.Close() })
|
||||
b.mu.Lock()
|
||||
b.stop = stop
|
||||
b.mu.Unlock()
|
||||
return b
|
||||
}
|
||||
|
||||
func (b *callbackBody) Read(p []byte) (int, error) {
|
||||
n, err := b.body.Read(p)
|
||||
if err != nil {
|
||||
_ = b.Close()
|
||||
}
|
||||
return n, err
|
||||
}
|
||||
|
||||
func (b *callbackBody) Close() error {
|
||||
var err error
|
||||
b.once.Do(func() {
|
||||
b.mu.Lock()
|
||||
stop := b.stop
|
||||
b.mu.Unlock()
|
||||
if stop != nil {
|
||||
stop()
|
||||
}
|
||||
err = b.body.Close()
|
||||
b.release()
|
||||
})
|
||||
return err
|
||||
}
|
||||
@@ -1,365 +0,0 @@
|
||||
package aliyundrive_open
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
)
|
||||
|
||||
func TestLinkSeparatesRedirectAndProxyRepresentations(t *testing.T) {
|
||||
oldConf := conf.Conf
|
||||
conf.Conf = &conf.Config{}
|
||||
t.Cleanup(func() { conf.Conf = oldConf })
|
||||
base.InitClient()
|
||||
|
||||
var server *httptest.Server
|
||||
server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/adrive/v1.0/user/getDriveInfo":
|
||||
_, _ = fmt.Fprint(w, `{"user_id":"user-1","resource_drive_id":"drive-1"}`)
|
||||
case "/adrive/v1.0/openFile/getDownloadUrl":
|
||||
_, _ = fmt.Fprintf(w, `{"url":%q}`, server.URL+"/callback")
|
||||
default:
|
||||
http.NotFound(w, r)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
oldAPIURL := API_URL
|
||||
API_URL = server.URL
|
||||
defer func() { API_URL = oldAPIURL }()
|
||||
|
||||
d := &AliyundriveOpen{Addition: Addition{AccessToken: "token"}}
|
||||
if err := d.Init(t.Context()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer d.Drop(context.Background())
|
||||
if d.CallbackConcurrency != defaultCallbackConcurrency {
|
||||
t.Fatalf("normalized callback concurrency = %d, want %d", d.CallbackConcurrency, defaultCallbackConcurrency)
|
||||
}
|
||||
|
||||
link, err := d.Link(t.Context(), &model.Object{ID: "file-1", Name: "file", Size: 1}, model.LinkArgs{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if link.RangeReader == nil {
|
||||
t.Fatal("proxy link must own callback acquisition through a range reader")
|
||||
}
|
||||
if _, ok := link.RangeReader.(stream.RateLimitRangeReaderFunc); !ok {
|
||||
t.Fatalf("proxy range reader type = %T, want server-rate-limited reader", link.RangeReader)
|
||||
}
|
||||
direct, err := d.Link(t.Context(), &model.Object{ID: "file-1", Name: "file", Size: 1}, model.LinkArgs{Redirect: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if direct.URL == "" || direct.RangeReader != nil {
|
||||
t.Fatal("redirect link must remain URL-only")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackRangeHoldsPermitUntilBodyClose(t *testing.T) {
|
||||
oldConf := conf.Conf
|
||||
conf.Conf = &conf.Config{}
|
||||
t.Cleanup(func() { conf.Conf = oldConf })
|
||||
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Length", "1")
|
||||
w.Header().Set("Content-Range", "bytes 0-0/1")
|
||||
w.WriteHeader(http.StatusPartialContent)
|
||||
_, _ = io.WriteString(w, "x")
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
registration := registerCallbackLimiter(t.Name(), 1)
|
||||
t.Cleanup(registration.unregister)
|
||||
d := &AliyundriveOpen{callback: registration}
|
||||
body, err := d.callbackRangeReader(server.URL, 1).RangeRead(t.Context(), http_range.Range{Length: 1})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
registration.limiter.mu.Lock()
|
||||
active := registration.limiter.active
|
||||
registration.limiter.mu.Unlock()
|
||||
if active != 1 {
|
||||
t.Fatalf("active callback bodies = %d, want 1", active)
|
||||
}
|
||||
if err := body.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
registration.limiter.mu.Lock()
|
||||
active = registration.limiter.active
|
||||
registration.limiter.mu.Unlock()
|
||||
if active != 0 {
|
||||
t.Fatalf("active callback bodies after Close = %d, want 0", active)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackLimiterUsesMinimumRegisteredLimit(t *testing.T) {
|
||||
firstRegistration := registerCallbackLimiter(t.Name(), 2)
|
||||
t.Cleanup(firstRegistration.unregister)
|
||||
first, err := firstRegistration.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
second, err := firstRegistration.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer first.release()
|
||||
defer second.release()
|
||||
|
||||
lowerRegistration := registerCallbackLimiter(t.Name(), 1)
|
||||
t.Cleanup(lowerRegistration.unregister)
|
||||
acquired := make(chan *callbackPermit, 1)
|
||||
go func() {
|
||||
permit, acquireErr := lowerRegistration.acquire(t.Context())
|
||||
if acquireErr == nil {
|
||||
acquired <- permit
|
||||
}
|
||||
}()
|
||||
|
||||
first.release()
|
||||
select {
|
||||
case permit := <-acquired:
|
||||
permit.release()
|
||||
t.Fatal("lowering the shared limit must wait for all excess bodies to drain")
|
||||
case <-time.After(100 * time.Millisecond):
|
||||
}
|
||||
second.release()
|
||||
select {
|
||||
case permit := <-acquired:
|
||||
permit.release()
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("admission did not resume after active bodies drained below the new limit")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackLimiterSeparatesUsers(t *testing.T) {
|
||||
firstUser := registerCallbackLimiter(t.Name()+"-first", 1)
|
||||
secondUser := registerCallbackLimiter(t.Name()+"-second", 1)
|
||||
t.Cleanup(firstUser.unregister)
|
||||
t.Cleanup(secondUser.unregister)
|
||||
first, err := firstUser.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer first.release()
|
||||
second, err := secondUser.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatalf("independent user was blocked: %v", err)
|
||||
}
|
||||
second.release()
|
||||
}
|
||||
|
||||
func TestCallbackLimiterReconfigureWaitsForOldBodies(t *testing.T) {
|
||||
userID := t.Name()
|
||||
oldRegistration := registerCallbackLimiter(userID, 2)
|
||||
first, err := oldRegistration.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
second, err := oldRegistration.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
oldRegistration.unregister()
|
||||
|
||||
newRegistration := registerCallbackLimiter(userID, 1)
|
||||
t.Cleanup(newRegistration.unregister)
|
||||
acquired := make(chan *callbackPermit, 1)
|
||||
go func() {
|
||||
permit, acquireErr := newRegistration.acquire(t.Context())
|
||||
if acquireErr == nil {
|
||||
acquired <- permit
|
||||
}
|
||||
}()
|
||||
first.release()
|
||||
select {
|
||||
case permit := <-acquired:
|
||||
permit.release()
|
||||
t.Fatal("reconfigured limiter admitted while an old body still occupied the new limit")
|
||||
case <-time.After(100 * time.Millisecond):
|
||||
}
|
||||
second.release()
|
||||
select {
|
||||
case permit := <-acquired:
|
||||
permit.release()
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("reconfigured limiter did not admit after old bodies drained")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackLimiterDistinguishesTimeoutAndCancellation(t *testing.T) {
|
||||
registration := registerCallbackLimiter(t.Name(), 1)
|
||||
t.Cleanup(registration.unregister)
|
||||
permit, err := registration.acquire(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer permit.release()
|
||||
|
||||
started := time.Now()
|
||||
_, err = registration.acquire(t.Context())
|
||||
if !errors.Is(err, errs.TemporaryCapacity) {
|
||||
t.Fatalf("admission timeout error = %v, want TemporaryCapacity", err)
|
||||
}
|
||||
if time.Since(started) < callbackAcquireTimeout {
|
||||
t.Fatal("admission timed out before the configured wait elapsed")
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
cancel()
|
||||
_, err = registration.acquire(ctx)
|
||||
if !errors.Is(err, context.Canceled) || errors.Is(err, errs.TemporaryCapacity) {
|
||||
t.Fatalf("canceled admission error = %v, want only context.Canceled", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackCapacityRejectionRequiresBothExactMarkers(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
body string
|
||||
want bool
|
||||
}{
|
||||
{name: "both", body: `{"code":"RequestDeniedByCallback","message":"ExceedMaxConcurrency"}`, want: true},
|
||||
{name: "code only", body: `{"code":"RequestDeniedByCallback"}`},
|
||||
{name: "message only", body: `{"message":"ExceedMaxConcurrency"}`},
|
||||
{name: "case differs", body: `{"code":"requestdeniedbycallback","message":"ExceedMaxConcurrency"}`},
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if got := isCallbackCapacityRejection(http.StatusForbidden, []byte(test.body)); got != test.want {
|
||||
t.Fatalf("classification = %v, want %v", got, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
if isCallbackCapacityRejection(http.StatusTooManyRequests, []byte(`RequestDeniedByCallback ExceedMaxConcurrency`)) {
|
||||
t.Fatal("non-403 response must not be classified as callback capacity")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackRangeRetriesOnlyVerifiedCapacityRejections(t *testing.T) {
|
||||
var requests atomic.Int32
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
requests.Add(1)
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_, _ = io.WriteString(w, `{"code":"RequestDeniedByCallback","message":"ExceedMaxConcurrency"}`)
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
registration := registerCallbackLimiter(t.Name(), 1)
|
||||
t.Cleanup(registration.unregister)
|
||||
d := &AliyundriveOpen{callback: registration}
|
||||
_, err := d.callbackRangeReader(server.URL+"?token=secret", 1).RangeRead(t.Context(), http_range.Range{Length: 1})
|
||||
if !errors.Is(err, errs.TemporaryCapacity) {
|
||||
t.Fatalf("verified rejection error = %v, want TemporaryCapacity", err)
|
||||
}
|
||||
if requests.Load() != callbackRequestAttempts {
|
||||
t.Fatalf("requests = %d, want %d", requests.Load(), callbackRequestAttempts)
|
||||
}
|
||||
if strings.Contains(err.Error(), "secret") {
|
||||
t.Fatal("capacity error leaked the signed callback URL")
|
||||
}
|
||||
permit, acquireErr := registration.acquire(t.Context())
|
||||
if acquireErr != nil {
|
||||
t.Fatalf("capacity retries leaked admission: %v", acquireErr)
|
||||
}
|
||||
permit.release()
|
||||
|
||||
requests.Store(0)
|
||||
permanent := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
requests.Add(1)
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_, _ = io.WriteString(w, `{"code":"RequestDeniedByCallback","message":"denied"}`)
|
||||
}))
|
||||
defer permanent.Close()
|
||||
_, err = d.callbackRangeReader(permanent.URL+"?token=secret", 1).RangeRead(t.Context(), http_range.Range{Length: 1})
|
||||
if errors.Is(err, errs.TemporaryCapacity) {
|
||||
t.Fatalf("permanent 403 error = %v, must not be TemporaryCapacity", err)
|
||||
}
|
||||
if requests.Load() != 1 {
|
||||
t.Fatalf("permanent 403 requests = %d, want 1", requests.Load())
|
||||
}
|
||||
if strings.Contains(err.Error(), "secret") {
|
||||
t.Fatal("permanent error leaked the signed callback URL")
|
||||
}
|
||||
}
|
||||
|
||||
type countingReadCloser struct {
|
||||
reader io.Reader
|
||||
closed atomic.Int32
|
||||
}
|
||||
|
||||
func (r *countingReadCloser) Read(p []byte) (int, error) { return r.reader.Read(p) }
|
||||
func (r *countingReadCloser) Close() error {
|
||||
r.closed.Add(1)
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestCallbackBodyReleasesExactlyOnce(t *testing.T) {
|
||||
underlying := &countingReadCloser{reader: strings.NewReader("x")}
|
||||
var released atomic.Int32
|
||||
body := newCallbackBody(t.Context(), underlying, func() { released.Add(1) })
|
||||
_, _ = io.ReadAll(body)
|
||||
if err := body.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := body.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if underlying.closed.Load() != 1 || released.Load() != 1 {
|
||||
t.Fatalf("close count = %d, release count = %d; want 1, 1", underlying.closed.Load(), released.Load())
|
||||
}
|
||||
}
|
||||
|
||||
type failingReadCloser struct {
|
||||
closed atomic.Int32
|
||||
}
|
||||
|
||||
func (*failingReadCloser) Read([]byte) (int, error) { return 0, errors.New("read failed") }
|
||||
func (r *failingReadCloser) Close() error {
|
||||
r.closed.Add(1)
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestCallbackBodyReadFailureReleasesPermit(t *testing.T) {
|
||||
underlying := &failingReadCloser{}
|
||||
var released atomic.Int32
|
||||
body := newCallbackBody(t.Context(), underlying, func() { released.Add(1) })
|
||||
if _, err := body.Read(make([]byte, 1)); err == nil {
|
||||
t.Fatal("read unexpectedly succeeded")
|
||||
}
|
||||
if underlying.closed.Load() != 1 || released.Load() != 1 {
|
||||
t.Fatalf("close count = %d, release count = %d; want 1, 1", underlying.closed.Load(), released.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallbackBodyCancellationReleasesPermit(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
underlying := &countingReadCloser{reader: strings.NewReader("x")}
|
||||
released := make(chan struct{}, 1)
|
||||
_ = newCallbackBody(ctx, underlying, func() { released <- struct{}{} })
|
||||
cancel()
|
||||
select {
|
||||
case <-released:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("context cancellation did not release callback admission")
|
||||
}
|
||||
if underlying.closed.Load() != 1 {
|
||||
t.Fatalf("underlying close count = %d, want 1", underlying.closed.Load())
|
||||
}
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
package aliyundrive_open
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
DriveType string `json:"drive_type" type:"select" options:"default,resource,backup" default:"resource"`
|
||||
driver.RootID
|
||||
RefreshToken string `json:"refresh_token" required:"true"`
|
||||
OrderBy string `json:"order_by" type:"select" options:"name,size,updated_at,created_at"`
|
||||
OrderDirection string `json:"order_direction" type:"select" options:"ASC,DESC"`
|
||||
UseOnlineAPI bool `json:"use_online_api" default:"true"`
|
||||
AlipanType string `json:"alipan_type" required:"true" type:"select" default:"default" options:"default,alipanTV"`
|
||||
APIAddress string `json:"api_url_address" default:"https://api.oplist.org/alicloud/renewapi"`
|
||||
ClientID string `json:"client_id" help:"Keep it empty if you don't have one"`
|
||||
ClientSecret string `json:"client_secret" help:"Keep it empty if you don't have one"`
|
||||
RemoveWay string `json:"remove_way" required:"true" type:"select" options:"trash,delete"`
|
||||
RapidUpload bool `json:"rapid_upload" help:"If you enable this option, the file will be uploaded to the server first, so the progress will be incorrect"`
|
||||
InternalUpload bool `json:"internal_upload" help:"If you are using Aliyun ECS is located in Beijing, you can turn it on to boost the upload speed"`
|
||||
LIVPDownloadFormat string `json:"livp_download_format" type:"select" options:"jpeg,mov" default:"jpeg"`
|
||||
CallbackConcurrency int `json:"callback_concurrency" type:"number" default:"1" help:"Maximum active proxied downloads shared by Aliyun user"`
|
||||
AccessToken string
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "AliyundriveOpen",
|
||||
DefaultRoot: "root",
|
||||
NoOverwriteUpload: true,
|
||||
}
|
||||
var API_URL = "https://openapi.alipan.com"
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &AliyundriveOpen{}
|
||||
})
|
||||
}
|
||||
@@ -1,169 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/antchfx/htmlquery"
|
||||
"github.com/antchfx/xpath"
|
||||
"github.com/pkg/errors"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
type AutoIndex struct {
|
||||
model.Storage
|
||||
Addition
|
||||
itemXPath *xpath.Expr
|
||||
nameXPath *xpath.Expr
|
||||
modifiedXPath *xpath.Expr
|
||||
sizeXPath *xpath.Expr
|
||||
ignores map[string]any
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *AutoIndex) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Init(ctx context.Context) error {
|
||||
var err error
|
||||
d.itemXPath, err = xpath.Compile(d.ItemXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Item XPath")
|
||||
}
|
||||
d.nameXPath, err = xpath.Compile(d.NameXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Name XPath")
|
||||
}
|
||||
if len(d.ModifiedXPath) > 0 {
|
||||
d.modifiedXPath, err = xpath.Compile(d.ModifiedXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Modified XPath")
|
||||
}
|
||||
}
|
||||
if len(d.SizeXPath) > 0 {
|
||||
d.sizeXPath, err = xpath.Compile(d.SizeXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Size XPath")
|
||||
}
|
||||
}
|
||||
ignores := strings.Split(d.IgnoreFileNames, "\n")
|
||||
d.ignores = make(map[string]any, len(ignores))
|
||||
for _, i := range ignores {
|
||||
i = strings.TrimSpace(i)
|
||||
if len(i) == 0 {
|
||||
continue
|
||||
}
|
||||
d.ignores[i] = struct{}{}
|
||||
}
|
||||
hasScheme := strings.Contains(d.URL, "://")
|
||||
hasSuffix := strings.HasSuffix(d.URL, "/")
|
||||
if !hasScheme || !hasSuffix {
|
||||
if !hasSuffix {
|
||||
d.URL = d.URL + "/"
|
||||
}
|
||||
if !hasScheme {
|
||||
d.URL = "https://" + d.URL
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) GetRoot(ctx context.Context) (model.Obj, error) {
|
||||
return &model.Object{
|
||||
Name: op.RootName,
|
||||
Path: d.URL,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Locked,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
res, err := base.RestyClient.R().
|
||||
SetContext(ctx).
|
||||
SetDoNotParseResponse(true).
|
||||
Get(dir.GetPath())
|
||||
if err != nil {
|
||||
return nil, errors.WithMessagef(err, "failed to get url [%s]", dir.GetPath())
|
||||
}
|
||||
defer res.RawResponse.Body.Close()
|
||||
doc, err := htmlquery.Parse(res.RawBody())
|
||||
if err != nil {
|
||||
return nil, errors.WithMessagef(err, "failed to parse [%s]", dir.GetPath())
|
||||
}
|
||||
itemsIter := d.itemXPath.Select(htmlquery.CreateXPathNavigator(doc))
|
||||
var objs []model.Obj
|
||||
for itemsIter.MoveNext() {
|
||||
nameFull, err := parseString(d.nameXPath.Evaluate(itemsIter.Current().Copy()))
|
||||
if err != nil {
|
||||
log.Warnf("skip invalid name evaluating result: %v", err)
|
||||
continue
|
||||
}
|
||||
nameFull = strings.TrimSpace(nameFull)
|
||||
name, isDir := strings.CutSuffix(nameFull, "/")
|
||||
if _, ok := d.ignores[name]; ok {
|
||||
continue
|
||||
}
|
||||
var size int64 = 0
|
||||
exact := false
|
||||
modified := time.Now()
|
||||
if d.sizeXPath != nil {
|
||||
size, exact, err = parseSize(d.sizeXPath.Evaluate(itemsIter.Current().Copy()))
|
||||
if err != nil {
|
||||
log.Errorf("failed to parse size of %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
if d.modifiedXPath != nil {
|
||||
modified, err = parseTime(d.modifiedXPath.Evaluate(itemsIter.Current().Copy()), d.ModifiedTimeFormat)
|
||||
if err != nil {
|
||||
log.Errorf("failed to parse modified time of %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
var o model.Obj = &model.Object{
|
||||
Name: name,
|
||||
IsFolder: isDir,
|
||||
Path: dir.GetPath() + nameFull,
|
||||
Modified: modified,
|
||||
Size: size,
|
||||
}
|
||||
if exact {
|
||||
o = &exactSizeObj{Obj: o}
|
||||
}
|
||||
objs = append(objs, o)
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if _, ok := file.(*exactSizeObj); ok || args.Redirect {
|
||||
return &model.Link{URL: file.GetPath()}, nil
|
||||
}
|
||||
res, err := base.RestyClient.R().
|
||||
SetContext(ctx).
|
||||
SetDoNotParseResponse(true).
|
||||
Head(file.GetPath())
|
||||
if err != nil {
|
||||
return nil, errors.WithMessagef(err, "failed to head [%s]", file.GetPath())
|
||||
}
|
||||
_ = res.RawResponse.Body.Close()
|
||||
return &model.Link{
|
||||
URL: file.GetPath(),
|
||||
ContentLength: res.RawResponse.ContentLength,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*AutoIndex)(nil)
|
||||
@@ -1,29 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
URL string `json:"url" required:"true"`
|
||||
ItemXPath string `json:"item_xpath" required:"true"`
|
||||
NameXPath string `json:"name_xpath" required:"true"`
|
||||
ModifiedXPath string `json:"modified_xpath"`
|
||||
SizeXPath string `json:"size_xpath"`
|
||||
IgnoreFileNames string `json:"ignore_file_names" type:"text" default:".\n..\nParent Directory\nUp"`
|
||||
ModifiedTimeFormat string `json:"modified_time_format" default:"02-Jan-2006 15:04" help:"Must be based on the time point Mon Jan 2 15:04:05 -0700 MST 2006"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "AutoIndex",
|
||||
LocalSort: true,
|
||||
CheckStatus: true,
|
||||
NoUpload: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &AutoIndex{}
|
||||
})
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
var (
|
||||
errEmptyEvaluateResult = fmt.Errorf("empty result")
|
||||
)
|
||||
|
||||
type exactSizeObj struct{ model.Obj }
|
||||
@@ -1,116 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/antchfx/xpath"
|
||||
"github.com/pkg/errors"
|
||||
)
|
||||
|
||||
var units = map[string]int64{
|
||||
"": 1,
|
||||
"b": 1,
|
||||
"byte": 1,
|
||||
"bytes": 1,
|
||||
"k": 1 << 10,
|
||||
"kb": 1 << 10,
|
||||
"kib": 1 << 10,
|
||||
"m": 1 << 20,
|
||||
"mb": 1 << 20,
|
||||
"mib": 1 << 20,
|
||||
"g": 1 << 30,
|
||||
"gb": 1 << 30,
|
||||
"gib": 1 << 30,
|
||||
"t": 1 << 40,
|
||||
"tb": 1 << 40,
|
||||
"tib": 1 << 40,
|
||||
"p": 1 << 50,
|
||||
"pb": 1 << 50,
|
||||
"pib": 1 << 50,
|
||||
}
|
||||
|
||||
func splitUnit(s string) (string, string) {
|
||||
for i := len(s) - 1; i >= 0; i-- {
|
||||
if s[i] >= '0' && s[i] <= '9' {
|
||||
return strings.TrimSpace(s[:i+1]), strings.TrimSpace(s[i+1:])
|
||||
}
|
||||
}
|
||||
return "", s
|
||||
}
|
||||
|
||||
func parseSize(a any) (int64, bool, error) {
|
||||
// 第二个返回值exact表示大小是否精确
|
||||
if f, ok := a.(float64); ok {
|
||||
return int64(f), false, nil
|
||||
}
|
||||
s, err := parseString(a)
|
||||
if errors.Is(err, errEmptyEvaluateResult) {
|
||||
// 可能是错误，也可能确实大小为0
|
||||
// 如果确实大小为0，大概率不会下载，exact返回false也不会有什么性能损失
|
||||
// 如果是错误，exact返回true会导致本地代理出错，综合来看返回false更好
|
||||
return 0, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return 0, false, err
|
||||
}
|
||||
s = strings.TrimSpace(s)
|
||||
if s == "-" {
|
||||
return 0, false, nil
|
||||
}
|
||||
nbs, unit := splitUnit(s)
|
||||
mul, ok := units[strings.ToLower(unit)]
|
||||
exact := mul == 1
|
||||
if !ok {
|
||||
mul = 1
|
||||
// 推测无单位，exact应为false
|
||||
}
|
||||
nb, err := strconv.ParseInt(nbs, 10, 64)
|
||||
if err != nil {
|
||||
fnb, err := strconv.ParseFloat(nbs, 64)
|
||||
if err != nil {
|
||||
return 0, false, fmt.Errorf("failed to convert %s to number", nbs)
|
||||
}
|
||||
nb = int64(fnb * float64(mul))
|
||||
exact = false
|
||||
} else {
|
||||
nb = nb * mul
|
||||
}
|
||||
return nb, exact, nil
|
||||
}
|
||||
|
||||
func parseString(res any) (string, error) {
|
||||
if r, ok := res.(string); ok {
|
||||
if len(r) == 0 {
|
||||
return "", errEmptyEvaluateResult
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
n, ok := res.(*xpath.NodeIterator)
|
||||
if !ok {
|
||||
return "", fmt.Errorf("unsupported evaluating result")
|
||||
}
|
||||
if !n.MoveNext() {
|
||||
return "", fmt.Errorf("no matched nodes")
|
||||
}
|
||||
ns := n.Current().Value()
|
||||
if len(ns) == 0 {
|
||||
return "", errEmptyEvaluateResult
|
||||
}
|
||||
return ns, nil
|
||||
}
|
||||
|
||||
func parseTime(res any, format string) (time.Time, error) {
|
||||
s, err := parseString(res)
|
||||
if err != nil {
|
||||
return time.Now(), err
|
||||
}
|
||||
s = strings.TrimSpace(s)
|
||||
t, err := time.Parse(format, s)
|
||||
if err != nil {
|
||||
return time.Now(), errors.WithMessagef(err, "failed to convert %s to time", s)
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
type wantType struct {
|
||||
v int64
|
||||
exact bool
|
||||
error bool
|
||||
}
|
||||
|
||||
func TestParseSize(t *testing.T) {
|
||||
tests := []struct {
|
||||
input string
|
||||
want wantType
|
||||
}{
|
||||
{"100", wantType{100, true, false}},
|
||||
{"1k", wantType{1024, false, false}},
|
||||
{"1kb", wantType{1024, false, false}},
|
||||
{"1K", wantType{1024, false, false}}, // case insensitive
|
||||
{"1.5m", wantType{1572864, false, false}}, // 1.5 * 1024^2
|
||||
{"500 bytes", wantType{500, true, false}},
|
||||
{"-", wantType{0, false, false}},
|
||||
{"", wantType{0, false, false}},
|
||||
{"abc", wantType{0, false, true}},
|
||||
{"1.5GB", wantType{1610612736, false, false}}, // 1.5 * 1024^3
|
||||
{"2t", wantType{2199023255552, false, false}}, // 2 * 1024^4
|
||||
{"1p", wantType{1125899906842624, false, false}}, // 1 * 1024^5
|
||||
{"0", wantType{0, true, false}},
|
||||
{" 100 ", wantType{100, true, false}}, // trimmed
|
||||
{"100b", wantType{100, true, false}},
|
||||
{"1gib", wantType{1073741824, false, false}}, // 1024^3
|
||||
{"1z", wantType{1, false, false}}, // invalid unit, mul=1
|
||||
{"1.5", wantType{1, false, false}}, // float without unit, truncated
|
||||
{"2.7k", wantType{2764, false, false}}, // 2.7 * 1024 truncated
|
||||
{"1.0g", wantType{1073741824, false, false}}, // 1.0 * 1024^3
|
||||
{"invalid", wantType{0, false, true}},
|
||||
{"123xyz", wantType{123, false, false}}, // unit not found, mul=1
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.input, func(t *testing.T) {
|
||||
got, exact, err := parseSize(tt.input)
|
||||
if got != tt.want.v || exact != tt.want.exact || (err != nil) != tt.want.error {
|
||||
t.Errorf("ParseSize(%q) = (%d, %t, %t), want (%d, %t, %t)", tt.input, got, exact, err != nil, tt.want.v, tt.want.exact, tt.want.error)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,226 +0,0 @@
|
||||
package bunny_storage
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type BunnyStorage struct {
|
||||
model.Storage
|
||||
Addition
|
||||
client *resty.Client
|
||||
endpoint *url.URL
|
||||
cdnBase *url.URL
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Config() driver.Config {
|
||||
cfg := config
|
||||
if d.StorageZoneName != "" && d.CDNBaseURL == "" {
|
||||
cfg.OnlyProxy = true
|
||||
cfg.PreferProxy = true
|
||||
}
|
||||
if d.CDNTokenKey != "" && d.CDNTokenIncludeIP {
|
||||
cfg.LinkCacheMode = driver.LinkCacheIP
|
||||
}
|
||||
return cfg
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Init(ctx context.Context) error {
|
||||
if d.RootFolderPath == "" {
|
||||
d.RootFolderPath = "/"
|
||||
}
|
||||
if d.Endpoint == "" {
|
||||
d.Endpoint = defaultEndpoint
|
||||
}
|
||||
if d.SignURLExpire <= 0 {
|
||||
d.SignURLExpire = 4
|
||||
}
|
||||
if d.CDNTokenMethod == "" {
|
||||
d.CDNTokenMethod = cdnTokenMethodSHA256
|
||||
}
|
||||
endpoint, err := normalizeBaseURL(d.Endpoint, defaultEndpoint)
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid endpoint: %w", err)
|
||||
}
|
||||
d.endpoint = endpoint
|
||||
if d.CDNBaseURL != "" {
|
||||
cdnBase, err := normalizeBaseURL(d.CDNBaseURL, "")
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid cdn_base_url: %w", err)
|
||||
}
|
||||
d.cdnBase = cdnBase
|
||||
}
|
||||
d.client = base.RestyClient
|
||||
if d.client == nil {
|
||||
d.client = base.NewRestyClient()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
var items []bunnyObject
|
||||
resp, err := d.authRequest().
|
||||
SetContext(ctx).
|
||||
SetResult(&items).
|
||||
Get(d.storageURL(dir.GetPath(), true))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := d.handleResponseError(resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := make([]model.Obj, 0, len(items))
|
||||
placeholder := d.placeholderName()
|
||||
for _, item := range items {
|
||||
if item.ObjectName == "" {
|
||||
continue
|
||||
}
|
||||
if !args.S3ShowPlaceholder && !item.IsDirectory && item.ObjectName == placeholder {
|
||||
continue
|
||||
}
|
||||
result = append(result, d.toObj(dir.GetPath(), item))
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if file.IsDir() {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
cacheTTL := time.Duration(0)
|
||||
if d.cdnBase != nil {
|
||||
linkURL := d.cdnURL(d.cdnObjectPath(file.GetPath()))
|
||||
link := &model.Link{
|
||||
URL: linkURL,
|
||||
ContentLength: file.GetSize(),
|
||||
Expiration: &cacheTTL,
|
||||
}
|
||||
if d.CDNTokenKey != "" {
|
||||
signedURL, _, err := d.signCDNURL(linkURL, args.IP)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
link.URL = signedURL
|
||||
}
|
||||
return link, nil
|
||||
}
|
||||
return &model.Link{
|
||||
URL: d.storageURL(file.GetPath(), false),
|
||||
Header: http.Header{"AccessKey": []string{d.AccessKey}},
|
||||
ContentLength: file.GetSize(),
|
||||
Expiration: &cacheTTL,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
dirPath := stdpath.Join(parentDir.GetPath(), dirName)
|
||||
placeholderPath := stdpath.Join(dirPath, d.placeholderName())
|
||||
if err := d.putReader(ctx, placeholderPath, bytes.NewReader(nil), 0, "application/octet-stream", nil); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
now := time.Now()
|
||||
return &model.Object{
|
||||
Path: dirPath,
|
||||
Name: dirName,
|
||||
Modified: now,
|
||||
Ctime: now,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Remove(ctx context.Context, obj model.Obj) error {
|
||||
resp, err := d.authRequest().
|
||||
SetContext(ctx).
|
||||
Delete(d.storageURL(obj.GetPath(), obj.IsDir()))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return d.handleResponseError(resp)
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
if up == nil {
|
||||
up = func(float64) {}
|
||||
}
|
||||
dstPath := stdpath.Join(dstDir.GetPath(), file.GetName())
|
||||
err := d.putReader(ctx, dstPath, driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: file,
|
||||
UpdateProgress: up,
|
||||
}), file.GetSize(), file.GetMimetype(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
now := time.Now()
|
||||
return &model.Object{
|
||||
Path: dstPath,
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
Modified: now,
|
||||
Ctime: now,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) putReader(ctx context.Context, path string, body any, size int64, contentType string, extraHeaders http.Header) error {
|
||||
if contentType == "" {
|
||||
contentType = "application/octet-stream"
|
||||
}
|
||||
req := d.authRequest().
|
||||
SetContext(ctx).
|
||||
SetBody(body).
|
||||
SetHeader("Content-Type", contentType)
|
||||
if size >= 0 {
|
||||
req.SetHeader("Content-Length", fmt.Sprint(size))
|
||||
}
|
||||
for key, values := range extraHeaders {
|
||||
for _, value := range values {
|
||||
req.SetHeader(key, value)
|
||||
}
|
||||
}
|
||||
resp, err := req.Put(d.storageURL(path, false))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return d.handleResponseError(resp)
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) Get(ctx context.Context, path string) (model.Obj, error) {
|
||||
fullPath := stdpath.Join(d.GetRootPath(), path)
|
||||
parentPath, name := stdpath.Split(fullPath)
|
||||
parentPath = strings.TrimSuffix(parentPath, "/")
|
||||
if parentPath == "" {
|
||||
parentPath = "/"
|
||||
}
|
||||
objs, err := d.List(ctx, &model.Object{Path: parentPath, IsFolder: true}, model.ListArgs{S3ShowPlaceholder: true})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, obj := range objs {
|
||||
if obj.GetName() == name {
|
||||
return obj, nil
|
||||
}
|
||||
}
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*BunnyStorage)(nil)
|
||||
var _ driver.Getter = (*BunnyStorage)(nil)
|
||||
@@ -1,32 +0,0 @@
|
||||
package bunny_storage
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
StorageZoneName string `json:"storage_zone_name" required:"true"`
|
||||
AccessKey string `json:"access_key" required:"true"`
|
||||
Endpoint string `json:"endpoint" required:"true" default:"storage.bunnycdn.com"`
|
||||
CDNBaseURL string `json:"cdn_base_url"`
|
||||
CDNTokenKey string `json:"cdn_token_key"`
|
||||
CDNTokenMethod string `json:"cdn_token_method" type:"select" options:"sha256,hmac_sha256" default:"sha256"`
|
||||
CDNTokenIncludeIP bool `json:"cdn_token_include_ip" default:"false"`
|
||||
SignURLExpire int `json:"sign_url_expire" type:"number" default:"4"`
|
||||
Placeholder string `json:"placeholder" default:".openlist"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Bunny Storage",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "/",
|
||||
CheckStatus: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &BunnyStorage{}
|
||||
})
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
package bunny_storage
|
||||
|
||||
import "time"
|
||||
|
||||
type bunnyObject struct {
|
||||
Guid string `json:"Guid"`
|
||||
StorageZoneName string `json:"StorageZoneName"`
|
||||
Path string `json:"Path"`
|
||||
ObjectName string `json:"ObjectName"`
|
||||
Length int64 `json:"Length"`
|
||||
LastChanged string `json:"LastChanged"`
|
||||
IsDirectory bool `json:"IsDirectory"`
|
||||
ServerID int `json:"ServerId"`
|
||||
UserID string `json:"UserId"`
|
||||
DateCreated string `json:"DateCreated"`
|
||||
StorageZoneID int64 `json:"StorageZoneId"`
|
||||
}
|
||||
|
||||
type apiError struct {
|
||||
HttpCode int `json:"HttpCode"`
|
||||
Message string `json:"Message"`
|
||||
}
|
||||
|
||||
type parsedTimes struct {
|
||||
modified time.Time
|
||||
created time.Time
|
||||
}
|
||||
@@ -1,275 +0,0 @@
|
||||
package bunny_storage
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultEndpoint = "storage.bunnycdn.com"
|
||||
defaultPlaceholder = ".openlist"
|
||||
|
||||
cdnTokenMethodSHA256 = "sha256"
|
||||
cdnTokenMethodHMACSHA256 = "hmac_sha256"
|
||||
)
|
||||
|
||||
func normalizeBaseURL(raw string, fallback string) (*url.URL, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if raw == "" {
|
||||
raw = fallback
|
||||
}
|
||||
if raw == "" {
|
||||
return nil, fmt.Errorf("empty url")
|
||||
}
|
||||
if !strings.Contains(raw, "://") {
|
||||
raw = "https://" + raw
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if u.Host == "" {
|
||||
return nil, fmt.Errorf("invalid url: %s", raw)
|
||||
}
|
||||
u.Path = strings.TrimRight(u.Path, "/")
|
||||
return u, nil
|
||||
}
|
||||
|
||||
func cleanObjectPath(path string) string {
|
||||
if path == "" {
|
||||
return "/"
|
||||
}
|
||||
return stdpath.Clean("/" + strings.TrimPrefix(path, "/"))
|
||||
}
|
||||
|
||||
func stripObjectPathPrefix(path string, prefix string) (string, bool) {
|
||||
path = cleanObjectPath(path)
|
||||
prefix = cleanObjectPath(prefix)
|
||||
if prefix == "/" {
|
||||
return path, false
|
||||
}
|
||||
if path == prefix {
|
||||
return "/", true
|
||||
}
|
||||
if strings.HasPrefix(path, prefix+"/") {
|
||||
return cleanObjectPath(strings.TrimPrefix(path, prefix)), true
|
||||
}
|
||||
return path, false
|
||||
}
|
||||
|
||||
func isObjectPathOrChild(path string, parent string) bool {
|
||||
path = cleanObjectPath(path)
|
||||
parent = cleanObjectPath(parent)
|
||||
return path == parent || strings.HasPrefix(path, parent+"/")
|
||||
}
|
||||
|
||||
func trimCDNBasePath(path string, mountPath string) string {
|
||||
path = cleanObjectPath(path)
|
||||
if path == "/" {
|
||||
return ""
|
||||
}
|
||||
if stripped, ok := stripObjectPathPrefix(path, mountPath); ok {
|
||||
path = stripped
|
||||
}
|
||||
if path == "/" {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimRight(path, "/")
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) cdnObjectPath(path string) string {
|
||||
objectPath := cleanObjectPath(path)
|
||||
if stripped, ok := stripObjectPathPrefix(objectPath, d.GetStorage().MountPath); ok {
|
||||
objectPath = stripped
|
||||
}
|
||||
rootPath := cleanObjectPath(d.GetRootPath())
|
||||
if rootPath != "/" && !isObjectPathOrChild(objectPath, rootPath) {
|
||||
objectPath = cleanObjectPath(stdpath.Join(rootPath, objectPath))
|
||||
}
|
||||
return objectPath
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) placeholderName() string {
|
||||
if d.Placeholder == "" {
|
||||
return defaultPlaceholder
|
||||
}
|
||||
return d.Placeholder
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) storageURL(path string, dir bool) string {
|
||||
u := *d.endpoint
|
||||
cleanPath := cleanObjectPath(path)
|
||||
zone := strings.Trim(d.StorageZoneName, "/")
|
||||
if cleanPath == "/" {
|
||||
u.Path = "/" + zone + "/"
|
||||
return u.String()
|
||||
}
|
||||
u.Path = "/" + zone + "/" + strings.TrimPrefix(cleanPath, "/")
|
||||
if dir && !strings.HasSuffix(u.Path, "/") {
|
||||
u.Path += "/"
|
||||
}
|
||||
return u.String()
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) cdnURL(path string) string {
|
||||
u := *d.cdnBase
|
||||
cleanPath := cleanObjectPath(path)
|
||||
basePath := trimCDNBasePath(u.Path, d.GetStorage().MountPath)
|
||||
if cleanPath == "/" {
|
||||
if basePath == "" {
|
||||
u.Path = "/"
|
||||
} else {
|
||||
u.Path = basePath + "/"
|
||||
}
|
||||
return u.String()
|
||||
}
|
||||
u.Path = basePath + "/" + strings.TrimPrefix(cleanPath, "/")
|
||||
return u.String()
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) authRequest() *resty.Request {
|
||||
return d.client.R().SetHeader("AccessKey", d.AccessKey)
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) handleResponseError(resp *resty.Response) error {
|
||||
if resp == nil {
|
||||
return fmt.Errorf("empty response")
|
||||
}
|
||||
if resp.StatusCode() >= http.StatusOK && resp.StatusCode() < http.StatusMultipleChoices {
|
||||
return nil
|
||||
}
|
||||
message := strings.TrimSpace(resp.String())
|
||||
var apiErrors []apiError
|
||||
if err := json.Unmarshal(resp.Body(), &apiErrors); err == nil && len(apiErrors) > 0 && apiErrors[0].Message != "" {
|
||||
message = apiErrors[0].Message
|
||||
}
|
||||
switch resp.StatusCode() {
|
||||
case http.StatusUnauthorized, http.StatusForbidden:
|
||||
return errs.NewErr(errs.PermissionDenied, "bunny storage request failed: %s", message)
|
||||
case http.StatusNotFound:
|
||||
return errs.NewErr(errs.ObjectNotFound, "bunny storage request failed: %s", message)
|
||||
default:
|
||||
return fmt.Errorf("bunny storage request failed: %s: %s", resp.Status(), message)
|
||||
}
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) parseTimes(item bunnyObject) parsedTimes {
|
||||
return parsedTimes{
|
||||
modified: parseBunnyTime(item.LastChanged, d.Modified),
|
||||
created: parseBunnyTime(item.DateCreated, time.Time{}),
|
||||
}
|
||||
}
|
||||
|
||||
func parseBunnyTime(value string, fallback time.Time) time.Time {
|
||||
if value == "" {
|
||||
return fallback
|
||||
}
|
||||
if t, err := time.Parse(time.RFC3339Nano, value); err == nil {
|
||||
return t
|
||||
}
|
||||
if t, err := time.Parse("2006-01-02T15:04:05.999999999", value); err == nil {
|
||||
return t
|
||||
}
|
||||
if t, err := time.Parse("2006-01-02T15:04:05", value); err == nil {
|
||||
return t
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) toObj(parentPath string, item bunnyObject) model.Obj {
|
||||
times := d.parseTimes(item)
|
||||
return &model.Object{
|
||||
ID: item.Guid,
|
||||
Path: stdpath.Join(parentPath, item.ObjectName),
|
||||
Name: item.ObjectName,
|
||||
Size: item.Length,
|
||||
Modified: times.modified,
|
||||
Ctime: times.created,
|
||||
IsFolder: item.IsDirectory,
|
||||
}
|
||||
}
|
||||
|
||||
func canonicalQuery(values url.Values) (string, error) {
|
||||
keys := make([]string, 0, len(values))
|
||||
for key := range values {
|
||||
if key == "token" || key == "expires" {
|
||||
continue
|
||||
}
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
parts := make([]string, 0, len(keys))
|
||||
for _, key := range keys {
|
||||
vals := values[key]
|
||||
if len(vals) > 1 {
|
||||
return "", fmt.Errorf("duplicate query parameter %q is not supported", key)
|
||||
}
|
||||
value := ""
|
||||
if len(vals) == 1 {
|
||||
value = vals[0]
|
||||
}
|
||||
parts = append(parts, key+"="+value)
|
||||
}
|
||||
return strings.Join(parts, "&"), nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) signCDNURL(rawURL string, clientIP string) (string, time.Duration, error) {
|
||||
return d.signCDNURLAt(rawURL, clientIP, time.Now())
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) signCDNURLAt(rawURL string, clientIP string, now time.Time) (string, time.Duration, error) {
|
||||
expire := time.Hour * time.Duration(d.SignURLExpire)
|
||||
if expire <= 0 {
|
||||
expire = 4 * time.Hour
|
||||
}
|
||||
expires := now.Add(expire).Unix()
|
||||
u, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
query := u.Query()
|
||||
parameterData, err := canonicalQuery(query)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
signaturePath, err := url.PathUnescape(u.EscapedPath())
|
||||
if err != nil {
|
||||
signaturePath = u.Path
|
||||
}
|
||||
if !d.CDNTokenIncludeIP {
|
||||
clientIP = ""
|
||||
}
|
||||
token := d.signCDNToken(signaturePath, strconv.FormatInt(expires, 10), parameterData, clientIP)
|
||||
query.Set("token", token)
|
||||
query.Set("expires", strconv.FormatInt(expires, 10))
|
||||
u.RawQuery = query.Encode()
|
||||
return u.String(), expire, nil
|
||||
}
|
||||
|
||||
func (d *BunnyStorage) signCDNToken(signaturePath string, expires string, parameterData string, clientIP string) string {
|
||||
switch strings.ToLower(strings.TrimSpace(d.CDNTokenMethod)) {
|
||||
case cdnTokenMethodHMACSHA256:
|
||||
message := signaturePath + expires + parameterData + clientIP
|
||||
mac := hmac.New(sha256.New, []byte(d.CDNTokenKey))
|
||||
_, _ = mac.Write([]byte(message))
|
||||
return "HS256-" + base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
|
||||
default:
|
||||
hashableBase := d.CDNTokenKey + signaturePath + expires + parameterData + clientIP
|
||||
sum := sha256.Sum256([]byte(hashableBase))
|
||||
return base64.RawURLEncoding.EncodeToString(sum[:])
|
||||
}
|
||||
}
|
||||
@@ -1,255 +0,0 @@
|
||||
package bunny_storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/url"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
func TestStorageURL(t *testing.T) {
|
||||
endpoint, err := normalizeBaseURL("ny.storage.bunnycdn.com", defaultEndpoint)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
driver := &BunnyStorage{
|
||||
Addition: Addition{
|
||||
StorageZoneName: "my-zone",
|
||||
},
|
||||
endpoint: endpoint,
|
||||
}
|
||||
if got, want := driver.storageURL("/", true), "https://ny.storage.bunnycdn.com/my-zone/"; got != want {
|
||||
t.Fatalf("root list url = %q, want %q", got, want)
|
||||
}
|
||||
if got, want := driver.storageURL("/dir/a file.txt", false), "https://ny.storage.bunnycdn.com/my-zone/dir/a%20file.txt"; got != want {
|
||||
t.Fatalf("file url = %q, want %q", got, want)
|
||||
}
|
||||
if got, want := driver.storageURL("/dir", true), "https://ny.storage.bunnycdn.com/my-zone/dir/"; got != want {
|
||||
t.Fatalf("dir url = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCDNURLWithBasePath(t *testing.T) {
|
||||
cdnBase, err := normalizeBaseURL("https://cdn.example.com/prefix/", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
driver := &BunnyStorage{cdnBase: cdnBase}
|
||||
if got, want := driver.cdnURL("/dir/a file.txt"), "https://cdn.example.com/prefix/dir/a%20file.txt"; got != want {
|
||||
t.Fatalf("cdn url = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCDNURLUsesObjectPathWithoutMountPath(t *testing.T) {
|
||||
cdnBase, err := normalizeBaseURL("https://cdn.firmant.me", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
driver := &BunnyStorage{
|
||||
Storage: model.Storage{MountPath: "/BS"},
|
||||
cdnBase: cdnBase,
|
||||
}
|
||||
if got, want := driver.cdnURL(driver.cdnObjectPath("/BS/Video")), "https://cdn.firmant.me/Video"; got != want {
|
||||
t.Fatalf("cdn url = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCDNURLDropsMountPathFromBaseURL(t *testing.T) {
|
||||
cdnBase, err := normalizeBaseURL("https://cdn.firmant.me/BS/", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
driver := &BunnyStorage{
|
||||
Storage: model.Storage{MountPath: "/BS"},
|
||||
cdnBase: cdnBase,
|
||||
}
|
||||
if got, want := driver.cdnURL(driver.cdnObjectPath("/Video")), "https://cdn.firmant.me/Video"; got != want {
|
||||
t.Fatalf("cdn url = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCDNObjectPathKeepsRootFolderPath(t *testing.T) {
|
||||
driver := &BunnyStorage{}
|
||||
driver.MountPath = "/BS"
|
||||
driver.RootFolderPath = "/library"
|
||||
if got, want := driver.cdnObjectPath("/BS/Video"), "/library/Video"; got != want {
|
||||
t.Fatalf("cdn object path = %q, want %q", got, want)
|
||||
}
|
||||
if got, want := driver.cdnObjectPath("/library/Video"), "/library/Video"; got != want {
|
||||
t.Fatalf("cdn object path = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinkDisablesLongLivedCache(t *testing.T) {
|
||||
cdnBase, err := normalizeBaseURL("https://cdn.example.com", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
driver := &BunnyStorage{cdnBase: cdnBase}
|
||||
link, err := driver.Link(context.Background(), &model.Object{
|
||||
Path: "/video.mp4",
|
||||
Name: "video.mp4",
|
||||
Size: 123,
|
||||
}, model.LinkArgs{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if link.Expiration == nil || *link.Expiration != 0 {
|
||||
t.Fatalf("link expiration = %v, want immediate cache expiration", link.Expiration)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignCDNURL(t *testing.T) {
|
||||
driver := &BunnyStorage{
|
||||
Addition: Addition{
|
||||
CDNTokenKey: "secret",
|
||||
CDNTokenIncludeIP: true,
|
||||
SignURLExpire: 1,
|
||||
},
|
||||
}
|
||||
signed, expire, err := driver.signCDNURLAt("https://zone.b-cdn.net/video.mp4?quality=high", "192.0.2.1", time.Unix(1700000000, 0))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if expire <= 0 {
|
||||
t.Fatal("expected positive expiration")
|
||||
}
|
||||
parsed, err := url.Parse(signed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
token := parsed.Query().Get("token")
|
||||
if want := "FxSpFem88zFo6uHFziwTuoMQTgDaD2PEn5n1zTMBUBI"; token != want {
|
||||
t.Fatalf("token = %q, want %q", token, want)
|
||||
}
|
||||
if parsed.Query().Get("expires") != "1700003600" {
|
||||
t.Fatalf("expires = %q, want 1700003600", parsed.Query().Get("expires"))
|
||||
}
|
||||
if parsed.Query().Get("quality") != "high" {
|
||||
t.Fatal("expected existing query parameters to be preserved")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignCDNURLSupportsHMACSHA256(t *testing.T) {
|
||||
driver := &BunnyStorage{
|
||||
Addition: Addition{
|
||||
CDNTokenKey: "secret",
|
||||
CDNTokenMethod: cdnTokenMethodHMACSHA256,
|
||||
CDNTokenIncludeIP: true,
|
||||
SignURLExpire: 1,
|
||||
},
|
||||
}
|
||||
signed, _, err := driver.signCDNURLAt("https://zone.b-cdn.net/video.mp4?quality=high", "192.0.2.1", time.Unix(1700000000, 0))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
parsed, err := url.Parse(signed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, want := parsed.Query().Get("token"), "HS256-sdrSSJE2JVwhSk2AoDUrmTV1muH6R5UHpZVcVfHeNxg"; got != want {
|
||||
t.Fatalf("token = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignCDNURLUsesDecodedPathForSHA256(t *testing.T) {
|
||||
driver := &BunnyStorage{
|
||||
Addition: Addition{
|
||||
CDNTokenKey: "secret",
|
||||
SignURLExpire: 1,
|
||||
},
|
||||
}
|
||||
signed, _, err := driver.signCDNURLAt("https://zone.b-cdn.net/%E8%A7%86%E9%A2%91/%5Ba%20b%5D.mp4", "", time.Unix(1700000000, 0))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
parsed, err := url.Parse(signed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, want := parsed.Query().Get("token"), "yq1evD7klw0e3DjCbv8dJptbW4S4JwVW3GKLnxfeKGM"; got != want {
|
||||
t.Fatalf("token = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignCDNURLTreatsPlusAsLiteralPathCharacter(t *testing.T) {
|
||||
driver := &BunnyStorage{
|
||||
Addition: Addition{
|
||||
CDNTokenKey: "secret",
|
||||
SignURLExpire: 1,
|
||||
},
|
||||
}
|
||||
now := time.Unix(1700000000, 0)
|
||||
literal, _, err := driver.signCDNURLAt("https://zone.b-cdn.net/a+b.mp4", "", now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
encoded, _, err := driver.signCDNURLAt("https://zone.b-cdn.net/a%2Bb.mp4", "", now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
space, _, err := driver.signCDNURLAt("https://zone.b-cdn.net/a%20b.mp4", "", now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
literalURL, err := url.Parse(literal)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
encodedURL, err := url.Parse(encoded)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
spaceURL, err := url.Parse(space)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, want := literalURL.Query().Get("token"), encodedURL.Query().Get("token"); got != want {
|
||||
t.Fatalf("literal plus token = %q, encoded plus token = %q", got, want)
|
||||
}
|
||||
if got, notWant := literalURL.Query().Get("token"), spaceURL.Query().Get("token"); got == notWant {
|
||||
t.Fatalf("literal plus token = %q, space token should differ", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignCDNURLRejectsDuplicateQueryParameters(t *testing.T) {
|
||||
driver := &BunnyStorage{
|
||||
Addition: Addition{
|
||||
CDNTokenKey: "secret",
|
||||
SignURLExpire: 1,
|
||||
},
|
||||
}
|
||||
_, _, err := driver.signCDNURLAt("https://zone.b-cdn.net/video.mp4?quality=high&quality=low", "", time.Unix(1700000000, 0))
|
||||
if err == nil {
|
||||
t.Fatal("expected duplicate query parameters to be rejected")
|
||||
}
|
||||
if got, want := err.Error(), `duplicate query parameter "quality" is not supported`; got != want {
|
||||
t.Fatalf("error = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseBunnyTimeSupportsFractionalSecondsWithoutTimezone(t *testing.T) {
|
||||
fallback := time.Unix(1, 0)
|
||||
got := parseBunnyTime("2023-03-21T13:38:31.693", fallback)
|
||||
want := time.Date(2023, 3, 21, 13, 38, 31, 693000000, time.UTC)
|
||||
if !got.Equal(want) {
|
||||
t.Fatalf("parsed time = %s, want %s", got.Format(time.RFC3339Nano), want.Format(time.RFC3339Nano))
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigProxyMode(t *testing.T) {
|
||||
registeredConfig := (&BunnyStorage{}).Config()
|
||||
if registeredConfig.OnlyProxy {
|
||||
t.Fatal("driver registration config should allow users to choose proxy policy")
|
||||
}
|
||||
withoutCDN := (&BunnyStorage{Addition: Addition{StorageZoneName: "my-zone"}}).Config()
|
||||
if !withoutCDN.OnlyProxy {
|
||||
t.Fatal("storage API links require AccessKey headers and should be proxied without CDN")
|
||||
}
|
||||
withCDN := (&BunnyStorage{Addition: Addition{StorageZoneName: "my-zone", CDNBaseURL: "https://zone.b-cdn.net"}}).Config()
|
||||
if withCDN.OnlyProxy {
|
||||
t.Fatal("CDN links should be allowed to redirect directly")
|
||||
}
|
||||
}
|
||||
@@ -1,545 +0,0 @@
|
||||
package chunk
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
stdpath "path"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/fs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/sign"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/avast/retry-go"
|
||||
)
|
||||
|
||||
type Chunk struct {
|
||||
model.Storage
|
||||
Addition
|
||||
}
|
||||
|
||||
func (d *Chunk) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Chunk) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Chunk) Init(ctx context.Context) error {
|
||||
if d.PartSize <= 0 {
|
||||
return errors.New("part size must be positive")
|
||||
}
|
||||
if len(d.ChunkPrefix) <= 0 {
|
||||
return errors.New("chunk folder prefix must not be empty")
|
||||
}
|
||||
d.RemotePath = utils.FixAndCleanPath(d.RemotePath)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Chunk) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (*Chunk) GetRootPath() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (d *Chunk) Get(ctx context.Context, path string) (model.Obj, error) {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
remoteActualPath = stdpath.Join(remoteActualPath, path)
|
||||
if remoteObj, err := op.Get(ctx, remoteStorage, remoteActualPath); err == nil {
|
||||
return &model.Object{
|
||||
Path: path,
|
||||
Name: remoteObj.GetName(),
|
||||
Size: remoteObj.GetSize(),
|
||||
Modified: remoteObj.ModTime(),
|
||||
IsFolder: remoteObj.IsDir(),
|
||||
HashInfo: remoteObj.GetHash(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
remoteActualDir, name := stdpath.Split(remoteActualPath)
|
||||
chunkName := d.ChunkPrefix + name
|
||||
chunkObjs, err := op.List(ctx, remoteStorage, stdpath.Join(remoteActualDir, chunkName), model.ListArgs{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var totalSize int64
|
||||
// 0号块默认为-1 以支持空文件
|
||||
chunkSizes := []int64{-1}
|
||||
h := make(map[*utils.HashType]string)
|
||||
var first model.Obj
|
||||
for _, o := range chunkObjs {
|
||||
if o.IsDir() {
|
||||
continue
|
||||
}
|
||||
if after, ok := strings.CutPrefix(o.GetName(), "hash_"); ok {
|
||||
hn, value, ok := strings.Cut(strings.TrimSuffix(after, d.CustomExt), "_")
|
||||
if ok {
|
||||
ht, ok := utils.GetHashByName(hn)
|
||||
if ok {
|
||||
h[ht] = value
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
idx, err := strconv.Atoi(strings.TrimSuffix(o.GetName(), d.CustomExt))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
totalSize += o.GetSize()
|
||||
if len(chunkSizes) > idx {
|
||||
if idx == 0 {
|
||||
first = o
|
||||
}
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
} else if len(chunkSizes) == idx {
|
||||
chunkSizes = append(chunkSizes, o.GetSize())
|
||||
} else {
|
||||
newChunkSizes := make([]int64, idx+1)
|
||||
copy(newChunkSizes, chunkSizes)
|
||||
chunkSizes = newChunkSizes
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
}
|
||||
}
|
||||
reqDir, _ := stdpath.Split(path)
|
||||
// 文件块不完整时，返回文件夹对象
|
||||
if chunkSizes[0] == -1 {
|
||||
return &model.Object{
|
||||
Path: stdpath.Join(reqDir, chunkName),
|
||||
Name: chunkName,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
}, nil
|
||||
}
|
||||
for i, l := 1, len(chunkSizes)-1; i < l; i++ {
|
||||
if chunkSizes[i] == 0 {
|
||||
return &model.Object{
|
||||
Path: stdpath.Join(reqDir, chunkName),
|
||||
Name: chunkName,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
objRes := chunkObject{
|
||||
Object: model.Object{
|
||||
Path: stdpath.Join(reqDir, chunkName),
|
||||
Name: name,
|
||||
Size: totalSize,
|
||||
Modified: first.ModTime(),
|
||||
Ctime: first.CreateTime(),
|
||||
},
|
||||
chunkSizes: chunkSizes,
|
||||
}
|
||||
if len(h) > 0 {
|
||||
objRes.HashInfo = utils.NewHashInfoByMap(h)
|
||||
}
|
||||
return &objRes, nil
|
||||
}
|
||||
|
||||
func (d *Chunk) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
remoteActualDir := stdpath.Join(remoteActualPath, dir.GetPath())
|
||||
remoteObjs, err := op.List(ctx, remoteStorage, remoteActualDir, model.ListArgs{
|
||||
ReqPath: args.ReqPath,
|
||||
Refresh: args.Refresh,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := make([]model.Obj, 0, len(remoteObjs))
|
||||
listG, listCtx := errgroup.NewGroupWithContext(ctx, d.NumListWorkers, retry.Attempts(3))
|
||||
for _, obj := range remoteObjs {
|
||||
if utils.IsCanceled(listCtx) {
|
||||
break
|
||||
}
|
||||
rawName := obj.GetName()
|
||||
if obj.IsDir() {
|
||||
if name, ok := strings.CutPrefix(rawName, d.ChunkPrefix); ok {
|
||||
resultIdx := len(result)
|
||||
result = append(result, nil)
|
||||
listG.Go(func(ctx context.Context) error {
|
||||
chunkObjs, err := op.List(ctx, remoteStorage, stdpath.Join(remoteActualDir, rawName), model.ListArgs{
|
||||
Refresh: args.Refresh,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var totalSize int64
|
||||
// 0号块默认为-1 以支持空文件
|
||||
chunkSizes := []int64{-1}
|
||||
h := make(map[*utils.HashType]string)
|
||||
var first model.Obj
|
||||
for _, o := range chunkObjs {
|
||||
if o.IsDir() {
|
||||
continue
|
||||
}
|
||||
if after, ok := strings.CutPrefix(o.GetName(), "hash_"); ok {
|
||||
hn, value, ok := strings.Cut(strings.TrimSuffix(after, d.CustomExt), "_")
|
||||
if ok {
|
||||
ht, ok := utils.GetHashByName(hn)
|
||||
if ok {
|
||||
h[ht] = value
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
idx, err := strconv.Atoi(strings.TrimSuffix(o.GetName(), d.CustomExt))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
totalSize += o.GetSize()
|
||||
if len(chunkSizes) > idx {
|
||||
if idx == 0 {
|
||||
first = o
|
||||
}
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
} else if len(chunkSizes) == idx {
|
||||
chunkSizes = append(chunkSizes, o.GetSize())
|
||||
} else {
|
||||
newChunkSizes := make([]int64, idx+1)
|
||||
copy(newChunkSizes, chunkSizes)
|
||||
chunkSizes = newChunkSizes
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
}
|
||||
}
|
||||
// 文件块不完整时，返回文件夹对象
|
||||
if chunkSizes[0] == -1 {
|
||||
result[resultIdx] = &model.Object{
|
||||
Name: rawName,
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: true,
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for i, l := 1, len(chunkSizes)-1; i < l; i++ {
|
||||
if chunkSizes[i] == 0 {
|
||||
result[resultIdx] = &model.Object{
|
||||
Name: rawName,
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: true,
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
objRes := model.Object{
|
||||
Name: name,
|
||||
Size: totalSize,
|
||||
Modified: first.ModTime(),
|
||||
Ctime: first.CreateTime(),
|
||||
}
|
||||
if len(h) > 0 {
|
||||
objRes.HashInfo = utils.NewHashInfoByMap(h)
|
||||
}
|
||||
if !d.Thumbnail {
|
||||
result[resultIdx] = &objRes
|
||||
} else {
|
||||
thumbPath := stdpath.Join(args.ReqPath, ".thumbnails", name+".webp")
|
||||
thumb := fmt.Sprintf("%s/d%s?sign=%s",
|
||||
common.GetApiUrl(ctx),
|
||||
utils.EncodePath(thumbPath, true),
|
||||
sign.Sign(thumbPath))
|
||||
result[resultIdx] = &model.ObjThumb{
|
||||
Object: objRes,
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumb,
|
||||
},
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
if !d.ShowHidden && strings.HasPrefix(rawName, ".") {
|
||||
continue
|
||||
}
|
||||
thumb, ok := model.GetThumb(obj)
|
||||
objRes := model.Object{
|
||||
Name: rawName,
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: obj.IsDir(),
|
||||
HashInfo: obj.GetHash(),
|
||||
}
|
||||
if !ok {
|
||||
result = append(result, &objRes)
|
||||
} else {
|
||||
result = append(result, &model.ObjThumb{
|
||||
Object: objRes,
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumb,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
if err = listG.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (d *Chunk) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
chunkFile, ok := file.(*chunkObject)
|
||||
remoteActualPath = stdpath.Join(remoteActualPath, file.GetPath())
|
||||
if !ok {
|
||||
l, _, err := op.Link(ctx, remoteStorage, remoteActualPath, args)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return l.Clone(), nil
|
||||
}
|
||||
// 检查0号块不等于-1 以支持空文件
|
||||
// 如果块数量大于1 最后一块不可能为0
|
||||
// 只检查中间块是否有0
|
||||
if chunkFile.chunkSizes[0] == -1 {
|
||||
return nil, fmt.Errorf("chunk part[%d] are missing", 0)
|
||||
}
|
||||
for i, l := 1, len(chunkFile.chunkSizes)-1; i < l; i++ {
|
||||
if chunkFile.chunkSizes[i] == 0 {
|
||||
return nil, fmt.Errorf("chunk part[%d] are missing", i)
|
||||
}
|
||||
}
|
||||
fileSize := chunkFile.GetSize()
|
||||
mergedRrf := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
|
||||
start := httpRange.Start
|
||||
length := httpRange.Length
|
||||
if length < 0 || start+length > fileSize {
|
||||
length = fileSize - start
|
||||
}
|
||||
if length == 0 {
|
||||
return io.NopCloser(strings.NewReader("")), nil
|
||||
}
|
||||
rs := make([]io.Reader, 0)
|
||||
cs := make(utils.Closers, 0)
|
||||
var (
|
||||
rc io.ReadCloser
|
||||
readFrom bool
|
||||
)
|
||||
for idx, chunkSize := range chunkFile.chunkSizes {
|
||||
if readFrom {
|
||||
l, o, err := op.Link(ctx, remoteStorage, stdpath.Join(remoteActualPath, d.getPartName(idx)), args)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
cs = append(cs, l)
|
||||
chunkSize2 := l.ContentLength
|
||||
if chunkSize2 <= 0 {
|
||||
chunkSize2 = o.GetSize()
|
||||
}
|
||||
if chunkSize2 != chunkSize {
|
||||
_ = cs.Close()
|
||||
return nil, fmt.Errorf("chunk part[%d] size not match", idx)
|
||||
}
|
||||
rrf, err := stream.GetRangeReaderFromLink(chunkSize2, l)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
newLength := length - chunkSize2
|
||||
if newLength >= 0 {
|
||||
length = newLength
|
||||
rc, err = rrf.RangeRead(ctx, http_range.Range{Length: -1})
|
||||
} else {
|
||||
rc, err = rrf.RangeRead(ctx, http_range.Range{Length: length})
|
||||
}
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
rs = append(rs, rc)
|
||||
cs = append(cs, rc)
|
||||
if newLength <= 0 {
|
||||
return utils.ReadCloser{
|
||||
Reader: io.MultiReader(rs...),
|
||||
Closer: &cs,
|
||||
}, nil
|
||||
}
|
||||
} else if newStart := start - chunkSize; newStart >= 0 {
|
||||
start = newStart
|
||||
} else {
|
||||
l, o, err := op.Link(ctx, remoteStorage, stdpath.Join(remoteActualPath, d.getPartName(idx)), args)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
cs = append(cs, l)
|
||||
chunkSize2 := l.ContentLength
|
||||
if chunkSize2 <= 0 {
|
||||
chunkSize2 = o.GetSize()
|
||||
}
|
||||
if chunkSize2 != chunkSize {
|
||||
_ = cs.Close()
|
||||
return nil, fmt.Errorf("chunk part[%d] size not match", idx)
|
||||
}
|
||||
rrf, err := stream.GetRangeReaderFromLink(chunkSize2, l)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
rc, err = rrf.RangeRead(ctx, http_range.Range{Start: start, Length: -1})
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
length -= chunkSize2 - start
|
||||
cs = append(cs, rc)
|
||||
if length <= 0 {
|
||||
return utils.ReadCloser{
|
||||
Reader: rc,
|
||||
Closer: &cs,
|
||||
}, nil
|
||||
}
|
||||
rs = append(rs, rc)
|
||||
readFrom = true
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("invalid range: start=%d,length=%d,fileSize=%d", httpRange.Start, httpRange.Length, fileSize)
|
||||
}
|
||||
return &model.Link{
|
||||
RangeReader: stream.RangeReaderFunc(mergedRrf),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *Chunk) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
path := stdpath.Join(d.RemotePath, parentDir.GetPath(), dirName)
|
||||
return fs.MakeDir(ctx, path)
|
||||
}
|
||||
|
||||
func (d *Chunk) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
src := stdpath.Join(d.RemotePath, srcObj.GetPath())
|
||||
dst := stdpath.Join(d.RemotePath, dstDir.GetPath())
|
||||
_, err := fs.Move(ctx, src, dst)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Chunk) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
if _, ok := srcObj.(*chunkObject); ok {
|
||||
newName = d.ChunkPrefix + newName
|
||||
}
|
||||
return fs.Rename(ctx, stdpath.Join(d.RemotePath, srcObj.GetPath()), newName)
|
||||
}
|
||||
|
||||
func (d *Chunk) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
dst := stdpath.Join(d.RemotePath, dstDir.GetPath())
|
||||
src := stdpath.Join(d.RemotePath, srcObj.GetPath())
|
||||
_, err := fs.Copy(ctx, src, dst)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Chunk) Remove(ctx context.Context, obj model.Obj) error {
|
||||
return fs.Remove(ctx, stdpath.Join(d.RemotePath, obj.GetPath()))
|
||||
}
|
||||
|
||||
func (d *Chunk) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if (d.Thumbnail && dstDir.GetName() == ".thumbnails") || (d.ChunkLargeFileOnly && file.GetSize() <= d.PartSize) {
|
||||
return op.Put(ctx, remoteStorage, stdpath.Join(remoteActualPath, dstDir.GetPath()), file, up)
|
||||
}
|
||||
upReader := &driver.ReaderUpdatingProgress{
|
||||
Reader: file,
|
||||
UpdateProgress: up,
|
||||
}
|
||||
dst := stdpath.Join(remoteActualPath, dstDir.GetPath(), d.ChunkPrefix+file.GetName())
|
||||
skipHookCtx := context.WithValue(ctx, conf.SkipHookKey, struct{}{})
|
||||
if d.StoreHash {
|
||||
for ht, value := range file.GetHash().All() {
|
||||
_ = op.Put(skipHookCtx, remoteStorage, dst, &stream.FileStream{
|
||||
Obj: &model.Object{
|
||||
Name: fmt.Sprintf("hash_%s_%s%s", ht.Name, value, d.CustomExt),
|
||||
Size: 1,
|
||||
Modified: file.ModTime(),
|
||||
},
|
||||
Mimetype: "application/octet-stream",
|
||||
Reader: bytes.NewReader([]byte{0}), // 兼容不支持空文件的驱动
|
||||
}, nil)
|
||||
}
|
||||
}
|
||||
fullPartCount := int(file.GetSize() / d.PartSize)
|
||||
tailSize := file.GetSize() % d.PartSize
|
||||
if tailSize == 0 && fullPartCount > 0 {
|
||||
fullPartCount--
|
||||
tailSize = d.PartSize
|
||||
}
|
||||
partIndex := 0
|
||||
for partIndex < fullPartCount {
|
||||
err = op.Put(skipHookCtx, remoteStorage, dst, &stream.FileStream{
|
||||
Obj: &model.Object{
|
||||
Name: d.getPartName(partIndex),
|
||||
Size: d.PartSize,
|
||||
Modified: file.ModTime(),
|
||||
},
|
||||
Mimetype: file.GetMimetype(),
|
||||
Reader: io.LimitReader(upReader, d.PartSize),
|
||||
}, nil)
|
||||
if err != nil {
|
||||
_ = op.Remove(ctx, remoteStorage, dst)
|
||||
return err
|
||||
}
|
||||
partIndex++
|
||||
}
|
||||
err = op.Put(ctx, remoteStorage, dst, &stream.FileStream{
|
||||
Obj: &model.Object{
|
||||
Name: d.getPartName(fullPartCount),
|
||||
Size: tailSize,
|
||||
Modified: file.ModTime(),
|
||||
},
|
||||
Mimetype: file.GetMimetype(),
|
||||
Reader: upReader,
|
||||
}, nil)
|
||||
if err != nil {
|
||||
_ = op.Remove(ctx, remoteStorage, dst)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Chunk) getPartName(part int) string {
|
||||
return fmt.Sprintf("%d%s", part, d.CustomExt)
|
||||
}
|
||||
|
||||
func (d *Chunk) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
remoteStorage, err := fs.GetStorage(d.RemotePath, &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
remoteDetails, err := op.GetStorageDetails(ctx, remoteStorage)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: remoteDetails.DiskUsage,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Chunk)(nil)
|
||||
@@ -1,39 +0,0 @@
|
||||
package chunk
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
RemotePath string `json:"remote_path" required:"true"`
|
||||
PartSize int64 `json:"part_size" required:"true" type:"number" help:"bytes"`
|
||||
ChunkLargeFileOnly bool `json:"chunk_large_file_only" default:"false" help:"chunk only if file size > part_size"`
|
||||
ChunkPrefix string `json:"chunk_prefix" type:"string" default:"[openlist_chunk]" help:"the prefix of chunk folder"`
|
||||
CustomExt string `json:"custom_ext" type:"string"`
|
||||
StoreHash bool `json:"store_hash" type:"bool" default:"true"`
|
||||
NumListWorkers int `json:"num_list_workers" required:"true" type:"number" default:"5"`
|
||||
|
||||
Thumbnail bool `json:"thumbnail" required:"true" default:"false" help:"enable thumbnail which pre-generated under .thumbnails folder"`
|
||||
ShowHidden bool `json:"show_hidden" default:"true" required:"false" help:"show hidden directories and files"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Chunk",
|
||||
LocalSort: true,
|
||||
OnlyProxy: true,
|
||||
NoCache: true,
|
||||
DefaultRoot: "/",
|
||||
NoLinkURL: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Chunk{
|
||||
Addition: Addition{
|
||||
ChunkPrefix: "[openlist_chunk]",
|
||||
NumListWorkers: 5,
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
package chunk
|
||||
|
||||
import "github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
|
||||
type chunkObject struct {
|
||||
model.Object
|
||||
chunkSizes []int64
|
||||
}
|
||||
@@ -1,168 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/cache"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type CFImgBed struct {
|
||||
model.Storage
|
||||
Addition
|
||||
client *resty.Client
|
||||
virtualDir *cache.WeakCacheMap[string, model.Object]
|
||||
publicUrlPrefix string
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Config() driver.Config { return config }
|
||||
func (d *CFImgBed) GetAddition() driver.Additional { return &d.Addition }
|
||||
|
||||
func (d *CFImgBed) Init(ctx context.Context) error {
|
||||
d.UploadThread = min(d.UploadThread, 32)
|
||||
if d.UploadThread < 1 {
|
||||
d.UploadThread = 3
|
||||
}
|
||||
d.Address = strings.TrimRight(d.Address, "/")
|
||||
|
||||
d.client = base.NewRestyClient().
|
||||
SetBaseURL(d.Address).
|
||||
SetHeader("Authorization", "Bearer "+d.Token).
|
||||
SetDebug(false)
|
||||
|
||||
// 连通性测试：尝试获取根目录单条数据
|
||||
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"start": "0",
|
||||
"count": "1",
|
||||
"dir": "/",
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("init verification failed: %w", err)
|
||||
}
|
||||
d.virtualDir = cache.NewWeakCacheMap[string, model.Object]()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Drop(ctx context.Context) error {
|
||||
if d.virtualDir != nil {
|
||||
d.virtualDir.Clear()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
// if !args.Refresh && model.ObjHasMask(dir, model.Virtual) {
|
||||
// if _, ok := d.virtualDir.Load(dir.GetPath()); ok {
|
||||
// return nil, nil
|
||||
// }
|
||||
// }
|
||||
|
||||
var dirSeen map[string]bool
|
||||
var fileSeen map[string]bool
|
||||
var objs []model.Obj
|
||||
|
||||
start := 0
|
||||
for {
|
||||
var resp ListResponse
|
||||
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"dir": dir.GetPath(),
|
||||
"start": fmt.Sprintf("%d", start),
|
||||
"count": fmt.Sprintf("%d", listPageSize),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(resp.Files) == 0 && len(resp.Directories) == 0 {
|
||||
break
|
||||
}
|
||||
|
||||
if start == 0 {
|
||||
dirSeen = make(map[string]bool, len(resp.Directories))
|
||||
fileSeen = make(map[string]bool, len(resp.Files))
|
||||
objs = make([]model.Obj, 0, len(resp.Directories)+len(resp.Files))
|
||||
}
|
||||
|
||||
for _, rawDir := range resp.Directories {
|
||||
rawDir = "/" + strings.TrimRight(rawDir, "/")
|
||||
if !dirSeen[rawDir] {
|
||||
dirSeen[rawDir] = true
|
||||
objs = append(objs, &model.Object{
|
||||
Path: rawDir,
|
||||
Name: path.Base(rawDir),
|
||||
Modified: d.Modified,
|
||||
IsFolder: true,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
for _, item := range resp.Files {
|
||||
if !fileSeen[item.Name] {
|
||||
fileSeen[item.Name] = true
|
||||
objs = append(objs, parseFile(item))
|
||||
}
|
||||
}
|
||||
|
||||
// 如果当前获取的数量少于分页大小，说明已加载完毕
|
||||
if len(resp.Files)+len(resp.Directories) < listPageSize {
|
||||
break
|
||||
}
|
||||
start += listPageSize
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if d.publicUrlPrefix != "" {
|
||||
return &model.Link{URL: d.publicUrlPrefix + utils.EncodePath(file.GetPath())}, nil
|
||||
}
|
||||
return &model.Link{URL: d.Address + "/file" + utils.EncodePath(file.GetPath())}, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Get(ctx context.Context, pathStr string) (model.Obj, error) {
|
||||
fullPath := path.Join(d.RootFolderPath, pathStr)
|
||||
if obj, found := d.virtualDir.Load(fullPath); found {
|
||||
return obj, nil
|
||||
}
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
|
||||
// MakeDir 在图床中通常是虚拟的，此处返回虚拟目录对象以支持上传时的路径展示
|
||||
func (d *CFImgBed) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
fullPath := path.Join(parentDir.GetPath(), dirName)
|
||||
temp := &model.Object{
|
||||
Path: fullPath,
|
||||
Name: dirName,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Virtual,
|
||||
}
|
||||
d.virtualDir.Store(fullPath, temp)
|
||||
return temp, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Remove(ctx context.Context, obj model.Obj) error {
|
||||
reqPath := obj.GetPath()
|
||||
if model.ObjHasMask(obj, model.Virtual) {
|
||||
d.virtualDir.Delete(reqPath)
|
||||
return nil
|
||||
}
|
||||
_, err := d.doRequest(ctx, http.MethodPost, deleteApi+utils.EncodePath(reqPath), func(req *resty.Request) {
|
||||
req.SetQueryParam("folder", fmt.Sprintf("%t", obj.IsDir()))
|
||||
}, nil)
|
||||
return err
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*CFImgBed)(nil)
|
||||
@@ -1,27 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Address string `json:"address" required:"true" help:"Backend API address of the image hosting service, e.g., https://img.example.com"`
|
||||
Token string `json:"token" required:"true" help:"Authentication Token"`
|
||||
SmallChannelName string `json:"smallChannelName" help:"Channel name for regular files (typically <20MB)"`
|
||||
LargeChannelName string `json:"largeChannelName" help:"Channel name for large files"`
|
||||
LargeChannelType string `json:"largeChannelType" type:"select" options:",huggingface,telegram,cfr2,s3,discord" help:"Large File Channel Type: Hugging Face (Direct Upload)、telegram/cfr2/s3/discord(Multipart Upload)"`
|
||||
UploadThread int `json:"uploadThread" type:"number" default:"3" help:"Concurrent thread count for HuggingFace chunked direct upload"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "cloudflare_imgbed",
|
||||
LocalSort: true,
|
||||
NoUpload: false,
|
||||
DefaultRoot: "/",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver { return &CFImgBed{} })
|
||||
}
|
||||
@@ -1,117 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
const listPageSize = 1000
|
||||
|
||||
// ListResponse 列表接口响应
|
||||
type ListResponse struct {
|
||||
Files []FileItem `json:"files"`
|
||||
Directories []string `json:"directories"`
|
||||
}
|
||||
|
||||
type FileItem struct {
|
||||
Name string `json:"name"`
|
||||
Metadata map[string]interface{} `json:"metadata"` // 存储文件大小、哈希、时间戳等
|
||||
}
|
||||
|
||||
type apiError struct {
|
||||
Error string `json:"error"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// standardUploadResp 标准上传成功返回的数组
|
||||
type standardUploadResp []struct {
|
||||
Src string `json:"src"`
|
||||
PublicUrl string `json:"publicUrl"`
|
||||
}
|
||||
|
||||
// hfGetUrlResp 获取 HF 直传授权地址的响应
|
||||
type hfGetUrlResp struct {
|
||||
Success bool `json:"success"`
|
||||
FullID string `json:"fullId"`
|
||||
FilePath string `json:"filePath"`
|
||||
ChannelName string `json:"channelName"`
|
||||
Repo string `json:"repo"`
|
||||
NeedsLfs bool `json:"needsLfs"` // 是否需要进行 LFS 物理上传
|
||||
AlreadyExists bool `json:"alreadyExists"` // 是否秒传成功
|
||||
Oid string `json:"oid"` // Git LFS 对象 ID (SHA256)
|
||||
UploadAction *UploadAction `json:"uploadAction"`
|
||||
}
|
||||
|
||||
type UploadAction struct {
|
||||
Href string `json:"href"`
|
||||
Header map[string]string `json:"header"`
|
||||
}
|
||||
|
||||
type hfCommitResp struct {
|
||||
Success bool `json:"success"`
|
||||
Src string `json:"src"`
|
||||
PublicUrl string `json:"publicUrl"`
|
||||
FileUrl string `json:"fileUrl"`
|
||||
FullID string `json:"fullId"`
|
||||
}
|
||||
|
||||
// 辅助函数：从 map 中安全提取字符串/数值
|
||||
func getString(m map[string]interface{}, keys ...string) string {
|
||||
for _, k := range keys {
|
||||
if v, ok := m[k]; ok {
|
||||
switch val := v.(type) {
|
||||
case string:
|
||||
return val
|
||||
case float64:
|
||||
return strconv.FormatInt(int64(val), 10)
|
||||
default:
|
||||
return fmt.Sprintf("%v", val)
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func getInt64(m map[string]interface{}, keys ...string) int64 {
|
||||
for _, k := range keys {
|
||||
if v, ok := m[k]; ok {
|
||||
switch val := v.(type) {
|
||||
case string:
|
||||
n, _ := strconv.ParseInt(val, 10, 64)
|
||||
return n
|
||||
case float64:
|
||||
return int64(val)
|
||||
case int64:
|
||||
return val
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func parseFile(item FileItem) *model.Object {
|
||||
name := path.Base(item.Name)
|
||||
var size int64
|
||||
var modTime time.Time
|
||||
|
||||
if item.Metadata != nil {
|
||||
size = getInt64(item.Metadata, "FileSizeBytes", "File-Size")
|
||||
ts := getInt64(item.Metadata, "TimeStamp")
|
||||
if ts > 0 {
|
||||
modTime = time.UnixMilli(ts)
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: "/" + strings.TrimRight(item.Name, "/"),
|
||||
Name: name,
|
||||
Size: size,
|
||||
Modified: modTime,
|
||||
IsFolder: false,
|
||||
}
|
||||
}
|
||||
@@ -1,560 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/avast/retry-go"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func (d *CFImgBed) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (newObj model.Obj, err error) {
|
||||
if file.GetSize() < hfDirectThreshold {
|
||||
newObj, err = d.standardUpload(ctx, dstDir, file, up)
|
||||
} else {
|
||||
switch d.LargeChannelType {
|
||||
case "huggingface":
|
||||
newObj, err = d.hfDirectUpload(ctx, dstDir, file, up)
|
||||
case "telegram", "cfr2", "s3", "discord":
|
||||
newObj, err = d.chunkedUpload(ctx, dstDir, file, up, d.LargeChannelType, d.LargeChannelName)
|
||||
default:
|
||||
newObj, err = d.standardUpload(ctx, dstDir, file, up)
|
||||
}
|
||||
}
|
||||
if newObj != nil && model.ObjHasMask(dstDir, model.Virtual) {
|
||||
key := dstDir.GetPath()
|
||||
for d.virtualDir.Delete(key) {
|
||||
key = path.Dir(key)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// standardUpload 通过普通 multipart 表单上传。
|
||||
// 使用 io.MultiReader 实现虚拟拼接，避免将整个大文件读入内存构建表单。
|
||||
func (d *CFImgBed) standardUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
|
||||
channelName := d.SmallChannelName
|
||||
if file.GetSize() >= hfDirectThreshold {
|
||||
channelName = d.LargeChannelName
|
||||
log.WithField("size", file.GetSize()).Warn("large file falls back to standard upload, consider configuring LargeChannelType")
|
||||
}
|
||||
if channelName == "" {
|
||||
return nil, fmt.Errorf("channel name not configured")
|
||||
}
|
||||
|
||||
// 1. 将参数放入 Query String
|
||||
reqUrl, err := url.Parse(d.Address + uploadApi)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
q := reqUrl.Query()
|
||||
q.Set("returnFormat", "default")
|
||||
q.Set("channelName", channelName)
|
||||
q.Set("uploadFolder", dstDir.GetPath())
|
||||
q.Set("autoRetry", "true")
|
||||
reqUrl.RawQuery = q.Encode()
|
||||
|
||||
// 2. 构建 multipart 表单的头部
|
||||
b := bytes.NewBuffer(make([]byte, 0, 164+len(file.GetName()))) // 预估头部大小，避免频繁扩容
|
||||
w := multipart.NewWriter(b)
|
||||
_, err = w.CreateFormFile("file", file.GetName())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
headSize := b.Len()
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
|
||||
// 3. 将 [表单头 + 文件流 + 表单尾] 组合成单一 Reader
|
||||
rateLimitedReader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: &driver.SimpleReaderWithSize{
|
||||
Reader: io.MultiReader(head, file, tail),
|
||||
Size: int64(b.Len()) + file.GetSize(),
|
||||
},
|
||||
UpdateProgress: up,
|
||||
})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, reqUrl.String(), rateLimitedReader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+d.Token)
|
||||
req.ContentLength = int64(b.Len()) + file.GetSize()
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
|
||||
b.Reset()
|
||||
_, err = b.ReadFrom(res.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("upload failed %d: %s", res.StatusCode, b.String())
|
||||
}
|
||||
|
||||
var resp standardUploadResp
|
||||
if err := json.Unmarshal(b.Bytes(), &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(resp) == 0 || resp[0].Src == "" {
|
||||
return nil, fmt.Errorf("no src returned")
|
||||
}
|
||||
|
||||
srcPath := strings.TrimPrefix(resp[0].Src, "/file/")
|
||||
srcPath = strings.TrimPrefix(srcPath, "/")
|
||||
|
||||
if resp[0].PublicUrl != "" {
|
||||
if u, err := url.Parse(resp[0].PublicUrl); err == nil {
|
||||
d.publicUrlPrefix = u.Scheme + "://" + u.Host
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: srcPath,
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
Modified: file.ModTime(),
|
||||
IsFolder: false,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) chunkedUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress, channelType, channelName string) (model.Obj, error) {
|
||||
if channelName == "" {
|
||||
return nil, fmt.Errorf("channel name not configured for chunked upload")
|
||||
}
|
||||
|
||||
fileSize := file.GetSize()
|
||||
fileMime := file.GetMimetype()
|
||||
fileName := file.GetName()
|
||||
|
||||
var chunkSizeMap = map[string]int64{
|
||||
|
||||
}
|
||||
chunkSize := chunkSizeMap[channelType]
|
||||
if chunkSize == 0 {
|
||||
chunkSize = 5 * 1024 * 1024
|
||||
}
|
||||
totalChunks := int((fileSize + chunkSize - 1) / chunkSize)
|
||||
|
||||
// 第一步：initChunked
|
||||
var initResp struct {
|
||||
Success bool `json:"success"`
|
||||
UploadId string `json:"uploadId"`
|
||||
}
|
||||
_, err := d.doRequest(ctx, http.MethodPost, uploadApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"initChunked": "true",
|
||||
"uploadChannel": channelType,
|
||||
"channelName": channelName,
|
||||
})
|
||||
req.SetFormData(map[string]string{
|
||||
"originalFileName": fileName,
|
||||
"originalFileType": fileMime,
|
||||
"totalChunks": strconv.Itoa(totalChunks),
|
||||
})
|
||||
}, &initResp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initChunked failed: %w", err)
|
||||
}
|
||||
if !initResp.Success || initResp.UploadId == "" {
|
||||
return nil, fmt.Errorf("initChunked returned no uploadId")
|
||||
}
|
||||
uploadId := initResp.UploadId
|
||||
|
||||
// 第二步：逐块上传
|
||||
ss, err := stream.NewStreamSectionReader(file, int(chunkSize), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
reqUrl := d.Address + uploadApi + "?chunked=true&uploadChannel=" + channelType + "&channelName=" + channelName
|
||||
b := bytes.NewBuffer(make([]byte, 0, 2048))
|
||||
|
||||
g, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, min(d.UploadThread, totalChunks),
|
||||
retry.Attempts(3),
|
||||
retry.Delay(time.Second),
|
||||
retry.DelayType(retry.BackOffDelay))
|
||||
|
||||
for i := 0; i < totalChunks; i++ {
|
||||
if utils.IsCanceled(uploadCtx) {
|
||||
break
|
||||
}
|
||||
chunkIndex := i
|
||||
offset := int64(chunkIndex) * chunkSize
|
||||
sizeToRead := chunkSize
|
||||
if offset+sizeToRead > fileSize {
|
||||
sizeToRead = fileSize - offset
|
||||
}
|
||||
|
||||
var reader io.ReadSeeker
|
||||
g.GoWithLifecycle(errgroup.Lifecycle{
|
||||
Before: func(ctx context.Context) (err error) {
|
||||
reader, err = ss.GetSectionReader(offset, sizeToRead)
|
||||
return
|
||||
},
|
||||
After: func(err error) {
|
||||
ss.FreeSectionReader(reader)
|
||||
},
|
||||
Do: func(ctx context.Context) (err error) {
|
||||
_, err = reader.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
b.Reset()
|
||||
w := multipart.NewWriter(b)
|
||||
_ = w.WriteField("uploadId", uploadId)
|
||||
_ = w.WriteField("chunkIndex", strconv.Itoa(chunkIndex))
|
||||
_ = w.WriteField("totalChunks", strconv.Itoa(totalChunks))
|
||||
_ = w.WriteField("originalFileName", fileName)
|
||||
_ = w.WriteField("originalFileType", fileMime)
|
||||
_, _ = w.CreateFormFile("file", fileName)
|
||||
headSize := b.Len()
|
||||
_ = w.Close()
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
|
||||
rateLimitedRd := driver.NewLimitedUploadStream(ctx, io.MultiReader(head, reader, tail))
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, reqUrl, rateLimitedRd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+d.Token)
|
||||
req.ContentLength = int64(headSize) + sizeToRead + int64(b.Len()-headSize)
|
||||
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("chunk %d upload failed: %d", chunkIndex, res.StatusCode)
|
||||
}
|
||||
|
||||
up(90 * float64(chunkIndex+1) / float64(totalChunks))
|
||||
return nil
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 第三步：merge
|
||||
var mergeResp standardUploadResp
|
||||
_, err = d.doRequest(ctx, http.MethodPost, uploadApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"chunked": "true",
|
||||
"merge": "true",
|
||||
"uploadChannel": channelType,
|
||||
"channelName": channelName,
|
||||
"returnFormat": "default",
|
||||
"uploadFolder": dstDir.GetPath(),
|
||||
})
|
||||
req.SetFormData(map[string]string{
|
||||
"uploadId": uploadId,
|
||||
"totalChunks": strconv.Itoa(totalChunks),
|
||||
"originalFileName": fileName,
|
||||
"originalFileType": fileMime,
|
||||
})
|
||||
}, &mergeResp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("merge failed: %w", err)
|
||||
}
|
||||
if len(mergeResp) == 0 || mergeResp[0].Src == "" {
|
||||
return nil, fmt.Errorf("merge returned no src")
|
||||
}
|
||||
|
||||
up(95)
|
||||
|
||||
srcPath := strings.TrimPrefix(mergeResp[0].Src, "/file/")
|
||||
srcPath = strings.TrimPrefix(srcPath, "/")
|
||||
|
||||
if mergeResp[0].PublicUrl != "" {
|
||||
if u, err := url.Parse(mergeResp[0].PublicUrl); err == nil {
|
||||
d.publicUrlPrefix = u.Scheme + "://" + u.Host
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: srcPath,
|
||||
Name: fileName,
|
||||
Size: fileSize,
|
||||
Modified: file.ModTime(),
|
||||
IsFolder: false,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// hfDirectUpload 处理 HuggingFace 的 LFS 直传逻辑（申请授权 -> 物理上传 -> 后端 Commit）
|
||||
func (d *CFImgBed) hfDirectUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
channelName := d.LargeChannelName
|
||||
if channelName == "" {
|
||||
return nil, errors.New("LargeChannelName not configured")
|
||||
}
|
||||
|
||||
sha256Hash := file.GetHash().GetHash(utils.SHA256)
|
||||
if len(sha256Hash) != utils.SHA256.Width {
|
||||
var err error
|
||||
_, sha256Hash, err = stream.CacheFullAndHash(file, &up, utils.SHA256)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
fileSize := file.GetSize()
|
||||
sampleSize := min(fileSize, fileSampleSize)
|
||||
sampleRd, err := file.RangeRead(http_range.Range{Start: 0, Length: sampleSize})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sampleBuf := make([]byte, sampleSize)
|
||||
_, err = io.ReadFull(sampleRd, sampleBuf)
|
||||
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
|
||||
return nil, err
|
||||
}
|
||||
fileSample := base64.StdEncoding.EncodeToString(sampleBuf)
|
||||
|
||||
fileMime := file.GetMimetype()
|
||||
// 1. 请求图床后端获取 HF 授权地址
|
||||
reqBody := map[string]interface{}{
|
||||
"fileName": file.GetName(),
|
||||
"fileType": fileMime,
|
||||
"fileSize": fileSize,
|
||||
"sha256": sha256Hash,
|
||||
"fileSample": fileSample,
|
||||
"channelName": channelName,
|
||||
"uploadFolder": dstDir.GetPath(),
|
||||
}
|
||||
|
||||
var getUrlResp hfGetUrlResp
|
||||
_, err = d.doRequest(ctx, http.MethodPost, hfGetUrlApi, func(req *resty.Request) {
|
||||
req.SetBody(reqBody)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
}, &getUrlResp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 秒传逻辑
|
||||
if getUrlResp.AlreadyExists || !getUrlResp.NeedsLfs {
|
||||
up(100)
|
||||
return d.hfCommit(ctx, getUrlResp, file.GetName(), fileSize, fileMime, file.ModTime())
|
||||
}
|
||||
|
||||
if getUrlResp.UploadAction == nil {
|
||||
return nil, fmt.Errorf("HF upload action is nil")
|
||||
}
|
||||
|
||||
headers := getUrlResp.UploadAction.Header
|
||||
href := getUrlResp.UploadAction.Href
|
||||
|
||||
// 2. 根据响应判断是执行分片上传还是单文件上传
|
||||
chunkSizeStr, needChunk := headers["chunk_size"]
|
||||
if needChunk {
|
||||
// 分片直传 (AWS S3 Multipart 风格)
|
||||
chunkSize, _ := strconv.ParseInt(chunkSizeStr, 10, 64)
|
||||
if chunkSize <= 0 {
|
||||
chunkSize = 20 * 1024 * 1024
|
||||
}
|
||||
|
||||
partUrls := make(map[int]string)
|
||||
for k, v := range headers {
|
||||
if k != "chunk_size" {
|
||||
if idx, err := strconv.Atoi(k); err == nil {
|
||||
partUrls[idx] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
totalParts := len(partUrls)
|
||||
|
||||
ss, err := stream.NewStreamSectionReader(file, int(chunkSize), &up)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
g, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, min(d.UploadThread, totalParts),
|
||||
retry.Attempts(3),
|
||||
retry.Delay(time.Second),
|
||||
retry.DelayType(retry.BackOffDelay))
|
||||
|
||||
parts := make([]map[string]any, totalParts)
|
||||
|
||||
for partNumber := range partUrls {
|
||||
if utils.IsCanceled(uploadCtx) {
|
||||
break
|
||||
}
|
||||
partUrl := partUrls[partNumber]
|
||||
offset := int64(partNumber-1) * chunkSize
|
||||
sizeToRead := chunkSize
|
||||
if offset+sizeToRead > fileSize {
|
||||
sizeToRead = fileSize - offset
|
||||
}
|
||||
|
||||
var reader io.ReadSeeker
|
||||
g.GoWithLifecycle(errgroup.Lifecycle{
|
||||
Before: func(ctx context.Context) (err error) {
|
||||
reader, err = ss.GetSectionReader(offset, sizeToRead)
|
||||
return
|
||||
},
|
||||
After: func(err error) {
|
||||
ss.FreeSectionReader(reader)
|
||||
},
|
||||
Do: func(ctx context.Context) (err error) {
|
||||
_, err = reader.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
limitedReader := driver.NewLimitedUploadStream(ctx, reader)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, partUrl, limitedReader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
req.ContentLength = sizeToRead
|
||||
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("chunk %d failed: %d", partNumber, res.StatusCode)
|
||||
}
|
||||
|
||||
etag := res.Header.Get("ETag")
|
||||
parts[partNumber-1] = map[string]any{"partNumber": partNumber, "etag": etag}
|
||||
|
||||
up(95 * float64(g.Success()+1) / float64(totalParts))
|
||||
return nil
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 合并分片
|
||||
// sort.Slice(parts, func(i, j int) bool { return parts[i]["partNumber"].(int) < parts[j]["partNumber"].(int) })
|
||||
mergeBody, _ := json.Marshal(map[string]any{"oid": getUrlResp.Oid, "parts": parts})
|
||||
mergeReq, err := http.NewRequestWithContext(ctx, http.MethodPost, href, bytes.NewReader(mergeBody))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mergeReq.Header.Set("Content-Type", "application/vnd.git-lfs+json")
|
||||
if d.Token != "" {
|
||||
mergeReq.Header.Set("Authorization", "Bearer "+d.Token)
|
||||
}
|
||||
|
||||
res, err := base.HttpClient.Do(mergeReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
body, _ := io.ReadAll(res.Body) // 读取 HF 返回的错误详情
|
||||
if res.StatusCode != http.StatusOK {
|
||||
log.WithField("status", res.StatusCode).WithField("response", string(body)).Error("HF merge chunks failed")
|
||||
return nil, fmt.Errorf("merge chunks failed: %s", string(body))
|
||||
}
|
||||
up(97)
|
||||
|
||||
} else {
|
||||
// 单文件直传 (PUT)
|
||||
limitedReader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: file,
|
||||
UpdateProgress: model.UpdateProgressWithRange(up, 0, 97),
|
||||
})
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, href, limitedReader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.ContentLength = fileSize
|
||||
for k, v := range headers {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("direct upload failed")
|
||||
}
|
||||
}
|
||||
|
||||
defer up(100)
|
||||
|
||||
// 3. 通知图床后端完成文件登记
|
||||
return d.hfCommit(ctx, getUrlResp, file.GetName(), fileSize, fileMime, file.ModTime())
|
||||
}
|
||||
|
||||
func (d *CFImgBed) hfCommit(ctx context.Context, getUrlResp hfGetUrlResp, fileName string, fileSize int64, fileMime string, modTime time.Time) (model.Obj, error) {
|
||||
commitBody := map[string]interface{}{
|
||||
"fullId": getUrlResp.FullID,
|
||||
"filePath": getUrlResp.FilePath,
|
||||
"sha256": getUrlResp.Oid,
|
||||
"fileSize": fileSize,
|
||||
"fileName": fileName,
|
||||
"fileType": fileMime,
|
||||
"channelName": getUrlResp.ChannelName,
|
||||
}
|
||||
var commitResp hfCommitResp
|
||||
_, err := d.doRequest(ctx, http.MethodPost, hfCommitApi, func(req *resty.Request) {
|
||||
req.SetBody(commitBody)
|
||||
}, &commitResp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("HF commit request failed: %w", err)
|
||||
}
|
||||
if !commitResp.Success {
|
||||
return nil, fmt.Errorf("HF commit failed: success=false")
|
||||
}
|
||||
|
||||
srcPath := strings.TrimPrefix(commitResp.Src, "/file/")
|
||||
srcPath = strings.TrimPrefix(srcPath, "/")
|
||||
|
||||
if commitResp.PublicUrl != "" {
|
||||
if u, err := url.Parse(commitResp.PublicUrl); err == nil {
|
||||
d.publicUrlPrefix = u.Scheme + "://" + u.Host
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: srcPath,
|
||||
Name: fileName,
|
||||
Size: fileSize,
|
||||
Modified: modTime,
|
||||
IsFolder: false,
|
||||
}, nil
|
||||
}
|
||||
@@ -1,70 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
listApi = "/api/manage/list"
|
||||
deleteApi = "/api/manage/delete"
|
||||
uploadApi = "/upload"
|
||||
hfGetUrlApi = "/upload/huggingface/getUploadUrl"
|
||||
hfCommitApi = "/upload/huggingface/commitUpload"
|
||||
hfDirectThreshold int64 = 20 * 1024 * 1024
|
||||
fileSampleSize = 512 // HF 申请上传地址时需提供文件前 512 字节的 Sample
|
||||
)
|
||||
|
||||
// doRequest 通用请求封装，包含重试和 API 错误解析
|
||||
func (d *CFImgBed) doRequest(ctx context.Context, method, urlPath string, callback func(*resty.Request), resp interface{}) ([]byte, error) {
|
||||
maxRetries := 3
|
||||
for i := 0; i < maxRetries; i++ {
|
||||
req := d.client.R()
|
||||
req.SetContext(ctx)
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
if resp != nil {
|
||||
req.SetResult(resp)
|
||||
}
|
||||
|
||||
res, err := req.Execute(method, urlPath)
|
||||
if err != nil {
|
||||
log.WithError(err).Warnf("request %s %s failed, attempt %d/%d", method, urlPath, i+1, maxRetries)
|
||||
if i < maxRetries-1 {
|
||||
time.Sleep(time.Duration(i+1) * time.Second)
|
||||
continue
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Retry on rate limit before attempting to interpret the body as an API error.
|
||||
if res.StatusCode() == 429 {
|
||||
time.Sleep(time.Duration(i+1) * 2 * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
body := res.Body()
|
||||
var apiErr apiError
|
||||
if err := json.Unmarshal(body, &apiErr); err == nil {
|
||||
if apiErr.Error != "" || apiErr.Message != "" {
|
||||
msg := apiErr.Error
|
||||
if msg == "" {
|
||||
msg = apiErr.Message
|
||||
}
|
||||
return nil, fmt.Errorf("API error: %s", msg)
|
||||
}
|
||||
}
|
||||
|
||||
if res.IsError() {
|
||||
return nil, fmt.Errorf("HTTP %d", res.StatusCode())
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
return nil, fmt.Errorf("max retries exceeded")
|
||||
}
|
||||
@@ -1,232 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type CnbReleases struct {
|
||||
model.Storage
|
||||
Addition
|
||||
ref *CnbReleases
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *CnbReleases) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Init(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) InitReference(storage driver.Driver) error {
|
||||
refStorage, ok := storage.(*CnbReleases)
|
||||
if ok {
|
||||
d.ref = refStorage
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("ref: storage is not CnbReleases")
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Drop(ctx context.Context) error {
|
||||
d.ref = nil
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
dirID := dir.GetID()
|
||||
if dirID == "" {
|
||||
// get all releases for root dir
|
||||
var resp ReleaseList
|
||||
|
||||
err := d.Request(http.MethodGet, "/{repo}/-/releases", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return utils.SliceConvert(resp, func(src Release) (model.Obj, error) {
|
||||
name := src.Name
|
||||
if d.UseTagName {
|
||||
name = src.TagName
|
||||
}
|
||||
return &model.Object{
|
||||
ID: src.ID,
|
||||
Name: name,
|
||||
Size: d.sumAssetsSize(src.Assets),
|
||||
Ctime: src.CreatedAt,
|
||||
Modified: src.UpdatedAt,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
})
|
||||
}
|
||||
|
||||
var resp Release
|
||||
err := d.Request(http.MethodGet, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", dirID)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return utils.SliceConvert(resp.Assets, func(src ReleaseAsset) (model.Obj, error) {
|
||||
return &Object{
|
||||
Object: model.Object{
|
||||
ID: src.ID,
|
||||
Path: src.Path,
|
||||
Name: src.Name,
|
||||
Size: src.Size,
|
||||
Ctime: src.CreatedAt,
|
||||
Modified: src.UpdatedAt,
|
||||
IsFolder: false,
|
||||
},
|
||||
ParentID: dirID,
|
||||
}, nil
|
||||
})
|
||||
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
return &model.Link{
|
||||
URL: "https://cnb.cool" + file.GetPath(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
if parentDir.GetPath() == "/" {
|
||||
// create a new release
|
||||
branch := d.DefaultBranch
|
||||
if branch == "" {
|
||||
branch = "main" // fallback to "main" if not set
|
||||
}
|
||||
return d.Request(http.MethodPost, "/{repo}/-/releases", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetBody(base.Json{
|
||||
"name": dirName,
|
||||
"tag_name": dirName,
|
||||
"target_commitish": branch,
|
||||
})
|
||||
}, nil)
|
||||
}
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
if srcObj.IsDir() && !d.UseTagName {
|
||||
return d.Request(http.MethodPatch, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", srcObj.GetID())
|
||||
req.SetFormData(map[string]string{
|
||||
"name": newName,
|
||||
})
|
||||
}, nil)
|
||||
}
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Remove(ctx context.Context, obj model.Obj) error {
|
||||
if obj.IsDir() {
|
||||
return d.Request(http.MethodDelete, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", obj.GetID())
|
||||
}, nil)
|
||||
}
|
||||
if o, ok := obj.(*Object); ok {
|
||||
return d.Request(http.MethodDelete, "/{repo}/-/releases/{release_id}/assets/{asset_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", o.ParentID)
|
||||
req.SetPathParam("asset_id", obj.GetID())
|
||||
}, nil)
|
||||
} else {
|
||||
return fmt.Errorf("unable to get release ID")
|
||||
}
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
// 1. get upload info
|
||||
var resp ReleaseAssetUploadURL
|
||||
err := d.Request(http.MethodPost, "/{repo}/-/releases/{release_id}/asset-upload-url", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", dstDir.GetID())
|
||||
req.SetBody(base.Json{
|
||||
"asset_name": file.GetName(),
|
||||
"overwrite": true,
|
||||
"size": file.GetSize(),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 2. upload file
|
||||
// use multipart to create form file
|
||||
var b bytes.Buffer
|
||||
w := multipart.NewWriter(&b)
|
||||
_, err = w.CreateFormFile("file", file.GetName())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
headSize := b.Len()
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
r := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: &driver.SimpleReaderWithSize{
|
||||
Reader: io.MultiReader(head, file, tail),
|
||||
Size: int64(b.Len()) + file.GetSize(),
|
||||
},
|
||||
UpdateProgress: up,
|
||||
})
|
||||
|
||||
// use net/http to upload file
|
||||
ctxWithTimeout, cancel := context.WithTimeout(ctx, time.Duration(resp.ExpiresInSec+1)*time.Second)
|
||||
defer cancel()
|
||||
req, err := http.NewRequestWithContext(ctxWithTimeout, http.MethodPost, resp.UploadURL, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("User-Agent", base.UserAgent)
|
||||
httpResp, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer httpResp.Body.Close()
|
||||
if httpResp.StatusCode != http.StatusNoContent {
|
||||
return fmt.Errorf("upload file failed: %s", httpResp.Status)
|
||||
}
|
||||
|
||||
// 3. verify upload
|
||||
return d.Request(http.MethodPost, resp.VerifyURL, nil, nil)
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*CnbReleases)(nil)
|
||||
@@ -1,25 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
Repo string `json:"repo" type:"string" required:"true"`
|
||||
Token string `json:"token" type:"string" required:"true"`
|
||||
UseTagName bool `json:"use_tag_name" type:"bool" default:"false" help:"Use tag name instead of release name"`
|
||||
DefaultBranch string `json:"default_branch" type:"string" default:"main" help:"Default branch for new releases"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "CNB Releases",
|
||||
LocalSort: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &CnbReleases{}
|
||||
})
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
type Object struct {
|
||||
model.Object
|
||||
ParentID string
|
||||
}
|
||||
|
||||
type TagList []Tag
|
||||
|
||||
type Tag struct {
|
||||
Commit struct {
|
||||
Author UserInfo `json:"author"`
|
||||
Commit CommitObject `json:"commit"`
|
||||
Committer UserInfo `json:"committer"`
|
||||
Parents []CommitParent `json:"parents"`
|
||||
Sha string `json:"sha"`
|
||||
} `json:"commit"`
|
||||
Name string `json:"name"`
|
||||
Target string `json:"target"`
|
||||
TargetType string `json:"target_type"`
|
||||
Verification TagObjectVerification `json:"verification"`
|
||||
}
|
||||
|
||||
type UserInfo struct {
|
||||
Freeze bool `json:"freeze"`
|
||||
Nickname string `json:"nickname"`
|
||||
Username string `json:"username"`
|
||||
}
|
||||
|
||||
type CommitObject struct {
|
||||
Author Signature `json:"author"`
|
||||
CommentCount int `json:"comment_count"`
|
||||
Committer Signature `json:"committer"`
|
||||
Message string `json:"message"`
|
||||
Tree CommitObjectTree `json:"tree"`
|
||||
Verification CommitObjectVerification `json:"verification"`
|
||||
}
|
||||
|
||||
type Signature struct {
|
||||
Date time.Time `json:"date"`
|
||||
Email string `json:"email"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type CommitObjectTree struct {
|
||||
Sha string `json:"sha"`
|
||||
}
|
||||
|
||||
type CommitObjectVerification struct {
|
||||
Payload string `json:"payload"`
|
||||
Reason string `json:"reason"`
|
||||
Signature string `json:"signature"`
|
||||
Verified bool `json:"verified"`
|
||||
VerifiedAt string `json:"verified_at"`
|
||||
}
|
||||
|
||||
type CommitParent = CommitObjectTree
|
||||
|
||||
type TagObjectVerification = CommitObjectVerification
|
||||
|
||||
type ReleaseList []Release
|
||||
|
||||
type Release struct {
|
||||
Assets []ReleaseAsset `json:"assets"`
|
||||
Author UserInfo `json:"author"`
|
||||
Body string `json:"body"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Draft bool `json:"draft"`
|
||||
ID string `json:"id"`
|
||||
IsLatest bool `json:"is_latest"`
|
||||
Name string `json:"name"`
|
||||
Prerelease bool `json:"prerelease"`
|
||||
PublishedAt time.Time `json:"published_at"`
|
||||
TagCommitish string `json:"tag_commitish"`
|
||||
TagName string `json:"tag_name"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type ReleaseAsset struct {
|
||||
ContentType string `json:"content_type"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Size int64 `json:"size"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Uploader UserInfo `json:"uploader"`
|
||||
}
|
||||
|
||||
type ReleaseAssetUploadURL struct {
|
||||
UploadURL string `json:"upload_url"`
|
||||
ExpiresInSec int `json:"expires_in_sec"`
|
||||
VerifyURL string `json:"verify_url"`
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
// do others that not defined in Driver interface
|
||||
|
||||
func (d *CnbReleases) Request(method string, path string, callback base.ReqCallback, resp any) error {
|
||||
if d.ref != nil {
|
||||
return d.ref.Request(method, path, callback, resp)
|
||||
}
|
||||
var url string
|
||||
if strings.HasPrefix(path, "http") {
|
||||
url = path
|
||||
} else {
|
||||
url = "https://api.cnb.cool" + path
|
||||
}
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeader("Accept", "application/json")
|
||||
req.SetAuthScheme("Bearer")
|
||||
req.SetAuthToken(d.Token)
|
||||
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
res, err := req.Execute(method, url)
|
||||
log.Debugln(res.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if res.StatusCode() != http.StatusOK && res.StatusCode() != http.StatusCreated && res.StatusCode() != http.StatusNoContent {
|
||||
return fmt.Errorf("failed to request %s, status code: %d, message: %s", url, res.StatusCode(), res.String())
|
||||
}
|
||||
|
||||
if resp != nil {
|
||||
err = json.Unmarshal(res.Body(), resp)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) sumAssetsSize(assets []ReleaseAsset) int64 {
|
||||
var size int64
|
||||
for _, asset := range assets {
|
||||
size += asset.Size
|
||||
}
|
||||
return size
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
package crypt
|
||||
|
||||
import (
|
||||
stdpath "path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// will give the best guessing based on the path
|
||||
func guessPath(path string) (isFolder, secondTry bool) {
|
||||
if strings.HasSuffix(path, "/") {
|
||||
//confirmed a folder
|
||||
return true, false
|
||||
}
|
||||
lastSlash := strings.LastIndex(path, "/")
|
||||
if !strings.Contains(path[lastSlash:], ".") {
|
||||
//no dot, try folder then try file
|
||||
return true, true
|
||||
}
|
||||
return false, true
|
||||
}
|
||||
|
||||
func (d *Crypt) encryptPath(path string, isFolder bool) string {
|
||||
if isFolder {
|
||||
return d.cipher.EncryptDirName(path)
|
||||
}
|
||||
dir, fileName := filepath.Split(path)
|
||||
return stdpath.Join(d.cipher.EncryptDirName(dir), d.cipher.EncryptFileName(fileName))
|
||||
}
|
||||
@@ -1,224 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type Degoo struct {
|
||||
model.Storage
|
||||
Addition
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func (d *Degoo) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Degoo) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Degoo) Init(ctx context.Context) error {
|
||||
|
||||
d.client = base.HttpClient
|
||||
|
||||
// Ensure we have a valid token (will login if needed or refresh if expired)
|
||||
if err := d.ensureValidToken(ctx); err != nil {
|
||||
return fmt.Errorf("failed to initialize token: %w", err)
|
||||
}
|
||||
|
||||
return d.getDevices(ctx)
|
||||
}
|
||||
|
||||
func (d *Degoo) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
items, err := d.getAllFileChildren5(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return utils.MustSliceConvert(items, func(s DegooFileItem) model.Obj {
|
||||
isFolder := s.Category == 2 || s.Category == 1 || s.Category == 10
|
||||
|
||||
createTime, modTime, _ := humanReadableTimes(s.CreationTime, s.LastModificationTime, s.LastUploadTime)
|
||||
|
||||
size, err := strconv.ParseInt(s.Size, 10, 64)
|
||||
if err != nil {
|
||||
size = 0 // Default to 0 if size parsing fails
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
ID: s.ID,
|
||||
Path: s.FilePath,
|
||||
Name: s.Name,
|
||||
Size: size,
|
||||
Modified: modTime,
|
||||
Ctime: createTime,
|
||||
IsFolder: isFolder,
|
||||
}
|
||||
}), nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
item, err := d.getOverlay4(ctx, file.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &model.Link{URL: item.URL}, nil
|
||||
}
|
||||
|
||||
func (d *Degoo) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
// This is done by calling the setUploadFile3 API with a special checksum and size.
|
||||
const query = `mutation SetUploadFile3($Token: String!, $FileInfos: [FileInfoUpload3]!) { setUploadFile3(Token: $Token, FileInfos: $FileInfos) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"FileInfos": []map[string]interface{}{
|
||||
{
|
||||
"Checksum": folderChecksum,
|
||||
"Name": dirName,
|
||||
"CreationTime": time.Now().UnixMilli(),
|
||||
"ParentID": parentDir.GetID(),
|
||||
"Size": 0,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetUploadFile3", query, variables)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
const query = `mutation SetMoveFile($Token: String!, $Copy: Boolean, $NewParentID: String!, $FileIDs: [String]!) { setMoveFile(Token: $Token, Copy: $Copy, NewParentID: $NewParentID, FileIDs: $FileIDs) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"Copy": false,
|
||||
"NewParentID": dstDir.GetID(),
|
||||
"FileIDs": []string{srcObj.GetID()},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetMoveFile", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
const query = `mutation SetRenameFile($Token: String!, $FileRenames: [FileRenameInfo]!) { setRenameFile(Token: $Token, FileRenames: $FileRenames) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"FileRenames": []DegooFileRenameInfo{
|
||||
{
|
||||
ID: srcObj.GetID(),
|
||||
NewName: newName,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetRenameFile", query, variables)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
// Copy is not implemented, Degoo API does not support direct copy.
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Degoo) Remove(ctx context.Context, obj model.Obj) error {
|
||||
// Remove deletes a file or folder (moves to trash).
|
||||
const query = `mutation SetDeleteFile5($Token: String!, $IsInRecycleBin: Boolean!, $IDs: [IDType]!) { setDeleteFile5(Token: $Token, IsInRecycleBin: $IsInRecycleBin, IDs: $IDs) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"IsInRecycleBin": false,
|
||||
"IDs": []map[string]string{{"FileID": obj.GetID()}},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetDeleteFile5", query, variables)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Degoo) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
tmpF, err := file.CacheFullAndWriter(&up, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
parentID := dstDir.GetID()
|
||||
|
||||
// Calculate the checksum for the file.
|
||||
checksum, err := d.checkSum(tmpF)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 1. Get upload authorization via getBucketWriteAuth4.
|
||||
auths, err := d.getBucketWriteAuth4(ctx, file, parentID, checksum)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 2. Upload file.
|
||||
// support rapid upload
|
||||
if auths.GetBucketWriteAuth4[0].Error != "Already exist!" {
|
||||
err = d.uploadS3(ctx, auths, tmpF, file, checksum)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Register metadata with setUploadFile3.
|
||||
data, err := d.SetUploadFile3(ctx, file, parentID, checksum)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !data.SetUploadFile3 {
|
||||
return fmt.Errorf("setUploadFile3 failed: %v", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
quota, err := d.getUserInfo(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
used, err := strconv.ParseInt(quota.GetUserInfo3.UsedQuota, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse used quota: %v", err)
|
||||
}
|
||||
total, err := strconv.ParseInt(quota.GetUserInfo3.TotalQuota, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse total quota: %v", err)
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: total,
|
||||
UsedSpace: used,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
Username string `json:"username" help:"Your Degoo account email"`
|
||||
Password string `json:"password" help:"Your Degoo account password"`
|
||||
RefreshToken string `json:"refresh_token" help:"Refresh token for automatic token renewal, obtained automatically"`
|
||||
AccessToken string `json:"access_token" help:"Access token for Degoo API, obtained automatically"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Degoo",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "0",
|
||||
NoOverwriteUpload: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Degoo{}
|
||||
})
|
||||
}
|
||||
@@ -1,133 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
)
|
||||
|
||||
// DegooLoginRequest represents the login request body.
|
||||
type DegooLoginRequest struct {
|
||||
GenerateToken bool `json:"GenerateToken"`
|
||||
Username string `json:"Username"`
|
||||
Password string `json:"Password"`
|
||||
}
|
||||
|
||||
// DegooLoginResponse represents a successful login response.
|
||||
type DegooLoginResponse struct {
|
||||
Token string `json:"Token"`
|
||||
RefreshToken string `json:"RefreshToken"`
|
||||
}
|
||||
|
||||
// DegooAccessTokenRequest represents the token refresh request body.
|
||||
type DegooAccessTokenRequest struct {
|
||||
RefreshToken string `json:"RefreshToken"`
|
||||
}
|
||||
|
||||
// DegooAccessTokenResponse represents the token refresh response.
|
||||
type DegooAccessTokenResponse struct {
|
||||
AccessToken string `json:"AccessToken"`
|
||||
}
|
||||
|
||||
// DegooFileItem represents a Degoo file or folder.
|
||||
type DegooFileItem struct {
|
||||
ID string `json:"ID"`
|
||||
ParentID string `json:"ParentID"`
|
||||
Name string `json:"Name"`
|
||||
Category int `json:"Category"`
|
||||
Size string `json:"Size"`
|
||||
URL string `json:"URL"`
|
||||
CreationTime string `json:"CreationTime"`
|
||||
LastModificationTime string `json:"LastModificationTime"`
|
||||
LastUploadTime string `json:"LastUploadTime"`
|
||||
MetadataID string `json:"MetadataID"`
|
||||
DeviceID int64 `json:"DeviceID"`
|
||||
FilePath string `json:"FilePath"`
|
||||
IsInRecycleBin bool `json:"IsInRecycleBin"`
|
||||
}
|
||||
|
||||
type DegooErrors struct {
|
||||
Path []string `json:"path"`
|
||||
Data interface{} `json:"data"`
|
||||
ErrorType string `json:"errorType"`
|
||||
ErrorInfo interface{} `json:"errorInfo"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// DegooGraphqlResponse is the common structure for GraphQL API responses.
|
||||
type DegooGraphqlResponse struct {
|
||||
Data json.RawMessage `json:"data"`
|
||||
Errors []DegooErrors `json:"errors,omitempty"`
|
||||
}
|
||||
|
||||
// DegooGetChildren5Data is the data field for getFileChildren5.
|
||||
type DegooGetChildren5Data struct {
|
||||
GetFileChildren5 struct {
|
||||
Items []DegooFileItem `json:"Items"`
|
||||
NextToken string `json:"NextToken"`
|
||||
} `json:"getFileChildren5"`
|
||||
}
|
||||
|
||||
// DegooGetOverlay4Data is the data field for getOverlay4.
|
||||
type DegooGetOverlay4Data struct {
|
||||
GetOverlay4 DegooFileItem `json:"getOverlay4"`
|
||||
}
|
||||
|
||||
// DegooFileRenameInfo represents a file rename operation.
|
||||
type DegooFileRenameInfo struct {
|
||||
ID string `json:"ID"`
|
||||
NewName string `json:"NewName"`
|
||||
}
|
||||
|
||||
// DegooFileIDs represents a list of file IDs for move operations.
|
||||
type DegooFileIDs struct {
|
||||
FileIDs []string `json:"FileIDs"`
|
||||
}
|
||||
|
||||
// DegooGetBucketWriteAuth4Data is the data field for GetBucketWriteAuth4.
|
||||
type DegooGetBucketWriteAuth4Data struct {
|
||||
GetBucketWriteAuth4 []struct {
|
||||
AuthData struct {
|
||||
PolicyBase64 string `json:"PolicyBase64"`
|
||||
Signature string `json:"Signature"`
|
||||
BaseURL string `json:"BaseURL"`
|
||||
KeyPrefix string `json:"KeyPrefix"`
|
||||
AccessKey struct {
|
||||
Key string `json:"Key"`
|
||||
Value string `json:"Value"`
|
||||
} `json:"AccessKey"`
|
||||
ACL string `json:"ACL"`
|
||||
AdditionalBody []struct {
|
||||
Key string `json:"Key"`
|
||||
Value string `json:"Value"`
|
||||
} `json:"AdditionalBody"`
|
||||
} `json:"AuthData"`
|
||||
Error interface{} `json:"Error"`
|
||||
} `json:"getBucketWriteAuth4"`
|
||||
}
|
||||
|
||||
// DegooSetUploadFile3Data is the data field for SetUploadFile3.
|
||||
type DegooSetUploadFile3Data struct {
|
||||
SetUploadFile3 bool `json:"setUploadFile3"`
|
||||
}
|
||||
|
||||
type DegooGetUserInfo3Data struct {
|
||||
GetUserInfo3 struct {
|
||||
// ID string
|
||||
// FirstName string
|
||||
// LastName string
|
||||
// Email string
|
||||
// AvatarURL string
|
||||
// CountryCode string = CN
|
||||
// LanguageCode string = zh-cn
|
||||
// Phone string
|
||||
// AccountType int
|
||||
UsedQuota string `json:"UsedQuota"`
|
||||
TotalQuota string `json:"TotalQuota"`
|
||||
// OAuth2Provider
|
||||
// GPMigrationStatus int
|
||||
// FeatureNoAds bool
|
||||
// FeatureTopSecret bool
|
||||
// FeatureDownsampling bool
|
||||
// FeatureAutomaticVideoUploads bool
|
||||
// FileSizeLimit string
|
||||
} `json:"getUserInfo3"`
|
||||
}
|
||||
@@ -1,198 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
func (d *Degoo) getBucketWriteAuth4(ctx context.Context, file model.FileStreamer, parentID string, checksum string) (*DegooGetBucketWriteAuth4Data, error) {
|
||||
const query = `query GetBucketWriteAuth4(
|
||||
$Token: String!
|
||||
$ParentID: String!
|
||||
$StorageUploadInfos: [StorageUploadInfo2]
|
||||
) {
|
||||
getBucketWriteAuth4(
|
||||
Token: $Token
|
||||
ParentID: $ParentID
|
||||
StorageUploadInfos: $StorageUploadInfos
|
||||
) {
|
||||
AuthData {
|
||||
PolicyBase64
|
||||
Signature
|
||||
BaseURL
|
||||
KeyPrefix
|
||||
AccessKey {
|
||||
Key
|
||||
Value
|
||||
}
|
||||
ACL
|
||||
AdditionalBody {
|
||||
Key
|
||||
Value
|
||||
}
|
||||
}
|
||||
Error
|
||||
}
|
||||
}`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ParentID": parentID,
|
||||
"StorageUploadInfos": []map[string]string{{
|
||||
"FileName": file.GetName(),
|
||||
"Checksum": checksum,
|
||||
"Size": strconv.FormatInt(file.GetSize(), 10),
|
||||
}}}
|
||||
|
||||
data, err := d.apiCall(ctx, "GetBucketWriteAuth4", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp DegooGetBucketWriteAuth4Data
|
||||
err = json.Unmarshal(data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
// checkSum calculates the SHA1-based checksum for Degoo upload API.
|
||||
func (d *Degoo) checkSum(file io.Reader) (string, error) {
|
||||
seed := []byte{13, 7, 2, 2, 15, 40, 75, 117, 13, 10, 19, 16, 29, 23, 3, 36}
|
||||
hasher := sha1.New()
|
||||
hasher.Write(seed)
|
||||
|
||||
if _, err := utils.CopyWithBuffer(hasher, file); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
cs := hasher.Sum(nil)
|
||||
|
||||
csBytes := []byte{10, byte(len(cs))}
|
||||
csBytes = append(csBytes, cs...)
|
||||
csBytes = append(csBytes, 16, 0)
|
||||
|
||||
return strings.ReplaceAll(base64.StdEncoding.EncodeToString(csBytes), "/", "_"), nil
|
||||
}
|
||||
|
||||
func (d *Degoo) uploadS3(ctx context.Context, auths *DegooGetBucketWriteAuth4Data, tmpF model.File, file model.FileStreamer, checksum string) error {
|
||||
a := auths.GetBucketWriteAuth4[0].AuthData
|
||||
|
||||
_, err := tmpF.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ext := utils.Ext(file.GetName())
|
||||
key := fmt.Sprintf("%s%s/%s.%s", a.KeyPrefix, ext, checksum, ext)
|
||||
|
||||
var b bytes.Buffer
|
||||
w := multipart.NewWriter(&b)
|
||||
err = w.WriteField("key", key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField("acl", a.ACL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField("policy", a.PolicyBase64)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField("signature", a.Signature)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField(a.AccessKey.Key, a.AccessKey.Value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, additional := range a.AdditionalBody {
|
||||
err = w.WriteField(additional.Key, additional.Value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
err = w.WriteField("Content-Type", "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = w.CreateFormFile("file", key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
headSize := b.Len()
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
|
||||
rateLimitedRd := driver.NewLimitedUploadStream(ctx, io.MultiReader(head, tmpF, tail))
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.BaseURL, rateLimitedRd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Add("ngsw-bypass", "1")
|
||||
req.Header.Add("Content-Type", w.FormDataContentType())
|
||||
|
||||
res, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusNoContent {
|
||||
return fmt.Errorf("upload failed with status code %d", res.StatusCode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Degoo)(nil)
|
||||
|
||||
func (d *Degoo) SetUploadFile3(ctx context.Context, file model.FileStreamer, parentID string, checksum string) (*DegooSetUploadFile3Data, error) {
|
||||
const query = `mutation SetUploadFile3($Token: String!, $FileInfos: [FileInfoUpload3]!) {
|
||||
setUploadFile3(Token: $Token, FileInfos: $FileInfos)
|
||||
}`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"FileInfos": []map[string]string{{
|
||||
"Checksum": checksum,
|
||||
"CreationTime": strconv.FormatInt(file.CreateTime().UnixMilli(), 10),
|
||||
"Name": file.GetName(),
|
||||
"ParentID": parentID,
|
||||
"Size": strconv.FormatInt(file.GetSize(), 10),
|
||||
}}}
|
||||
|
||||
data, err := d.apiCall(ctx, "SetUploadFile3", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp DegooSetUploadFile3Data
|
||||
err = json.Unmarshal(data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
@@ -1,478 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
// Thanks to https://github.com/bernd-wechner/Degoo for API research.
|
||||
|
||||
const (
|
||||
// API endpoints
|
||||
loginURL = "https://rest-api.degoo.com/login"
|
||||
accessTokenURL = "https://rest-api.degoo.com/access-token/v2"
|
||||
apiURL = "https://production-appsync.degoo.com/graphql"
|
||||
|
||||
// API configuration
|
||||
apiKey = "da2-vs6twz5vnjdavpqndtbzg3prra"
|
||||
folderChecksum = "CgAQAg"
|
||||
|
||||
// Token management
|
||||
tokenRefreshThreshold = 5 * time.Minute
|
||||
|
||||
// Rate limiting
|
||||
minRequestInterval = 1 * time.Second
|
||||
|
||||
// Error messages
|
||||
errRateLimited = "rate limited (429), please try again later"
|
||||
errUnauthorized = "unauthorized access"
|
||||
)
|
||||
|
||||
var (
|
||||
// Global rate limiting - protects against concurrent API calls
|
||||
lastRequestTime time.Time
|
||||
requestMutex sync.Mutex
|
||||
)
|
||||
|
||||
// JWT payload structure for token expiration checking
|
||||
type JWTPayload struct {
|
||||
UserID string `json:"userID"`
|
||||
Exp int64 `json:"exp"`
|
||||
Iat int64 `json:"iat"`
|
||||
}
|
||||
|
||||
// Rate limiting helper functions
|
||||
|
||||
// applyRateLimit ensures minimum interval between API requests
|
||||
func applyRateLimit() {
|
||||
requestMutex.Lock()
|
||||
defer requestMutex.Unlock()
|
||||
|
||||
if !lastRequestTime.IsZero() {
|
||||
if elapsed := time.Since(lastRequestTime); elapsed < minRequestInterval {
|
||||
time.Sleep(minRequestInterval - elapsed)
|
||||
}
|
||||
}
|
||||
lastRequestTime = time.Now()
|
||||
}
|
||||
|
||||
// HTTP request helper functions
|
||||
|
||||
// createJSONRequest creates a new HTTP request with JSON body
|
||||
func createJSONRequest(ctx context.Context, method, url string, body interface{}) (*http.Request, error) {
|
||||
jsonBody, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to marshal request body: %w", err)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, method, url, bytes.NewBuffer(jsonBody))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create request: %w", err)
|
||||
}
|
||||
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", base.UserAgent)
|
||||
return req, nil
|
||||
}
|
||||
|
||||
// checkHTTPResponse checks for common HTTP error conditions
|
||||
func checkHTTPResponse(resp *http.Response, operation string) error {
|
||||
if resp.StatusCode == http.StatusTooManyRequests {
|
||||
return fmt.Errorf("%s %s", operation, errRateLimited)
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("%s failed: %s", operation, resp.Status)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// isTokenExpired checks if the JWT token is expired or will expire soon
|
||||
func (d *Degoo) isTokenExpired() bool {
|
||||
if d.AccessToken == "" {
|
||||
return true
|
||||
}
|
||||
|
||||
payload, err := extractJWTPayload(d.AccessToken)
|
||||
if err != nil {
|
||||
return true // Invalid token format
|
||||
}
|
||||
|
||||
// Check if token expires within the threshold
|
||||
expireTime := time.Unix(payload.Exp, 0)
|
||||
return time.Now().Add(tokenRefreshThreshold).After(expireTime)
|
||||
}
|
||||
|
||||
// extractJWTPayload extracts and parses JWT payload
|
||||
func extractJWTPayload(token string) (*JWTPayload, error) {
|
||||
parts := strings.Split(token, ".")
|
||||
if len(parts) != 3 {
|
||||
return nil, fmt.Errorf("invalid JWT format")
|
||||
}
|
||||
|
||||
// Decode the payload (second part)
|
||||
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to decode JWT payload: %w", err)
|
||||
}
|
||||
|
||||
var jwtPayload JWTPayload
|
||||
if err := json.Unmarshal(payload, &jwtPayload); err != nil {
|
||||
return nil, fmt.Errorf("failed to parse JWT payload: %w", err)
|
||||
}
|
||||
|
||||
return &jwtPayload, nil
|
||||
}
|
||||
|
||||
// refreshToken attempts to refresh the access token using the refresh token
|
||||
func (d *Degoo) refreshToken(ctx context.Context) error {
|
||||
if d.RefreshToken == "" {
|
||||
return fmt.Errorf("no refresh token available")
|
||||
}
|
||||
|
||||
// Create request
|
||||
tokenReq := DegooAccessTokenRequest{RefreshToken: d.RefreshToken}
|
||||
req, err := createJSONRequest(ctx, "POST", accessTokenURL, tokenReq)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create refresh token request: %w", err)
|
||||
}
|
||||
|
||||
// Execute request
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("refresh token request failed: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Check response
|
||||
if err := checkHTTPResponse(resp, "refresh token"); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var accessTokenResp DegooAccessTokenResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&accessTokenResp); err != nil {
|
||||
return fmt.Errorf("failed to parse access token response: %w", err)
|
||||
}
|
||||
|
||||
if accessTokenResp.AccessToken == "" {
|
||||
return fmt.Errorf("empty access token received")
|
||||
}
|
||||
|
||||
d.AccessToken = accessTokenResp.AccessToken
|
||||
// Save the updated token to storage
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ensureValidToken ensures we have a valid, non-expired token
|
||||
func (d *Degoo) ensureValidToken(ctx context.Context) error {
|
||||
// Check if token is expired or will expire soon
|
||||
if d.isTokenExpired() {
|
||||
// Try to refresh token first if we have a refresh token
|
||||
if d.RefreshToken != "" {
|
||||
if refreshErr := d.refreshToken(ctx); refreshErr == nil {
|
||||
return nil // Successfully refreshed
|
||||
} else {
|
||||
// If refresh failed, fall back to full login
|
||||
fmt.Printf("Token refresh failed, falling back to full login: %v\n", refreshErr)
|
||||
}
|
||||
}
|
||||
|
||||
// Perform full login
|
||||
if d.Username != "" && d.Password != "" {
|
||||
return d.login(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// login performs the login process and retrieves the access token.
|
||||
func (d *Degoo) login(ctx context.Context) error {
|
||||
if d.Username == "" || d.Password == "" {
|
||||
return fmt.Errorf("username or password not provided")
|
||||
}
|
||||
|
||||
creds := DegooLoginRequest{
|
||||
GenerateToken: true,
|
||||
Username: d.Username,
|
||||
Password: d.Password,
|
||||
}
|
||||
|
||||
jsonCreds, err := json.Marshal(creds)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to serialize login credentials: %w", err)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, "POST", loginURL, bytes.NewBuffer(jsonCreds))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create login request: %w", err)
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", base.UserAgent)
|
||||
req.Header.Set("Origin", "https://app.degoo.com")
|
||||
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("login request failed: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Handle rate limiting (429 Too Many Requests)
|
||||
if resp.StatusCode == http.StatusTooManyRequests {
|
||||
return fmt.Errorf("login rate limited (429), please try again later")
|
||||
}
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("login failed: %s", resp.Status)
|
||||
}
|
||||
|
||||
var loginResp DegooLoginResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&loginResp); err != nil {
|
||||
return fmt.Errorf("failed to parse login response: %w", err)
|
||||
}
|
||||
|
||||
if loginResp.RefreshToken != "" {
|
||||
tokenReq := DegooAccessTokenRequest{RefreshToken: loginResp.RefreshToken}
|
||||
jsonTokenReq, err := json.Marshal(tokenReq)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to serialize access token request: %w", err)
|
||||
}
|
||||
|
||||
tokenReqHTTP, err := http.NewRequestWithContext(ctx, "POST", accessTokenURL, bytes.NewBuffer(jsonTokenReq))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create access token request: %w", err)
|
||||
}
|
||||
|
||||
tokenReqHTTP.Header.Set("User-Agent", base.UserAgent)
|
||||
|
||||
tokenResp, err := d.client.Do(tokenReqHTTP)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get access token: %w", err)
|
||||
}
|
||||
defer tokenResp.Body.Close()
|
||||
|
||||
var accessTokenResp DegooAccessTokenResponse
|
||||
if err := json.NewDecoder(tokenResp.Body).Decode(&accessTokenResp); err != nil {
|
||||
return fmt.Errorf("failed to parse access token response: %w", err)
|
||||
}
|
||||
d.AccessToken = accessTokenResp.AccessToken
|
||||
d.RefreshToken = loginResp.RefreshToken // Save refresh token
|
||||
} else if loginResp.Token != "" {
|
||||
d.AccessToken = loginResp.Token
|
||||
d.RefreshToken = "" // Direct token, no refresh token available
|
||||
} else {
|
||||
return fmt.Errorf("login failed, no valid token returned")
|
||||
}
|
||||
|
||||
// Save the updated tokens to storage
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// apiCall performs a Degoo GraphQL API request.
|
||||
func (d *Degoo) apiCall(ctx context.Context, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
|
||||
// Apply rate limiting
|
||||
applyRateLimit()
|
||||
|
||||
// Ensure we have a valid token before making the API call
|
||||
if err := d.ensureValidToken(ctx); err != nil {
|
||||
return nil, fmt.Errorf("failed to ensure valid token: %w", err)
|
||||
}
|
||||
|
||||
// Update the Token in variables if it exists (after potential refresh)
|
||||
d.updateTokenInVariables(variables)
|
||||
|
||||
return d.executeGraphQLRequest(ctx, operationName, query, variables)
|
||||
}
|
||||
|
||||
// updateTokenInVariables updates the Token field in GraphQL variables
|
||||
func (d *Degoo) updateTokenInVariables(variables map[string]interface{}) {
|
||||
if variables != nil {
|
||||
if _, hasToken := variables["Token"]; hasToken {
|
||||
variables["Token"] = d.AccessToken
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// executeGraphQLRequest executes a GraphQL request with retry logic
|
||||
func (d *Degoo) executeGraphQLRequest(ctx context.Context, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
|
||||
reqBody := map[string]interface{}{
|
||||
"operationName": operationName,
|
||||
"query": query,
|
||||
"variables": variables,
|
||||
}
|
||||
|
||||
// Create and configure request
|
||||
req, err := createJSONRequest(ctx, "POST", apiURL, reqBody)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set Degoo-specific headers
|
||||
req.Header.Set("x-api-key", apiKey)
|
||||
if d.AccessToken != "" {
|
||||
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", d.AccessToken))
|
||||
}
|
||||
|
||||
// Execute request
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("GraphQL API request failed: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Check for HTTP errors
|
||||
if err := checkHTTPResponse(resp, "GraphQL API"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Parse GraphQL response
|
||||
var degooResp DegooGraphqlResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(°ooResp); err != nil {
|
||||
return nil, fmt.Errorf("failed to decode GraphQL response: %w", err)
|
||||
}
|
||||
|
||||
// Handle GraphQL errors
|
||||
if len(degooResp.Errors) > 0 {
|
||||
return d.handleGraphQLError(ctx, degooResp.Errors[0], operationName, query, variables)
|
||||
}
|
||||
|
||||
return degooResp.Data, nil
|
||||
}
|
||||
|
||||
// handleGraphQLError handles GraphQL-level errors with retry logic
|
||||
func (d *Degoo) handleGraphQLError(ctx context.Context, gqlError DegooErrors, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
|
||||
if gqlError.ErrorType == "Unauthorized" {
|
||||
// Re-login and retry
|
||||
if err := d.login(ctx); err != nil {
|
||||
return nil, fmt.Errorf("%s, login failed: %w", errUnauthorized, err)
|
||||
}
|
||||
|
||||
// Update token in variables and retry
|
||||
d.updateTokenInVariables(variables)
|
||||
return d.apiCall(ctx, operationName, query, variables)
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("GraphQL API error: %s", gqlError.Message)
|
||||
}
|
||||
|
||||
// humanReadableTimes converts Degoo timestamps to Go time.Time.
|
||||
func humanReadableTimes(creation, modification, upload string) (cTime, mTime, uTime time.Time) {
|
||||
cTime, _ = time.Parse(time.RFC3339, creation)
|
||||
if modification != "" {
|
||||
modMillis, _ := strconv.ParseInt(modification, 10, 64)
|
||||
mTime = time.Unix(0, modMillis*int64(time.Millisecond))
|
||||
}
|
||||
if upload != "" {
|
||||
upMillis, _ := strconv.ParseInt(upload, 10, 64)
|
||||
uTime = time.Unix(0, upMillis*int64(time.Millisecond))
|
||||
}
|
||||
return cTime, mTime, uTime
|
||||
}
|
||||
|
||||
// getDevices fetches and caches top-level devices and folders.
|
||||
func (d *Degoo) getDevices(ctx context.Context) error {
|
||||
const query = `query GetFileChildren5($Token: String! $ParentID: String $AllParentIDs: [String] $Limit: Int! $Order: Int! $NextToken: String ) { getFileChildren5(Token: $Token ParentID: $ParentID AllParentIDs: $AllParentIDs Limit: $Limit Order: $Order NextToken: $NextToken) { Items { ParentID } NextToken } }`
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ParentID": "0",
|
||||
"Limit": 10,
|
||||
"Order": 3,
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetFileChildren5", query, variables)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp DegooGetChildren5Data
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return fmt.Errorf("failed to parse device list: %w", err)
|
||||
}
|
||||
if d.RootFolderID == "0" {
|
||||
if len(resp.GetFileChildren5.Items) > 0 {
|
||||
d.RootFolderID = resp.GetFileChildren5.Items[0].ParentID
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// getAllFileChildren5 fetches all children of a directory with pagination.
|
||||
func (d *Degoo) getAllFileChildren5(ctx context.Context, parentID string) ([]DegooFileItem, error) {
|
||||
const query = `query GetFileChildren5($Token: String! $ParentID: String $AllParentIDs: [String] $Limit: Int! $Order: Int! $NextToken: String ) { getFileChildren5(Token: $Token ParentID: $ParentID AllParentIDs: $AllParentIDs Limit: $Limit Order: $Order NextToken: $NextToken) { Items { ID ParentID Name Category Size CreationTime LastModificationTime LastUploadTime FilePath IsInRecycleBin DeviceID MetadataID } NextToken } }`
|
||||
var allItems []DegooFileItem
|
||||
nextToken := ""
|
||||
for {
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ParentID": parentID,
|
||||
"Limit": 1000,
|
||||
"Order": 3,
|
||||
}
|
||||
if nextToken != "" {
|
||||
variables["NextToken"] = nextToken
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetFileChildren5", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var resp DegooGetChildren5Data
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
allItems = append(allItems, resp.GetFileChildren5.Items...)
|
||||
if resp.GetFileChildren5.NextToken == "" {
|
||||
break
|
||||
}
|
||||
nextToken = resp.GetFileChildren5.NextToken
|
||||
}
|
||||
return allItems, nil
|
||||
}
|
||||
|
||||
// getOverlay4 fetches metadata for a single item by ID.
|
||||
func (d *Degoo) getOverlay4(ctx context.Context, id string) (DegooFileItem, error) {
|
||||
const query = `query GetOverlay4($Token: String!, $ID: IDType!) { getOverlay4(Token: $Token, ID: $ID) { ID ParentID Name Category Size CreationTime LastModificationTime LastUploadTime URL FilePath IsInRecycleBin DeviceID MetadataID } }`
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ID": map[string]string{
|
||||
"FileID": id,
|
||||
},
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetOverlay4", query, variables)
|
||||
if err != nil {
|
||||
return DegooFileItem{}, err
|
||||
}
|
||||
var resp DegooGetOverlay4Data
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return DegooFileItem{}, fmt.Errorf("failed to parse item metadata: %w", err)
|
||||
}
|
||||
return resp.GetOverlay4, nil
|
||||
}
|
||||
|
||||
func (d *Degoo) getUserInfo(ctx context.Context) (DegooGetUserInfo3Data, error) {
|
||||
const query = "query GetUserInfo3($Token: String!) { getUserInfo3(Token: $Token) { UsedQuota TotalQuota } }"
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetUserInfo3", query, variables)
|
||||
var resp DegooGetUserInfo3Data
|
||||
if err != nil {
|
||||
return resp, err
|
||||
}
|
||||
if err = json.Unmarshal(data, &resp); err != nil {
|
||||
return resp, fmt.Errorf("failed to parse user info: %w", err)
|
||||
}
|
||||
return resp, nil
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
package doubao
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
// Usually one of two
|
||||
// driver.RootPath
|
||||
driver.RootID
|
||||
// define other
|
||||
Cookie string `json:"cookie" type:"text"`
|
||||
UploadThread string `json:"upload_thread" default:"3"`
|
||||
DownloadApi string `json:"download_api" type:"select" options:"get_file_url,get_download_info" default:"get_file_url"`
|
||||
LimitRate float64 `json:"limit_rate" type:"float" default:"2" help:"limit all api request rate ([limit]r/1s)"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Doubao",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "0",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Doubao{
|
||||
Addition: Addition{
|
||||
LimitRate: 2,
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,597 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/big"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
|
||||
"github.com/go-resty/resty/v2"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/crypto/pbkdf2"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultAuthRefreshAheadSeconds = int64(120)
|
||||
defaultDpopRefreshAheadSeconds = int64(5)
|
||||
)
|
||||
|
||||
type Clock interface {
|
||||
Now() (int64, error)
|
||||
}
|
||||
|
||||
type SystemClock struct{}
|
||||
|
||||
func (SystemClock) Now() (int64, error) { return time.Now().Unix(), nil }
|
||||
|
||||
type DPoPTokenInput struct {
|
||||
KeyPair *ecdsa.PrivateKey
|
||||
ExpiresIn int64 // 默认 15
|
||||
|
||||
JTI string
|
||||
HTM string
|
||||
HTU string
|
||||
IAT int64
|
||||
Nonce string
|
||||
Clock Clock
|
||||
}
|
||||
|
||||
type DPoPTokenOutput struct {
|
||||
DPoPToken string `json:"dpopToken"`
|
||||
ExpiredTime int64 `json:"expiredTime"`
|
||||
ExpiresIn int64 `json:"expiresIn"`
|
||||
}
|
||||
|
||||
type JWTPayload struct {
|
||||
Exp int64 `json:"exp,omitempty"`
|
||||
Iat int64 `json:"iat,omitempty"`
|
||||
Nbf int64 `json:"nbf,omitempty"`
|
||||
Jti string `json:"jti,omitempty"`
|
||||
Htm string `json:"htm,omitempty"`
|
||||
Htu string `json:"htu,omitempty"`
|
||||
Nonce string `json:"nonce,omitempty"`
|
||||
Sub string `json:"sub,omitempty"`
|
||||
}
|
||||
|
||||
type jwkECPrivateKey struct {
|
||||
Kty string `json:"kty"`
|
||||
Crv string `json:"crv"`
|
||||
X string `json:"x"`
|
||||
Y string `json:"y"`
|
||||
D string `json:"d"`
|
||||
}
|
||||
|
||||
type dpopKeyPairEnvelope struct {
|
||||
PrivateKey *jwkECPrivateKey `json:"privateKey"`
|
||||
KeyPair *jwkECPrivateKey `json:"keyPair"`
|
||||
JWK *jwkECPrivateKey `json:"jwk"`
|
||||
}
|
||||
|
||||
type encryptedDpopKeyPair struct {
|
||||
Data string `json:"data"`
|
||||
Ciphertext string `json:"ciphertext"`
|
||||
Encrypted string `json:"encrypted"`
|
||||
Secret string `json:"secret"`
|
||||
Password string `json:"password"`
|
||||
Passphrase string `json:"passphrase"`
|
||||
}
|
||||
|
||||
func GenerateDPoPToken(in DPoPTokenInput) (*DPoPTokenOutput, error) {
|
||||
if in.KeyPair == nil {
|
||||
return nil, errors.New("keyPair required")
|
||||
}
|
||||
if in.KeyPair.Curve != elliptic.P256() {
|
||||
return nil, errors.New("ES256 requires P-256 key")
|
||||
}
|
||||
if in.Clock == nil {
|
||||
in.Clock = SystemClock{}
|
||||
}
|
||||
if in.ExpiresIn <= 0 {
|
||||
in.ExpiresIn = 15
|
||||
}
|
||||
|
||||
now, err := in.Clock.Now()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
payload := map[string]any{
|
||||
"jti": pickStr(in.JTI, uuid.NewString()),
|
||||
"htm": pickStr(in.HTM, ""),
|
||||
"htu": pickStr(in.HTU, ""),
|
||||
"iat": pickI64(in.IAT, now),
|
||||
"nonce": pickStr(in.Nonce, uuid.NewString()),
|
||||
}
|
||||
if in.ExpiresIn > 0 {
|
||||
payload["exp"] = payload["iat"].(int64) + in.ExpiresIn
|
||||
}
|
||||
|
||||
pub := in.KeyPair.PublicKey
|
||||
header := map[string]any{
|
||||
"typ": "dpop+jwt",
|
||||
"alg": "ES256",
|
||||
"jwk": map[string]string{
|
||||
"kty": "EC",
|
||||
"crv": "P-256",
|
||||
"x": b64url(pad32(pub.X.Bytes())),
|
||||
"y": b64url(pad32(pub.Y.Bytes())),
|
||||
},
|
||||
}
|
||||
|
||||
hb, err := json.Marshal(header)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pb, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
hEnc := b64url(hb)
|
||||
pEnc := b64url(pb)
|
||||
signingInput := hEnc + "." + pEnc
|
||||
|
||||
sum := sha256.Sum256([]byte(signingInput))
|
||||
r, s, err := ecdsa.Sign(rand.Reader, in.KeyPair, sum[:])
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
sig := append(pad32(r.Bytes()), pad32(s.Bytes())...)
|
||||
token := signingInput + "." + b64url(sig)
|
||||
|
||||
iat := payload["iat"].(int64)
|
||||
return &DPoPTokenOutput{
|
||||
DPoPToken: token,
|
||||
ExpiredTime: iat + in.ExpiresIn,
|
||||
ExpiresIn: in.ExpiresIn,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func GenerateDPoPKeyPair() (*ecdsa.PrivateKey, error) {
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return validateP256Key(key)
|
||||
}
|
||||
|
||||
func ParseJWTPayload(token string, out any) error {
|
||||
token = strings.TrimSpace(trimTokenScheme(token))
|
||||
parts := strings.Split(token, ".")
|
||||
if len(parts) < 2 {
|
||||
return fmt.Errorf("invalid JWT format")
|
||||
}
|
||||
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to decode JWT payload: %w", err)
|
||||
}
|
||||
if err := json.Unmarshal(payload, out); err != nil {
|
||||
return fmt.Errorf("failed to parse JWT payload: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func parseECPrivateKeyJWK(raw string) (*ecdsa.PrivateKey, error) {
|
||||
var jwk jwkECPrivateKey
|
||||
if err := json.Unmarshal([]byte(raw), &jwk); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if jwk.D == "" || jwk.X == "" || jwk.Y == "" {
|
||||
var env dpopKeyPairEnvelope
|
||||
if err := json.Unmarshal([]byte(raw), &env); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch {
|
||||
case env.PrivateKey != nil:
|
||||
jwk = *env.PrivateKey
|
||||
case env.KeyPair != nil:
|
||||
jwk = *env.KeyPair
|
||||
case env.JWK != nil:
|
||||
jwk = *env.JWK
|
||||
default:
|
||||
return nil, errors.New("missing private key JWK")
|
||||
}
|
||||
}
|
||||
|
||||
if jwk.Kty != "" && jwk.Kty != "EC" {
|
||||
return nil, errors.New("unsupported JWK kty")
|
||||
}
|
||||
if jwk.Crv != "" && jwk.Crv != "P-256" {
|
||||
return nil, errors.New("unsupported JWK curve")
|
||||
}
|
||||
if jwk.D == "" || jwk.X == "" || jwk.Y == "" {
|
||||
return nil, errors.New("incomplete JWK")
|
||||
}
|
||||
|
||||
xBytes, err := base64.RawURLEncoding.DecodeString(jwk.X)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid jwk x: %w", err)
|
||||
}
|
||||
yBytes, err := base64.RawURLEncoding.DecodeString(jwk.Y)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid jwk y: %w", err)
|
||||
}
|
||||
dBytes, err := base64.RawURLEncoding.DecodeString(jwk.D)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid jwk d: %w", err)
|
||||
}
|
||||
|
||||
key := &ecdsa.PrivateKey{
|
||||
PublicKey: ecdsa.PublicKey{
|
||||
Curve: elliptic.P256(),
|
||||
X: new(big.Int).SetBytes(xBytes),
|
||||
Y: new(big.Int).SetBytes(yBytes),
|
||||
},
|
||||
D: new(big.Int).SetBytes(dBytes),
|
||||
}
|
||||
return validateP256Key(key)
|
||||
}
|
||||
|
||||
func parseEncryptedDPoPKeyPair(raw, secret string) (*ecdsa.PrivateKey, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if raw == "" {
|
||||
return nil, errors.New("empty encrypted key pair")
|
||||
}
|
||||
|
||||
var payload encryptedDpopKeyPair
|
||||
ciphertext := raw
|
||||
if strings.HasPrefix(raw, "{") {
|
||||
if err := json.Unmarshal([]byte(raw), &payload); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch {
|
||||
case strings.TrimSpace(payload.Data) != "":
|
||||
ciphertext = strings.TrimSpace(payload.Data)
|
||||
case strings.TrimSpace(payload.Ciphertext) != "":
|
||||
ciphertext = strings.TrimSpace(payload.Ciphertext)
|
||||
case strings.TrimSpace(payload.Encrypted) != "":
|
||||
ciphertext = strings.TrimSpace(payload.Encrypted)
|
||||
default:
|
||||
return nil, errors.New("missing encrypted dpop payload")
|
||||
}
|
||||
}
|
||||
|
||||
decoded, err := decodeBase64Loose(ciphertext)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(decoded) <= 12 {
|
||||
return nil, errors.New("encrypted dpop payload too short")
|
||||
}
|
||||
|
||||
plain, err := decryptDoubaoKeyPair(decoded, secret)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to decrypt with secret: %w", err)
|
||||
}
|
||||
return parseECPrivateKeyJWK(string(plain))
|
||||
}
|
||||
|
||||
func decryptDoubaoKeyPair(ciphertext []byte, secret string) ([]byte, error) {
|
||||
key := pbkdf2.Key([]byte(secret), []byte("fixed-salt"), 100000, 32, sha256.New)
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
aead, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
nonceSize := aead.NonceSize()
|
||||
if len(ciphertext) <= nonceSize {
|
||||
return nil, errors.New("ciphertext too short")
|
||||
}
|
||||
nonce := ciphertext[:nonceSize]
|
||||
enc := ciphertext[nonceSize:]
|
||||
return aead.Open(nil, nonce, enc, nil)
|
||||
}
|
||||
|
||||
func decodeBase64Loose(raw string) ([]byte, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
raw = strings.ReplaceAll(raw, "\n", "")
|
||||
raw = strings.ReplaceAll(raw, "\r", "")
|
||||
raw = strings.ReplaceAll(raw, "\t", "")
|
||||
raw = strings.ReplaceAll(raw, " ", "")
|
||||
|
||||
encodings := []*base64.Encoding{
|
||||
base64.StdEncoding,
|
||||
base64.RawStdEncoding,
|
||||
base64.URLEncoding,
|
||||
base64.RawURLEncoding,
|
||||
}
|
||||
var lastErr error
|
||||
for _, enc := range encodings {
|
||||
decoded, err := enc.DecodeString(raw)
|
||||
if err == nil {
|
||||
return decoded, nil
|
||||
}
|
||||
lastErr = err
|
||||
}
|
||||
if lastErr == nil {
|
||||
lastErr = errors.New("invalid base64")
|
||||
}
|
||||
return nil, lastErr
|
||||
}
|
||||
|
||||
func validateP256Key(key *ecdsa.PrivateKey) (*ecdsa.PrivateKey, error) {
|
||||
if key == nil {
|
||||
return nil, errors.New("nil private key")
|
||||
}
|
||||
if key.Curve != elliptic.P256() {
|
||||
return nil, errors.New("ES256 requires P-256 key")
|
||||
}
|
||||
if key.PublicKey.X == nil || key.PublicKey.Y == nil || key.D == nil {
|
||||
return nil, errors.New("invalid private key")
|
||||
}
|
||||
if !key.Curve.IsOnCurve(key.PublicKey.X, key.PublicKey.Y) {
|
||||
return nil, errors.New("public key is not on P-256 curve")
|
||||
}
|
||||
return key, nil
|
||||
}
|
||||
|
||||
func trimTokenScheme(token string) string {
|
||||
token = strings.TrimSpace(token)
|
||||
if i := strings.IndexByte(token, ' '); i > 0 {
|
||||
scheme := strings.ToLower(strings.TrimSpace(token[:i]))
|
||||
if scheme == "bearer" || scheme == "dpop" {
|
||||
return strings.TrimSpace(token[i+1:])
|
||||
}
|
||||
}
|
||||
return token
|
||||
}
|
||||
|
||||
func b64url(b []byte) string {
|
||||
return base64.RawURLEncoding.EncodeToString(b)
|
||||
}
|
||||
|
||||
func pad32(b []byte) []byte {
|
||||
if len(b) >= 32 {
|
||||
return b[len(b)-32:]
|
||||
}
|
||||
out := make([]byte, 32)
|
||||
copy(out[32-len(b):], b)
|
||||
return out
|
||||
}
|
||||
|
||||
func pickStr(v, def string) string {
|
||||
if v != "" {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
func pickI64(v, def int64) int64 {
|
||||
if v != 0 {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveAuthorization() string {
|
||||
auth := trimTokenScheme(d.Authorization)
|
||||
if auth == "" {
|
||||
return ""
|
||||
}
|
||||
return "DPoP " + auth
|
||||
}
|
||||
|
||||
func shouldRefreshJWT(token string) bool {
|
||||
if token == "" {
|
||||
return true
|
||||
}
|
||||
var payload JWTPayload
|
||||
if err := ParseJWTPayload(token, &payload); err != nil {
|
||||
return true
|
||||
}
|
||||
if payload.Exp <= 0 {
|
||||
return false
|
||||
}
|
||||
return payload.Exp <= time.Now().Unix()+defaultAuthRefreshAheadSeconds
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) fetchBizAuth(dpop string, public bool) (string, error) {
|
||||
var reqUrl string
|
||||
client := base.RestyClient.Clone()
|
||||
req := client.R()
|
||||
req.SetHeader("accept", "application/json, text/javascript")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("content-type", "application/x-www-form-urlencoded")
|
||||
if public {
|
||||
reqUrl = DoubaoURL + "/passport/anonymity_user/biz_auth/"
|
||||
} else {
|
||||
reqUrl = DoubaoURL + "/passport/user/biz_auth/"
|
||||
if d.Cookie != "" {
|
||||
req.SetHeader("cookie", d.Cookie)
|
||||
if csrf := strings.TrimSpace(cookie.GetStr(d.Cookie, "passport_csrf_token")); csrf != "" {
|
||||
req.SetHeader("x-tt-passport-csrf-token", csrf)
|
||||
}
|
||||
}
|
||||
if oldAuth := d.resolveAuthorization(); oldAuth != "" {
|
||||
req.SetHeader("authorization", oldAuth)
|
||||
}
|
||||
}
|
||||
if dpop != "" {
|
||||
req.SetHeader("dpop", dpop)
|
||||
}
|
||||
values := url.Values{}
|
||||
values.Set("client_id", d.AuthClientID)
|
||||
values.Set("client_type", d.AuthClientType)
|
||||
values.Set("scope", d.AuthScope)
|
||||
values.Set("d_pop", dpop)
|
||||
req.SetBody(values.Encode())
|
||||
req.SetQueryParam("aid", d.AppID)
|
||||
req.SetQueryParam("account_sdk_source", d.AuthSDKSource)
|
||||
req.SetQueryParam("sdk_version", d.AuthSDKVersion)
|
||||
|
||||
res, err := req.Post(reqUrl)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
var resp bizAuthResp
|
||||
if err = json.Unmarshal(res.Body(), &resp); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if resp.Message != "success" || resp.Data.AccessToken == "" {
|
||||
return "", fmt.Errorf("[doubao_new] %s: %s", resp.Message, resp.Data.Description)
|
||||
}
|
||||
return resp.Data.AccessToken, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) refreshAuthorizationWithDPoP(dpop string) (string, error) {
|
||||
token, err := d.fetchBizAuth(dpop, false)
|
||||
if err == nil && token != "" {
|
||||
return token, nil
|
||||
}
|
||||
if err == nil {
|
||||
err = errors.New("biz auth refresh failed")
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveDpopForRequest(method, rawURL string) (string, error) {
|
||||
if d.DPoPKeyPair != nil {
|
||||
proof, err := GenerateDPoPToken(DPoPTokenInput{
|
||||
KeyPair: d.DPoPKeyPair,
|
||||
HTM: strings.ToUpper(strings.TrimSpace(method)),
|
||||
HTU: normalizeDPoPURL(rawURL),
|
||||
})
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return proof.DPoPToken, nil
|
||||
}
|
||||
|
||||
static := d.DPoP
|
||||
if static == "" {
|
||||
return "", nil
|
||||
}
|
||||
if !d.IgnoreJWTCheck {
|
||||
if payload, err := parseDPoPPayload(static); err == nil && payload.Exp > 0 {
|
||||
now := time.Now().Unix()
|
||||
if payload.Exp <= now+defaultDpopRefreshAheadSeconds {
|
||||
return "", errors.New("static dpop token expired or near expiry; configure dpop_key_pair for automatic refresh")
|
||||
}
|
||||
}
|
||||
}
|
||||
return static, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) ensureAuthAdditons() bool {
|
||||
return d.DPoPKeySecret != "" && d.AuthClientID != "" && d.AuthClientType != "" &&
|
||||
d.AuthScope != "" && d.AuthSDKSource != "" && d.AuthSDKVersion != ""
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveAuthorizationForRequest(method, rawURL string) (string, error) {
|
||||
if !shouldRefreshJWT(d.Authorization) {
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
if d.DPoPKeyPair == nil || strings.TrimSpace(d.Cookie) == "" || !d.ensureAuthAdditons() {
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
d.authRefreshMu.Lock()
|
||||
defer d.authRefreshMu.Unlock()
|
||||
|
||||
if !shouldRefreshJWT(d.Authorization) {
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
refreshDpop, err := d.resolveDpopForRequest(method, rawURL)
|
||||
if err != nil || refreshDpop == "" {
|
||||
return "", err
|
||||
}
|
||||
|
||||
newToken, err := d.refreshAuthorizationWithDPoP(refreshDpop)
|
||||
if err != nil {
|
||||
if auth := d.resolveAuthorization(); auth != "" {
|
||||
return auth, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
d.Authorization = trimTokenScheme(newToken)
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveAuthorizationForPublic() (dpop string, auth string, err error) {
|
||||
if d.DPoPPublic != "" && !shouldRefreshJWT(d.AuthorizationPublic) {
|
||||
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
|
||||
}
|
||||
|
||||
if !d.ensureAuthAdditons() {
|
||||
return "", "", fmt.Errorf("[doubao_new] missing auth additions, please fill them all")
|
||||
}
|
||||
|
||||
d.authRefreshPublicMu.Lock()
|
||||
defer d.authRefreshPublicMu.Unlock()
|
||||
|
||||
if d.DPoPPublic != "" && !shouldRefreshJWT(d.AuthorizationPublic) {
|
||||
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
|
||||
}
|
||||
|
||||
// generate new public dpop
|
||||
keypair, err := GenerateDPoPKeyPair()
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
proof, err := GenerateDPoPToken(DPoPTokenInput{
|
||||
KeyPair: keypair,
|
||||
})
|
||||
d.DPoPPublic = proof.DPoPToken
|
||||
|
||||
// get authorization token
|
||||
d.AuthorizationPublic, err = d.fetchBizAuth(proof.DPoPToken, true)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
|
||||
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) applyAuthHeaders(req *resty.Request, method, rawURL string) error {
|
||||
auth, err := d.resolveAuthorizationForRequest(method, rawURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if auth != "" {
|
||||
req.SetHeader("authorization", auth)
|
||||
}
|
||||
dpop, err := d.resolveDpopForRequest(method, rawURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if dpop != "" {
|
||||
req.SetHeader("dpop", dpop)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func normalizeDPoPURL(rawURL string) string {
|
||||
u, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return rawURL
|
||||
}
|
||||
u.Fragment = ""
|
||||
return u.String()
|
||||
}
|
||||
|
||||
func parseDPoPPayload(token string) (*JWTPayload, error) {
|
||||
var payload JWTPayload
|
||||
if err := ParseJWTPayload(token, &payload); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &payload, nil
|
||||
}
|
||||
@@ -1,533 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/ecdsa"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type DoubaoNew struct {
|
||||
model.Storage
|
||||
Addition
|
||||
TtLogid string
|
||||
|
||||
// DPoP access token (Authorization header value, without DPoP prefix)
|
||||
Authorization string
|
||||
AuthorizationPublic string
|
||||
// DPoP header value
|
||||
DPoP string
|
||||
DPoPPublic string
|
||||
// DPoP key pair for generating DPoP
|
||||
DPoPKeyPairStr string
|
||||
DPoPKeyPair *ecdsa.PrivateKey
|
||||
|
||||
authRefreshMu sync.Mutex
|
||||
authRefreshPublicMu sync.Mutex
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Init(ctx context.Context) error {
|
||||
if cookieStr := strings.TrimSpace(d.Cookie); cookieStr != "" {
|
||||
d.Cookie = cookieStr
|
||||
auth := trimTokenScheme(cookie.GetStr(d.Cookie, "LARK_SUITE_ACCESS_TOKEN"))
|
||||
if auth != "" {
|
||||
d.Authorization = auth
|
||||
}
|
||||
dpop := strings.TrimSpace(cookie.GetStr(d.Cookie, "LARK_SUITE_DPOP"))
|
||||
if dpop != "" {
|
||||
d.DPoP = dpop
|
||||
}
|
||||
keypair := strings.TrimSpace(cookie.GetStr(d.Cookie, "feishu_dpop_keypair"))
|
||||
if keypair != "" && d.DPoPKeySecret != "" {
|
||||
d.DPoPKeyPairStr = keypair
|
||||
d.DPoPKeyPair, _ = parseEncryptedDPoPKeyPair(keypair, d.DPoPKeySecret)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Drop(ctx context.Context) error {
|
||||
if d.Authorization != "" {
|
||||
d.Cookie = cookie.SetStr(d.Cookie, "LARK_SUITE_ACCESS_TOKEN", d.Authorization)
|
||||
}
|
||||
if d.DPoP != "" {
|
||||
d.Cookie = cookie.SetStr(d.Cookie, "LARK_SUITE_DPOP", d.DPoP)
|
||||
}
|
||||
if d.DPoPKeyPairStr != "" {
|
||||
d.Cookie = cookie.SetStr(d.Cookie, "feishu_dpop_keypair", d.DPoPKeyPairStr)
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
nodes, err := d.listAllChildren(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
objs := make([]model.Obj, 0, len(nodes))
|
||||
for _, node := range nodes {
|
||||
if node.NodeToken == "" || node.ObjToken == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
size := parseSize(node.Extra.Size)
|
||||
isFolder := node.Type == 0
|
||||
if isFolder && node.NodeToken == dir.GetID() {
|
||||
continue
|
||||
}
|
||||
|
||||
obj := &Object{
|
||||
Object: model.Object{
|
||||
ID: node.NodeToken,
|
||||
Path: dir.GetID(),
|
||||
Name: node.Name,
|
||||
Size: size,
|
||||
Modified: time.Unix(node.EditTime, 0),
|
||||
Ctime: time.Unix(node.CreateTime, 0),
|
||||
IsFolder: isFolder,
|
||||
},
|
||||
ObjToken: node.ObjToken,
|
||||
NodeType: node.NodeType,
|
||||
ObjType: node.Type,
|
||||
URL: node.URL,
|
||||
}
|
||||
objs = append(objs, obj)
|
||||
}
|
||||
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
obj, ok := file.(*Object)
|
||||
if !ok {
|
||||
return nil, errors.New("unsupported object type")
|
||||
}
|
||||
if obj.IsFolder {
|
||||
return nil, fmt.Errorf("link is directory")
|
||||
}
|
||||
var (
|
||||
err error
|
||||
auth, dpop string
|
||||
)
|
||||
if d.ShareLink {
|
||||
err := d.createShare(ctx, obj)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
dpop, auth, err = d.resolveAuthorizationForPublic()
|
||||
} else {
|
||||
// TODO: append previewLink() with auth args to support ShareLink
|
||||
if args.Type == "preview" || args.Type == "thumb" {
|
||||
if link, err := d.previewLink(ctx, obj, args); err == nil {
|
||||
return link, nil
|
||||
}
|
||||
}
|
||||
auth = d.resolveAuthorization()
|
||||
dpop, err = d.resolveDpopForRequest(http.MethodGet, DownloadBaseURL+"/space/api/box/stream/download/all/"+obj.ObjToken+"/")
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if auth == "" || dpop == "" {
|
||||
return nil, errors.New("missing authorization or dpop")
|
||||
}
|
||||
if obj.ObjToken == "" {
|
||||
return nil, errors.New("missing obj_token")
|
||||
}
|
||||
|
||||
query := url.Values{}
|
||||
query.Set("authorization", auth)
|
||||
query.Set("dpop", dpop)
|
||||
|
||||
downloadURL := DownloadBaseURL + "/space/api/box/stream/download/all/" + obj.ObjToken + "/?" + query.Encode()
|
||||
|
||||
headers := http.Header{
|
||||
"Referer": []string{DoubaoURL + "/"},
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
URL: downloadURL,
|
||||
Header: headers,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
node, err := d.createFolder(ctx, parentDir.GetID(), dirName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Object{
|
||||
Object: model.Object{
|
||||
ID: node.NodeToken,
|
||||
Path: parentDir.GetID(),
|
||||
Name: node.Name,
|
||||
Size: parseSize(node.Extra.Size),
|
||||
Modified: time.Unix(node.EditTime, 0),
|
||||
Ctime: time.Unix(node.CreateTime, 0),
|
||||
IsFolder: true,
|
||||
},
|
||||
ObjToken: node.ObjToken,
|
||||
NodeType: node.NodeType,
|
||||
ObjType: node.Type,
|
||||
URL: node.URL,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
if srcObj == nil {
|
||||
return nil, errors.New("nil source object")
|
||||
}
|
||||
if dstDir == nil {
|
||||
return nil, errors.New("nil destination dir")
|
||||
}
|
||||
srcToken := srcObj.GetID()
|
||||
if srcToken == "" {
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
srcToken = obj.ObjToken
|
||||
}
|
||||
}
|
||||
if srcToken == "" {
|
||||
return nil, errors.New("missing source token")
|
||||
}
|
||||
if err := d.moveObj(ctx, srcToken, dstDir.GetID()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
clone := *obj
|
||||
clone.Path = dstDir.GetID()
|
||||
return &clone, nil
|
||||
}
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
|
||||
if srcObj == nil {
|
||||
return nil, errors.New("nil source object")
|
||||
}
|
||||
if srcObj.IsDir() {
|
||||
if err := d.renameFolder(ctx, srcObj.GetID(), newName); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
fileToken := ""
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
fileToken = obj.ObjToken
|
||||
}
|
||||
if fileToken == "" {
|
||||
fileToken = srcObj.GetID()
|
||||
}
|
||||
if err := d.renameFile(ctx, fileToken, newName); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
clone := *obj
|
||||
clone.Name = newName
|
||||
return &clone, nil
|
||||
}
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
// TODO copy obj, optional
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Remove(ctx context.Context, obj model.Obj) error {
|
||||
if obj == nil {
|
||||
return errors.New("nil object")
|
||||
}
|
||||
token := obj.GetID()
|
||||
if token == "" {
|
||||
if o, ok := obj.(*Object); ok {
|
||||
token = o.ObjToken
|
||||
}
|
||||
}
|
||||
if token == "" {
|
||||
return errors.New("missing object token")
|
||||
}
|
||||
return d.removeObj(ctx, []string{token})
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
if file == nil {
|
||||
return nil, errors.New("nil file")
|
||||
}
|
||||
if file.GetSize() <= 0 {
|
||||
return nil, errors.New("invalid file size")
|
||||
}
|
||||
|
||||
uploadPrep, err := d.prepareUpload(ctx, file.GetName(), file.GetSize(), dstDir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if uploadPrep.BlockSize <= 0 {
|
||||
return nil, errors.New("invalid block size from prepare")
|
||||
}
|
||||
|
||||
tmpFile, err := utils.CreateTempFile(file, file.GetSize())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() {
|
||||
_ = tmpFile.Close()
|
||||
_ = os.Remove(tmpFile.Name())
|
||||
}()
|
||||
|
||||
blockSize := uploadPrep.BlockSize
|
||||
totalSize := file.GetSize()
|
||||
numBlocks := int((totalSize + blockSize - 1) / blockSize)
|
||||
blocks := make([]UploadBlock, 0, numBlocks)
|
||||
blockMeta := make(map[int]UploadBlock, numBlocks)
|
||||
|
||||
for seq := 0; seq < numBlocks; seq++ {
|
||||
offset := int64(seq) * blockSize
|
||||
length := blockSize
|
||||
if remain := totalSize - offset; remain < length {
|
||||
length = remain
|
||||
}
|
||||
buf := make([]byte, int(length))
|
||||
n, err := tmpFile.ReadAt(buf, offset)
|
||||
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
|
||||
return nil, err
|
||||
}
|
||||
buf = buf[:n]
|
||||
sum := sha256.Sum256(buf)
|
||||
hash := base64.StdEncoding.EncodeToString(sum[:])
|
||||
checksum := adler32String(buf)
|
||||
|
||||
block := UploadBlock{
|
||||
Hash: hash,
|
||||
Seq: seq,
|
||||
Size: int64(n),
|
||||
Checksum: checksum,
|
||||
IsUploaded: true,
|
||||
}
|
||||
blocks = append(blocks, block)
|
||||
blockMeta[seq] = block
|
||||
}
|
||||
|
||||
needed, err := d.uploadBlocks(ctx, uploadPrep.UploadID, blocks, "explorer")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(needed.NeededUploadBlocks) > 0 {
|
||||
sort.Slice(needed.NeededUploadBlocks, func(i, j int) bool {
|
||||
return needed.NeededUploadBlocks[i].Seq < needed.NeededUploadBlocks[j].Seq
|
||||
})
|
||||
const maxMergeBlockCount = 20
|
||||
var (
|
||||
groupSeqs []int
|
||||
groupChecksums []string
|
||||
groupSizes []int64
|
||||
groupRealSize int64
|
||||
groupExpectSum int64
|
||||
groupBuf bytes.Buffer
|
||||
uploadedBytes int64
|
||||
)
|
||||
|
||||
flushGroup := func() error {
|
||||
if len(groupSeqs) == 0 {
|
||||
return nil
|
||||
}
|
||||
data := groupBuf.Bytes()
|
||||
expectLen := groupExpectSum
|
||||
if int64(len(data)) != expectLen {
|
||||
return fmt.Errorf("[doubao_new] merge blocks invalid body len: got=%d expect=%d seqs=%v", len(data), expectLen, groupSeqs)
|
||||
}
|
||||
mergeResp, err := d.mergeUploadBlocks(ctx, uploadPrep.UploadID, groupSeqs, groupChecksums, groupSizes, blockSize, data)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(mergeResp.SuccessSeqList) != len(groupSeqs) {
|
||||
return fmt.Errorf("[doubao_new] merge blocks incomplete: %v", mergeResp.SuccessSeqList)
|
||||
}
|
||||
success := make(map[int]bool, len(mergeResp.SuccessSeqList))
|
||||
for _, seq := range mergeResp.SuccessSeqList {
|
||||
success[seq] = true
|
||||
}
|
||||
for _, seq := range groupSeqs {
|
||||
if !success[seq] {
|
||||
return fmt.Errorf("[doubao_new] merge blocks missing seq %d", seq)
|
||||
}
|
||||
}
|
||||
|
||||
uploadedBytes += groupRealSize
|
||||
groupSeqs = groupSeqs[:0]
|
||||
groupChecksums = groupChecksums[:0]
|
||||
groupSizes = groupSizes[:0]
|
||||
groupRealSize = 0
|
||||
groupExpectSum = 0
|
||||
groupBuf.Reset()
|
||||
if up != nil {
|
||||
percent := float64(uploadedBytes) / float64(totalSize) * 100
|
||||
up(percent)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
for _, item := range needed.NeededUploadBlocks {
|
||||
if _, ok := blockMeta[item.Seq]; !ok {
|
||||
return nil, fmt.Errorf("[doubao_new] missing block meta for seq %d", item.Seq)
|
||||
}
|
||||
if item.Size <= 0 {
|
||||
return nil, fmt.Errorf("[doubao_new] invalid block size from needed list: seq=%d size=%d", item.Seq, item.Size)
|
||||
}
|
||||
offset := int64(item.Seq) * blockSize
|
||||
buf := make([]byte, int(item.Size))
|
||||
n, err := tmpFile.ReadAt(buf, offset)
|
||||
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
|
||||
return nil, err
|
||||
}
|
||||
if n != len(buf) {
|
||||
return nil, fmt.Errorf("[doubao_new] short read: seq=%d want=%d got=%d", item.Seq, len(buf), n)
|
||||
}
|
||||
buf = buf[:n]
|
||||
realAdler := adler32String(buf)
|
||||
if realAdler != item.Checksum {
|
||||
return nil, fmt.Errorf("[doubao_new] block checksum mismatch: seq=%d offset=%d adler32=%s step2=%s", item.Seq, offset, realAdler, item.Checksum)
|
||||
}
|
||||
payloadStart := groupBuf.Len()
|
||||
groupBuf.Write(buf)
|
||||
payloadEnd := groupBuf.Len()
|
||||
payloadAdler := adler32String(groupBuf.Bytes()[payloadStart:payloadEnd])
|
||||
if payloadAdler != item.Checksum {
|
||||
return nil, fmt.Errorf("[doubao_new] payload checksum mismatch: seq=%d start=%d end=%d adler32=%s step2=%s", item.Seq, payloadStart, payloadEnd, payloadAdler, item.Checksum)
|
||||
}
|
||||
groupSeqs = append(groupSeqs, item.Seq)
|
||||
groupChecksums = append(groupChecksums, item.Checksum)
|
||||
groupSizes = append(groupSizes, item.Size)
|
||||
groupRealSize += int64(n)
|
||||
groupExpectSum += item.Size
|
||||
if len(groupSeqs) >= maxMergeBlockCount {
|
||||
if err := flushGroup(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if err := flushGroup(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if up != nil {
|
||||
up(100)
|
||||
}
|
||||
} else if up != nil {
|
||||
up(100)
|
||||
}
|
||||
|
||||
numBlocksFinish := uploadPrep.NumBlocks
|
||||
if numBlocksFinish <= 0 {
|
||||
numBlocksFinish = numBlocks
|
||||
}
|
||||
finish, err := d.finishUpload(ctx, uploadPrep.UploadID, numBlocksFinish, "explorer")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
nodeToken := finish.Extra.NodeToken
|
||||
if nodeToken == "" {
|
||||
nodeToken = finish.FileToken
|
||||
}
|
||||
now := time.Now()
|
||||
return &Object{
|
||||
Object: model.Object{
|
||||
ID: nodeToken,
|
||||
Path: dstDir.GetID(),
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
Modified: now,
|
||||
Ctime: now,
|
||||
IsFolder: false,
|
||||
},
|
||||
ObjToken: finish.FileToken,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
data, err := d.getUserStorage(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: data.TotalSizeLimitBytes,
|
||||
UsedSpace: data.UsedSizeBytes,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
|
||||
switch args.Method {
|
||||
case "doubao_preview", "preview":
|
||||
obj, ok := args.Obj.(*Object)
|
||||
if !ok {
|
||||
return nil, errors.New("unsupported object type")
|
||||
}
|
||||
info, err := d.getFileInfo(ctx, obj.ObjToken)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entry, ok := info.PreviewMeta.Data["22"]
|
||||
if !ok || entry.Status != 0 {
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
|
||||
imgExt := ".webp"
|
||||
pageNums := 1
|
||||
if entry.Extra != "" {
|
||||
var extra PreviewImageExtra
|
||||
if err := json.Unmarshal([]byte(entry.Extra), &extra); err == nil {
|
||||
if extra.ImgExt != "" {
|
||||
imgExt = extra.ImgExt
|
||||
}
|
||||
if extra.PageNums > 0 {
|
||||
pageNums = extra.PageNums
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return base.Json{
|
||||
"version": info.Version,
|
||||
"img_ext": imgExt,
|
||||
"page_nums": pageNums,
|
||||
}, nil
|
||||
default:
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*DoubaoNew)(nil)
|
||||
@@ -1,39 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
// Usually one of two
|
||||
driver.RootID
|
||||
// define other
|
||||
Cookie string `json:"cookie" required:"true" help:"Web Cookie"`
|
||||
AppID string `json:"app_id" required:"true" default:"497858" help:"Doubao App ID"`
|
||||
DPoPKeySecret string `json:"dpop_key_secret" help:"DPoP Key Secret for generating DPoP token"`
|
||||
AuthClientID string `json:"auth_client_id" help:"Doubao Biz Auth Client ID"`
|
||||
AuthClientType string `json:"auth_client_type" help:"Doubao Biz Auth Client Type"`
|
||||
AuthScope string `json:"auth_scope" help:"Doubao Biz Auth Scope"`
|
||||
AuthSDKSource string `json:"auth_sdk_source" help:"Doubao Biz Auth SDK Source"`
|
||||
AuthSDKVersion string `json:"auth_sdk_version" help:"Doubao Biz Auth SDK Version"`
|
||||
ShareLink bool `json:"share_link" help:"Whether to use share link for download"`
|
||||
IgnoreJWTCheck bool `json:"ignore_jwt_check" help:"Whether to ignore JWT check to prevent time issue"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "DoubaoNew",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "",
|
||||
Alert: `danger|Do not use 302 if the storage is public accessible.
|
||||
Otherwise, the download link may leak sensitive information such as access token or signature.
|
||||
Others may use the leaked link to access all your files.`,
|
||||
NoOverwriteUpload: false,
|
||||
PreferProxy: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &DoubaoNew{}
|
||||
})
|
||||
}
|
||||
@@ -1,192 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import "github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
|
||||
type BaseResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg,omitempty"`
|
||||
Message string `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
type ListResp struct {
|
||||
BaseResp
|
||||
Data ListData `json:"data"`
|
||||
}
|
||||
|
||||
type ListData struct {
|
||||
HasMore bool `json:"has_more"`
|
||||
LastLabel string `json:"last_label"`
|
||||
NodeList []string `json:"node_list"`
|
||||
Entities struct {
|
||||
Nodes map[string]Node `json:"nodes"`
|
||||
Users map[string]User `json:"users"`
|
||||
} `json:"entities"`
|
||||
}
|
||||
|
||||
type Node struct {
|
||||
Token string `json:"token"`
|
||||
NodeToken string `json:"node_token"`
|
||||
ObjToken string `json:"obj_token"`
|
||||
Name string `json:"name"`
|
||||
Type int `json:"type"`
|
||||
NodeType int `json:"node_type"`
|
||||
OwnerID string `json:"owner_id"`
|
||||
EditUID string `json:"edit_uid"`
|
||||
CreateTime int64 `json:"create_time"`
|
||||
EditTime int64 `json:"edit_time"`
|
||||
URL string `json:"url"`
|
||||
Extra struct {
|
||||
Size string `json:"size"`
|
||||
} `json:"extra"`
|
||||
}
|
||||
|
||||
type User struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type Object struct {
|
||||
model.Object
|
||||
ObjToken string
|
||||
NodeType int
|
||||
ObjType int
|
||||
URL string
|
||||
}
|
||||
|
||||
type CreateFolderResp struct {
|
||||
BaseResp
|
||||
Data struct {
|
||||
Entities struct {
|
||||
Nodes map[string]Node `json:"nodes"`
|
||||
} `json:"entities"`
|
||||
NodeList []string `json:"node_list"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type FileInfoResp struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Data FileInfo `json:"data"`
|
||||
}
|
||||
|
||||
type FileInfo struct {
|
||||
Name string `json:"name"`
|
||||
NumBlocks int `json:"num_blocks"`
|
||||
Version string `json:"version"`
|
||||
MimeType string `json:"mime_type"`
|
||||
MountPoint string `json:"mount_point"`
|
||||
PreviewMeta PreviewMeta `json:"preview_meta"`
|
||||
}
|
||||
|
||||
type PreviewMeta struct {
|
||||
Data map[string]PreviewMetaEntry `json:"data"`
|
||||
}
|
||||
|
||||
type PreviewMetaEntry struct {
|
||||
Status int `json:"status"`
|
||||
Extra string `json:"extra"`
|
||||
PreviewFileSize int64 `json:"preview_file_size"`
|
||||
}
|
||||
|
||||
type PreviewImageExtra struct {
|
||||
ImgExt string `json:"img_ext"`
|
||||
PageNums int `json:"page_nums"`
|
||||
}
|
||||
|
||||
type UserStorageResp struct {
|
||||
BaseResp
|
||||
Data UserStorageData `json:"data"`
|
||||
}
|
||||
|
||||
type UserStorageData struct {
|
||||
ShowSizeLimit bool `json:"show_size_limit"`
|
||||
TotalSizeLimitBytes int64 `json:"total_size_limit_bytes"`
|
||||
UsedSizeBytes int64 `json:"used_size_bytes"`
|
||||
}
|
||||
|
||||
type UploadPrepareResp struct {
|
||||
BaseResp
|
||||
Data UploadPrepareData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadPrepareData struct {
|
||||
BlockSize int64 `json:"block_size"`
|
||||
NumBlocks int `json:"num_blocks"`
|
||||
OptionBlockSize int64 `json:"option_block_size"`
|
||||
DedupeSupport bool `json:"dedupe_support"`
|
||||
UploadID string `json:"upload_id"`
|
||||
}
|
||||
|
||||
type UploadBlock struct {
|
||||
Hash string `json:"hash"`
|
||||
Seq int `json:"seq"`
|
||||
Size int64 `json:"size"`
|
||||
Checksum string `json:"checksum"`
|
||||
IsUploaded bool `json:"isUploaded"`
|
||||
}
|
||||
|
||||
type UploadBlocksResp struct {
|
||||
BaseResp
|
||||
Data UploadBlocksData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadBlocksData struct {
|
||||
NeededUploadBlocks []UploadBlockNeed `json:"needed_upload_blocks"`
|
||||
}
|
||||
|
||||
type UploadBlockNeed struct {
|
||||
Seq int `json:"seq"`
|
||||
Size int64 `json:"size"`
|
||||
Checksum string `json:"checksum"`
|
||||
Hash string `json:"hash"`
|
||||
}
|
||||
|
||||
type UploadMergeResp struct {
|
||||
BaseResp
|
||||
Data UploadMergeData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadMergeData struct {
|
||||
SuccessSeqList []int `json:"success_seq_list"`
|
||||
}
|
||||
|
||||
type UploadFinishResp struct {
|
||||
BaseResp
|
||||
Data UploadFinishData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadFinishData struct {
|
||||
Version string `json:"version"`
|
||||
DataVersion string `json:"data_version"`
|
||||
Extra struct {
|
||||
NodeToken string `json:"node_token"`
|
||||
} `json:"extra"`
|
||||
FileToken string `json:"file_token"`
|
||||
}
|
||||
|
||||
type RemoveResp struct {
|
||||
BaseResp
|
||||
Data struct {
|
||||
TaskID string `json:"task_id"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type TaskStatusResp struct {
|
||||
BaseResp
|
||||
Data TaskStatusData `json:"data"`
|
||||
}
|
||||
|
||||
type TaskStatusData struct {
|
||||
IsFinish bool `json:"is_finish"`
|
||||
IsFail bool `json:"is_fail"`
|
||||
}
|
||||
|
||||
type bizAuthResp struct {
|
||||
Data struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
AuthScheme string `json:"auth_scheme"`
|
||||
ExpiresIn int64 `json:"expires_in"`
|
||||
Description string `json:"description,omitempty"`
|
||||
} `json:"data"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
@@ -1,283 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
func (d *DoubaoNew) prepareUpload(ctx context.Context, name string, size int64, mountNodeToken string) (UploadPrepareData, error) {
|
||||
var resp UploadPrepareResp
|
||||
_, err := d.request(ctx, "/space/api/box/upload/prepare/", http.MethodPost, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
req.SetQueryParamsFromValues(values)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetHeader("x-command", "space.api.box.upload.prepare")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("cache-control", "no-cache")
|
||||
req.SetHeader("pragma", "no-cache")
|
||||
body := base.Json{
|
||||
"mount_point": "explorer",
|
||||
"mount_node_token": "",
|
||||
"name": name,
|
||||
"size": size,
|
||||
"size_checker": true,
|
||||
}
|
||||
if mountNodeToken != "" {
|
||||
body["mount_node_token"] = mountNodeToken
|
||||
}
|
||||
req.SetBody(body)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return UploadPrepareData{}, err
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) uploadBlocks(ctx context.Context, uploadID string, blocks []UploadBlock, mountPoint string) (UploadBlocksData, error) {
|
||||
if uploadID == "" {
|
||||
return UploadBlocksData{}, fmt.Errorf("[doubao_new] upload blocks missing upload_id")
|
||||
}
|
||||
if mountPoint == "" {
|
||||
mountPoint = "explorer"
|
||||
}
|
||||
var resp UploadBlocksResp
|
||||
_, err := d.request(ctx, "/space/api/box/upload/blocks/", http.MethodPost, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
req.SetQueryParamsFromValues(values)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetHeader("x-command", "space.api.box.upload.blocks")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("cache-control", "no-cache")
|
||||
req.SetHeader("pragma", "no-cache")
|
||||
req.SetBody(base.Json{
|
||||
"blocks": blocks,
|
||||
"upload_id": uploadID,
|
||||
"mount_point": mountPoint,
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return UploadBlocksData{}, err
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) mergeUploadBlocks(ctx context.Context, uploadID string, seqList []int, checksumList []string, sizeList []int64, blockOriginSize int64, data []byte) (UploadMergeData, error) {
|
||||
if uploadID == "" {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks missing upload_id")
|
||||
}
|
||||
if len(seqList) == 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty seq list")
|
||||
}
|
||||
if len(checksumList) == 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty checksum list")
|
||||
}
|
||||
if len(sizeList) != len(seqList) {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks size list mismatch")
|
||||
}
|
||||
if blockOriginSize <= 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks invalid block origin size")
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty data")
|
||||
}
|
||||
|
||||
seqHeader := joinIntComma(seqList)
|
||||
checksumHeader := buildCommaHeader(checksumList)
|
||||
|
||||
client := base.NewRestyClient()
|
||||
client.SetCookieJar(nil)
|
||||
req := client.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("content-type", "application/octet-stream")
|
||||
req.SetHeader("x-block-list-checksum", checksumHeader)
|
||||
req.SetHeader("x-seq-list", seqHeader)
|
||||
req.SetHeader("x-block-origin-size", strconv.FormatInt(blockOriginSize, 10))
|
||||
req.SetHeader("x-command", "space.api.box.stream.upload.merge_block")
|
||||
req.SetHeader("x-csrftoken", "")
|
||||
reqID := ""
|
||||
if buf := make([]byte, 16); true {
|
||||
if _, err := rand.Read(buf); err == nil {
|
||||
reqID = hex.EncodeToString(buf)
|
||||
}
|
||||
}
|
||||
if reqID != "" {
|
||||
req.SetHeader("x-request-id", reqID)
|
||||
}
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("upload_id", uploadID)
|
||||
values.Set("mount_point", "explorer")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
urlStr := DownloadBaseURL + "/space/api/box/stream/upload/merge_block/?" + values.Encode()
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, urlStr); err != nil {
|
||||
return UploadMergeData{}, err
|
||||
}
|
||||
req.Header.Del("cookie")
|
||||
if req.Header.Get("x-command") == "" {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks missing x-command header")
|
||||
}
|
||||
req.SetBody(data)
|
||||
|
||||
res, err := req.Execute(http.MethodPost, urlStr)
|
||||
if err != nil {
|
||||
return UploadMergeData{}, err
|
||||
}
|
||||
if v := res.Header().Get("X-Tt-Logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
} else if v := res.Header().Get("x-tt-logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
}
|
||||
body := res.Body()
|
||||
var resp UploadMergeResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return UploadMergeData{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
if res != nil && res.StatusCode() == http.StatusBadRequest && resp.Code == 2 {
|
||||
success := make([]int, 0, len(seqList))
|
||||
offset := 0
|
||||
for i, seq := range seqList {
|
||||
size := sizeList[i]
|
||||
if size <= 0 {
|
||||
return UploadMergeData{SuccessSeqList: success}, fmt.Errorf("[doubao_new] v3 fallback invalid size: seq=%d size=%d", seq, size)
|
||||
}
|
||||
if offset+int(size) > len(data) {
|
||||
return UploadMergeData{SuccessSeqList: success}, fmt.Errorf("[doubao_new] v3 fallback payload out of range: seq=%d offset=%d size=%d total=%d", seq, offset, size, len(data))
|
||||
}
|
||||
payload := data[offset : offset+int(size)]
|
||||
block := UploadBlockNeed{
|
||||
Seq: seq,
|
||||
Size: size,
|
||||
Checksum: checksumList[i],
|
||||
}
|
||||
if err := d.uploadBlockV3(ctx, uploadID, block, payload); err != nil {
|
||||
return UploadMergeData{SuccessSeqList: success}, err
|
||||
}
|
||||
success = append(success, seq)
|
||||
offset += int(size)
|
||||
}
|
||||
return UploadMergeData{SuccessSeqList: success}, nil
|
||||
}
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) uploadBlockV3(ctx context.Context, uploadID string, block UploadBlockNeed, data []byte) error {
|
||||
if uploadID == "" {
|
||||
return fmt.Errorf("[doubao_new] upload v3 block missing upload_id")
|
||||
}
|
||||
if block.Seq < 0 {
|
||||
return fmt.Errorf("[doubao_new] upload v3 block invalid seq")
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return fmt.Errorf("[doubao_new] upload v3 block empty data")
|
||||
}
|
||||
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("x-block-seq", strconv.Itoa(block.Seq))
|
||||
req.SetHeader("x-block-checksum", block.Checksum)
|
||||
req.SetMultipartFormData(map[string]string{
|
||||
"upload_id": uploadID,
|
||||
"size": strconv.FormatInt(int64(len(data)), 10),
|
||||
})
|
||||
req.SetMultipartField("file", "blob", "application/octet-stream", bytes.NewReader(data))
|
||||
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("upload_id", uploadID)
|
||||
values.Set("seq", strconv.Itoa(block.Seq))
|
||||
values.Set("size", strconv.FormatInt(int64(len(data)), 10))
|
||||
values.Set("checksum", block.Checksum)
|
||||
values.Set("mount_point", "explorer")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
urlStr := DownloadBaseURL + "/space/api/box/stream/upload/v3/block/?" + values.Encode()
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, urlStr); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
res, err := req.Execute(http.MethodPost, urlStr)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body := res.Body()
|
||||
if err := decodeBaseResp(body, res); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) finishUpload(ctx context.Context, uploadID string, numBlocks int, mountPoint string) (UploadFinishData, error) {
|
||||
if uploadID == "" {
|
||||
return UploadFinishData{}, fmt.Errorf("[doubao_new] finish upload missing upload_id")
|
||||
}
|
||||
if numBlocks <= 0 {
|
||||
return UploadFinishData{}, fmt.Errorf("[doubao_new] finish upload invalid num_blocks")
|
||||
}
|
||||
if mountPoint == "" {
|
||||
mountPoint = "explorer"
|
||||
}
|
||||
var resp UploadFinishResp
|
||||
_, err := d.request(ctx, "/space/api/box/upload/finish/", http.MethodPost, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
req.SetQueryParamsFromValues(values)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetHeader("x-command", "space.api.box.upload.finish")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("cache-control", "no-cache")
|
||||
req.SetHeader("pragma", "no-cache")
|
||||
req.SetHeader("biz-scene", "file_upload")
|
||||
req.SetHeader("biz-ua-type", "Web")
|
||||
req.SetBody(base.Json{
|
||||
"upload_id": uploadID,
|
||||
"num_blocks": numBlocks,
|
||||
"mount_point": mountPoint,
|
||||
"push_open_history_record": 1,
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return UploadFinishData{}, err
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
@@ -1,713 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"hash/adler32"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
const (
|
||||
BaseURL = "https://my.feishu.cn"
|
||||
DownloadBaseURL = "https://internal-api-drive-stream.feishu.cn"
|
||||
DoubaoURL = "https://www.doubao.com"
|
||||
)
|
||||
|
||||
var defaultObjTypes = []string{"124", "0", "12", "30", "123", "22"}
|
||||
|
||||
func (d *DoubaoNew) request(ctx context.Context, path string, method string, callback base.ReqCallback, resp interface{}) ([]byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, method, BaseURL+path); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
|
||||
res, err := req.Execute(method, BaseURL+path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if res != nil {
|
||||
if v := res.Header().Get("X-Tt-Logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
} else if v := res.Header().Get("x-tt-logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
}
|
||||
}
|
||||
|
||||
body := res.Body()
|
||||
var common BaseResp
|
||||
if err = json.Unmarshal(body, &common); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return body, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if common.Code != 0 {
|
||||
errMsg := common.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = common.Message
|
||||
}
|
||||
return body, fmt.Errorf("[doubao_new] API error (code: %d): %s", common.Code, errMsg)
|
||||
}
|
||||
if resp != nil {
|
||||
if err = json.Unmarshal(body, resp); err != nil {
|
||||
return body, err
|
||||
}
|
||||
}
|
||||
|
||||
return body, nil
|
||||
}
|
||||
|
||||
func adler32String(data []byte) string {
|
||||
sum := adler32.Checksum(data)
|
||||
return strconv.FormatUint(uint64(sum), 10)
|
||||
}
|
||||
|
||||
func buildCommaHeader(items []string) string {
|
||||
return strings.Join(items, ",")
|
||||
}
|
||||
|
||||
func joinIntComma(items []int) string {
|
||||
if len(items) == 0 {
|
||||
return ""
|
||||
}
|
||||
var sb strings.Builder
|
||||
for i, v := range items {
|
||||
if i > 0 {
|
||||
sb.WriteByte(',')
|
||||
}
|
||||
sb.WriteString(strconv.Itoa(v))
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
func previewList(items []string, n int) string {
|
||||
if n <= 0 || len(items) == 0 {
|
||||
return ""
|
||||
}
|
||||
if len(items) < n {
|
||||
n = len(items)
|
||||
}
|
||||
return strings.Join(items[:n], ",")
|
||||
}
|
||||
|
||||
func parseSize(size string) int64 {
|
||||
if size == "" {
|
||||
return 0
|
||||
}
|
||||
val, err := strconv.ParseInt(size, 10, 64)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) listChildren(ctx context.Context, parentToken string, lastLabel string, length int) (ListData, error) {
|
||||
var resp ListResp
|
||||
_, err := d.request(ctx, "/space/api/explorer/doubao/children/list/", http.MethodGet, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
for _, t := range defaultObjTypes {
|
||||
values.Add("obj_type", t)
|
||||
}
|
||||
values.Set("length", strconv.Itoa(length))
|
||||
values.Set("rank", "0")
|
||||
values.Set("asc", "0")
|
||||
values.Set("min_length", "40")
|
||||
values.Set("thumbnail_width", "1028")
|
||||
values.Set("thumbnail_height", "1028")
|
||||
values.Set("thumbnail_policy", "4")
|
||||
if parentToken != "" {
|
||||
values.Set("token", parentToken)
|
||||
}
|
||||
if lastLabel != "" {
|
||||
values.Set("last_label", lastLabel)
|
||||
}
|
||||
req.SetQueryParamsFromValues(values)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return ListData{}, err
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) listAllChildren(ctx context.Context, parentToken string) ([]Node, error) {
|
||||
length := 50
|
||||
nodes := make([]Node, 0, length)
|
||||
lastLabel := ""
|
||||
for range 100 {
|
||||
data, err := d.listChildren(ctx, parentToken, lastLabel, length)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(data.NodeList) > 0 {
|
||||
for _, token := range data.NodeList {
|
||||
node, ok := data.Entities.Nodes[token]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
nodes = append(nodes, node)
|
||||
}
|
||||
} else {
|
||||
for _, node := range data.Entities.Nodes {
|
||||
nodes = append(nodes, node)
|
||||
}
|
||||
}
|
||||
|
||||
if !data.HasMore || data.LastLabel == "" || data.LastLabel == lastLabel {
|
||||
break
|
||||
}
|
||||
lastLabel = data.LastLabel
|
||||
}
|
||||
|
||||
if len(nodes) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
return nodes, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) getFileInfo(ctx context.Context, fileToken string) (FileInfo, error) {
|
||||
var resp FileInfoResp
|
||||
_, err := d.request(ctx, "/space/api/box/file/info/", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"caller": "explorer",
|
||||
"file_token": fileToken,
|
||||
"mount_point": "explorer",
|
||||
"option_params": []string{"preview_meta", "check_cipher"},
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return FileInfo{}, err
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) previewLink(ctx context.Context, obj *Object, args model.LinkArgs) (*model.Link, error) {
|
||||
auth := d.resolveAuthorization()
|
||||
dpop, err := d.resolveDpopForRequest(http.MethodGet, fmt.Sprintf("%s/space/api/box/stream/download/preview_sub/%s", BaseURL, obj.ObjToken))
|
||||
if auth == "" || dpop == "" {
|
||||
return nil, errors.New("missing authorization or dpop")
|
||||
}
|
||||
if obj.ObjToken == "" {
|
||||
return nil, errors.New("missing obj_token")
|
||||
}
|
||||
info, err := d.getFileInfo(ctx, obj.ObjToken)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
entry, ok := info.PreviewMeta.Data["22"]
|
||||
if !ok || entry.Status != 0 {
|
||||
return nil, errors.New("preview not available")
|
||||
}
|
||||
|
||||
subID := ""
|
||||
pageIndex := 0
|
||||
|
||||
if subID == "" {
|
||||
imgExt := ".webp"
|
||||
pageNums := 0
|
||||
if entry.Extra != "" {
|
||||
var extra PreviewImageExtra
|
||||
if err := json.Unmarshal([]byte(entry.Extra), &extra); err == nil {
|
||||
if extra.ImgExt != "" {
|
||||
imgExt = extra.ImgExt
|
||||
}
|
||||
pageNums = extra.PageNums
|
||||
}
|
||||
}
|
||||
if pageNums > 0 && pageIndex >= pageNums {
|
||||
pageIndex = pageNums - 1
|
||||
}
|
||||
subID = fmt.Sprintf("img_%d%s", pageIndex, imgExt)
|
||||
}
|
||||
|
||||
query := url.Values{}
|
||||
query.Set("preview_type", "22")
|
||||
query.Set("sub_id", subID)
|
||||
if info.Version != "" {
|
||||
query.Set("version", info.Version)
|
||||
}
|
||||
previewURL := fmt.Sprintf("%s/space/api/box/stream/download/preview_sub/%s?%s", BaseURL, obj.ObjToken, query.Encode())
|
||||
|
||||
headers := http.Header{
|
||||
"Referer": []string{DoubaoURL + "/"},
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
"Authorization": []string{auth},
|
||||
"Dpop": []string{dpop},
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
URL: previewURL,
|
||||
Header: headers,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) createShare(ctx context.Context, obj *Object) error {
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/suite/permission/public/update.v5/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"external_access_entity": 1,
|
||||
"link_share_entity": 4,
|
||||
"token": obj.ObjToken,
|
||||
"type": obj.ObjType,
|
||||
})
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/suite/permission/public/update.v5/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := decodeBaseResp(body, res); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) createFolder(ctx context.Context, parentToken, name string) (Node, error) {
|
||||
data := url.Values{}
|
||||
data.Set("name", name)
|
||||
data.Set("source", "0")
|
||||
if parentToken != "" {
|
||||
data.Set("parent_token", parentToken)
|
||||
}
|
||||
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/create/folder/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.SetBody(data.Encode())
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/create/folder/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return Node{}, err
|
||||
}
|
||||
if err := decodeBaseResp(body, res); err != nil {
|
||||
return Node{}, err
|
||||
}
|
||||
|
||||
var resp CreateFolderResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return Node{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
|
||||
var node Node
|
||||
if len(resp.Data.NodeList) > 0 {
|
||||
if n, ok := resp.Data.Entities.Nodes[resp.Data.NodeList[0]]; ok {
|
||||
node = n
|
||||
}
|
||||
}
|
||||
if node.Token == "" {
|
||||
for _, n := range resp.Data.Entities.Nodes {
|
||||
node = n
|
||||
break
|
||||
}
|
||||
}
|
||||
if node.Token == "" && node.ObjToken == "" && node.NodeToken == "" {
|
||||
return Node{}, fmt.Errorf("[doubao_new] create folder failed: empty response")
|
||||
}
|
||||
if node.NodeToken == "" {
|
||||
if node.Token != "" {
|
||||
node.NodeToken = node.Token
|
||||
} else if node.ObjToken != "" {
|
||||
node.NodeToken = node.ObjToken
|
||||
}
|
||||
}
|
||||
if node.ObjToken == "" && node.Token != "" {
|
||||
node.ObjToken = node.Token
|
||||
}
|
||||
return node, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) renameFolder(ctx context.Context, token, name string) error {
|
||||
if token == "" {
|
||||
return fmt.Errorf("[doubao_new] rename folder missing token")
|
||||
}
|
||||
data := url.Values{}
|
||||
data.Set("token", token)
|
||||
data.Set("name", name)
|
||||
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/rename/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.SetBody(data.Encode())
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/rename/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return decodeBaseResp(body, res)
|
||||
}
|
||||
|
||||
func isCsrfTokenError(body []byte, res *resty.Response) bool {
|
||||
if len(body) == 0 {
|
||||
return false
|
||||
}
|
||||
if strings.Contains(strings.ToLower(string(body)), "csrf token error") {
|
||||
return true
|
||||
}
|
||||
if res != nil && res.StatusCode() == http.StatusForbidden {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func doRequestWithCsrf(doRequest func(csrfToken string) (*resty.Response, []byte, error)) (*resty.Response, []byte, error) {
|
||||
res, body, err := doRequest("")
|
||||
if err != nil {
|
||||
return res, body, err
|
||||
}
|
||||
if isCsrfTokenError(body, res) {
|
||||
csrfToken := extractCsrfTokenFromResponse(res)
|
||||
if csrfToken != "" {
|
||||
return doRequest(csrfToken)
|
||||
}
|
||||
}
|
||||
return res, body, err
|
||||
}
|
||||
|
||||
func extractCsrfTokenFromResponse(res *resty.Response) string {
|
||||
if res == nil || res.Request == nil {
|
||||
return ""
|
||||
}
|
||||
if res.Request.RawRequest != nil {
|
||||
if csrf := cookie.GetStr(res.Request.RawRequest.Header.Get("Cookie"), "_csrf_token"); csrf != "" {
|
||||
return csrf
|
||||
}
|
||||
}
|
||||
if csrf := cookie.GetStr(res.Request.Header.Get("Cookie"), "_csrf_token"); csrf != "" {
|
||||
return csrf
|
||||
}
|
||||
for _, c := range res.Cookies() {
|
||||
if c.Name == "_csrf_token" {
|
||||
return c.Value
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func decodeBaseResp(body []byte, res *resty.Response) error {
|
||||
var common BaseResp
|
||||
if err := json.Unmarshal(body, &common); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
if common.Code != 0 {
|
||||
errMsg := common.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = common.Message
|
||||
}
|
||||
return fmt.Errorf("[doubao_new] API error (code: %d): %s", common.Code, errMsg)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) renameFile(ctx context.Context, fileToken, name string) error {
|
||||
if fileToken == "" {
|
||||
return fmt.Errorf("[doubao_new] rename file missing file token")
|
||||
}
|
||||
_, err := d.request(ctx, "/space/api/box/file/update_info/", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"file_token": fileToken,
|
||||
"name": name,
|
||||
})
|
||||
}, nil)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) moveObj(ctx context.Context, srcToken, destToken string) error {
|
||||
if srcToken == "" {
|
||||
return fmt.Errorf("[doubao_new] move missing src token")
|
||||
}
|
||||
data := url.Values{}
|
||||
data.Set("src_token", srcToken)
|
||||
if destToken != "" {
|
||||
data.Set("dest_token", destToken)
|
||||
}
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/move/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.SetBody(data.Encode())
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/move/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return decodeBaseResp(body, res)
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) removeObj(ctx context.Context, tokens []string) error {
|
||||
if len(tokens) == 0 {
|
||||
return fmt.Errorf("[doubao_new] remove missing tokens")
|
||||
}
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v3/remove/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"tokens": tokens,
|
||||
"apply": 1,
|
||||
})
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v3/remove/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp RemoveResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
if resp.Data.TaskID == "" {
|
||||
return nil
|
||||
}
|
||||
return d.waitTask(ctx, resp.Data.TaskID)
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) getUserStorage(ctx context.Context) (UserStorageData, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("agw-js-conv", "str")
|
||||
req.SetHeader("content-type", "application/json")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, DoubaoURL+"/alice/aispace/facade/get_user_storage"); err != nil {
|
||||
return UserStorageData{}, err
|
||||
}
|
||||
if d.Cookie != "" {
|
||||
req.SetHeader("cookie", d.Cookie)
|
||||
}
|
||||
req.SetBody(base.Json{})
|
||||
|
||||
res, err := req.Execute(http.MethodPost, DoubaoURL+"/alice/aispace/facade/get_user_storage")
|
||||
if err != nil {
|
||||
return UserStorageData{}, err
|
||||
}
|
||||
|
||||
body := res.Body()
|
||||
var resp UserStorageResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return UserStorageData{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return UserStorageData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) waitTask(ctx context.Context, taskID string) error {
|
||||
const (
|
||||
taskPollInterval = time.Second
|
||||
taskPollMaxAttempts = 120
|
||||
)
|
||||
var lastErr error
|
||||
for attempt := 0; attempt < taskPollMaxAttempts; attempt++ {
|
||||
if attempt > 0 {
|
||||
if err := waitWithContext(ctx, taskPollInterval); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
status, err := d.getTaskStatus(ctx, taskID)
|
||||
if err != nil {
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
if status.IsFail {
|
||||
return fmt.Errorf("[doubao_new] remove task failed: %s", taskID)
|
||||
}
|
||||
if status.IsFinish {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if lastErr != nil {
|
||||
return lastErr
|
||||
}
|
||||
return fmt.Errorf("[doubao_new] remove task timed out: %s", taskID)
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) getTaskStatus(ctx context.Context, taskID string) (TaskStatusData, error) {
|
||||
if taskID == "" {
|
||||
return TaskStatusData{}, fmt.Errorf("[doubao_new] task status missing task_id")
|
||||
}
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodGet, BaseURL+"/space/api/explorer/v2/task/"); err != nil {
|
||||
return TaskStatusData{}, err
|
||||
}
|
||||
req.SetQueryParam("task_id", taskID)
|
||||
res, err := req.Execute(http.MethodGet, BaseURL+"/space/api/explorer/v2/task/")
|
||||
if err != nil {
|
||||
return TaskStatusData{}, err
|
||||
}
|
||||
body := res.Body()
|
||||
var resp TaskStatusResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return TaskStatusData{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return TaskStatusData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func waitWithContext(ctx context.Context, d time.Duration) error {
|
||||
timer := time.NewTimer(d)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@@ -1,261 +0,0 @@
|
||||
package emby
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Emby struct {
|
||||
model.Storage
|
||||
Addition
|
||||
|
||||
client *http.Client
|
||||
token string
|
||||
userID string
|
||||
}
|
||||
|
||||
func (d *Emby) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Emby) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Emby) Init(ctx context.Context) error {
|
||||
d.URL = strings.TrimRight(strings.TrimSpace(d.URL), "/")
|
||||
if d.URL == "" {
|
||||
return fmt.Errorf("url is required")
|
||||
}
|
||||
|
||||
if strings.TrimSpace(d.RootFolderID) == "" {
|
||||
d.RootFolderID = "1"
|
||||
}
|
||||
|
||||
d.client = base.HttpClient
|
||||
d.token = strings.TrimSpace(d.ApiKey)
|
||||
d.userID = strings.TrimSpace(d.UserID)
|
||||
|
||||
if d.token != "" {
|
||||
if d.userID == "" {
|
||||
return fmt.Errorf("user_id is required when api_key is set")
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
if strings.TrimSpace(d.Username) == "" {
|
||||
return fmt.Errorf("please provide api_key+user_id or username(+password)")
|
||||
}
|
||||
|
||||
if err := d.login(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
d.ApiKey = d.token
|
||||
d.UserID = d.userID
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Emby) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Emby) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
parentID := strings.TrimSpace(d.RootFolderID)
|
||||
if dir != nil && strings.TrimSpace(dir.GetID()) != "" {
|
||||
parentID = strings.TrimSpace(dir.GetID())
|
||||
}
|
||||
|
||||
items, err := d.getItems(ctx, parentID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
parentPath := "/"
|
||||
if dir != nil && strings.TrimSpace(dir.GetPath()) != "" {
|
||||
parentPath = dir.GetPath()
|
||||
}
|
||||
|
||||
objs := make([]model.Obj, 0, len(items.Items))
|
||||
for _, it := range items.Items {
|
||||
modified := time.Now()
|
||||
if it.DateCreated != "" {
|
||||
if t, parseErr := time.Parse(time.RFC3339Nano, it.DateCreated); parseErr == nil {
|
||||
modified = t
|
||||
}
|
||||
}
|
||||
|
||||
name := strings.TrimSpace(it.Name)
|
||||
id := strings.TrimSpace(it.ID)
|
||||
displayName := name
|
||||
if name != "" && id != "" {
|
||||
if it.IsFolder {
|
||||
displayName = fmt.Sprintf("%s (ID%s)", name, id)
|
||||
} else {
|
||||
ext := path.Ext(strings.TrimSpace(it.Path))
|
||||
if ext == "" {
|
||||
ext = path.Ext(name)
|
||||
}
|
||||
|
||||
base := strings.TrimSpace(strings.TrimSuffix(name, ext))
|
||||
episodeCode := ""
|
||||
if m := episodeCodeRegexp.FindString(base); m != "" {
|
||||
episodeCode = strings.ToUpper(m)
|
||||
} else if it.ParentIndex > 0 && it.IndexNumber > 0 {
|
||||
episodeCode = fmt.Sprintf("S%02dE%02d", it.ParentIndex, it.IndexNumber)
|
||||
}
|
||||
|
||||
title := strings.TrimSpace(base)
|
||||
if episodeCode != "" {
|
||||
title = strings.TrimSpace(episodeCodeRegexp.ReplaceAllString(title, ""))
|
||||
title = strings.TrimSpace(strings.Trim(title, "-_:[]() "))
|
||||
}
|
||||
|
||||
series := strings.TrimSpace(it.SeriesName)
|
||||
if series == "" && episodeCode != "" {
|
||||
if idx := strings.Index(title, " - "); idx > 0 {
|
||||
series = strings.TrimSpace(title[:idx])
|
||||
title = strings.TrimSpace(title[idx+3:])
|
||||
}
|
||||
}
|
||||
|
||||
core := title
|
||||
if series != "" {
|
||||
if title == "" || strings.EqualFold(series, title) {
|
||||
core = series
|
||||
} else {
|
||||
core = series + " " + title
|
||||
}
|
||||
}
|
||||
if core == "" {
|
||||
core = base
|
||||
}
|
||||
|
||||
if episodeCode != "" {
|
||||
core = fmt.Sprintf("%s - [%s]", core, episodeCode)
|
||||
}
|
||||
if ext == "" {
|
||||
displayName = fmt.Sprintf("%s (ID%s)", core, id)
|
||||
} else {
|
||||
displayName = fmt.Sprintf("%s (ID%s)%s", core, id, ext)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
obj := &model.Object{
|
||||
ID: id,
|
||||
Name: displayName,
|
||||
Path: path.Join(parentPath, displayName),
|
||||
Size: it.Size,
|
||||
Modified: modified,
|
||||
IsFolder: it.IsFolder,
|
||||
}
|
||||
if it.IsFolder {
|
||||
obj.Size = 0
|
||||
}
|
||||
objs = append(objs, obj)
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *Emby) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if file.IsDir() {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
fileID := strings.TrimSpace(file.GetID())
|
||||
if fileID == "" {
|
||||
return nil, fmt.Errorf("invalid file id")
|
||||
}
|
||||
|
||||
u, err := url.Parse(d.URL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
linkMethod := strings.ToLower(strings.TrimSpace(d.LinkMethod))
|
||||
useDownload := linkMethod == "download"
|
||||
|
||||
mediaSourceID := ""
|
||||
mediaContainer := ""
|
||||
if !useDownload {
|
||||
detailURL, parseErr := url.Parse(d.URL + "/Users/" + d.userID + "/Items/" + fileID)
|
||||
if parseErr == nil {
|
||||
q := detailURL.Query()
|
||||
q.Set("Fields", "MediaSources")
|
||||
q.Set("api_key", d.token)
|
||||
detailURL.RawQuery = q.Encode()
|
||||
|
||||
req, reqErr := http.NewRequestWithContext(ctx, http.MethodGet, detailURL.String(), nil)
|
||||
if reqErr == nil {
|
||||
resp, doErr := d.client.Do(req)
|
||||
if doErr == nil {
|
||||
func() {
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return
|
||||
}
|
||||
var detail itemDetailResp
|
||||
if decodeErr := json.NewDecoder(resp.Body).Decode(&detail); decodeErr != nil || len(detail.MediaSources) == 0 {
|
||||
return
|
||||
}
|
||||
for i := range detail.MediaSources {
|
||||
if strings.TrimSpace(detail.MediaSources[i].ID) != "" && detail.MediaSources[i].SupportsDirectStream {
|
||||
mediaSourceID = strings.TrimSpace(detail.MediaSources[i].ID)
|
||||
mediaContainer = strings.TrimSpace(detail.MediaSources[i].Container)
|
||||
return
|
||||
}
|
||||
}
|
||||
for i := range detail.MediaSources {
|
||||
if strings.TrimSpace(detail.MediaSources[i].ID) != "" {
|
||||
mediaSourceID = strings.TrimSpace(detail.MediaSources[i].ID)
|
||||
mediaContainer = strings.TrimSpace(detail.MediaSources[i].Container)
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if useDownload {
|
||||
u.Path = path.Join(u.Path, "/Items", fileID, "Download")
|
||||
} else {
|
||||
if mediaContainer != "" {
|
||||
u.Path = path.Join(u.Path, "/Videos", fileID, "stream."+mediaContainer)
|
||||
} else {
|
||||
u.Path = path.Join(u.Path, "/Videos", fileID, "stream")
|
||||
}
|
||||
}
|
||||
q := u.Query()
|
||||
q.Set("api_key", d.token)
|
||||
if mediaSourceID != "" {
|
||||
q.Set("MediaSourceId", mediaSourceID)
|
||||
}
|
||||
if !useDownload {
|
||||
q.Set("Static", "true")
|
||||
}
|
||||
u.RawQuery = q.Encode()
|
||||
|
||||
return &model.Link{
|
||||
URL: u.String(),
|
||||
Header: http.Header{
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Emby)(nil)
|
||||
@@ -1,30 +0,0 @@
|
||||
package emby
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
URL string `json:"url" required:"true"`
|
||||
ApiKey string `json:"api_key"`
|
||||
UserID string `json:"user_id"`
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
LinkMethod string `json:"link_method" type:"select" options:"stream,download" default:"stream"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Emby",
|
||||
LocalSort: true,
|
||||
NoUpload: true,
|
||||
DefaultRoot: "1",
|
||||
CheckStatus: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Emby{}
|
||||
})
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
package emby
|
||||
|
||||
type authReq struct {
|
||||
Username string `json:"Username"`
|
||||
Pw string `json:"Pw"`
|
||||
}
|
||||
|
||||
type authResp struct {
|
||||
AccessToken string `json:"AccessToken"`
|
||||
User struct {
|
||||
ID string `json:"Id"`
|
||||
} `json:"User"`
|
||||
}
|
||||
|
||||
type listResp struct {
|
||||
Items []embyItem `json:"Items"`
|
||||
TotalRecordCount int `json:"TotalRecordCount"`
|
||||
}
|
||||
|
||||
type embyItem struct {
|
||||
Name string `json:"Name"`
|
||||
ID string `json:"Id"`
|
||||
Type string `json:"Type"`
|
||||
Path string `json:"Path"`
|
||||
SeriesName string `json:"SeriesName"`
|
||||
IndexNumber int `json:"IndexNumber"`
|
||||
ParentIndex int `json:"ParentIndexNumber"`
|
||||
IsFolder bool `json:"IsFolder"`
|
||||
Size int64 `json:"Size"`
|
||||
DateCreated string `json:"DateCreated"`
|
||||
}
|
||||
|
||||
type itemDetailResp struct {
|
||||
MediaSources []embyMediaSource `json:"MediaSources"`
|
||||
}
|
||||
|
||||
type embyMediaSource struct {
|
||||
ID string `json:"Id"`
|
||||
Container string `json:"Container"`
|
||||
SupportsDirectStream bool `json:"SupportsDirectStream"`
|
||||
}
|
||||
@@ -1,91 +0,0 @@
|
||||
package emby
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var episodeCodeRegexp = regexp.MustCompile(`(?i)\bS\d{1,2}E\d{1,2}\b`)
|
||||
|
||||
func (d *Emby) login(ctx context.Context) error {
|
||||
payload, err := json.Marshal(authReq{
|
||||
Username: d.Username,
|
||||
Pw: d.Password,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
endpoint := d.URL + "/Users/AuthenticateByName"
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("X-Emby-Authorization", `MediaBrowser Client="OpenList", Device="OpenList", DeviceId="openlist-emby", Version="1.0.0"`)
|
||||
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return fmt.Errorf("emby auth failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(body)))
|
||||
}
|
||||
|
||||
var data authResp
|
||||
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(data.AccessToken) == "" || strings.TrimSpace(data.User.ID) == "" {
|
||||
return fmt.Errorf("emby auth response missing access token or user id")
|
||||
}
|
||||
|
||||
d.token = data.AccessToken
|
||||
d.userID = data.User.ID
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Emby) getItems(ctx context.Context, parentID string) (*listResp, error) {
|
||||
u, err := url.Parse(d.URL + "/Users/" + d.userID + "/Items")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
q := u.Query()
|
||||
q.Set("ParentId", parentID)
|
||||
q.Set("Recursive", "false")
|
||||
q.Set("Fields", "Path,Size,DateCreated,SeriesName,IndexNumber,ParentIndexNumber")
|
||||
q.Set("api_key", d.token)
|
||||
u.RawQuery = q.Encode()
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return nil, fmt.Errorf("emby list failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(body)))
|
||||
}
|
||||
|
||||
var data listResp
|
||||
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &data, nil
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
package github
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Token string `json:"token" type:"string" required:"true"`
|
||||
Owner string `json:"owner" type:"string" required:"true"`
|
||||
Repo string `json:"repo" type:"string" required:"true"`
|
||||
Ref string `json:"ref" type:"string" help:"A branch, a tag or a commit SHA, main branch by default."`
|
||||
AccurateModifiedTime bool `json:"accurate_modified_time" type:"bool" default:"false" help:"Best-effort accurate modified time for small directory listings. Default disabled. Adds one GitHub GraphQL request and falls back to legacy zero-time values on failure."`
|
||||
GitHubProxy string `json:"gh_proxy" type:"string" help:"GitHub proxy, e.g. https://ghproxy.net/raw.githubusercontent.com or https://gh-proxy.com/raw.githubusercontent.com"`
|
||||
GPGPrivateKey string `json:"gpg_private_key" type:"text"`
|
||||
GPGKeyPassphrase string `json:"gpg_key_passphrase" type:"string"`
|
||||
CommitterName string `json:"committer_name" type:"string"`
|
||||
CommitterEmail string `json:"committer_email" type:"string"`
|
||||
AuthorName string `json:"author_name" type:"string"`
|
||||
AuthorEmail string `json:"author_email" type:"string"`
|
||||
MkdirCommitMsg string `json:"mkdir_commit_message" type:"text" default:"{{.UserName}} mkdir {{.ObjPath}}"`
|
||||
DeleteCommitMsg string `json:"delete_commit_message" type:"text" default:"{{.UserName}} remove {{.ObjPath}}"`
|
||||
PutCommitMsg string `json:"put_commit_message" type:"text" default:"{{.UserName}} upload {{.ObjPath}}"`
|
||||
RenameCommitMsg string `json:"rename_commit_message" type:"text" default:"{{.UserName}} rename {{.ObjPath}} to {{.TargetName}}"`
|
||||
CopyCommitMsg string `json:"copy_commit_message" type:"text" default:"{{.UserName}} copy {{.ObjPath}} to {{.TargetPath}}"`
|
||||
MoveCommitMsg string `json:"move_commit_message" type:"text" default:"{{.UserName}} move {{.ObjPath}} to {{.TargetPath}}"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "GitHub API",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "/",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Github{}
|
||||
})
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
package github
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
mtimeMaxEntries = 200
|
||||
githubGraphQLEndpoint = "https://api.github.com/graphql"
|
||||
)
|
||||
|
||||
var githubZeroTime = time.Unix(0, 0)
|
||||
|
||||
type graphQLHistory struct {
|
||||
Nodes []struct {
|
||||
CommittedDate time.Time `json:"committedDate"`
|
||||
} `json:"nodes"`
|
||||
}
|
||||
|
||||
type graphQLMtimeResponse struct {
|
||||
Data struct {
|
||||
Repository struct {
|
||||
Commit map[string]graphQLHistory `json:"commit"`
|
||||
} `json:"repository"`
|
||||
} `json:"data"`
|
||||
Errors []struct{} `json:"errors"`
|
||||
}
|
||||
|
||||
func quoteGraphQLString(value string) string {
|
||||
quoted, _ := json.Marshal(value)
|
||||
return string(quoted)
|
||||
}
|
||||
|
||||
func buildMtimeQuery(owner, repo, ref string, objs []model.Obj) string {
|
||||
histories := make([]string, 0, len(objs))
|
||||
for i, obj := range objs {
|
||||
path := strings.TrimPrefix(obj.GetPath(), "/")
|
||||
histories = append(histories, fmt.Sprintf(`p%d: history(first: 1, path: %s) { nodes { committedDate } }`, i, quoteGraphQLString(path)))
|
||||
}
|
||||
|
||||
return fmt.Sprintf(`query {
|
||||
repository(owner: %s, name: %s) {
|
||||
commit: object(expression: %s) {
|
||||
... on Commit {
|
||||
%s
|
||||
}
|
||||
}
|
||||
}
|
||||
}`,
|
||||
quoteGraphQLString(owner),
|
||||
quoteGraphQLString(repo),
|
||||
quoteGraphQLString(ref+"^{commit}"),
|
||||
strings.Join(histories, "\n\t\t\t\t"),
|
||||
)
|
||||
}
|
||||
|
||||
func (d *Github) fetchAccurateModifiedTimes(ctx context.Context, dirPath string, objs []model.Obj) {
|
||||
token := strings.TrimSpace(d.Token)
|
||||
if !d.AccurateModifiedTime || token == "" || len(objs) == 0 || len(objs) > mtimeMaxEntries {
|
||||
return
|
||||
}
|
||||
|
||||
res, err := d.client.R().
|
||||
SetContext(ctx).
|
||||
SetHeader("Accept", "application/vnd.github+json").
|
||||
SetHeader("Authorization", "Bearer "+token).
|
||||
SetBody(map[string]string{"query": buildMtimeQuery(d.Owner, d.Repo, d.Ref, objs)}).
|
||||
Post(githubGraphQLEndpoint)
|
||||
if err != nil {
|
||||
log.WithError(err).Warnf("github accurate mtime failed for %s: transport", dirPath)
|
||||
return
|
||||
}
|
||||
if res.StatusCode() != http.StatusOK {
|
||||
log.Warnf("github accurate mtime failed for %s: http_%d", dirPath, res.StatusCode())
|
||||
return
|
||||
}
|
||||
|
||||
var response graphQLMtimeResponse
|
||||
if err := utils.Json.Unmarshal(res.Body(), &response); err != nil {
|
||||
log.WithError(err).Warnf("github accurate mtime failed for %s: graphql", dirPath)
|
||||
return
|
||||
}
|
||||
if len(response.Errors) > 0 || response.Data.Repository.Commit == nil {
|
||||
log.Warnf("github accurate mtime failed for %s: graphql", dirPath)
|
||||
return
|
||||
}
|
||||
|
||||
for i, obj := range objs {
|
||||
history := response.Data.Repository.Commit[fmt.Sprintf("p%d", i)]
|
||||
if len(history.Nodes) == 0 {
|
||||
continue
|
||||
}
|
||||
if raw, ok := obj.(*model.Object); ok {
|
||||
raw.Modified = history.Nodes[0].CommittedDate
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,378 +0,0 @@
|
||||
package github
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
odriver "github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
func TestDriverInfoIncludesAccurateModifiedTimeDefault(t *testing.T) {
|
||||
info := op.GetDriverInfoMap()["GitHub API"]
|
||||
for _, item := range info.Additional {
|
||||
if item.Name != "accurate_modified_time" {
|
||||
continue
|
||||
}
|
||||
if item.Default != "false" {
|
||||
t.Fatalf("unexpected default: %q", item.Default)
|
||||
}
|
||||
return
|
||||
}
|
||||
t.Fatal("accurate_modified_time item not registered")
|
||||
}
|
||||
|
||||
type roundTripFunc func(*http.Request) (*http.Response, error)
|
||||
|
||||
func (f roundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) {
|
||||
return f(r)
|
||||
}
|
||||
|
||||
func newGithubTestDriver(rt roundTripFunc, token string, enabled bool) *Github {
|
||||
return &Github{
|
||||
Storage: model.Storage{MountPath: "/github-test", CacheExpiration: 10},
|
||||
Addition: Addition{
|
||||
RootPath: odriver.RootPath{RootFolderPath: "/"},
|
||||
Token: token,
|
||||
Owner: "owner",
|
||||
Repo: "repo",
|
||||
Ref: "main",
|
||||
AccurateModifiedTime: enabled,
|
||||
},
|
||||
client: resty.New().SetTransport(rt),
|
||||
}
|
||||
}
|
||||
|
||||
func newJSONResponse(status int, body string) *http.Response {
|
||||
return &http.Response{
|
||||
StatusCode: status,
|
||||
Header: make(http.Header),
|
||||
Body: io.NopCloser(strings.NewReader(body)),
|
||||
}
|
||||
}
|
||||
|
||||
func mustJSON(t *testing.T, value any) string {
|
||||
t.Helper()
|
||||
data, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal json: %v", err)
|
||||
}
|
||||
return string(data)
|
||||
}
|
||||
|
||||
func graphQLQueryFromRequest(t *testing.T, r *http.Request) string {
|
||||
t.Helper()
|
||||
body, err := io.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
t.Fatalf("read graphql request body: %v", err)
|
||||
}
|
||||
var payload map[string]string
|
||||
if err := json.Unmarshal(body, &payload); err != nil {
|
||||
t.Fatalf("decode graphql request body: %v", err)
|
||||
}
|
||||
query := payload["query"]
|
||||
if query == "" {
|
||||
t.Fatalf("graphql request missing query: %s", string(body))
|
||||
}
|
||||
return query
|
||||
}
|
||||
|
||||
func newContentsPayload(t *testing.T, entries []Object) string {
|
||||
t.Helper()
|
||||
return mustJSON(t, map[string]any{
|
||||
"type": "dir",
|
||||
"sha": "tree-sha",
|
||||
"entries": entries,
|
||||
})
|
||||
}
|
||||
|
||||
func newTreePayload(t *testing.T, sha string, trees []TreeObjResp) string {
|
||||
t.Helper()
|
||||
return mustJSON(t, map[string]any{
|
||||
"sha": sha,
|
||||
"truncated": false,
|
||||
"tree": trees,
|
||||
})
|
||||
}
|
||||
|
||||
func newCommitGraphQLPayload(t *testing.T, histories map[string][]string) string {
|
||||
t.Helper()
|
||||
commit := make(map[string]any, len(histories))
|
||||
for alias, dates := range histories {
|
||||
nodes := make([]map[string]string, 0, len(dates))
|
||||
for _, date := range dates {
|
||||
nodes = append(nodes, map[string]string{"committedDate": date})
|
||||
}
|
||||
commit[alias] = map[string]any{"nodes": nodes}
|
||||
}
|
||||
return mustJSON(t, map[string]any{
|
||||
"data": map[string]any{
|
||||
"repository": map[string]any{"commit": commit},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func newSequentialEntries(count int) []Object {
|
||||
entries := make([]Object, 0, count)
|
||||
for i := range count {
|
||||
name := fmt.Sprintf("%03d.md", i)
|
||||
entries = append(entries, Object{Name: name, Path: "docs/" + name, Type: "file", Size: 1})
|
||||
}
|
||||
return entries
|
||||
}
|
||||
|
||||
func mustObject(t *testing.T, obj model.Obj) *model.Object {
|
||||
t.Helper()
|
||||
raw, ok := model.UnwrapObjName(obj).(*model.Object)
|
||||
if !ok {
|
||||
t.Fatalf("unexpected obj type %T", obj)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
func TestListAppliesAccurateModifiedTimeInOneRequest(t *testing.T) {
|
||||
stamp := time.Date(2025, 12, 22, 4, 52, 41, 0, time.UTC)
|
||||
entries := []Object{
|
||||
{Name: "a.md", Path: "docs/a.md", Type: "file", Size: 1},
|
||||
{Name: ".gitkeep", Path: "docs/.gitkeep", Type: "file"},
|
||||
{Name: `quote " 文.md`, Path: `docs/quote " 文.md`, Type: "file", Size: 1},
|
||||
{Name: "control.md", Path: "docs/control\x01.md", Type: "file", Size: 1},
|
||||
}
|
||||
graphqlCalls := 0
|
||||
var query string
|
||||
drv := newGithubTestDriver(roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/contents/"):
|
||||
return newJSONResponse(http.StatusOK, newContentsPayload(t, entries)), nil
|
||||
case r.Method == http.MethodPost && r.URL.String() == githubGraphQLEndpoint:
|
||||
graphqlCalls++
|
||||
query = graphQLQueryFromRequest(t, r)
|
||||
if got := r.Header.Get("Authorization"); got != "Bearer token" {
|
||||
t.Fatalf("unexpected authorization header: %q", got)
|
||||
}
|
||||
return newJSONResponse(http.StatusOK, newCommitGraphQLPayload(t, map[string][]string{
|
||||
"p0": {stamp.Format(time.RFC3339)},
|
||||
"p1": {},
|
||||
"p2": {},
|
||||
})), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected request %s %s", r.Method, r.URL.String())
|
||||
}
|
||||
}), "token", true)
|
||||
|
||||
objs, err := drv.List(context.Background(), &model.Object{Path: "/docs", Name: "docs", IsFolder: true}, model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if graphqlCalls != 1 {
|
||||
t.Fatalf("expected one GraphQL request, got %d", graphqlCalls)
|
||||
}
|
||||
if strings.Count(query, "history(first: 1") != 3 ||
|
||||
!strings.Contains(query, `object(expression: "main^{commit}")`) ||
|
||||
!strings.Contains(query, `p0: history(first: 1, path: "docs/a.md")`) ||
|
||||
!strings.Contains(query, `p1: history(first: 1, path: "docs/quote \" 文.md")`) ||
|
||||
!strings.Contains(query, `p2: history(first: 1, path: "docs/control\u0001.md")`) {
|
||||
t.Fatalf("query should peel the ref and contain all listed paths once:\n%s", query)
|
||||
}
|
||||
if len(objs) != 3 {
|
||||
t.Fatalf("expected three objects after .gitkeep filtering, got %d", len(objs))
|
||||
}
|
||||
if first := mustObject(t, objs[0]); !first.ModTime().Equal(stamp) || !first.CreateTime().Equal(githubZeroTime) {
|
||||
t.Fatalf("unexpected first timestamps: mod=%v create=%v", first.ModTime(), first.CreateTime())
|
||||
}
|
||||
if second := mustObject(t, objs[1]); !second.ModTime().Equal(githubZeroTime) || !second.CreateTime().Equal(githubZeroTime) {
|
||||
t.Fatalf("unmatched entry should retain legacy timestamps: mod=%v create=%v", second.ModTime(), second.CreateTime())
|
||||
}
|
||||
}
|
||||
|
||||
func TestListSkipsAccurateModifiedTime(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
enabled bool
|
||||
token string
|
||||
entries int
|
||||
}{
|
||||
{name: "disabled", enabled: false, token: "token", entries: 1},
|
||||
{name: "missing token", enabled: true, token: "", entries: 1},
|
||||
{name: "over entry limit", enabled: true, token: "token", entries: 201},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
graphqlCalls := 0
|
||||
payload := newContentsPayload(t, newSequentialEntries(tc.entries))
|
||||
drv := newGithubTestDriver(roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/contents/"):
|
||||
return newJSONResponse(http.StatusOK, payload), nil
|
||||
case r.Method == http.MethodPost && r.URL.String() == githubGraphQLEndpoint:
|
||||
graphqlCalls++
|
||||
return newJSONResponse(http.StatusOK, newCommitGraphQLPayload(t, nil)), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected request %s %s", r.Method, r.URL.String())
|
||||
}
|
||||
}), tc.token, tc.enabled)
|
||||
|
||||
objs, err := drv.List(context.Background(), &model.Object{Path: "/docs", Name: "docs", IsFolder: true}, model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if graphqlCalls != 0 {
|
||||
t.Fatalf("expected zero GraphQL requests, got %d", graphqlCalls)
|
||||
}
|
||||
for _, obj := range objs {
|
||||
raw := mustObject(t, obj)
|
||||
if !raw.ModTime().Equal(githubZeroTime) || !raw.CreateTime().Equal(githubZeroTime) {
|
||||
t.Fatalf("legacy timestamps should be preserved: mod=%v create=%v", raw.ModTime(), raw.CreateTime())
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestListFallsBackWhenGraphQLFails(t *testing.T) {
|
||||
graphqlCalls := 0
|
||||
drv := newGithubTestDriver(roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/contents/"):
|
||||
return newJSONResponse(http.StatusOK, newContentsPayload(t, newSequentialEntries(1))), nil
|
||||
case r.Method == http.MethodPost && r.URL.String() == githubGraphQLEndpoint:
|
||||
graphqlCalls++
|
||||
return newJSONResponse(http.StatusOK, `{"errors":[{"message":"rate limited"}]}`), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected request %s %s", r.Method, r.URL.String())
|
||||
}
|
||||
}), "token", true)
|
||||
|
||||
objs, err := drv.List(context.Background(), &model.Object{Path: "/docs", Name: "docs", IsFolder: true}, model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("GraphQL failure should be best-effort: %v", err)
|
||||
}
|
||||
if graphqlCalls != 1 {
|
||||
t.Fatalf("expected one GraphQL request, got %d", graphqlCalls)
|
||||
}
|
||||
obj := mustObject(t, objs[0])
|
||||
if !obj.ModTime().Equal(githubZeroTime) || !obj.CreateTime().Equal(githubZeroTime) {
|
||||
t.Fatalf("failed GraphQL request should retain legacy timestamps: mod=%v create=%v", obj.ModTime(), obj.CreateTime())
|
||||
}
|
||||
}
|
||||
|
||||
func TestListUsesOneGraphQLRequestAtEntryLimit(t *testing.T) {
|
||||
stamp := time.Date(2025, 12, 22, 4, 52, 41, 0, time.UTC)
|
||||
graphqlCalls := 0
|
||||
var query string
|
||||
drv := newGithubTestDriver(roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/contents/"):
|
||||
return newJSONResponse(http.StatusOK, newContentsPayload(t, newSequentialEntries(200))), nil
|
||||
case r.Method == http.MethodPost && r.URL.String() == githubGraphQLEndpoint:
|
||||
graphqlCalls++
|
||||
query = graphQLQueryFromRequest(t, r)
|
||||
return newJSONResponse(http.StatusOK, newCommitGraphQLPayload(t, map[string][]string{
|
||||
"p199": {stamp.Format(time.RFC3339)},
|
||||
})), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected request %s %s", r.Method, r.URL.String())
|
||||
}
|
||||
}), "token", true)
|
||||
|
||||
objs, err := drv.List(context.Background(), &model.Object{Path: "/docs", Name: "docs", IsFolder: true}, model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if graphqlCalls != 1 || strings.Count(query, "history(first: 1") != 200 {
|
||||
t.Fatalf("200 entries should share one request: calls=%d histories=%d", graphqlCalls, strings.Count(query, "history(first: 1"))
|
||||
}
|
||||
if !strings.Contains(query, `p199: history(first: 1, path: "docs/199.md")`) {
|
||||
t.Fatalf("query missing final entry:\n%s", query)
|
||||
}
|
||||
if len(objs) != 200 || !mustObject(t, objs[199]).ModTime().Equal(stamp) {
|
||||
t.Fatalf("unexpected final object timestamp")
|
||||
}
|
||||
}
|
||||
|
||||
func TestListKeepsTreeFallbackOnLegacyPath(t *testing.T) {
|
||||
entries := make([]Object, 0, 1000)
|
||||
for i := range 1000 {
|
||||
name := fmt.Sprintf("dir-%d", i)
|
||||
entries = append(entries, Object{Name: name, Path: "docs/" + name, Type: "dir"})
|
||||
}
|
||||
graphqlCalls := 0
|
||||
drv := newGithubTestDriver(roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/contents/"):
|
||||
return newJSONResponse(http.StatusOK, newContentsPayload(t, entries)), nil
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/git/trees/"):
|
||||
return newJSONResponse(http.StatusOK, newTreePayload(t, "tree-sha", []TreeObjResp{{TreeObjReq: TreeObjReq{Path: "child.md", Mode: "100644", Type: "blob", Sha: "blob-sha"}, Size: 1, URL: "https://example.invalid/blob"}})), nil
|
||||
case r.Method == http.MethodPost && r.URL.String() == githubGraphQLEndpoint:
|
||||
graphqlCalls++
|
||||
return newJSONResponse(http.StatusOK, newCommitGraphQLPayload(t, nil)), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected request %s %s", r.Method, r.URL.String())
|
||||
}
|
||||
}), "token", true)
|
||||
|
||||
objs, err := drv.List(context.Background(), &model.Object{Path: "/docs", Name: "docs", IsFolder: true}, model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(objs) != 1 {
|
||||
t.Fatalf("unexpected tree fallback result length: %d", len(objs))
|
||||
}
|
||||
first := mustObject(t, objs[0])
|
||||
if first.GetPath() != "/child.md" {
|
||||
t.Fatalf("unexpected tree fallback path: %s", first.GetPath())
|
||||
}
|
||||
if !first.ModTime().Equal(githubZeroTime) || !first.CreateTime().Equal(githubZeroTime) {
|
||||
t.Fatalf("tree fallback should preserve legacy timestamps: mod=%v create=%v", first.ModTime(), first.CreateTime())
|
||||
}
|
||||
if graphqlCalls != 0 {
|
||||
t.Fatalf("tree fallback should skip GraphQL, got %d calls", graphqlCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpListCacheHitDoesNotRepeatGraphQL(t *testing.T) {
|
||||
op.Cache.ClearAll()
|
||||
defer op.Cache.ClearAll()
|
||||
stamp := time.Date(2025, 12, 22, 4, 52, 41, 0, time.UTC)
|
||||
graphqlCalls := 0
|
||||
drv := newGithubTestDriver(roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/contents/"):
|
||||
return newJSONResponse(http.StatusOK, newContentsPayload(t, []Object{{Name: "a.md", Path: "a.md", Type: "file", Size: 1}})), nil
|
||||
case r.Method == http.MethodPost && r.URL.String() == githubGraphQLEndpoint:
|
||||
graphqlCalls++
|
||||
return newJSONResponse(http.StatusOK, newCommitGraphQLPayload(t, map[string][]string{"p0": {stamp.Format(time.RFC3339)}})), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unexpected request %s %s", r.Method, r.URL.String())
|
||||
}
|
||||
}), "token", true)
|
||||
|
||||
first, err := op.List(context.Background(), drv, "/", model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected first list error: %v", err)
|
||||
}
|
||||
second, err := op.List(context.Background(), drv, "/", model.ListArgs{})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected second list error: %v", err)
|
||||
}
|
||||
if len(first) != 1 || len(second) != 1 {
|
||||
t.Fatalf("unexpected cached results: first=%d second=%d", len(first), len(second))
|
||||
}
|
||||
if graphqlCalls != 1 {
|
||||
t.Fatalf("expected one GraphQL call across cached lists, got %d", graphqlCalls)
|
||||
}
|
||||
if !mustObject(t, first[0]).ModTime().Equal(stamp) {
|
||||
t.Fatalf("expected first list to include backfilled modified time, got %v", mustObject(t, first[0]).ModTime())
|
||||
}
|
||||
if !mustObject(t, second[0]).ModTime().Equal(stamp) {
|
||||
t.Fatalf("expected cached list to retain modified time, got %v", mustObject(t, second[0]).ModTime())
|
||||
}
|
||||
}
|
||||
@@ -1,210 +0,0 @@
|
||||
package github_releases
|
||||
|
||||
import (
|
||||
"path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
// MountPoint 表示一个仓库挂载点
|
||||
type MountPoint struct {
|
||||
Point string // 挂载点路径
|
||||
Repo string // 仓库名 owner/repo
|
||||
}
|
||||
|
||||
// Release 转为 File 列表
|
||||
func releaseToFiles(point string, release *Release) []File {
|
||||
if release == nil {
|
||||
return nil
|
||||
}
|
||||
files := make([]File, 0, len(release.Assets))
|
||||
for _, asset := range release.Assets {
|
||||
files = append(files, File{
|
||||
Path: path.Join(point, asset.Name),
|
||||
FileName: asset.Name,
|
||||
Size: asset.Size,
|
||||
Type: "file",
|
||||
UpdateAt: asset.UpdatedAt,
|
||||
CreateAt: asset.CreatedAt,
|
||||
Url: asset.BrowserDownloadUrl,
|
||||
})
|
||||
}
|
||||
return files
|
||||
}
|
||||
|
||||
// 计算 release 的 asset 总大小
|
||||
func releaseSize(release *Release) int64 {
|
||||
if release == nil {
|
||||
return 0
|
||||
}
|
||||
size := int64(0)
|
||||
for _, asset := range release.Assets {
|
||||
size += asset.Size
|
||||
}
|
||||
return size
|
||||
}
|
||||
|
||||
// Releases 列表转为版本目录 File 列表
|
||||
func releasesToVersionDirs(point string, releases []Release) []File {
|
||||
files := make([]File, 0, len(releases))
|
||||
for _, release := range releases {
|
||||
files = append(files, File{
|
||||
Path: path.Join(point, release.TagName),
|
||||
FileName: release.TagName,
|
||||
Size: releaseSize(&release),
|
||||
Type: "dir",
|
||||
UpdateAt: release.PublishedAt,
|
||||
CreateAt: release.CreatedAt,
|
||||
Url: release.HtmlUrl,
|
||||
})
|
||||
}
|
||||
return files
|
||||
}
|
||||
|
||||
// 根据 tagName 查找 release 的 asset 文件列表
|
||||
func releaseAssetsByTag(point, tagName string, releases []Release) []File {
|
||||
for _, item := range releases {
|
||||
if item.TagName == tagName {
|
||||
files := make([]File, 0, len(item.Assets))
|
||||
for _, asset := range item.Assets {
|
||||
files = append(files, File{
|
||||
Path: path.Join(point, tagName, asset.Name),
|
||||
FileName: asset.Name,
|
||||
Size: asset.Size,
|
||||
Type: "file",
|
||||
UpdateAt: asset.UpdatedAt,
|
||||
CreateAt: asset.CreatedAt,
|
||||
Url: asset.BrowserDownloadUrl,
|
||||
})
|
||||
}
|
||||
return files
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// 根据 tagName 计算 asset 总大小
|
||||
func releasesSizeByTag(releases []Release, tagName string) int64 {
|
||||
for _, item := range releases {
|
||||
if item.TagName == tagName {
|
||||
return releaseSize(&item)
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// 计算所有 releases 的 asset 总大小
|
||||
func releasesTotalSize(releases []Release) int64 {
|
||||
size := int64(0)
|
||||
for _, release := range releases {
|
||||
size += releaseSize(&release)
|
||||
}
|
||||
return size
|
||||
}
|
||||
|
||||
// Source code 文件
|
||||
func sourceCodeFiles(point string, release *Release) []File {
|
||||
if release == nil {
|
||||
return nil
|
||||
}
|
||||
return []File{
|
||||
{
|
||||
Path: path.Join(point, "Source code (zip)"),
|
||||
FileName: "Source code (zip)",
|
||||
Size: 1,
|
||||
Type: "file",
|
||||
UpdateAt: release.CreatedAt,
|
||||
CreateAt: release.CreatedAt,
|
||||
Url: release.ZipballUrl,
|
||||
},
|
||||
{
|
||||
Path: path.Join(point, "Source code (tar.gz)"),
|
||||
FileName: "Source code (tar.gz)",
|
||||
Size: 1,
|
||||
Type: "file",
|
||||
UpdateAt: release.CreatedAt,
|
||||
CreateAt: release.CreatedAt,
|
||||
Url: release.TarballUrl,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// 根据 tagName 获取 Source Code 文件
|
||||
func sourceCodeFilesByTag(point string, releases []Release, tagName string) []File {
|
||||
for _, item := range releases {
|
||||
if item.TagName == tagName {
|
||||
return sourceCodeFiles(point, &item)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// 仓库根目录下的 README/LICENSE 文件
|
||||
func otherFiles(point string, fileInfos []FileInfo) []File {
|
||||
files := make([]File, 0)
|
||||
defaultTime := "1970-01-01T00:00:00Z"
|
||||
for _, file := range fileInfos {
|
||||
if file.Type == "dir" {
|
||||
continue
|
||||
}
|
||||
name := file.Name
|
||||
if strings.EqualFold(name, "README.md") || strings.HasPrefix(name, "LICENSE") {
|
||||
files = append(files, File{
|
||||
Path: path.Join(point, file.Name),
|
||||
FileName: file.Name,
|
||||
Size: file.Size,
|
||||
Type: "file",
|
||||
UpdateAt: defaultTime,
|
||||
CreateAt: defaultTime,
|
||||
Url: file.DownloadUrl,
|
||||
})
|
||||
}
|
||||
}
|
||||
return files
|
||||
}
|
||||
|
||||
type File struct {
|
||||
Path string // 文件路径
|
||||
FileName string // 文件名
|
||||
Size int64 // 文件大小
|
||||
Type string // 文件类型
|
||||
UpdateAt string // 更新时间 eg:"2025-01-27T16:10:16Z"
|
||||
CreateAt string // 创建时间
|
||||
Url string // 下载链接
|
||||
}
|
||||
|
||||
func (f File) GetHash() utils.HashInfo {
|
||||
return utils.HashInfo{}
|
||||
}
|
||||
|
||||
func (f File) GetPath() string {
|
||||
return f.Path
|
||||
}
|
||||
|
||||
func (f File) GetSize() int64 {
|
||||
return f.Size
|
||||
}
|
||||
|
||||
func (f File) GetName() string {
|
||||
return f.FileName
|
||||
}
|
||||
|
||||
func (f File) ModTime() time.Time {
|
||||
t, _ := time.Parse(time.RFC3339, f.CreateAt)
|
||||
return t
|
||||
}
|
||||
|
||||
func (f File) CreateTime() time.Time {
|
||||
t, _ := time.Parse(time.RFC3339, f.CreateAt)
|
||||
return t
|
||||
}
|
||||
|
||||
func (f File) IsDir() bool {
|
||||
return f.Type == "dir"
|
||||
}
|
||||
|
||||
func (f File) GetID() string {
|
||||
return f.Url
|
||||
}
|
||||
@@ -1,152 +0,0 @@
|
||||
package github_releases
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
// 发送 GET 请求
|
||||
func (d *GithubReleases) GetRequest(url string) (*resty.Response, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeader("Accept", "application/vnd.github+json")
|
||||
req.SetHeader("X-GitHub-Api-Version", "2022-11-28")
|
||||
if d.Addition.Token != "" {
|
||||
req.SetHeader("Authorization", fmt.Sprintf("Bearer %s", d.Addition.Token))
|
||||
}
|
||||
res, err := req.Get(url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if res.StatusCode() != 200 {
|
||||
return nil, fmt.Errorf("github api error: status %d", res.StatusCode())
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// 解析挂载结构
|
||||
func (d *GithubReleases) ParseRepos(text string) ([]MountPoint, error) {
|
||||
lines := strings.Split(text, "\n")
|
||||
points := make([]MountPoint, 0)
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
parts := strings.Split(line, ":")
|
||||
path, repo := "", ""
|
||||
if len(parts) == 1 {
|
||||
path = "/"
|
||||
repo = parts[0]
|
||||
} else if len(parts) == 2 {
|
||||
path = fmt.Sprintf("/%s", strings.Trim(parts[0], "/"))
|
||||
repo = parts[1]
|
||||
} else {
|
||||
return nil, fmt.Errorf("invalid format: %s", line)
|
||||
}
|
||||
|
||||
points = append(points, MountPoint{
|
||||
Point: path,
|
||||
Repo: repo,
|
||||
})
|
||||
}
|
||||
d.points = points
|
||||
return points, nil
|
||||
}
|
||||
|
||||
// 获取下一级目录
|
||||
func GetNextDir(wholePath string, basePath string) string {
|
||||
basePath = fmt.Sprintf("%s/", strings.TrimRight(basePath, "/"))
|
||||
if !strings.HasPrefix(wholePath, basePath) {
|
||||
return ""
|
||||
}
|
||||
remainingPath := strings.TrimLeft(strings.TrimPrefix(wholePath, basePath), "/")
|
||||
if remainingPath != "" {
|
||||
parts := strings.Split(remainingPath, "/")
|
||||
nextDir := parts[0]
|
||||
if strings.HasPrefix(wholePath, strings.TrimRight(basePath, "/")+"/"+nextDir) {
|
||||
return nextDir
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// getLatestRelease 获取最新 release
|
||||
func (d *GithubReleases) getLatestRelease(repo string) (*Release, error) {
|
||||
resp, err := d.GetRequest("https://api.github.com/repos/" + repo + "/releases/latest")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
release := new(Release)
|
||||
if err := json.Unmarshal(resp.Body(), release); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return release, nil
|
||||
}
|
||||
|
||||
// getAllReleases 获取所有 releases（支持自动翻页）
|
||||
func (d *GithubReleases) getAllReleases(repo string) ([]Release, error) {
|
||||
perPage := d.Addition.PerPage
|
||||
if perPage < 1 {
|
||||
perPage = 30
|
||||
} else if perPage > 100 {
|
||||
perPage = 100
|
||||
}
|
||||
|
||||
maxPage := d.Addition.MaxPage
|
||||
if maxPage < 0 {
|
||||
maxPage = 0
|
||||
}
|
||||
|
||||
allReleases := make([]Release, 0)
|
||||
page := 1
|
||||
|
||||
for {
|
||||
url := fmt.Sprintf("https://api.github.com/repos/%s/releases?per_page=%d&page=%d", repo, perPage, page)
|
||||
resp, err := d.GetRequest(url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
releases := make([]Release, 0)
|
||||
if err := json.Unmarshal(resp.Body(), &releases); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(releases) == 0 {
|
||||
break
|
||||
}
|
||||
|
||||
allReleases = append(allReleases, releases...)
|
||||
|
||||
// 达到最大页数限制
|
||||
if maxPage > 0 && page >= maxPage {
|
||||
break
|
||||
}
|
||||
|
||||
// 如果返回数量小于 perPage，说明是最后一页
|
||||
if len(releases) < perPage {
|
||||
break
|
||||
}
|
||||
|
||||
page++
|
||||
}
|
||||
|
||||
return allReleases, nil
|
||||
}
|
||||
|
||||
// fetchRepoFiles 获取仓库根目录文件列表
|
||||
func (d *GithubReleases) fetchRepoFiles(repo string) ([]FileInfo, error) {
|
||||
resp, err := d.GetRequest("https://api.github.com/repos/" + repo + "/contents")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
files := make([]FileInfo, 0)
|
||||
if err := json.Unmarshal(resp.Body(), &files); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package google_drive
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
RefreshToken string `json:"refresh_token" required:"true"`
|
||||
OrderBy string `json:"order_by" type:"string" help:"such as: folder,name,modifiedTime"`
|
||||
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc"`
|
||||
UseOnlineAPI bool `json:"use_online_api" default:"true"`
|
||||
APIAddress string `json:"api_url_address" default:"https://api.oplist.org/googleui/renewapi"`
|
||||
ClientID string `json:"client_id"`
|
||||
ClientSecret string `json:"client_secret"`
|
||||
ChunkSize int64 `json:"chunk_size" type:"number" default:"5" help:"chunk size while uploading (unit: MB)"`
|
||||
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "GoogleDrive",
|
||||
OnlyProxy: true,
|
||||
DefaultRoot: "root",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &GoogleDrive{}
|
||||
})
|
||||
}
|
||||
@@ -1,792 +0,0 @@
|
||||
package guangyapan
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/aliyun/aliyun-oss-go-sdk/oss"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
accountBaseURL = "https://account.guangyapan.com"
|
||||
apiBaseURL = "https://api.guangyapan.com"
|
||||
)
|
||||
|
||||
type GuangYaPan struct {
|
||||
model.Storage
|
||||
Addition
|
||||
|
||||
accountClient *resty.Client
|
||||
apiClient *resty.Client
|
||||
|
||||
resolvedRootFolderID string
|
||||
rootFolderResolved bool
|
||||
|
||||
// refreshMu protects concurrent access to AccessToken/RefreshToken during refresh.
|
||||
refreshMu sync.Mutex
|
||||
// statusTimer tracks delayed status updates for cancellation on Drop.
|
||||
statusTimer *time.Timer
|
||||
|
||||
// apiRateLimit throttles requests per API endpoint so that batch operations
|
||||
// (e.g. copying many files cross-storage) don't flood the upstream API.
|
||||
apiRateLimit sync.Map
|
||||
}
|
||||
|
||||
// apiRateInterval is the minimum gap between two requests to the same endpoint.
|
||||
const apiRateInterval = 500 * time.Millisecond
|
||||
|
||||
func (d *GuangYaPan) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Init(ctx context.Context) error {
|
||||
d.ClientID = strings.TrimSpace(d.ClientID)
|
||||
if d.ClientID == "" {
|
||||
return errors.New("client_id is required, please provide a valid client_id")
|
||||
}
|
||||
d.DeviceID = normalizeDeviceID(d.DeviceID)
|
||||
if d.DeviceID == "" {
|
||||
d.DeviceID = randomDeviceID()
|
||||
}
|
||||
deviceSign := strings.TrimSpace(d.DeviceSign)
|
||||
if deviceSign == "" {
|
||||
deviceSign = "wdi10." + d.DeviceID
|
||||
}
|
||||
if d.PageSize <= 0 {
|
||||
d.PageSize = 100
|
||||
}
|
||||
if d.OrderBy < 0 {
|
||||
d.OrderBy = 3
|
||||
}
|
||||
if d.SortType != 0 && d.SortType != 1 {
|
||||
d.SortType = 1
|
||||
}
|
||||
|
||||
d.RootPath = strings.TrimSpace(d.RootPath)
|
||||
d.AccessToken = strings.TrimSpace(d.AccessToken)
|
||||
d.RefreshToken = strings.TrimSpace(d.RefreshToken)
|
||||
d.PhoneNumber = strings.TrimSpace(d.PhoneNumber)
|
||||
d.VerifyCode = strings.TrimSpace(d.VerifyCode)
|
||||
d.CaptchaToken = strings.TrimSpace(d.CaptchaToken)
|
||||
d.VerificationID = strings.TrimSpace(d.VerificationID)
|
||||
d.resolvedRootFolderID = ""
|
||||
d.rootFolderResolved = false
|
||||
|
||||
d.accountClient = base.NewRestyClient().
|
||||
SetBaseURL(accountBaseURL).
|
||||
SetHeader("Accept", "application/json, text/plain, */*").
|
||||
SetHeader("Content-Type", "application/json").
|
||||
SetHeader("X-Device-Model", "chrome%2F147.0.0.0").
|
||||
SetHeader("X-Device-Name", "PC-Chrome").
|
||||
SetHeader("X-Device-Sign", deviceSign).
|
||||
SetHeader("X-Net-Work-Type", "NONE").
|
||||
SetHeader("X-OS-Version", "MacIntel").
|
||||
SetHeader("X-Platform-Version", "1").
|
||||
SetHeader("X-Protocol-Version", "301").
|
||||
SetHeader("X-Provider-Name", "NONE").
|
||||
SetHeader("X-SDK-Version", "9.0.2").
|
||||
SetHeader("X-Client-Id", d.ClientID).
|
||||
SetHeader("X-Client-Version", "0.0.1").
|
||||
SetHeader("X-Device-Id", d.DeviceID)
|
||||
|
||||
d.apiClient = base.NewRestyClient().
|
||||
SetBaseURL(apiBaseURL).
|
||||
SetHeader("Accept", "application/json, text/plain, */*").
|
||||
SetHeader("Content-Type", "application/json").
|
||||
SetHeader("Did", d.DeviceID).
|
||||
SetHeader("Dt", "4")
|
||||
|
||||
// Priority: access_token -> refresh_token -> sms login.
|
||||
if d.AccessToken != "" {
|
||||
if err := d.validateToken(ctx); err == nil {
|
||||
return d.prepareRootFolder(ctx)
|
||||
}
|
||||
d.AccessToken = ""
|
||||
}
|
||||
if d.RefreshToken != "" {
|
||||
if err := d.refreshToken(ctx); err == nil {
|
||||
if err2 := d.validateToken(ctx); err2 == nil {
|
||||
return d.prepareRootFolder(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Two-stage SMS flow:
|
||||
// 1) phone only + send_code=true: send code and cache verification_id (do not fail init).
|
||||
// 2) phone + verify_code: complete login and save tokens.
|
||||
if d.PhoneNumber != "" {
|
||||
if d.canSMSLogin() {
|
||||
if err := d.loginBySMSCode(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := d.validateToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
return d.prepareRootFolder(ctx)
|
||||
}
|
||||
if d.SendCode {
|
||||
d.setTempStatus("SMS sending in progress...")
|
||||
if err := d.prepareSMSCode(ctx); err != nil {
|
||||
d.setTempStatus(fmt.Sprintf("SMS send failed: %v. Please check captcha/meta and set send_code=true to retry.", err))
|
||||
log.Warnf("guangyapan: prepare sms code failed: %v", err)
|
||||
} else {
|
||||
d.setTempStatus("SMS sent successfully. Please fill verify_code and save to complete login.")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return errors.New("login failed: provide a valid access_token, or refresh_token, or phone_number + verify_code + captcha_token")
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Drop(ctx context.Context) error {
|
||||
if d.statusTimer != nil {
|
||||
d.statusTimer.Stop()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) GetRoot(ctx context.Context) (model.Obj, error) {
|
||||
rootID, err := d.getRootFolderID(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Object{
|
||||
ID: rootID,
|
||||
Path: "/",
|
||||
Name: "root",
|
||||
Size: 0,
|
||||
Modified: d.Modified,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
parentID := dir.GetID()
|
||||
|
||||
const maxPage = 10000
|
||||
res := make([]model.Obj, 0, d.PageSize)
|
||||
for page := 0; page < maxPage; page++ {
|
||||
var resp listResp
|
||||
body := map[string]any{
|
||||
"parentId": parentID,
|
||||
"page": page,
|
||||
"pageSize": d.PageSize,
|
||||
"orderBy": d.OrderBy,
|
||||
"sortType": d.SortType,
|
||||
}
|
||||
if err := d.postAPI(ctx, "/userres/v1/file/get_file_list", body, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, item := range resp.Data.List {
|
||||
res = append(res, &model.Object{
|
||||
ID: item.FileID,
|
||||
Path: parentID,
|
||||
Name: item.FileName,
|
||||
Size: item.FileSize,
|
||||
Modified: unixOrZero(item.UTime),
|
||||
Ctime: unixOrZero(item.CTime),
|
||||
IsFolder: item.ResType == 2,
|
||||
})
|
||||
}
|
||||
if len(resp.Data.List) < d.PageSize {
|
||||
break
|
||||
}
|
||||
if resp.Data.Total > 0 && len(res) >= resp.Data.Total {
|
||||
break
|
||||
}
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if file.IsDir() {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp downloadResp
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/get_res_download_url", map[string]any{
|
||||
"fileId": file.GetID(),
|
||||
}, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
url := strings.TrimSpace(resp.Data.SignedURL)
|
||||
if url == "" {
|
||||
url = strings.TrimSpace(resp.Data.DownloadURL)
|
||||
}
|
||||
if url == "" {
|
||||
return nil, errors.New("empty download url")
|
||||
}
|
||||
return &model.Link{URL: url}, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
name := strings.TrimSpace(dirName)
|
||||
if name == "" {
|
||||
return errors.New("dir name is empty")
|
||||
}
|
||||
|
||||
parentID := parentDir.GetID()
|
||||
|
||||
var out createDirResp
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/file/create_dir", map[string]any{
|
||||
"parentId": parentID,
|
||||
"dirName": name,
|
||||
}, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !isSuccessMsg(out.Msg) {
|
||||
return fmt.Errorf("make dir failed: %s", strings.TrimSpace(out.Msg))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fileID := strings.TrimSpace(srcObj.GetID())
|
||||
if fileID == "" {
|
||||
return errors.New("file id is empty")
|
||||
}
|
||||
name := strings.TrimSpace(newName)
|
||||
if name == "" {
|
||||
return errors.New("new name is empty")
|
||||
}
|
||||
|
||||
var out commonResp
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/file/rename", map[string]any{
|
||||
"fileId": fileID,
|
||||
"newName": name,
|
||||
}, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !isSuccessMsg(out.Msg) {
|
||||
return fmt.Errorf("rename failed: %s", strings.TrimSpace(out.Msg))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Remove(ctx context.Context, obj model.Obj) error {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fileID := strings.TrimSpace(obj.GetID())
|
||||
if fileID == "" {
|
||||
return errors.New("file id is empty")
|
||||
}
|
||||
|
||||
var del taskResp
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/file/delete_file", map[string]any{
|
||||
"fileIds": []string{fileID},
|
||||
}, &del); err != nil {
|
||||
return err
|
||||
}
|
||||
if !isSuccessMsg(del.Msg) {
|
||||
return fmt.Errorf("delete failed: %s", strings.TrimSpace(del.Msg))
|
||||
}
|
||||
|
||||
taskID := strings.TrimSpace(del.Data.TaskID)
|
||||
if taskID == "" {
|
||||
// Some backends may apply deletion synchronously.
|
||||
return nil
|
||||
}
|
||||
return d.waitTaskDone(ctx, taskID)
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fileID := strings.TrimSpace(srcObj.GetID())
|
||||
if fileID == "" {
|
||||
return errors.New("file id is empty")
|
||||
}
|
||||
parentID := dstDir.GetID()
|
||||
|
||||
var out taskResp
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/file/move_file", map[string]any{
|
||||
"fileIds": []string{fileID},
|
||||
"parentId": parentID,
|
||||
}, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !isSuccessMsg(out.Msg) {
|
||||
return fmt.Errorf("move failed: %s", strings.TrimSpace(out.Msg))
|
||||
}
|
||||
taskID := strings.TrimSpace(out.Data.TaskID)
|
||||
if taskID == "" {
|
||||
return nil
|
||||
}
|
||||
return d.waitTaskDone(ctx, taskID)
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fileID := strings.TrimSpace(srcObj.GetID())
|
||||
if fileID == "" {
|
||||
return errors.New("file id is empty")
|
||||
}
|
||||
parentID := dstDir.GetID()
|
||||
|
||||
var out taskResp
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/file/copy_file", map[string]any{
|
||||
"fileIds": []string{fileID},
|
||||
"parentId": parentID,
|
||||
}, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !isSuccessMsg(out.Msg) {
|
||||
return fmt.Errorf("copy failed: %s", strings.TrimSpace(out.Msg))
|
||||
}
|
||||
taskID := strings.TrimSpace(out.Data.TaskID)
|
||||
if taskID == "" {
|
||||
return nil
|
||||
}
|
||||
return d.waitTaskDone(ctx, taskID)
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if file == nil {
|
||||
return nil, errors.New("file is nil")
|
||||
}
|
||||
if file.GetSize() < 0 {
|
||||
return nil, errors.New("invalid file size")
|
||||
}
|
||||
name := strings.TrimSpace(file.GetName())
|
||||
if name == "" {
|
||||
return nil, errors.New("file name is empty")
|
||||
}
|
||||
|
||||
parentID := dstDir.GetID()
|
||||
|
||||
// 优先秒传：先计算文件 MD5，后端命中相同文件时直接秒传完成，无需真实上传。
|
||||
md5sum, err := d.fileMD5(file, up)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
token, code, err := d.getUploadToken(ctx, parentID, name, file.GetSize(), md5sum)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
taskID := strings.TrimSpace(token.TaskID)
|
||||
// code == 156 (instant upload) or AlreadyDone mean the backend has already
|
||||
// finished/imported the file; there is no OSS upload to perform.
|
||||
if code == 156 || token.AlreadyDone {
|
||||
if taskID == "" {
|
||||
return nil, errors.New("instant upload returns empty task id")
|
||||
}
|
||||
if err := d.waitUploadTaskInfo(ctx, taskID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
if token.ObjectPath == "" || token.BucketName == "" || token.EndPoint == "" || token.AccessKeyID == "" || token.SecretAccessKey == "" {
|
||||
return nil, errors.New("upload token is incomplete")
|
||||
}
|
||||
|
||||
ossEndpoint := normalizeOSSEndpoint(token.EndPoint, token.BucketName)
|
||||
client, err := oss.New(ossEndpoint, token.AccessKeyID, token.SecretAccessKey, oss.SecurityToken(token.SessionToken))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create oss client failed: %w", err)
|
||||
}
|
||||
bucket, err := client.Bucket(token.BucketName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create oss bucket failed: %w", err)
|
||||
}
|
||||
|
||||
if file.GetSize() == 0 {
|
||||
if err := bucket.PutObject(token.ObjectPath, strings.NewReader("")); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
if err := d.multipartUploadToOSS(ctx, bucket, token.ObjectPath, file, up); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if taskID == "" {
|
||||
return nil, nil
|
||||
}
|
||||
if err := d.waitUploadTaskInfo(ctx, taskID); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp assetsInfoResp
|
||||
if err := d.postAPI(ctx, "/nd.bizassets.s/v1/get_assets", nil, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.IsSuccess() && resp.Data.TotalSpaceSize > 0 {
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: resp.Data.TotalSpaceSize,
|
||||
UsedSpace: resp.Data.UsedSpaceSize,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
return nil, errors.New("failed to get storage details")
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) getRootFolderID(ctx context.Context) (string, error) {
|
||||
if d.rootFolderResolved {
|
||||
return d.resolvedRootFolderID, nil
|
||||
}
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := d.prepareRootFolder(ctx); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return d.resolvedRootFolderID, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) prepareRootFolder(ctx context.Context) error {
|
||||
rootID, err := d.resolveConfiguredRootFolderID(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.resolvedRootFolderID = rootID
|
||||
d.rootFolderResolved = true
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) resolveConfiguredRootFolderID(ctx context.Context) (string, error) {
|
||||
root := strings.TrimSpace(d.RootPath)
|
||||
if root == "" {
|
||||
return "", nil
|
||||
}
|
||||
return d.resolveFolderPath(ctx, root)
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) resolveFolderPath(ctx context.Context, rootPath string) (string, error) {
|
||||
cleanPath := strings.Trim(strings.ReplaceAll(strings.TrimSpace(rootPath), "\\", "/"), "/")
|
||||
if cleanPath == "" {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
parentID := ""
|
||||
for _, name := range strings.Split(cleanPath, "/") {
|
||||
if name == "" {
|
||||
continue
|
||||
}
|
||||
childID, err := d.findChildFolderID(ctx, parentID, name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
parentID = childID
|
||||
}
|
||||
return parentID, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) findChildFolderID(ctx context.Context, parentID, name string) (string, error) {
|
||||
pageSize := d.PageSize
|
||||
if pageSize <= 0 {
|
||||
pageSize = 100
|
||||
}
|
||||
|
||||
const maxPage = 10000
|
||||
seen := 0
|
||||
for page := 0; page < maxPage; page++ {
|
||||
var resp listResp
|
||||
body := map[string]any{
|
||||
"parentId": parentID,
|
||||
"page": page,
|
||||
"pageSize": pageSize,
|
||||
"orderBy": d.OrderBy,
|
||||
"sortType": d.SortType,
|
||||
}
|
||||
if err := d.postAPI(ctx, "/nd.bizuserres.s/v1/file/get_file_list", body, &resp); err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, item := range resp.Data.List {
|
||||
seen++
|
||||
if item.ResType == 2 && item.FileName == name {
|
||||
return item.FileID, nil
|
||||
}
|
||||
}
|
||||
if len(resp.Data.List) < pageSize {
|
||||
break
|
||||
}
|
||||
if resp.Data.Total > 0 && seen >= resp.Data.Total {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if parentID == "" {
|
||||
return "", fmt.Errorf("resolve root folder path failed: folder %q not found under /", name)
|
||||
}
|
||||
return "", fmt.Errorf("resolve root folder path failed: folder %q not found under parent %s", name, parentID)
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) ensureAccessToken(ctx context.Context) error {
|
||||
if strings.TrimSpace(d.AccessToken) != "" {
|
||||
return nil
|
||||
}
|
||||
if strings.TrimSpace(d.RefreshToken) == "" {
|
||||
return errors.New("not logged in, please re-init storage")
|
||||
}
|
||||
return d.refreshToken(ctx)
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) validateToken(ctx context.Context) error {
|
||||
var me userMeResp
|
||||
resp, err := d.accountClient.R().
|
||||
SetContext(ctx).
|
||||
SetHeader("Authorization", "Bearer "+d.AccessToken).
|
||||
SetResult(&me).
|
||||
Get("/v1/user/me")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.IsError() {
|
||||
return fmt.Errorf("validate token failed: status=%d body=%s", resp.StatusCode(), resp.String())
|
||||
}
|
||||
if strings.TrimSpace(me.Sub) == "" {
|
||||
return errors.New("validate token failed: empty user sub")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) refreshToken(ctx context.Context) error {
|
||||
if strings.TrimSpace(d.RefreshToken) == "" {
|
||||
return errors.New("refresh_token is empty")
|
||||
}
|
||||
|
||||
d.refreshMu.Lock()
|
||||
defer d.refreshMu.Unlock()
|
||||
|
||||
// Double-check after acquiring lock (may have been refreshed by another goroutine)
|
||||
if strings.TrimSpace(d.AccessToken) != "" {
|
||||
if err := d.validateToken(ctx); err == nil {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
var out tokenResp
|
||||
resp, err := d.accountClient.R().
|
||||
SetContext(ctx).
|
||||
SetBody(map[string]any{
|
||||
"client_id": d.ClientID,
|
||||
"grant_type": "refresh_token",
|
||||
"refresh_token": d.RefreshToken,
|
||||
}).
|
||||
SetResult(&out).
|
||||
Post("/v1/auth/token")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.IsError() || out.Error != "" || strings.TrimSpace(out.AccessToken) == "" {
|
||||
errMsg := strings.TrimSpace(out.ErrorDesc)
|
||||
if errMsg == "" {
|
||||
errMsg = strings.TrimSpace(out.Error)
|
||||
}
|
||||
if errMsg == "" {
|
||||
errMsg = strings.TrimSpace(resp.String())
|
||||
}
|
||||
if errMsg == "" {
|
||||
errMsg = fmt.Sprintf("status=%d", resp.StatusCode())
|
||||
}
|
||||
return fmt.Errorf("refresh token failed: %s", errMsg)
|
||||
}
|
||||
|
||||
d.AccessToken = strings.TrimSpace(out.AccessToken)
|
||||
if strings.TrimSpace(out.RefreshToken) != "" {
|
||||
d.RefreshToken = strings.TrimSpace(out.RefreshToken)
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) canSMSLogin() bool {
|
||||
return d.PhoneNumber != "" && d.VerifyCode != ""
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) loginBySMSCode(ctx context.Context) error {
|
||||
verificationID := strings.TrimSpace(d.VerificationID)
|
||||
if verificationID == "" {
|
||||
var err error
|
||||
verificationID, err = d.requestVerificationID(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
var step2 verifyResp
|
||||
resp, err := d.accountClient.R().
|
||||
SetContext(ctx).
|
||||
SetBody(map[string]any{
|
||||
"verification_id": verificationID,
|
||||
"verification_code": d.VerifyCode,
|
||||
"client_id": d.ClientID,
|
||||
}).
|
||||
SetResult(&step2).
|
||||
Post("/v1/auth/verification/verify")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.IsError() || step2.Error != "" || strings.TrimSpace(step2.VerificationToken) == "" {
|
||||
return fmt.Errorf("verify code failed: %s", d.accountErr(step2.ErrorDesc, step2.Error, resp))
|
||||
}
|
||||
|
||||
var out tokenResp
|
||||
resp, err = d.accountClient.R().
|
||||
SetContext(ctx).
|
||||
SetBody(map[string]any{
|
||||
"verification_code": d.VerifyCode,
|
||||
"verification_token": step2.VerificationToken,
|
||||
"username": normalizePhoneE164(d.PhoneNumber),
|
||||
"client_id": d.ClientID,
|
||||
}).
|
||||
SetResult(&out).
|
||||
Post("/v1/auth/signin")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.IsError() || out.Error != "" || strings.TrimSpace(out.AccessToken) == "" {
|
||||
return fmt.Errorf("signin failed: %s", d.accountErr(out.ErrorDesc, out.Error, resp))
|
||||
}
|
||||
|
||||
d.AccessToken = strings.TrimSpace(out.AccessToken)
|
||||
d.RefreshToken = strings.TrimSpace(out.RefreshToken)
|
||||
d.VerificationID = ""
|
||||
// One-time SMS code should not be reused after successful login.
|
||||
d.VerifyCode = ""
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) prepareSMSCode(ctx context.Context) error {
|
||||
// Explicit send action should always refresh verification_id.
|
||||
d.VerificationID = ""
|
||||
if err := d.ensureCaptchaToken(ctx, false); err != nil {
|
||||
return err
|
||||
}
|
||||
verificationID, err := d.requestVerificationID(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.VerificationID = verificationID
|
||||
d.SendCode = false
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) requestVerificationID(ctx context.Context) (string, error) {
|
||||
req := d.accountClient.R().SetContext(ctx)
|
||||
if d.CaptchaToken != "" {
|
||||
req.SetHeader("X-Captcha-Token", d.CaptchaToken)
|
||||
}
|
||||
|
||||
var step1 verificationResp
|
||||
resp, err := req.
|
||||
SetBody(map[string]any{
|
||||
"phone_number": normalizePhoneE164(d.PhoneNumber),
|
||||
"target": "ANY",
|
||||
"client_id": d.ClientID,
|
||||
}).
|
||||
SetResult(&step1).
|
||||
Post("/v1/auth/verification")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if resp.IsError() || step1.Error != "" || strings.TrimSpace(step1.VerificationID) == "" {
|
||||
// If captcha token is expired/invalid, refresh it once and retry.
|
||||
if strings.Contains(step1.Error, "captcha_invalid") || strings.Contains(step1.ErrorDesc, "captcha_token expired") {
|
||||
if err := d.ensureCaptchaToken(ctx, true); err == nil {
|
||||
return d.requestVerificationID(ctx)
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("request verification failed: %s", d.accountErr(step1.ErrorDesc, step1.Error, resp))
|
||||
}
|
||||
return strings.TrimSpace(step1.VerificationID), nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) ensureCaptchaToken(ctx context.Context, force bool) error {
|
||||
if !force && d.CaptchaToken != "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
var out captchaInitResp
|
||||
req := d.accountClient.R().SetContext(ctx)
|
||||
if d.CaptchaToken != "" {
|
||||
req.SetHeader("X-Captcha-Token", d.CaptchaToken)
|
||||
}
|
||||
resp, err := req.
|
||||
SetBody(map[string]any{
|
||||
"client_id": d.ClientID,
|
||||
"action": "POST:/v1/auth/verification",
|
||||
"device_id": d.DeviceID,
|
||||
"meta": map[string]any{
|
||||
"username": normalizePhoneE164(d.PhoneNumber),
|
||||
"phone_number": normalizePhoneE164(d.PhoneNumber),
|
||||
"VERIFICATION_PHONE": normalizePhoneE164(d.PhoneNumber),
|
||||
},
|
||||
}).
|
||||
SetResult(&out).
|
||||
Post("/v1/shield/captcha/init")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.IsError() || out.Error != "" || strings.TrimSpace(out.CaptchaToken) == "" {
|
||||
return fmt.Errorf("init captcha token failed: %s", d.accountErr(out.ErrorDesc, out.Error, resp))
|
||||
}
|
||||
d.CaptchaToken = strings.TrimSpace(out.CaptchaToken)
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Interface compliance checks
|
||||
var (
|
||||
_ driver.Driver = (*GuangYaPan)(nil)
|
||||
_ driver.GetRooter = (*GuangYaPan)(nil)
|
||||
_ driver.Mkdir = (*GuangYaPan)(nil)
|
||||
_ driver.Move = (*GuangYaPan)(nil)
|
||||
_ driver.Copy = (*GuangYaPan)(nil)
|
||||
_ driver.Rename = (*GuangYaPan)(nil)
|
||||
_ driver.Remove = (*GuangYaPan)(nil)
|
||||
_ driver.PutResult = (*GuangYaPan)(nil)
|
||||
_ driver.WithDetails = (*GuangYaPan)(nil)
|
||||
)
|
||||
@@ -1,37 +0,0 @@
|
||||
package guangyapan
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
RootPath string `json:"root_path" help:"Full path in GuangYaPan cloud drive"`
|
||||
PhoneNumber string `json:"phone_number" type:"text" help:"Phone number for SMS login, e.g. +86 13800000000"`
|
||||
CaptchaToken string `json:"captcha_token" help:"Captcha token required by /v1/auth/verification"`
|
||||
SendCode bool `json:"send_code" type:"bool" help:"Set true and save to send SMS code, it auto-resets to false after sending"`
|
||||
VerifyCode string `json:"verify_code" type:"text" help:"SMS verification code used with phone_number; fill then save to finish login"`
|
||||
VerificationID string `json:"verification_id" type:"text" help:"Auto-generated after sending SMS code; do not edit manually"`
|
||||
AccessToken string `json:"access_token" help:"Bearer access token (optional if refresh_token is provided)"`
|
||||
RefreshToken string `json:"refresh_token" help:"Refresh token for auto-login/auto-refresh"`
|
||||
ClientID string `json:"client_id" required:"true" help:"Client ID for GuangYaPan API, must be provided"`
|
||||
DeviceID string `json:"device_id" help:"Optional custom device id (32 hex chars), auto-generated when empty"`
|
||||
DeviceSign string `json:"device_sign" help:"Optional custom X-Device-Sign header (generated from device_id when empty)"`
|
||||
PageSize int `json:"page_size" type:"number" default:"100"`
|
||||
OrderBy int `json:"order_by" type:"number" options:"0,1,2,3,4" default:"3" help:"Sort field used by the file list"`
|
||||
SortType int `json:"sort_type" type:"number" options:"0,1" default:"1" help:"Sort direction used by the file list"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "GuangYaPan",
|
||||
DefaultRoot: "",
|
||||
CheckStatus: true,
|
||||
Alert: "info|Two-stage SMS login: (1) fill phone_number (+ captcha_token if needed), set send_code=true and save; (2) fill verify_code and save to finish login and auto-save access_token/refresh_token.",
|
||||
NoOverwriteUpload: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &GuangYaPan{}
|
||||
})
|
||||
}
|
||||
@@ -1,169 +0,0 @@
|
||||
package guangyapan
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
func (d *GuangYaPan) ResolveOfflineResource(ctx context.Context, fileURL string) (*OfflineResolveData, error) {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
fileURL = strings.TrimSpace(fileURL)
|
||||
if fileURL == "" {
|
||||
return nil, errors.New("offline url is empty")
|
||||
}
|
||||
|
||||
var resp offlineResolveResp
|
||||
if err := d.postAPI(ctx, "/cloudcollection/v1/resolve_res", map[string]any{
|
||||
"url": fileURL,
|
||||
}, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !isSuccessMsg(resp.Msg) {
|
||||
return nil, fmt.Errorf("resolve offline resource failed: %s", strings.TrimSpace(resp.Msg))
|
||||
}
|
||||
return &resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) OfflineDownload(ctx context.Context, fileURL string, parentDir model.Obj, fileName string) (*OfflineTask, error) {
|
||||
resolved, err := d.ResolveOfflineResource(ctx, fileURL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
parentID := parentDir.GetID()
|
||||
rootID, err := d.getRootFolderID(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if parentID == rootID {
|
||||
parentID = ""
|
||||
}
|
||||
|
||||
taskURL := strings.TrimSpace(resolved.URL)
|
||||
if taskURL == "" {
|
||||
taskURL = strings.TrimSpace(fileURL)
|
||||
}
|
||||
name := strings.TrimSpace(fileName)
|
||||
if name == "" {
|
||||
name = resolved.defaultName(taskURL)
|
||||
}
|
||||
|
||||
body := map[string]any{
|
||||
"url": taskURL,
|
||||
"parentId": parentID,
|
||||
"newName": name,
|
||||
}
|
||||
if indexes := resolved.fileIndexes(); len(indexes) > 0 {
|
||||
body["fileIndexes"] = indexes
|
||||
}
|
||||
|
||||
var resp offlineCreateResp
|
||||
if err := d.postAPI(ctx, "/cloudcollection/v1/create_task", body, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !isSuccessMsg(resp.Msg) {
|
||||
return nil, fmt.Errorf("create offline task failed: %s", strings.TrimSpace(resp.Msg))
|
||||
}
|
||||
taskID := strings.TrimSpace(resp.Data.TaskID)
|
||||
if taskID == "" {
|
||||
return nil, errors.New("create offline task failed: empty task id")
|
||||
}
|
||||
return &OfflineTask{
|
||||
TaskID: taskID,
|
||||
FileName: name,
|
||||
Res: taskURL,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) OfflineList(ctx context.Context, taskIDs []string, statuses []int, cursor string, pageSize int) ([]OfflineTask, error) {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body := map[string]any{}
|
||||
if len(taskIDs) > 0 {
|
||||
body["taskIds"] = taskIDs
|
||||
}
|
||||
if len(statuses) > 0 {
|
||||
body["status"] = statuses
|
||||
}
|
||||
if cursor = strings.TrimSpace(cursor); cursor != "" {
|
||||
body["cursor"] = cursor
|
||||
}
|
||||
if pageSize > 0 {
|
||||
body["pageSize"] = pageSize
|
||||
}
|
||||
|
||||
var resp offlineListResp
|
||||
if err := d.postAPI(ctx, "/cloudcollection/v1/list_task", body, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !isSuccessMsg(resp.Msg) {
|
||||
return nil, fmt.Errorf("list offline tasks failed: %s", strings.TrimSpace(resp.Msg))
|
||||
}
|
||||
return resp.Data.List, nil
|
||||
}
|
||||
|
||||
func (d *GuangYaPan) DeleteOfflineTasks(ctx context.Context, taskIDs []string) error {
|
||||
if err := d.ensureAccessToken(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(taskIDs) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
var resp offlineDeleteResp
|
||||
if err := d.postAPI(ctx, "/cloudcollection/v2/delete_task", map[string]any{
|
||||
"taskIds": taskIDs,
|
||||
}, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if !isSuccessMsg(resp.Msg) {
|
||||
return fmt.Errorf("delete offline tasks failed: %s", strings.TrimSpace(resp.Msg))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d OfflineResolveData) defaultName(fileURL string) string {
|
||||
if d.BTResInfo != nil && strings.TrimSpace(d.BTResInfo.FileName) != "" {
|
||||
return strings.TrimSpace(d.BTResInfo.FileName)
|
||||
}
|
||||
u, err := url.Parse(fileURL)
|
||||
if err == nil {
|
||||
name := strings.TrimSpace(stdpath.Base(u.Path))
|
||||
if name != "" && name != "." && name != "/" {
|
||||
if decoded, err := url.PathUnescape(name); err == nil {
|
||||
name = decoded
|
||||
}
|
||||
return name
|
||||
}
|
||||
}
|
||||
return "offline_download"
|
||||
}
|
||||
|
||||
func (d OfflineResolveData) fileIndexes() []int {
|
||||
if d.BTResInfo == nil || len(d.BTResInfo.Subfiles) == 0 {
|
||||
return nil
|
||||
}
|
||||
indexes := make([]int, 0, len(d.BTResInfo.Subfiles))
|
||||
for i, file := range d.BTResInfo.Subfiles {
|
||||
if file.FileIndex != nil {
|
||||
indexes = append(indexes, *file.FileIndex)
|
||||
continue
|
||||
}
|
||||
indexes = append(indexes, i)
|
||||
}
|
||||
return indexes
|
||||
}
|
||||
|
||||
func isSuccessMsg(msg string) bool {
|
||||
msg = strings.TrimSpace(msg)
|
||||
return msg == "" || strings.EqualFold(msg, "success")
|
||||
}
|
||||
@@ -1,232 +0,0 @@
|
||||
package guangyapan
|
||||
|
||||
import "time"
|
||||
|
||||
type tokenResp struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
RefreshToken string `json:"refresh_token"`
|
||||
TokenType string `json:"token_type"`
|
||||
ExpiresIn int64 `json:"expires_in"`
|
||||
Sub string `json:"sub"`
|
||||
Error string `json:"error"`
|
||||
ErrorCode int `json:"error_code"`
|
||||
ErrorDesc string `json:"error_description"`
|
||||
}
|
||||
|
||||
type verificationResp struct {
|
||||
VerificationID string `json:"verification_id"`
|
||||
Error string `json:"error"`
|
||||
ErrorCode int `json:"error_code"`
|
||||
ErrorDesc string `json:"error_description"`
|
||||
}
|
||||
|
||||
type captchaInitResp struct {
|
||||
CaptchaToken string `json:"captcha_token"`
|
||||
ExpiresIn int64 `json:"expires_in"`
|
||||
Error string `json:"error"`
|
||||
ErrorCode int `json:"error_code"`
|
||||
ErrorDesc string `json:"error_description"`
|
||||
}
|
||||
|
||||
type verifyResp struct {
|
||||
VerificationToken string `json:"verification_token"`
|
||||
Error string `json:"error"`
|
||||
ErrorCode int `json:"error_code"`
|
||||
ErrorDesc string `json:"error_description"`
|
||||
}
|
||||
|
||||
type userMeResp struct {
|
||||
Sub string `json:"sub"`
|
||||
}
|
||||
|
||||
type listResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
Total int `json:"total"`
|
||||
List []fileItem `json:"list"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type fileItem struct {
|
||||
FileID string `json:"fileId"`
|
||||
ParentID string `json:"parentId"`
|
||||
FileName string `json:"fileName"`
|
||||
FileSize int64 `json:"fileSize"`
|
||||
ResType int `json:"resType"`
|
||||
CTime int64 `json:"ctime"`
|
||||
UTime int64 `json:"utime"`
|
||||
}
|
||||
|
||||
type downloadResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
SignedURL string `json:"signedURL"`
|
||||
DownloadURL string `json:"downloadUrl"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type createDirResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
FileID string `json:"fileId"`
|
||||
FileName string `json:"fileName"`
|
||||
ResType int `json:"resType"`
|
||||
CTime int64 `json:"ctime"`
|
||||
UTime int64 `json:"utime"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type commonResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
}
|
||||
|
||||
// taskResp is used for async operations (delete, move, copy) that return a task ID.
|
||||
type taskResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
TaskID string `json:"taskId"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type taskStatusResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
Status int `json:"status"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type uploadTokenResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data uploadTokenData `json:"data"`
|
||||
}
|
||||
|
||||
type uploadTokenData struct {
|
||||
TaskID string `json:"taskId"`
|
||||
AlreadyDone bool `json:"-"`
|
||||
ObjectPath string `json:"objectPath"`
|
||||
Provider any `json:"provider"`
|
||||
Region string `json:"region"`
|
||||
BucketName string `json:"bucketName"`
|
||||
EndPoint string `json:"endPoint"`
|
||||
FullEndPoint string `json:"fullEndPoint"`
|
||||
CallbackVar string `json:"callbackVar"`
|
||||
AccessKeyID string `json:"accessKeyID"`
|
||||
SecretAccessKey string `json:"secretAccessKey"`
|
||||
SessionToken string `json:"sessionToken"`
|
||||
Creds struct {
|
||||
AccessKeyID string `json:"accessKeyID"`
|
||||
SecretAccessKey string `json:"secretAccessKey"`
|
||||
SessionToken string `json:"sessionToken"`
|
||||
} `json:"creds"`
|
||||
}
|
||||
|
||||
type taskInfoResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
FileID string `json:"fileId"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
// assetsInfoResp is the response for storage details query.
|
||||
type assetsInfoResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
TotalSpaceSize int64 `json:"totalSpaceSize"`
|
||||
UsedSpaceSize int64 `json:"usedSpaceSize"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
func (r assetsInfoResp) IsSuccess() bool {
|
||||
return isSuccessMsg(r.Msg)
|
||||
}
|
||||
|
||||
// Offline download types
|
||||
|
||||
type offlineResolveResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data OfflineResolveData `json:"data"`
|
||||
}
|
||||
|
||||
type OfflineResolveData struct {
|
||||
ResType int `json:"resType"`
|
||||
BTResInfo *OfflineBTResInfo `json:"btResInfo"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
type OfflineBTResInfo struct {
|
||||
InfoHash string `json:"infoHash"`
|
||||
FileName string `json:"fileName"`
|
||||
FileSize int64 `json:"fileSize"`
|
||||
SubfilesNum int `json:"subfilesNum"`
|
||||
Subfiles []OfflineSubfile `json:"subfiles"`
|
||||
CreateTime int64 `json:"createTime"`
|
||||
ExcludeIndices []int `json:"excludeIndices"`
|
||||
}
|
||||
|
||||
type OfflineSubfile struct {
|
||||
FileName string `json:"fileName"`
|
||||
FileIndex *int `json:"fileIndex"`
|
||||
FileSize int64 `json:"fileSize"`
|
||||
}
|
||||
|
||||
type offlineCreateResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
TaskID string `json:"taskId"`
|
||||
URL string `json:"url"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type offlineDeleteResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
TaskIDs []string `json:"taskIds"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type offlineListResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
Data struct {
|
||||
StatusCounts []struct {
|
||||
Status int `json:"status"`
|
||||
Count int `json:"count"`
|
||||
} `json:"statusCounts"`
|
||||
Cursor string `json:"cursor"`
|
||||
List []OfflineTask `json:"list"`
|
||||
Total int `json:"total"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type OfflineTask struct {
|
||||
TaskID string `json:"taskId"`
|
||||
FileName string `json:"fileName"`
|
||||
TotalSize int64 `json:"totalSize"`
|
||||
Status int `json:"status"`
|
||||
CreateTime int64 `json:"createTime"`
|
||||
Res string `json:"res"`
|
||||
ResType int `json:"resType"`
|
||||
Progress int `json:"progress"`
|
||||
FileID string `json:"fileId"`
|
||||
IsDir bool `json:"isDir"`
|
||||
Exist bool `json:"exist"`
|
||||
}
|
||||
|
||||
func unixOrZero(v int64) time.Time {
|
||||
if v <= 0 {
|
||||
return time.Time{}
|
||||
}
|
||||
return time.Unix(v, 0)
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user