mirror of
https://github.com/OpenListTeam/OpenList.git
synced 2026-10-10 13:03:09 +08:00
Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 55d3827dee | |||
| 1fbc9427df | |||
| bb3d139a47 | |||
| d227ab85d6 | |||
| 5342ae96d0 | |||
| 273e15a050 | |||
| 13aad2c2fa | |||
| 368dc65a6e | |||
| 8b4b6ba970 | |||
| 4d28e838ce | |||
| 3930d4789a |
@@ -1,114 +0,0 @@
|
||||
<!--
|
||||
PR title / PR 标题:
|
||||
- Use Conventional Commits: `type(scope): summary`
|
||||
- Allowed types: `feat`, `docs`, `fix`, `style`, `refactor`, `chore`
|
||||
- Scope is required by the current PR title check.
|
||||
- For breaking changes, add `!`: `feat(driver)!: change auth flow`
|
||||
-->
|
||||
|
||||
## Summary / 摘要
|
||||
|
||||
<!--
|
||||
Briefly describe what changed and why.
|
||||
简要说明改了什么,以及为什么需要改。
|
||||
-->
|
||||
|
||||
<!--
|
||||
- List user-visible behavior changes.
|
||||
- List important implementation changes.
|
||||
- Mention config, storage, API, or compatibility changes if any.
|
||||
|
||||
- 列出用户可感知的行为变化。
|
||||
- 列出重要实现变化。
|
||||
- 如涉及配置、存储、API 或兼容性变化,请明确说明。
|
||||
-->
|
||||
|
||||
- [ ] This PR has breaking changes.
|
||||
/ 此 PR 包含破坏性变更。
|
||||
- [ ] This PR changes public API, config, storage format, or migration behavior.
|
||||
/ 此 PR 修改了公开 API、配置、存储格式或迁移行为。
|
||||
- [ ] This PR requires corresponding changes in related repositories.
|
||||
/ 此 PR 需要关联仓库同步修改。
|
||||
|
||||
Related repository PRs / 关联仓库 PR:
|
||||
|
||||
- OpenList-Frontend:
|
||||
- OpenList-Docs:
|
||||
|
||||
## Related Issues / 关联 Issue
|
||||
|
||||
<!--
|
||||
Use `Closes #123`, `Fixes #123`, or `Relates to #123`.
|
||||
Remove this section if not applicable.
|
||||
使用 `Closes #123`、`Fixes #123` 或 `Relates to #123`。
|
||||
不适用时请删除本节。
|
||||
-->
|
||||
|
||||
## Testing / 测试
|
||||
|
||||
<!--
|
||||
Describe commands, platforms, and manual checks.
|
||||
If not tested, explain why.
|
||||
|
||||
说明执行过的命令、测试平台和手动验证。
|
||||
如果未测试,请说明原因。
|
||||
-->
|
||||
|
||||
- [ ] `go test ./...`
|
||||
- [ ] Manual test / 手动测试:
|
||||
|
||||
## Checklist / 检查清单
|
||||
|
||||
- [ ] I have read [CONTRIBUTING](https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md).
|
||||
/ 我已阅读 [CONTRIBUTING](https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md)。
|
||||
- [ ] I confirm this contribution follows the repository license, contribution policy, and code of conduct.
|
||||
/ 我确认此贡献符合仓库许可证、贡献规范和行为准则。
|
||||
- [ ] I have formatted the changed code with `gofmt`, `go fmt`, or `prettier` where applicable.
|
||||
/ 我已按适用情况使用 `gofmt`、`go fmt` 或 `prettier` 格式化变更代码。
|
||||
- [ ] I have requested review from relevant maintainers or code owners where applicable.
|
||||
/ 我已在适用情况下请求相关维护者或代码所有者审查。
|
||||
|
||||
## AI Disclosure / AI 使用声明
|
||||
|
||||
<!--
|
||||
Please disclose any substantial AI assistance used in this PR.
|
||||
Minor AI assistance, such as typo fixes, autocomplete, formatting suggestions,
|
||||
or wording polish, does not need to be disclosed.
|
||||
Remove this section if not applicable.
|
||||
|
||||
请披露此 PR 中使用的重要 AI 辅助内容。
|
||||
轻微 AI 辅助,例如拼写修正、自动补全、格式建议或文字润色,无需披露。
|
||||
如不适用,请删除本节。
|
||||
|
||||
Deliberate non-disclosure may be treated as a trust and compliance issue.
|
||||
|
||||
故意隐瞒 AI 使用情况可能被视为信任与合规问题。
|
||||
-->
|
||||
|
||||
- [ ] This PR includes AI-assisted content.
|
||||
/ 此 PR 包含 AI 辅助内容。
|
||||
|
||||
Tools used / 使用工具:
|
||||
|
||||
- [ ] ChatGPT
|
||||
- [ ] Codex
|
||||
- [ ] GitHub Copilot
|
||||
- [ ] Claude
|
||||
- [ ] Gemini
|
||||
- [ ] Other (please specify) / 其他(请注明):
|
||||
|
||||
Usage scope / 使用范围:
|
||||
|
||||
- [ ] Code generation / 代码生成
|
||||
- [ ] Refactoring / 重构
|
||||
- [ ] Documentation / 文档
|
||||
- [ ] Tests / 测试
|
||||
- [ ] Translation / 翻译
|
||||
- [ ] Review assistance / 审查辅助
|
||||
|
||||
- [ ] I have reviewed and validated all AI-assisted content included in this PR.
|
||||
/ 我已审核并验证此 PR 中的所有 AI 辅助内容。
|
||||
- [ ] I have ensured that all AI-assisted commits include `Co-Authored-By` attribution.
|
||||
/ 我已确保所有 AI 辅助提交都包含 `Co-Authored-By` 归属信息。
|
||||
- [ ] I can reproduce all AI-assisted content included in this PR without any AI tools.
|
||||
/ 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。
|
||||
@@ -1,215 +0,0 @@
|
||||
name: Container Registry Cleanup
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
package:
|
||||
description: 'The name of the package to clean up'
|
||||
required: true
|
||||
default: 'openlist-git'
|
||||
owner:
|
||||
description: 'The owner of the package (user or organization)'
|
||||
required: false
|
||||
default: 'openlistteam'
|
||||
older_than:
|
||||
description: 'Delete untagged images older than this many days'
|
||||
required: false
|
||||
default: '-1'
|
||||
untagged_timestamp_tolerance:
|
||||
description: 'Tolerance in milliseconds for untagged images close to tagged ones'
|
||||
required: false
|
||||
# We do this because multi-arch docker containers will be pushed as separate untagged images
|
||||
# to the container registry and GitHub cannot recognize them as part of a tagged image.
|
||||
# Setting this option large enough prevents accidental deletion of one of these images.
|
||||
# Here we set it to 10 seconds
|
||||
default: '10000'
|
||||
token:
|
||||
description: 'GitHub token with permissions to delete packages (leave empty to use default GITHUB_TOKEN)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
jobs:
|
||||
cleanup:
|
||||
runs-on: ubuntu-slim
|
||||
|
||||
steps:
|
||||
- name: Check inputs and tools
|
||||
id: setup
|
||||
shell: bash
|
||||
run: |
|
||||
PACKAGE="${{ github.event.inputs.package || 'openlist-git' }}"
|
||||
OLDER="${{ github.event.inputs.older_than || '-1' }}"
|
||||
TOLERANCE_VAL="${{ github.event.inputs.untagged_timestamp_tolerance || '10000' }}"
|
||||
TOKEN_INPUT="${{ github.event.inputs.token || '' }}"
|
||||
OWNER="${{ github.event.inputs.owner || 'openlistteam' }}"
|
||||
|
||||
echo "package_name=$PACKAGE" >> "$GITHUB_OUTPUT"
|
||||
echo "older_than=$OLDER" >> "$GITHUB_OUTPUT"
|
||||
echo "tolerance=$TOLERANCE_VAL" >> "$GITHUB_OUTPUT"
|
||||
echo "owner=$OWNER" >> "$GITHUB_OUTPUT"
|
||||
|
||||
if [[ -z "$TOKEN_INPUT" ]]; then
|
||||
echo "::notice:: No token provided, using GITHUB_TOKEN instead."
|
||||
echo "token=${{ secrets.GITHUB_TOKEN }}" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "::add-mask::$TOKEN_INPUT"
|
||||
echo "token=$TOKEN_INPUT" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
if [[ -z "$PACKAGE" || -z "$OWNER" || ( -z "$TOKEN_INPUT" && -z "${{ secrets.GITHUB_TOKEN }}" ) ]]; then
|
||||
echo "::error:: Missing required inputs (package, token, or owner context)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
command -v curl >/dev/null 2>&1 || { echo "::error::curl is required but not installed."; exit 1; }
|
||||
command -v jq >/dev/null 2>&1 || { echo "::error::jq is required but not installed."; exit 1; }
|
||||
|
||||
- name: Determine owner type (user or organization)
|
||||
id: owner_type
|
||||
shell: bash
|
||||
run: |
|
||||
OWNER="${{ steps.setup.outputs.owner }}"
|
||||
TOKEN="${{ steps.setup.outputs.token }}"
|
||||
|
||||
TYPE=$(curl -s -H "Authorization: token $TOKEN" \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
"https://api.github.com/users/${OWNER}" | jq -r '.type')
|
||||
|
||||
if [[ "$TYPE" == "Organization" ]]; then
|
||||
OWNER_PATH="orgs"
|
||||
else
|
||||
OWNER_PATH="users"
|
||||
fi
|
||||
|
||||
echo "owner_path=$OWNER_PATH" >> "$GITHUB_OUTPUT"
|
||||
echo "Detected owner type: $TYPE (using /${OWNER_PATH}/)"
|
||||
|
||||
- name: Access all versions from GitHub
|
||||
id: fetch
|
||||
shell: bash
|
||||
run: |
|
||||
PACKAGE_NAME="${{ steps.setup.outputs.package_name }}"
|
||||
OWNER="${{ steps.setup.outputs.owner }}"
|
||||
TOKEN="${{ steps.setup.outputs.token }}"
|
||||
OWNER_PATH="${{ steps.owner_type.outputs.owner_path }}"
|
||||
|
||||
BASE_URL="https://api.github.com/${OWNER_PATH}/${OWNER}/packages/container/${PACKAGE_NAME}/versions"
|
||||
URL="${BASE_URL}?per_page=100"
|
||||
TEMP_FILE="/tmp/all_versions.ndjson"
|
||||
|
||||
> "$TEMP_FILE"
|
||||
PAGE=1
|
||||
while [[ -n "$URL" && $PAGE -le 10 ]]; do
|
||||
echo "Fetching: $URL"
|
||||
|
||||
curl -s -H "Authorization: token $TOKEN" \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
-D headers.txt \
|
||||
"$URL" > page.json
|
||||
|
||||
jq -c '.[]' page.json >> "$TEMP_FILE"
|
||||
|
||||
URL=$(grep -i '^link:' headers.txt | grep -o '<[^>]*>;\s*rel="next"' | sed 's/<\(.*\)>.*/\1/' || true)
|
||||
PAGE=$((PAGE + 1))
|
||||
done
|
||||
rm -f headers.txt page.json
|
||||
|
||||
echo "temp_file=$TEMP_FILE" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Filter untagged versions
|
||||
id: filter
|
||||
shell: bash
|
||||
run: |
|
||||
OLDER_THAN="${{ steps.setup.outputs.older_than }}"
|
||||
TOLERANCE="${{ steps.setup.outputs.tolerance }}"
|
||||
TEMP_FILE="${{ steps.fetch.outputs.temp_file }}"
|
||||
|
||||
TO_DELETE=$(jq -c -s \
|
||||
--arg older_than "$OLDER_THAN" \
|
||||
--arg tolerance "$TOLERANCE" \
|
||||
'
|
||||
def abs: if . < 0 then -. else . end;
|
||||
def to_ms: sub("\\.[0-9]+Z$"; "Z") | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime * 1000;
|
||||
|
||||
. as $all |
|
||||
($older_than | tonumber) as $older_than_val |
|
||||
($tolerance | tonumber) as $tolerance_val |
|
||||
now as $now |
|
||||
(if $older_than_val > 0 then ($now - ($older_than_val * 86400)) * 1000 else 0 end) as $cutoff_ms |
|
||||
|
||||
[ $all[] | select((.metadata.container.tags // []) | length > 0) | .created_at | to_ms ] as $tagged_times |
|
||||
|
||||
$all[] |
|
||||
select((.metadata.container.tags // []) | length == 0) |
|
||||
. as $item |
|
||||
(.created_at | to_ms) as $created_ms |
|
||||
|
||||
(if $older_than_val > 0 then $created_ms < $cutoff_ms else true end) as $pass_age |
|
||||
|
||||
(if $tolerance_val <= 0 then true
|
||||
else [ $tagged_times[] | select(($created_ms - .) | abs < $tolerance_val) ] | length == 0
|
||||
end) as $pass_tolerance |
|
||||
|
||||
select($pass_age and $pass_tolerance) |
|
||||
{id, name}
|
||||
' "$TEMP_FILE")
|
||||
|
||||
rm -f "$TEMP_FILE"
|
||||
|
||||
if [[ -z "$TO_DELETE" ]]; then
|
||||
DELETE_COUNT=0
|
||||
else
|
||||
DELETE_COUNT=$(echo "$TO_DELETE" | wc -l | tr -d ' ')
|
||||
fi
|
||||
|
||||
echo "Found $DELETE_COUNT untagged images no longer necessary"
|
||||
|
||||
{
|
||||
echo 'to_delete<<EOF'
|
||||
echo "$TO_DELETE"
|
||||
echo 'EOF'
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "delete_count=$DELETE_COUNT" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Delete untagged versions
|
||||
id: delete
|
||||
shell: bash
|
||||
run: |
|
||||
PACKAGE_NAME="${{ steps.setup.outputs.package_name }}"
|
||||
OWNER="${{ steps.setup.outputs.owner }}"
|
||||
TOKEN="${{ steps.setup.outputs.token }}"
|
||||
OWNER_PATH="${{ steps.owner_type.outputs.owner_path }}"
|
||||
DELETE_COUNT="${{ steps.filter.outputs.delete_count }}"
|
||||
TO_DELETE='${{ steps.filter.outputs.to_delete }}'
|
||||
|
||||
if [[ "$DELETE_COUNT" -gt 0 && -n "$TO_DELETE" ]]; then
|
||||
while IFS= read -r item; do
|
||||
[[ -z "$item" ]] && continue
|
||||
|
||||
# $item 是完整的单行 JSON,如 {"id":123,"name":"sha256:..."}
|
||||
ID=$(echo "$item" | jq -r '.id')
|
||||
NAME=$(echo "$item" | jq -r '.name')
|
||||
|
||||
DELETE_URL="https://api.github.com/${OWNER_PATH}/${OWNER}/packages/container/${PACKAGE_NAME}/versions/${ID}"
|
||||
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X DELETE \
|
||||
-H "Authorization: token $TOKEN" \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
"$DELETE_URL")
|
||||
|
||||
if [[ "$HTTP_CODE" == "204" ]]; then
|
||||
echo "Deleted untagged container image '${NAME}'"
|
||||
else
|
||||
echo "::warning:: Failed to delete ${NAME} (ID: ${ID}). HTTP Status: ${HTTP_CODE}"
|
||||
fi
|
||||
done <<< "$TO_DELETE"
|
||||
else
|
||||
echo "No images to delete."
|
||||
fi
|
||||
|
||||
echo "Cleanup completed."
|
||||
@@ -1,101 +0,0 @@
|
||||
name: Issue or PR Auto Reply
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened]
|
||||
pull_request:
|
||||
types: [opened]
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
auto-reply:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'issues'
|
||||
steps:
|
||||
- name: Check issue for unchecked tasks and reply
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
let comment = "";
|
||||
const issueTitle = context.payload.issue.title || "";
|
||||
const titleNotEdited = /(请修改标题|Please modify the title)/i.test(issueTitle);
|
||||
if (titleNotEdited) {
|
||||
comment = "⚠️ 请修改标题以更好地描述您的问题或需求,并删除示例提示。当前 Issue 将被自动关闭。如需继续提交,请创建新的 Issue。\n";
|
||||
comment += "⚠️ Please modify the title to better describe your issue or request, and remove the example prompt. This issue will be automatically closed. If you wish to proceed, please create a new issue.\n";
|
||||
await github.rest.issues.createComment({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: comment
|
||||
});
|
||||
await github.rest.issues.update({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
state: 'closed',
|
||||
state_reason: 'not_planned',
|
||||
labels: ['invalid']
|
||||
});
|
||||
return;
|
||||
}
|
||||
const issueBody = context.payload.issue.body || "";
|
||||
const confirmHasRead = /- \[ \] (?!我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody);
|
||||
const confirmNotRead = /- \[[xX]\] (?:我没有阅读这个清单|I have not read these checkboxes)/.test(issueBody);
|
||||
if (confirmNotRead) {
|
||||
comment = "⚠️ 你的 Issue 不符合提交规则。请先阅读相关规范后再重新提交。当前 Issue 将被自动关闭。如需继续提交,请确认已了解规则后重新打开或创建新的 Issue。\n";
|
||||
comment += "⚠️ Your issue does not comply with the submission rules. Please read the guidelines before submitting again. This issue will be automatically closed. If you wish to proceed, please confirm that you have reviewed the rules before reopening or creating a new issue.\n";
|
||||
await github.rest.issues.createComment({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: comment
|
||||
});
|
||||
await github.rest.issues.update({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
state: 'closed',
|
||||
state_reason: 'not_planned',
|
||||
labels: ['invalid']
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (confirmHasRead) {
|
||||
comment = "感谢您联系OpenList。我们会尽快回复您。\n";
|
||||
comment += "Thanks for contacting OpenList. We will reply to you as soon as possible.\n\n";
|
||||
comment += "由于您提出的 Issue 中包含部分未确认的项目,为了更好地管理项目,在人工审核后可能会直接关闭此问题。\n";
|
||||
comment += "如果您能确认并补充相关未确认项目的信息,欢迎随时重新提交。我们会及时关注并处理。感谢您的理解与支持!\n";
|
||||
comment += "Since your issue contains some unchecked tasks, it may be closed after manual review.\n";
|
||||
comment += "If you can confirm and provide information for the unchecked tasks, feel free to resubmit.\n";
|
||||
comment += "We will pay attention and handle it in a timely manner.\n\n";
|
||||
comment += "感谢您的理解与支持!\n";
|
||||
comment += "Thank you for your understanding and support!\n";
|
||||
await github.rest.issues.createComment({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: comment
|
||||
});
|
||||
}
|
||||
|
||||
pr-title-check:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Check PR title for required prefix and comment
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const title = context.payload.pull_request.title || "";
|
||||
const ok = /^(feat|docs|fix|style|refactor|chore)\(.+?\)!?: /i.test(title);
|
||||
if (!ok) {
|
||||
let comment = "⚠️ PR 标题需以 `feat(): `, `docs(): `, `fix(): `, `style(): `, `refactor(): `, `chore(): ` 其中之一开头,例如:`feat(component): 新增功能`。\n";
|
||||
comment += "⚠️ The PR title must start with `feat(): `, `docs(): `, `fix(): `, `style(): `, or `refactor(): `, `chore(): `. For example: `feat(component): add new feature`.\n\n";
|
||||
comment += "如果跨多个组件,请使用主要组件作为前缀,并在标题中枚举、描述中说明。\n";
|
||||
comment += "If it spans multiple components, use the main component as the prefix and enumerate in the title, describe in the body.\n\n";
|
||||
comment += "如果是破坏性变更,请在类型后添加 `!`,例如 `feat(component)!: 破坏性变更`。\n";
|
||||
comment += "For breaking changes, add `!` after the type, e.g., `feat(component)!: breaking change`.\n\n";
|
||||
await github.rest.issues.createComment({
|
||||
...context.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: comment
|
||||
});
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
name: Sync to Gitee
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
sync:
|
||||
runs-on: ubuntu-latest
|
||||
name: Sync GitHub to Gitee
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup SSH
|
||||
run: |
|
||||
mkdir -p ~/.ssh
|
||||
echo "${{ secrets.GITEE_SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
|
||||
chmod 600 ~/.ssh/id_rsa
|
||||
ssh-keyscan gitee.com >> ~/.ssh/known_hosts
|
||||
|
||||
- name: Create single commit and push
|
||||
run: |
|
||||
git config user.name "GitHub Actions"
|
||||
git config user.email "actions@github.com"
|
||||
|
||||
# Create a new branch
|
||||
git checkout --orphan new-main
|
||||
git add .
|
||||
git commit -m "Sync from GitHub: $(date)"
|
||||
|
||||
# Add Gitee remote and force push
|
||||
git remote add gitee ${{ vars.GITEE_REPO_URL }}
|
||||
git push --force gitee new-main:main
|
||||
@@ -1,60 +0,0 @@
|
||||
# Agent Instructions
|
||||
|
||||
## Issues
|
||||
|
||||
Before creating an issue, review the available issue templates in the `.github` directory.
|
||||
|
||||
When drafting the issue:
|
||||
|
||||
- Use the most appropriate template.
|
||||
- Follow the template structure.
|
||||
- Fill in all required sections.
|
||||
- Remove sections that the template explicitly marks as optional or not applicable.
|
||||
- Do not invent reproduction steps, logs, screenshots, or expected behavior.
|
||||
|
||||
## Pull Requests
|
||||
|
||||
Before creating a pull request, read `.github/PULL_REQUEST_TEMPLATE.md`.
|
||||
|
||||
When drafting the pull request:
|
||||
|
||||
- Follow the template structure.
|
||||
- Use the title format required by the template.
|
||||
- Fill in or remove each section according to the template guidance.
|
||||
- Include testing details, or explain why testing was not run.
|
||||
- Do not invent testing results.
|
||||
- Do not claim validation, verification, or review steps that were not actually performed.
|
||||
|
||||
## Automated Contributions
|
||||
|
||||
Fully automated contributions are not considered equivalent to normal community participation.
|
||||
|
||||
A contribution may be considered fully automated if it is submitted through an automated agent, or if the submitting account participates in project discussions through an automated agent, without meaningful human review or intervention.
|
||||
|
||||
When making this determination, maintainers may consider the overall behavior of the account, including but not limited to disclosed agent usage, interaction patterns, response characteristics, and other available evidence. No single factor is determinative.
|
||||
|
||||
Maintainers reserve the right to accept, reject, modify, or reimplement any contribution independently of any action taken against the submitting account. Acceptance of a contribution does not imply acceptance of the submitting account or its contribution method. If an account is determined to be primarily operated through automated processes, we may need to restrict its future participation in contributions until that determination is rescinded.
|
||||
|
||||
## Git Commits
|
||||
|
||||
When creating commits, follow the repository `git-commit` skill rules:
|
||||
|
||||
- Use Conventional Commits title format: `type(scope): subject`.
|
||||
- Allowed types: `feat`, `fix`, `refactor`, `perf`, `docs`, `style`, `test`, `build`, `ci`, `chore`, `revert`.
|
||||
- Use a meaningful scope based on the main module, package, or feature.
|
||||
- Write the subject in imperative mood and describe the actual change.
|
||||
- Use a concise Markdown list in the commit body, with each item describing one key change.
|
||||
- Do not invent changes that are not present in the diff.
|
||||
- Do not describe behavior, refactors, fixes, or tests that are not reflected in the commit.
|
||||
|
||||
Include at most one `Co-authored-by` trailer that matches the AI assistant actually used to produce the change.
|
||||
|
||||
Examples:
|
||||
|
||||
- `Co-authored-by: Codex <267193182+codex@users.noreply.github.com>`
|
||||
- `Co-authored-by: GitHub Copilot <copilot@github.com>`
|
||||
- `Co-authored-by: Claude <81847+claude@users.noreply.github.com>`
|
||||
|
||||
If you are not one of the listed assistants, do not add a `Co-authored-by` trailer.
|
||||
|
||||
Instead, ask the human collaborator to provide the exact `Co-authored-by` trailer to use. Do not invent, infer, or generate one yourself.
|
||||
-110
@@ -1,110 +0,0 @@
|
||||
# Contributing
|
||||
|
||||
## Setup your machine
|
||||
|
||||
`OpenList` is written in [Go](https://golang.org/) and [SolidJS](https://www.solidjs.com/).
|
||||
|
||||
Prerequisites:
|
||||
|
||||
- [git](https://git-scm.com)
|
||||
- [Go](https://golang.org/doc/install) version declared in [`go.mod`](./go.mod)
|
||||
- [gcc](https://gcc.gnu.org/)
|
||||
- [nodejs](https://nodejs.org/)
|
||||
|
||||
## Cloning a fork
|
||||
|
||||
Fork and clone `OpenList` and `OpenList-Frontend` anywhere:
|
||||
|
||||
```shell
|
||||
git clone https://github.com/<your-username>/OpenList.git
|
||||
git clone --recurse-submodules https://github.com/<your-username>/OpenList-Frontend.git
|
||||
```
|
||||
|
||||
## Creating a branch
|
||||
|
||||
Create a new branch from the `main` branch, with an appropriate name.
|
||||
|
||||
```shell
|
||||
git checkout -b <branch-name>
|
||||
```
|
||||
|
||||
## Preview your change
|
||||
|
||||
### backend
|
||||
|
||||
```shell
|
||||
go run main.go
|
||||
```
|
||||
|
||||
### frontend
|
||||
|
||||
```shell
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
## Add a new driver
|
||||
|
||||
Copy `drivers/template` folder and rename it, and follow the comments in it.
|
||||
|
||||
## Community and policies
|
||||
|
||||
By contributing, you agree to follow the repository's code of conduct and license terms.
|
||||
|
||||
- Code of conduct: [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md)
|
||||
- License: [LICENSE](./LICENSE)
|
||||
- Security issues: please report privately according to [SECURITY.md](./SECURITY.md)
|
||||
|
||||
If your contribution includes substantial AI-assisted content, disclose the tools used and the scope of assistance in the pull request.
|
||||
|
||||
## Create a commit
|
||||
|
||||
Commit messages should be well formatted, and to make that "standardized".
|
||||
|
||||
Submit your pull request. For PR titles, follow [Conventional Commits](https://www.conventionalcommits.org).
|
||||
|
||||
<https://github.com/OpenListTeam/OpenList/issues/376>
|
||||
|
||||
It's suggested to sign your commits. See: [How to sign commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
|
||||
|
||||
## Submit a pull request
|
||||
|
||||
Please make sure your code has been formatted with `go fmt` or [prettier](https://prettier.io/) before submitting.
|
||||
|
||||
Push your branch to your `openlist` fork and open a pull request against the `main` branch.
|
||||
|
||||
## Merge your pull request
|
||||
|
||||
Your pull request will be merged after review. Please wait for the maintainer to merge your pull request after review.
|
||||
|
||||
At least 1 approving review is required by reviewers with write access. You can also request a review from maintainers.
|
||||
|
||||
## Delete your branch
|
||||
|
||||
(Optional) After your pull request is merged, you can delete your branch.
|
||||
|
||||
## AI Disclosure
|
||||
|
||||
If your pull request includes substantial AI-assisted content, disclose it in the PR description.
|
||||
|
||||
The pull request description must follow the repository's pull request template.
|
||||
|
||||
Fully automated contributions are not considered equivalent to normal community participation.
|
||||
|
||||
A contribution may be considered fully automated if it is submitted through an automated agent, or if the submitting account participates in project discussions through an automated agent, without meaningful human review or intervention.
|
||||
|
||||
When making this determination, maintainers may consider the overall behavior of the account, including but not limited to disclosed agent usage, interaction patterns, response characteristics, and other available evidence. No single factor is determinative.
|
||||
|
||||
Maintainers reserve the right to accept, reject, modify, or reimplement any contribution independently of any action taken against the submitting account. Acceptance of a contribution does not imply acceptance of the submitting account or its contribution method. If an account is determined to be primarily operated through automated processes, we may need to restrict its future participation in contributions until that determination is rescinded.
|
||||
|
||||
Please include:
|
||||
|
||||
- Tools used, such as ChatGPT, GitHub Copilot, Claude, Cursor, or other AI tools.
|
||||
- Usage scope, such as code generation, refactoring, documentation, tests, translation, or review assistance.
|
||||
- Confirmation that you have reviewed and validated all AI-assisted content before submission.
|
||||
- Confirmation that the submitted content complies with this repository's license and contribution policies.
|
||||
|
||||
Minor AI assistance, such as typo fixes, autocomplete, formatting suggestions, or wording polish, does not need to be disclosed.
|
||||
|
||||
---
|
||||
|
||||
Thank you for your contribution! Let's make OpenList better together!
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList هو تفرّع من AList يتميز بالمرونة والحوكمة طويلة الأمد ويقوده المجتمع، وقد بُني للدفاع عن البرمجيات مفتوحة المصدر ضد الهجمات القائمة على الثقة.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | العربية
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## إخلاء المسؤولية
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## الميزات
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## الوثائق
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## العرض التجريبي
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## النقاش
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## الراعي
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## المتبرعون
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## الترخيص
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## تواصل معنا
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## المساهمون
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList ist ein widerstandsfähiger, langfristig verwalteter und Community-getriebener Fork von AList, entwickelt zum Schutz von Open Source vor vertrauensbasierten Angriffen.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | Deutsch | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Haftungsausschluss
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Funktionen
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Dokumentation
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Demo
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Diskussion
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Sponsor
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Spender
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Lizenz
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Kontakt
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Mitwirkende
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList es un fork de AList resiliente, con gobernanza a largo plazo e impulsado por la comunidad, creado para defender el código abierto frente a ataques basados en la confianza.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | Español | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Aviso legal
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Características
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Documentación
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Demo
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Discusión
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Patrocinador
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Donantes
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Licencia
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Contacto
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Colaboradores
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList est un fork d’AList résilient, gouverné sur le long terme et porté par la communauté, conçu pour protéger l’open source contre les attaques fondées sur la confiance.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | Français | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Avertissement
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Fonctionnalités
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Documentation
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Démo
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Discussion
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Sponsor
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Donateurs
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Licence
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Nous contacter
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Contributeurs
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList는 신뢰 기반 공격으로부터 오픈 소스를 보호하기 위해 만들어진, 탄력적이고 장기 거버넌스를 지향하는 커뮤니티 주도 AList 포크입니다.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | 한국어 | [Deutsch](./README_de.md) | [Русский](./README_ru.md) | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## 면책 조항
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## 기능
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## 문서
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## 데모
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## 토론
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## 후원
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## 후원자
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## 라이선스
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## 문의
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## 기여자
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
@@ -1,136 +0,0 @@
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/OpenListTeam/Logo/main/logo.svg" width="128" height="128" alt="logo" />
|
||||
|
||||
<p><em>OpenList — устойчивый, управляемый сообществом форк AList с долгосрочным управлением, созданный для защиты open source от атак, основанных на доверии.</em></p>
|
||||
|
||||
<img src="https://goreportcard.com/badge/github.com/OpenListTeam/OpenList/v3" alt="latest version" />
|
||||
<a href="https://github.com/OpenListTeam/OpenList/blob/main/LICENSE"><img src="https://img.shields.io/github/license/OpenListTeam/OpenList" alt="License" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/actions?query=workflow%3ABuild"><img src="https://img.shields.io/github/actions/workflow/status/OpenListTeam/OpenList/build.yml?branch=main" alt="Build status" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/release/OpenListTeam/OpenList" alt="latest version" /></a>
|
||||
|
||||
<a href="https://github.com/OpenListTeam/OpenList/discussions"><img src="https://img.shields.io/github/discussions/OpenListTeam/OpenList?color=%23ED8936" alt="discussions" /></a>
|
||||
<a href="https://github.com/OpenListTeam/OpenList/releases"><img src="https://img.shields.io/github/downloads/OpenListTeam/OpenList/total?color=%239F7AEA&logo=github" alt="Downloads" /></a>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
- [English](../README.md) | [中文](./README_cn.md) | [日本語](./README_ja.md) | [Dutch](./README_nl.md) | [한국어](./README_ko.md) | [Deutsch](./README_de.md) | Русский | [Français](./README_fr.md) | [Español](./README_es.md) | [العربية](./README_ar.md)
|
||||
|
||||
- [Contributing](../CONTRIBUTING.md)
|
||||
- [CODE OF CONDUCT](../CODE_OF_CONDUCT.md)
|
||||
- [LICENSE](../LICENSE)
|
||||
|
||||
## Отказ от ответственности
|
||||
|
||||
OpenList is an open-source project independently maintained by the OpenList Team under the AGPL-3.0 license, with a commitment to code openness and modification transparency.
|
||||
|
||||
OpenList has no official association with third-party derivative projects that use similar names, including OpenListApp/OpenListApp or paid proprietary software using the same or similar names.
|
||||
|
||||
- All software, code, and services of this project are maintained by the OpenList Team and are freely available on GitHub.
|
||||
- Project documentation and API services primarily rely on charitable resources provided by Cloudflare. There are currently no paid plans or commercial deployments.
|
||||
- Downstream projects should not use the "OpenList" name for impersonation or commercial confusion, and must not distribute OpenList-based code as closed source in violation of the AGPL license.
|
||||
- This software is provided "as is" without warranties. Please comply with applicable laws and regulations when using it.
|
||||
|
||||
## Возможности
|
||||
|
||||
- [x] Multiple storages
|
||||
- [x] Local storage
|
||||
- [x] [Aliyundrive](https://www.alipan.com)
|
||||
- [x] OneDrive / Sharepoint ([Global](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [CN](https://portal.partner.microsoftonline.cn), DE, US)
|
||||
- [x] [189cloud](https://cloud.189.cn) (Personal, Family)
|
||||
- [x] [GoogleDrive](https://drive.google.com)
|
||||
- [x] [123pan](https://www.123pan.com)
|
||||
- [x] [FTP / SFTP](https://en.wikipedia.org/wiki/File_Transfer_Protocol)
|
||||
- [x] [PikPak](https://www.mypikpak.com)
|
||||
- [x] [S3](https://aws.amazon.com/s3)
|
||||
- [x] [Seafile](https://seafile.com)
|
||||
- [x] [UPYUN Storage Service](https://www.upyun.com/products/file-storage)
|
||||
- [x] [WebDAV](https://en.wikipedia.org/wiki/WebDAV)
|
||||
- [x] Teambition([China](https://www.teambition.com), [International](https://us.teambition.com))
|
||||
- [x] [MediaFire](https://www.mediafire.com)
|
||||
- [x] [Mediatrack](https://www.mediatrack.cn)
|
||||
- [x] [ProtonDrive](https://proton.me/drive)
|
||||
- [x] [139yun](https://yun.139.com) (Personal, Family, Group)
|
||||
- [x] [YandexDisk](https://disk.yandex.com)
|
||||
- [x] [BaiduNetdisk](http://pan.baidu.com)
|
||||
- [x] [Terabox](https://www.terabox.com/main)
|
||||
- [x] [UC](https://drive.uc.cn)
|
||||
- [x] [Quark](https://pan.quark.cn)
|
||||
- [x] [Thunder](https://pan.xunlei.com)
|
||||
- [x] [Lanzou](https://www.lanzou.com)
|
||||
- [x] [ILanzou](https://www.ilanzou.com)
|
||||
- [x] [Google photo](https://photos.google.com)
|
||||
- [x] [Mega.nz](https://mega.nz)
|
||||
- [x] [Baidu photo](https://photo.baidu.com)
|
||||
- [x] [SMB](https://en.wikipedia.org/wiki/Server_Message_Block)
|
||||
- [x] [115](https://115.com)
|
||||
- [x] [Cloudreve](https://cloudreve.org)
|
||||
- [x] [Dropbox](https://www.dropbox.com)
|
||||
- [x] [FeijiPan](https://www.feijipan.com)
|
||||
- [x] [dogecloud](https://www.dogecloud.com/product/oss)
|
||||
- [x] [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs)
|
||||
- [x] [Chaoxing](https://www.chaoxing.com)
|
||||
- [x] [CNB](https://cnb.cool/)
|
||||
- [x] [Degoo](https://degoo.com)
|
||||
- [x] [Doubao](https://www.doubao.com)
|
||||
- [x] [Febbox](https://www.febbox.com)
|
||||
- [x] [GitHub](https://github.com)
|
||||
- [x] [OpenList](https://github.com/OpenListTeam/OpenList)
|
||||
- [x] [Teldrive](https://github.com/tgdrive/teldrive)
|
||||
- [x] [Weiyun](https://www.weiyun.com)
|
||||
- [x] [DingTalk Docs](https://alidocs.dingtalk.com/)
|
||||
- [x] Easy to deploy and out-of-the-box
|
||||
- [x] File preview (PDF, markdown, code, plain text, ...)
|
||||
- [x] Image preview in gallery mode
|
||||
- [x] Video and audio preview, with lyrics and subtitles support
|
||||
- [x] Office document preview (docx, pptx, xlsx, ...)
|
||||
- [x] `README.md` preview rendering
|
||||
- [x] File permalink copy and direct file download
|
||||
- [x] Dark mode and i18n
|
||||
- [x] Protected routes, WebDAV, Docker deploy, and Cloudflare Workers proxy
|
||||
- [x] File/folder package download and web upload operations
|
||||
- [x] Offline download, cross-storage copy, and multi-thread download acceleration
|
||||
|
||||
## Документация
|
||||
|
||||
- 📘 [Docs](https://doc.oplist.org)
|
||||
- 🌏 [CN Mirror](https://doc.oplist.org.cn)
|
||||
- ⚖️ [Terms of Use](https://doc.oplist.org/terms)
|
||||
- 🔒 [Privacy Policy](https://doc.oplist.org/privacy)
|
||||
|
||||
## Демо
|
||||
|
||||
- 🌎 [Global Demo](https://demo.oplist.org)
|
||||
- 🇨🇳 [CN Demo](https://demo.oplist.org.cn)
|
||||
|
||||
## Обсуждение
|
||||
|
||||
Please refer to [Discussions](https://github.com/OpenListTeam/OpenList/discussions) for general questions; Issues are for bug reports and feature requests only.
|
||||
|
||||
## Спонсор
|
||||
|
||||
[](https://vps.town "VPS.Town - Trust, Effortlessly. Your Cloud, Reimagined.")
|
||||
|
||||
## Доноры
|
||||
|
||||
Thanks to the following donors for their generous support:
|
||||
|
||||
- [HisAtri](https://github.com/HisAtri)
|
||||
- 爱发电用户_7jTh
|
||||
- suka
|
||||
|
||||
## Лицензия
|
||||
|
||||
OpenList is open-source software licensed under the [AGPL-3.0](https://www.gnu.org/licenses/agpl-3.0.txt) license.
|
||||
|
||||
## Связаться с нами
|
||||
|
||||
- [@GitHub](https://github.com/OpenListTeam)
|
||||
- [Telegram Group](https://t.me/OpenListTeam)
|
||||
- [Telegram Channel](https://t.me/OpenListOfficial)
|
||||
|
||||
## Участники
|
||||
|
||||
We sincerely thank [Xhofe](https://github.com/Xhofe), author of the original project [AlistGo/alist](https://github.com/AlistGo/alist), and all other contributors.
|
||||
|
||||
[](https://github.com/OpenListTeam/OpenList/graphs/contributors)
|
||||
-89
@@ -1,89 +0,0 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
Only the latest stable release receives security patches. We strongly recommend always keeping OpenList up to date.
|
||||
|
||||
| Version | Supported |
|
||||
| -------------------- | ------------------ |
|
||||
| Latest stable (v4.x) | :white_check_mark: |
|
||||
| Older versions | :x: |
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
**Please do NOT report security vulnerabilities through public GitHub Issues.**
|
||||
|
||||
If you discover a security vulnerability in OpenList, please report it responsibly by using one of the following channels:
|
||||
|
||||
- **GitHub Private Security Advisory** (preferred): [Submit here](https://github.com/OpenListTeam/OpenList/security/advisories/new)
|
||||
- **Telegram**: Contact a maintainer privately via [@OpenListTeam](https://t.me/OpenListTeam)
|
||||
|
||||
When reporting, please include as much of the following as possible:
|
||||
|
||||
- A description of the vulnerability and its potential impact
|
||||
- The affected version(s)
|
||||
- Step-by-step instructions to reproduce the issue
|
||||
- Any proof-of-concept code or screenshots (if applicable)
|
||||
- Suggested mitigation or fix (optional but appreciated)
|
||||
|
||||
## Security Best Practices for Users
|
||||
|
||||
To keep your OpenList instance secure:
|
||||
|
||||
- Always update to the latest release.
|
||||
- Use a strong, unique admin password and change it after first login.
|
||||
- Enable HTTPS (TLS) for your deployment — do **not** expose OpenList over plain HTTP on the public internet.
|
||||
- Limit exposed ports using a reverse proxy (e.g., Nginx, Caddy).
|
||||
- Set up access controls and avoid enabling guest access unless necessary.
|
||||
- Regularly review mounted storage permissions and revoke unused API tokens.
|
||||
- When using Docker, avoid running the container as root if possible.
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
We sincerely thank all security researchers and community members who responsibly disclose vulnerabilities and help make OpenList safer for everyone.
|
||||
|
||||
---
|
||||
|
||||
# 安全政策
|
||||
|
||||
## 支持的版本
|
||||
|
||||
我们仅对最新稳定版本提供安全补丁。强烈建议始终保持 OpenList 为最新版本。
|
||||
|
||||
| 版本 | 是否支持 |
|
||||
| ------------------ | ------------------ |
|
||||
| 最新稳定版(v4.x) | :white_check_mark: |
|
||||
| 旧版本 | :x: |
|
||||
|
||||
## 报告漏洞
|
||||
|
||||
**请勿通过公开的 GitHub Issues 报告安全漏洞。**
|
||||
|
||||
如果您在 OpenList 中发现安全漏洞,请通过以下渠道之一负责任地进行报告:
|
||||
|
||||
- **GitHub 私密安全公告**(推荐):[点击提交](https://github.com/OpenListTeam/OpenList/security/advisories/new)
|
||||
- **Telegram**:通过 [@OpenListTeam](https://t.me/OpenListTeam) 私信联系维护者
|
||||
|
||||
报告时,请尽量提供以下信息:
|
||||
|
||||
- 漏洞描述及其潜在影响
|
||||
- 受影响的版本
|
||||
- 复现问题的详细步骤
|
||||
- 概念验证代码或截图(如有)
|
||||
- 建议的缓解措施或修复方案(可选,但非常欢迎)
|
||||
|
||||
## 用户安全最佳实践
|
||||
|
||||
为保障您的 OpenList 实例安全:
|
||||
|
||||
- 始终更新至最新版本。
|
||||
- 使用强且唯一的管理员密码,并在首次登录后立即修改。
|
||||
- 为您的部署启用 HTTPS(TLS)—— **请勿**在公网上以明文 HTTP 方式暴露 OpenList。
|
||||
- 使用反向代理(如 Nginx、Caddy)限制对外暴露的端口。
|
||||
- 配置访问控制,非必要情况下不要开启访客访问。
|
||||
- 定期检查已挂载存储的权限,并撤销未使用的 API 令牌。
|
||||
- 使用 Docker 部署时,尽可能避免以 root 用户运行容器。
|
||||
|
||||
## 致谢
|
||||
|
||||
我们衷心感谢所有负责任地披露漏洞、帮助 OpenList 变得更加安全的安全研究人员和社区成员。
|
||||
@@ -0,0 +1,11 @@
|
||||
version: v1
|
||||
plugins:
|
||||
- plugin: buf.build/protocolbuffers/go:v1.36.7
|
||||
out: .
|
||||
opt:
|
||||
- paths=source_relative
|
||||
- plugin: buf.build/grpc/go:v1.5.1
|
||||
out: .
|
||||
opt:
|
||||
- paths=source_relative
|
||||
- require_unimplemented_servers=false
|
||||
+29
-30
@@ -1,43 +1,42 @@
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"context"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
|
||||
"github.com/OpenListTeam/OpenList/v5/internal/bootstrap"
|
||||
"github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func Init() {
|
||||
bootstrap.Init()
|
||||
func Init(ctx context.Context) {
|
||||
if flags.Dev {
|
||||
flags.Debug = true
|
||||
}
|
||||
initLogrus()
|
||||
bootstrap.InitConfig()
|
||||
bootstrap.InitDriverPlugins()
|
||||
}
|
||||
|
||||
func Release() {
|
||||
bootstrap.Release()
|
||||
|
||||
}
|
||||
|
||||
var pid = -1
|
||||
var pidFile string
|
||||
|
||||
func initDaemon() {
|
||||
ex, err := os.Executable()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
exPath := filepath.Dir(ex)
|
||||
_ = os.MkdirAll(filepath.Join(exPath, "daemon"), 0700)
|
||||
pidFile = filepath.Join(exPath, "daemon/pid")
|
||||
if utils.Exists(pidFile) {
|
||||
bytes, err := os.ReadFile(pidFile)
|
||||
if err != nil {
|
||||
log.Fatal("failed to read pid file", err)
|
||||
}
|
||||
id, err := strconv.Atoi(string(bytes))
|
||||
if err != nil {
|
||||
log.Fatal("failed to parse pid data", err)
|
||||
}
|
||||
pid = id
|
||||
func initLog(l *logrus.Logger) {
|
||||
if flags.Debug {
|
||||
l.SetLevel(logrus.DebugLevel)
|
||||
l.SetReportCaller(true)
|
||||
} else {
|
||||
l.SetLevel(logrus.InfoLevel)
|
||||
l.SetReportCaller(false)
|
||||
}
|
||||
}
|
||||
func initLogrus() {
|
||||
formatter := logrus.TextFormatter{
|
||||
ForceColors: true,
|
||||
EnvironmentOverrideColors: true,
|
||||
TimestampFormat: "2006-01-02 15:04:05",
|
||||
FullTimestamp: true,
|
||||
}
|
||||
logrus.SetFormatter(&formatter)
|
||||
initLog(logrus.StandardLogger())
|
||||
}
|
||||
|
||||
+31
-2
@@ -1,11 +1,40 @@
|
||||
package flags
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var (
|
||||
DataDir string
|
||||
ConfigPath string
|
||||
ConfigFile string
|
||||
Debug bool
|
||||
NoPrefix bool
|
||||
Dev bool
|
||||
ForceBinDir bool
|
||||
LogStd bool
|
||||
|
||||
pwd string
|
||||
)
|
||||
|
||||
// Program working directory
|
||||
func PWD() string {
|
||||
if pwd != "" {
|
||||
return pwd
|
||||
}
|
||||
if ForceBinDir {
|
||||
ex, err := os.Executable()
|
||||
if err != nil {
|
||||
logrus.Fatal(err)
|
||||
}
|
||||
pwd = filepath.Dir(ex)
|
||||
return pwd
|
||||
}
|
||||
d, err := os.Getwd()
|
||||
if err != nil {
|
||||
logrus.Fatal(err)
|
||||
}
|
||||
pwd = d
|
||||
return d
|
||||
}
|
||||
|
||||
+4
-8
@@ -4,10 +4,7 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/cmd/flags"
|
||||
_ "github.com/OpenListTeam/OpenList/v4/drivers"
|
||||
_ "github.com/OpenListTeam/OpenList/v4/internal/archive"
|
||||
_ "github.com/OpenListTeam/OpenList/v4/internal/offline_download"
|
||||
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
@@ -27,11 +24,10 @@ func Execute() {
|
||||
}
|
||||
|
||||
func init() {
|
||||
RootCmd.PersistentFlags().StringVar(&flags.DataDir, "data", "data", "data directory (relative paths are resolved against the current working directory)")
|
||||
RootCmd.PersistentFlags().StringVar(&flags.ConfigPath, "config", "", "path to config.json (relative to current working directory; defaults to [data directory]/config.json, where [data directory] is set by --data)")
|
||||
RootCmd.PersistentFlags().StringVarP(&flags.ConfigFile, "config", "c", "data/config.json", "config file")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.Debug, "debug", false, "start with debug mode")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.NoPrefix, "no-prefix", false, "disable env prefix")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.Dev, "dev", false, "start with dev mode")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.ForceBinDir, "force-bin-dir", false, "Force to use the directory where the binary file is located as data directory")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.LogStd, "log-std", false, "Force to log to std")
|
||||
RootCmd.PersistentFlags().BoolVarP(&flags.ForceBinDir, "force-bin-dir", "f", false, "force to use the directory where the binary file is located as data directory")
|
||||
RootCmd.PersistentFlags().BoolVar(&flags.LogStd, "log-std", false, "force to log to std")
|
||||
}
|
||||
|
||||
+125
-19
@@ -1,13 +1,26 @@
|
||||
package cmd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/bootstrap"
|
||||
"github.com/OpenListTeam/OpenList/v5/cmd/flags"
|
||||
"github.com/OpenListTeam/OpenList/v5/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v5/server"
|
||||
"github.com/gin-gonic/gin"
|
||||
log "github.com/sirupsen/logrus"
|
||||
"github.com/spf13/cobra"
|
||||
"golang.org/x/net/http2"
|
||||
"golang.org/x/net/http2/h2c"
|
||||
)
|
||||
|
||||
// ServerCmd represents the server command
|
||||
@@ -16,34 +29,127 @@ var ServerCmd = &cobra.Command{
|
||||
Short: "Start the server at the specified address",
|
||||
Long: `Start the server at the specified address
|
||||
the address is defined in config file`,
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
bootstrap.Init()
|
||||
defer bootstrap.Release()
|
||||
bootstrap.Start()
|
||||
// Wait for interrupt signal to gracefully shutdown the server with
|
||||
// a timeout of 1 second.
|
||||
Run: func(_ *cobra.Command, args []string) {
|
||||
serverCtx, serverCancel := context.WithCancel(context.Background())
|
||||
defer serverCancel()
|
||||
Init(serverCtx)
|
||||
|
||||
if !flags.Debug {
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
}
|
||||
r := gin.New()
|
||||
r.Use(gin.LoggerWithWriter(log.StandardLogger().Out))
|
||||
r.Use(gin.RecoveryWithWriter(log.StandardLogger().Out))
|
||||
server.Init(r)
|
||||
|
||||
var httpHandler http.Handler = r
|
||||
if conf.Conf.Scheme.EnableH2c {
|
||||
httpHandler = h2c.NewHandler(r, &http2.Server{})
|
||||
}
|
||||
var httpSrv, httpsSrv, unixSrv *http.Server
|
||||
if conf.Conf.Scheme.HttpPort > 0 {
|
||||
httpBase := fmt.Sprintf("%s:%d", conf.Conf.Scheme.Address, conf.Conf.Scheme.HttpPort)
|
||||
log.Infoln("start HTTP server", "@", httpBase)
|
||||
httpSrv = &http.Server{Addr: httpBase, Handler: httpHandler}
|
||||
go func() {
|
||||
err := httpSrv.ListenAndServe()
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Errorln("start HTTP server", ":", err)
|
||||
serverCancel()
|
||||
}
|
||||
}()
|
||||
}
|
||||
if conf.Conf.Scheme.HttpsPort > 0 {
|
||||
httpsBase := fmt.Sprintf("%s:%d", conf.Conf.Scheme.Address, conf.Conf.Scheme.HttpsPort)
|
||||
log.Infoln("start HTTPS server", "@", httpsBase)
|
||||
httpsSrv = &http.Server{Addr: httpsBase, Handler: r}
|
||||
go func() {
|
||||
err := httpsSrv.ListenAndServeTLS(conf.Conf.Scheme.CertFile, conf.Conf.Scheme.KeyFile)
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Errorln("start HTTPS server", ":", err)
|
||||
serverCancel()
|
||||
}
|
||||
}()
|
||||
}
|
||||
if conf.Conf.Scheme.UnixFile != "" {
|
||||
log.Infoln("start Unix server", "@", conf.Conf.Scheme.UnixFile)
|
||||
unixSrv = &http.Server{Handler: httpHandler}
|
||||
go func() {
|
||||
listener, err := net.Listen("unix", conf.Conf.Scheme.UnixFile)
|
||||
if err != nil {
|
||||
log.Errorln("start Unix server", ":", err)
|
||||
serverCancel()
|
||||
return
|
||||
}
|
||||
|
||||
mode, err := strconv.ParseUint(conf.Conf.Scheme.UnixFilePerm, 8, 32)
|
||||
if err != nil {
|
||||
log.Errorln("parse unix_file_perm", ":", err)
|
||||
} else {
|
||||
err = os.Chmod(conf.Conf.Scheme.UnixFile, os.FileMode(mode))
|
||||
if err != nil {
|
||||
log.Errorln("chmod socket file", ":", err)
|
||||
}
|
||||
}
|
||||
|
||||
err = unixSrv.Serve(listener)
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Errorln("start Unix server", ":", err)
|
||||
serverCancel()
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
quit := make(chan os.Signal, 1)
|
||||
// kill (no param) default send syscanll.SIGTERM
|
||||
// kill -2 is syscall.SIGINT
|
||||
// kill -9 is syscall. SIGKILL but can"t be catch, so don't need add it
|
||||
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
|
||||
<-quit
|
||||
bootstrap.Shutdown(1 * time.Second)
|
||||
select {
|
||||
case <-quit:
|
||||
case <-serverCtx.Done():
|
||||
}
|
||||
|
||||
log.Println("shutdown server...")
|
||||
Release()
|
||||
|
||||
quitCtx, quitCancel := context.WithTimeout(context.Background(), time.Second)
|
||||
defer quitCancel()
|
||||
var wg sync.WaitGroup
|
||||
if httpSrv != nil {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := httpSrv.Shutdown(quitCtx); err != nil {
|
||||
log.Errorln("shutdown HTTP server", ":", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
if httpsSrv != nil {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := httpsSrv.Shutdown(quitCtx); err != nil {
|
||||
log.Errorln("shutdown HTTPS server", ":", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
if unixSrv != nil {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := unixSrv.Shutdown(quitCtx); err != nil {
|
||||
log.Errorln("shutdown Unix server", ":", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
log.Println("server exit")
|
||||
},
|
||||
}
|
||||
|
||||
func init() {
|
||||
RootCmd.AddCommand(ServerCmd)
|
||||
|
||||
// Here you will define your flags and configuration settings.
|
||||
|
||||
// Cobra supports Persistent Flags which will work for this command
|
||||
// and all subcommands, e.g.:
|
||||
// serverCmd.PersistentFlags().String("foo", "", "A help for foo")
|
||||
|
||||
// Cobra supports local flags which will only run when this command
|
||||
// is called directly, e.g.:
|
||||
// serverCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
|
||||
}
|
||||
|
||||
// OutOpenListInit 暴露用于外部启动server的函数
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
package _115
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
driver115 "github.com/SheltonZhu/115driver/pkg/driver"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var (
|
||||
md5Salt = "Qclm8MGWUv59TnrR0XPg"
|
||||
appVer = "35.6.0.3"
|
||||
)
|
||||
|
||||
func (d *Pan115) getAppVersion() (string, error) {
|
||||
result := VersionResp{}
|
||||
res, err := base.RestyClient.R().Get(driver115.ApiGetVersion)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
err = utils.Json.Unmarshal(res.Body(), &result)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(result.Error) > 0 {
|
||||
return "", errors.New(result.Error)
|
||||
}
|
||||
return result.Data.Win.Version, nil
|
||||
}
|
||||
|
||||
func (d *Pan115) getAppVer() string {
|
||||
ver, err := d.getAppVersion()
|
||||
if err != nil {
|
||||
log.Warnf("[115] get app version failed: %v", err)
|
||||
return appVer
|
||||
}
|
||||
if len(ver) > 0 {
|
||||
return ver
|
||||
}
|
||||
return appVer
|
||||
}
|
||||
|
||||
func (d *Pan115) initAppVer() {
|
||||
appVer = d.getAppVer()
|
||||
log.Debugf("use app version: %v", appVer)
|
||||
}
|
||||
|
||||
type VersionResp struct {
|
||||
Error string `json:"error,omitempty"`
|
||||
Data Versions `json:"data"`
|
||||
}
|
||||
|
||||
type Versions struct {
|
||||
Win Version `json:"win"`
|
||||
}
|
||||
|
||||
type Version struct {
|
||||
Version string `json:"version_code"`
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
package _115_open
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
func ParseInt64(v json.Number) (int64, error) {
|
||||
i, err := v.Int64()
|
||||
if err == nil {
|
||||
return i, nil
|
||||
}
|
||||
f, e1 := v.Float64()
|
||||
if e1 == nil {
|
||||
return int64(f), nil
|
||||
}
|
||||
return int64(0), err
|
||||
}
|
||||
@@ -1,129 +0,0 @@
|
||||
package _123_open
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
var (
|
||||
AccessToken = "https://open-api.123pan.com/api/v1/access_token"
|
||||
)
|
||||
|
||||
func expiresInToExpiredAt(expiresIn int64) (time.Time, error) {
|
||||
if expiresIn <= 0 {
|
||||
return time.Time{}, errors.New("invalid expires_in from official API")
|
||||
}
|
||||
return time.Now().UTC().Add(time.Duration(expiresIn) * time.Second), nil
|
||||
}
|
||||
|
||||
type tokenManager struct {
|
||||
// accessToken string
|
||||
expiredAt time.Time
|
||||
mu sync.Mutex
|
||||
blockRefresh bool
|
||||
}
|
||||
|
||||
func (d *Open123) getAccessToken(forceRefresh bool) (string, error) {
|
||||
tm := d.tm
|
||||
tm.mu.Lock()
|
||||
defer tm.mu.Unlock()
|
||||
if tm.blockRefresh {
|
||||
return "", errors.New("Authentication expired")
|
||||
}
|
||||
if !forceRefresh && d.AccessToken != "" && time.Now().Before(tm.expiredAt.Add(-5*time.Minute)) {
|
||||
return d.AccessToken, nil
|
||||
}
|
||||
if err := d.flushAccessToken(); err != nil {
|
||||
// token expired and failed to refresh, block further refresh attempts
|
||||
tm.blockRefresh = true
|
||||
return "", err
|
||||
}
|
||||
return d.AccessToken, nil
|
||||
}
|
||||
|
||||
func (d *Open123) flushAccessToken() error {
|
||||
// Official app renewapi response contains access_token, refresh_token and expires_in.
|
||||
if d.UseOnlineAPI && d.RefreshToken != "" && len(d.APIAddress) > 0 {
|
||||
var resp RefreshTokenResp
|
||||
_, err := base.RestyClient.R().
|
||||
SetResult(&resp).
|
||||
SetQueryParams(map[string]string{
|
||||
"refresh_ui": d.RefreshToken,
|
||||
"server_use": "true",
|
||||
"driver_txt": "123cloud_oa",
|
||||
}).
|
||||
Get(d.APIAddress)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if resp.AccessToken == "" || resp.RefreshToken == "" {
|
||||
errMessage := resp.ErrorDescription
|
||||
if errMessage == "" {
|
||||
errMessage = resp.Text
|
||||
}
|
||||
if errMessage == "" {
|
||||
errMessage = resp.Message
|
||||
}
|
||||
if errMessage == "" {
|
||||
errMessage = resp.Error
|
||||
}
|
||||
if errMessage != "" {
|
||||
return fmt.Errorf("failed to refresh token: %s", errMessage)
|
||||
}
|
||||
return fmt.Errorf("empty access_token or refresh_token returned from official API")
|
||||
}
|
||||
expiredAt, err := expiresInToExpiredAt(resp.ExpiresIn)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
d.AccessToken = resp.AccessToken
|
||||
d.RefreshToken = resp.RefreshToken
|
||||
d.tm.expiredAt = expiredAt
|
||||
op.MustSaveDriverStorage(d)
|
||||
d.tm.blockRefresh = false
|
||||
return nil
|
||||
}
|
||||
|
||||
// Developer API response contains code/message/data(accessToken, expiredAt).
|
||||
if d.ClientID != "" && d.ClientSecret != "" {
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeaders(map[string]string{
|
||||
"platform": "open_platform",
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
var resp AccessTokenResp
|
||||
req.SetBody(base.Json{
|
||||
"clientID": d.ClientID,
|
||||
"clientSecret": d.ClientSecret,
|
||||
})
|
||||
req.SetResult(&resp)
|
||||
_, err := req.Execute(http.MethodPost, AccessToken)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
return fmt.Errorf("get access token failed: %s", resp.Message)
|
||||
}
|
||||
if resp.Data.AccessToken == "" || resp.Data.ExpiredAt == "" {
|
||||
return errors.New("invalid token payload from developer API")
|
||||
}
|
||||
expiredAt, err := time.Parse(time.RFC3339, resp.Data.ExpiredAt)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse expire time failed: %w", err)
|
||||
}
|
||||
d.AccessToken = resp.Data.AccessToken
|
||||
d.tm.expiredAt = expiredAt.UTC()
|
||||
op.MustSaveDriverStorage(d)
|
||||
d.tm.blockRefresh = false
|
||||
return nil
|
||||
}
|
||||
return errors.New("no valid authentication method available")
|
||||
}
|
||||
@@ -1,278 +0,0 @@
|
||||
package _123_open
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/md5"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/go-resty/resty/v2"
|
||||
"github.com/google/uuid"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
var ( // 不同情况下获取的AccessTokenQPS限制不同 如下模块化易于拓展
|
||||
Api = "https://open-api.123pan.com"
|
||||
|
||||
UserInfo = InitApiInfo(Api+"/api/v1/user/info", 1)
|
||||
FileList = InitApiInfo(Api+"/api/v2/file/list", 3)
|
||||
DownloadInfo = InitApiInfo(Api+"/api/v1/file/download_info", 5)
|
||||
DirectLink = InitApiInfo(Api+"/api/v1/direct-link/url", 5)
|
||||
Mkdir = InitApiInfo(Api+"/upload/v1/file/mkdir", 2)
|
||||
Move = InitApiInfo(Api+"/api/v1/file/move", 1)
|
||||
Rename = InitApiInfo(Api+"/api/v1/file/name", 1)
|
||||
Trash = InitApiInfo(Api+"/api/v1/file/trash", 2)
|
||||
UploadCreate = InitApiInfo(Api+"/upload/v2/file/create", 2)
|
||||
UploadComplete = InitApiInfo(Api+"/upload/v2/file/upload_complete", 0)
|
||||
UploadSHA1Reuse = InitApiInfo(Api+"/upload/v2/file/sha1_reuse", 2)
|
||||
|
||||
OfflineDownload = InitApiInfo(Api+"/api/v1/offline/download", 1)
|
||||
OfflineDownloadProcess = InitApiInfo(Api+"/api/v1/offline/download/process", 5)
|
||||
)
|
||||
|
||||
func (d *Open123) Request(apiInfo *ApiInfo, method string, callback base.ReqCallback, resp interface{}) ([]byte, error) {
|
||||
for {
|
||||
token, err := d.getAccessToken(false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeaders(map[string]string{
|
||||
"authorization": "Bearer " + token,
|
||||
"platform": "open_platform",
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
if resp != nil {
|
||||
req.SetResult(resp)
|
||||
}
|
||||
|
||||
log.Debugf("API: %s, QPS: %d, NowLen: %d", apiInfo.url, apiInfo.qps, apiInfo.NowLen())
|
||||
|
||||
apiInfo.Require()
|
||||
defer apiInfo.Release()
|
||||
res, err := req.Execute(method, apiInfo.url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body := res.Body()
|
||||
|
||||
// 解析为通用响应
|
||||
var baseResp BaseResp
|
||||
if err = json.Unmarshal(body, &baseResp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if baseResp.Code == 0 {
|
||||
return body, nil
|
||||
} else if baseResp.Code == 401 {
|
||||
// 强制刷新Token, 有小概率会 race condition 导致多次刷新Token,但不影响正确运行
|
||||
if _, err := d.getAccessToken(true); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else if baseResp.Code == 429 {
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
log.Warningf("API: %s, QPS: %d, 请求太频繁,对应API提示过多请减小QPS", apiInfo.url, apiInfo.qps)
|
||||
} else {
|
||||
return nil, errors.New(baseResp.Message)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Open123) SignURL(originURL, privateKey string, uid uint64, validDuration time.Duration) (newURL string, err error) {
|
||||
// 生成Unix时间戳
|
||||
ts := time.Now().Add(validDuration).Unix()
|
||||
|
||||
// 生成随机数(建议使用UUID,不能包含中划线(-))
|
||||
rand := strings.ReplaceAll(uuid.New().String(), "-", "")
|
||||
|
||||
// 解析URL
|
||||
objURL, err := url.Parse(originURL)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// 待签名字符串,格式:path-timestamp-rand-uid-privateKey
|
||||
unsignedStr := fmt.Sprintf("%s-%d-%s-%d-%s", objURL.Path, ts, rand, uid, privateKey)
|
||||
md5Hash := md5.Sum([]byte(unsignedStr))
|
||||
// 生成鉴权参数,格式:timestamp-rand-uid-md5hash
|
||||
authKey := fmt.Sprintf("%d-%s-%d-%x", ts, rand, uid, md5Hash)
|
||||
|
||||
// 添加鉴权参数到URL查询参数
|
||||
v := objURL.Query()
|
||||
v.Add("auth_key", authKey)
|
||||
objURL.RawQuery = v.Encode()
|
||||
|
||||
return objURL.String(), nil
|
||||
}
|
||||
|
||||
func (d *Open123) getUserInfo(ctx context.Context) (*UserInfoResp, error) {
|
||||
var resp UserInfoResp
|
||||
|
||||
if _, err := d.Request(UserInfo, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getUID(ctx context.Context) (uint64, error) {
|
||||
if d.UID != 0 {
|
||||
return d.UID, nil
|
||||
}
|
||||
resp, err := d.getUserInfo(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
d.UID = resp.Data.UID
|
||||
return resp.Data.UID, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getFiles(parentFileId int64, limit int, lastFileId int64) (*FileListResp, error) {
|
||||
var resp FileListResp
|
||||
|
||||
_, err := d.Request(FileList, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(
|
||||
map[string]string{
|
||||
"parentFileId": strconv.FormatInt(parentFileId, 10),
|
||||
"limit": strconv.Itoa(limit),
|
||||
"lastFileId": strconv.FormatInt(lastFileId, 10),
|
||||
"trashed": "false",
|
||||
"searchMode": "",
|
||||
"searchData": "",
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getDownloadInfo(fileId int64) (*DownloadInfoResp, error) {
|
||||
var resp DownloadInfoResp
|
||||
|
||||
_, err := d.Request(DownloadInfo, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"fileId": strconv.FormatInt(fileId, 10),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) getDirectLink(fileId int64) (*DirectLinkResp, error) {
|
||||
var resp DirectLinkResp
|
||||
|
||||
_, err := d.Request(DirectLink, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"fileID": strconv.FormatInt(fileId, 10),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Open123) mkdir(parentID int64, name string) error {
|
||||
_, err := d.Request(Mkdir, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"parentID": strconv.FormatInt(parentID, 10),
|
||||
"name": name,
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) move(fileID, toParentFileID int64) error {
|
||||
_, err := d.Request(Move, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"fileIDs": []int64{fileID},
|
||||
"toParentFileID": toParentFileID,
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) rename(fileId int64, fileName string) error {
|
||||
_, err := d.Request(Rename, http.MethodPut, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"fileId": fileId,
|
||||
"fileName": fileName,
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) trash(fileId int64) error {
|
||||
_, err := d.Request(Trash, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(base.Json{
|
||||
"fileIDs": []int64{fileId},
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Open123) createOfflineDownloadTask(ctx context.Context, url string, dirID, callback string) (taskID int, err error) {
|
||||
body := base.Json{
|
||||
"url": url,
|
||||
"dirID": dirID,
|
||||
}
|
||||
if len(callback) > 0 {
|
||||
body["callBackUrl"] = callback
|
||||
}
|
||||
var resp OfflineDownloadResp
|
||||
_, err = d.Request(OfflineDownload, http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(body)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return resp.Data.TaskID, nil
|
||||
}
|
||||
|
||||
func (d *Open123) queryOfflineDownloadStatus(ctx context.Context, taskID int) (process float64, status int, err error) {
|
||||
var resp OfflineDownloadProcessResp
|
||||
_, err = d.Request(OfflineDownloadProcess, http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"taskID": strconv.Itoa(taskID),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return .0, 0, err
|
||||
}
|
||||
return resp.Data.Process, resp.Data.Status, nil
|
||||
}
|
||||
-1377
File diff suppressed because it is too large
Load Diff
@@ -1,149 +0,0 @@
|
||||
package _189
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
|
||||
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
|
||||
// 计算 sliceMD5
|
||||
sliceMD5 := fileMD5
|
||||
if len(sliceMD5s) > 1 {
|
||||
joined := strings.Join(sliceMD5s, "\n")
|
||||
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
|
||||
}
|
||||
|
||||
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
|
||||
t.Info.PieceLength = sliceSize
|
||||
t.SetPieces(pieceHashes)
|
||||
t.SetCASInfo(&torrent.CASInfo{
|
||||
FileMD5: fileMD5,
|
||||
SliceMD5: sliceMD5,
|
||||
SliceMD5s: sliceMD5s,
|
||||
SliceSize: sliceSize,
|
||||
Cloud: "189",
|
||||
})
|
||||
|
||||
return t.Encode()
|
||||
}
|
||||
|
||||
// RapidUploadFromTorrent 从 torrent 文件中提取 CAS 信息进行秒传
|
||||
func (d *Cloud189) RapidUploadFromTorrent(ctx context.Context, dstDir model.Obj, torrentData []byte) error {
|
||||
// 解析 torrent
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
// 检查是否包含 CAS 扩展信息
|
||||
if !t.HasCASInfo() {
|
||||
return fmt.Errorf("torrent 不包含 CAS 扩展信息,无法秒传")
|
||||
}
|
||||
|
||||
cas := t.CAS
|
||||
fileName := t.Info.Name
|
||||
fileSize := t.GetTotalSize()
|
||||
|
||||
// 获取 sessionKey
|
||||
sessionKey, err := d.getSessionKey()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.sessionKey = sessionKey
|
||||
|
||||
// 初始化上传
|
||||
res, err := d.uploadRequest("/person/initMultiUpload", map[string]string{
|
||||
"parentFolderId": dstDir.GetID(),
|
||||
"fileName": encode(fileName),
|
||||
"fileSize": fmt.Sprint(fileSize),
|
||||
"sliceSize": fmt.Sprint(cas.SliceSize),
|
||||
"lazyCheck": "1",
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("初始化上传失败: %w", err)
|
||||
}
|
||||
|
||||
uploadFileId := utils.Json.Get(res, "data", "uploadFileId").ToString()
|
||||
|
||||
// 提交上传(使用 CAS 信息秒传)
|
||||
_, err = d.uploadRequest("/person/commitMultiUploadFile", map[string]string{
|
||||
"uploadFileId": uploadFileId,
|
||||
"fileMd5": cas.FileMD5,
|
||||
"sliceMd5": cas.SliceMD5,
|
||||
"lazyCheck": "1",
|
||||
"opertype": "3",
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("秒传提交失败: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
|
||||
func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64, sliceSize int64) ([]byte, error) {
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = torrent.DefaultPieceSize
|
||||
}
|
||||
|
||||
hw := torrent.NewHashWriter(sliceSize, sliceSize)
|
||||
|
||||
buf := make([]byte, 32*1024)
|
||||
for {
|
||||
n, err := reader.Read(buf)
|
||||
if n > 0 {
|
||||
hw.Write(buf[:n])
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
hw.Finish()
|
||||
|
||||
fileMD5 := hw.GetFileMD5()
|
||||
sliceMD5s := hw.GetSliceMD5s()
|
||||
pieceHashes := hw.GetPieceHashes()
|
||||
|
||||
return GenerateTorrent(fileName, fileSize, fileMD5, sliceMD5s, sliceSize, pieceHashes)
|
||||
}
|
||||
|
||||
// ComputePieceSHA1 计算单个分片的 SHA-1 哈希
|
||||
func ComputePieceSHA1(data []byte) []byte {
|
||||
h := sha1.Sum(data)
|
||||
return h[:]
|
||||
}
|
||||
|
||||
// ExtractCASFromTorrent 从 torrent 数据中提取 CAS 信息
|
||||
func ExtractCASFromTorrent(torrentData []byte) (*torrent.CASInfo, string, int64, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, "", 0, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
if !t.HasCASInfo() {
|
||||
return nil, "", 0, fmt.Errorf("torrent 不包含 CAS 扩展信息")
|
||||
}
|
||||
|
||||
return t.CAS, t.Info.Name, t.GetTotalSize(), nil
|
||||
}
|
||||
|
||||
// GetInfoHashHex 获取 torrent 的 info_hash(十六进制字符串)
|
||||
func GetInfoHashHex(torrentData []byte) (string, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(t.InfoHash), nil
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
package _189pc
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
LoginType string `json:"login_type" type:"select" options:"password,qrcode" default:"password" required:"true"`
|
||||
Username string `json:"username" required:"true"`
|
||||
Password string `json:"password" required:"true"`
|
||||
VCode string `json:"validate_code"`
|
||||
AccessToken string `json:"access_token" required:"false"`
|
||||
RefreshToken string `json:"refresh_token" help:"To switch accounts, please clear this field"`
|
||||
driver.RootID
|
||||
OrderBy string `json:"order_by" type:"select" options:"filename,filesize,lastOpTime" default:"filename"`
|
||||
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc" default:"asc"`
|
||||
Type string `json:"type" type:"select" options:"personal,family" default:"personal"`
|
||||
FamilyID string `json:"family_id"`
|
||||
UploadMethod string `json:"upload_method" type:"select" options:"stream,rapid,old" default:"stream"`
|
||||
UploadThread string `json:"upload_thread" default:"3" help:"1<=thread<=32"`
|
||||
FamilyTransfer bool `json:"family_transfer"`
|
||||
RapidUpload bool `json:"rapid_upload"`
|
||||
NoUseOcr bool `json:"no_use_ocr"`
|
||||
GenerateTorrent bool `json:"generate_torrent" help:"Generate torrent file with CAS extension after upload"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "189CloudPC",
|
||||
DefaultRoot: "-11",
|
||||
CheckStatus: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Cloud189PC{}
|
||||
})
|
||||
}
|
||||
@@ -1,449 +0,0 @@
|
||||
package _189pc
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-resty/resty/v2"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/torrent"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
// GenerateTorrent 根据上传过程中收集的哈希信息生成包含 CAS 扩展的 torrent 文件
|
||||
// fileMD5: 整文件 MD5(大写十六进制)
|
||||
// sliceMD5s: 每个分片的 MD5 列表(大写十六进制)
|
||||
// sliceSize: 分片大小
|
||||
// pieceHashes: SHA-1 piece hashes 拼接(每 20 字节一个)
|
||||
// fileName: 文件名
|
||||
// fileSize: 文件大小
|
||||
func GenerateTorrent(fileName string, fileSize int64, fileMD5 string, sliceMD5s []string, sliceSize int64, pieceHashes []byte) ([]byte, error) {
|
||||
// 计算 sliceMD5
|
||||
sliceMD5 := fileMD5
|
||||
if len(sliceMD5s) > 1 {
|
||||
joined := strings.Join(sliceMD5s, "\n")
|
||||
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
|
||||
}
|
||||
|
||||
t := torrent.NewTorrent(fileName, fileSize, fileMD5)
|
||||
t.Info.PieceLength = sliceSize
|
||||
t.SetPieces(pieceHashes)
|
||||
t.SetCASInfo(&torrent.CASInfo{
|
||||
FileMD5: fileMD5,
|
||||
SliceMD5: sliceMD5,
|
||||
SliceMD5s: sliceMD5s,
|
||||
SliceSize: sliceSize,
|
||||
Cloud: "189",
|
||||
})
|
||||
|
||||
return t.Encode()
|
||||
}
|
||||
|
||||
// RapidUploadFromTorrent 从 torrent 文件中提取 CAS 信息进行秒传
|
||||
// 返回值:上传成功的文件对象、错误
|
||||
func (y *Cloud189PC) RapidUploadFromTorrent(ctx context.Context, dstDir model.Obj, torrentData []byte, overwrite bool) (model.Obj, error) {
|
||||
isFamily := y.isFamily()
|
||||
|
||||
// 解析 torrent
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
// 检查是否包含 CAS 扩展信息
|
||||
if !t.HasCASInfo() {
|
||||
return nil, fmt.Errorf("torrent 不包含 CAS 扩展信息,无法秒传")
|
||||
}
|
||||
|
||||
cas := t.CAS
|
||||
fileName := t.Info.Name
|
||||
fileSize := t.GetTotalSize()
|
||||
|
||||
// 统一 MD5 为大写(与正常上传保持一致,天翼云盘要求大写)
|
||||
fileMD5Upper := strings.ToUpper(cas.FileMD5)
|
||||
|
||||
// 优先使用 torrent 中嵌入的分片大小,与生成时保持一致
|
||||
sliceSize := cas.SliceSize
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = partSize(fileSize)
|
||||
}
|
||||
|
||||
// 计算 sliceMd5(与上传时一致的算法)
|
||||
// 优先使用 torrent 中已有的 SliceMD5;仅当有多分片列表时才重新计算
|
||||
sliceMd5Hex := strings.ToUpper(cas.SliceMD5)
|
||||
if sliceMd5Hex == "" {
|
||||
sliceMd5Hex = fileMD5Upper
|
||||
}
|
||||
if len(cas.SliceMD5s) > 1 {
|
||||
// 分片 MD5 也需要统一大写后再拼接计算
|
||||
upperSliceMD5s := make([]string, len(cas.SliceMD5s))
|
||||
for i, s := range cas.SliceMD5s {
|
||||
upperSliceMD5s[i] = strings.ToUpper(s)
|
||||
}
|
||||
sliceMd5Hex = strings.ToUpper(utils.GetMD5EncodeStr(strings.Join(upperSliceMD5s, "\n")))
|
||||
}
|
||||
|
||||
// 使用与 Web 端一致的三步秒传流程
|
||||
fullUrl := "https://upload.cloud.189.cn"
|
||||
if isFamily {
|
||||
fullUrl += "/family"
|
||||
} else {
|
||||
fullUrl += "/person"
|
||||
}
|
||||
|
||||
// Step 1: initMultiUpload(不传 fileMd5/sliceMd5,只传 lazyCheck)
|
||||
initParams := Params{
|
||||
"parentFolderId": dstDir.GetID(),
|
||||
"fileName": url.QueryEscape(fileName),
|
||||
"fileSize": fmt.Sprint(fileSize),
|
||||
"sliceSize": fmt.Sprint(sliceSize),
|
||||
"lazyCheck": "1",
|
||||
}
|
||||
if isFamily {
|
||||
initParams.Set("familyId", y.FamilyID)
|
||||
}
|
||||
|
||||
var uploadInfo InitMultiUploadResp
|
||||
_, err = y.request(fullUrl+"/initMultiUpload", "GET", func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, initParams, &uploadInfo, isFamily)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initMultiUpload 失败: %w", err)
|
||||
}
|
||||
|
||||
uploadFileId := uploadInfo.Data.UploadFileID
|
||||
|
||||
// Step 2: checkTransSecond(用 fileMd5 + sliceMd5 + uploadFileId 检查秒传)
|
||||
checkParams := Params{
|
||||
"fileMd5": fileMD5Upper,
|
||||
"sliceMd5": sliceMd5Hex,
|
||||
"uploadFileId": uploadFileId,
|
||||
}
|
||||
|
||||
var checkResp struct {
|
||||
Data struct {
|
||||
FileDataExists int `json:"fileDataExists"`
|
||||
} `json:"data"`
|
||||
}
|
||||
_, err = y.request(fullUrl+"/checkTransSecond", "GET", func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, checkParams, &checkResp, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Errorf("[RapidUpload] checkTransSecond 失败: uploadFileId=%s, err=%v", uploadFileId, err)
|
||||
return nil, fmt.Errorf("秒传检查失败: %w", err)
|
||||
}
|
||||
|
||||
if checkResp.Data.FileDataExists != 1 {
|
||||
return nil, fmt.Errorf("秒传失败:云端不存在该文件(fileMD5=%s, sliceMD5=%s, size=%d)", fileMD5Upper, sliceMd5Hex, fileSize)
|
||||
}
|
||||
|
||||
// Step 3: commitMultiUploadFile(传 fileMd5 + sliceMd5)
|
||||
|
||||
var resp CommitMultiUploadFileResp
|
||||
commitParams := Params{
|
||||
"uploadFileId": uploadFileId,
|
||||
"fileMd5": fileMD5Upper,
|
||||
"sliceMd5": sliceMd5Hex,
|
||||
"lazyCheck": "1",
|
||||
"opertype": IF(overwrite, "3", "1"),
|
||||
}
|
||||
|
||||
_, err = y.request(fullUrl+"/commitMultiUploadFile", "GET", func(req *resty.Request) {
|
||||
req.SetContext(ctx)
|
||||
}, commitParams, &resp, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Errorf("[RapidUpload] commitMultiUploadFile 失败: uploadFileId=%s, err=%v", uploadFileId, err)
|
||||
return nil, fmt.Errorf("提交上传失败: %w", err)
|
||||
}
|
||||
|
||||
// 秒传成功后,将 torrent 文件上传到目标目录(异步,不影响秒传结果)
|
||||
if y.Addition.GenerateTorrent {
|
||||
capturedDstDir := dstDir
|
||||
capturedIsFamily := isFamily
|
||||
go func() {
|
||||
torrentName := fileName + ".cas.torrent"
|
||||
infoHash, _ := GetInfoHashHex(torrentData)
|
||||
utils.Log.Infof("秒传成功,上传 torrent: %s (info_hash: %s, size: %d bytes)",
|
||||
torrentName, infoHash, len(torrentData))
|
||||
|
||||
torrentFileStream := &stream.FileStream{
|
||||
Ctx: context.Background(),
|
||||
Obj: &model.Object{
|
||||
Name: torrentName,
|
||||
Size: int64(len(torrentData)),
|
||||
IsFolder: false,
|
||||
},
|
||||
Reader: bytes.NewReader(torrentData),
|
||||
Mimetype: "application/x-bittorrent",
|
||||
}
|
||||
_, uploadErr := y.fastUpload(context.Background(), capturedDstDir, torrentFileStream, func(p float64) {}, capturedIsFamily, false, false)
|
||||
if uploadErr != nil {
|
||||
utils.Log.Warnf("上传 torrent 文件失败: %v", uploadErr)
|
||||
} else {
|
||||
utils.Log.Infof("torrent 文件已上传: %s", torrentName)
|
||||
op.Cache.DeleteDirectory(y, capturedDstDir.GetPath())
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
return resp.toFile(), nil
|
||||
}
|
||||
|
||||
// ComputeTorrentFromReader 从 io.Reader 计算并生成 torrent 文件
|
||||
// 适用于:已有文件需要生成 torrent 的场景(如下载完成后生成)
|
||||
func ComputeTorrentFromReader(reader io.Reader, fileName string, fileSize int64, sliceSize int64) ([]byte, error) {
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = torrent.DefaultPieceSize
|
||||
}
|
||||
|
||||
hw := torrent.NewHashWriter(sliceSize, sliceSize)
|
||||
|
||||
buf := make([]byte, 32*1024)
|
||||
for {
|
||||
n, err := reader.Read(buf)
|
||||
if n > 0 {
|
||||
hw.Write(buf[:n])
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
hw.Finish()
|
||||
|
||||
fileMD5 := hw.GetFileMD5()
|
||||
sliceMD5s := hw.GetSliceMD5s()
|
||||
pieceHashes := hw.GetPieceHashes()
|
||||
|
||||
return GenerateTorrent(fileName, fileSize, fileMD5, sliceMD5s, sliceSize, pieceHashes)
|
||||
}
|
||||
|
||||
// ComputePieceSHA1 计算单个分片的 SHA-1 哈希
|
||||
func ComputePieceSHA1(data []byte) []byte {
|
||||
h := sha1.Sum(data)
|
||||
return h[:]
|
||||
}
|
||||
|
||||
// ExtractCASFromTorrent 从 torrent 数据中提取 CAS 信息
|
||||
// 返回:CAS 信息、文件名、文件大小、错误
|
||||
func ExtractCASFromTorrent(torrentData []byte) (*torrent.CASInfo, string, int64, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, "", 0, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
if !t.HasCASInfo() {
|
||||
return nil, "", 0, fmt.Errorf("torrent 不包含 CAS 扩展信息")
|
||||
}
|
||||
|
||||
return t.CAS, t.Info.Name, t.GetTotalSize(), nil
|
||||
}
|
||||
|
||||
// InjectCASIntoTorrent 向已有的 torrent 文件注入 CAS 扩展信息
|
||||
// 用于:下载完成后,计算了 MD5 信息,写回到 torrent 中
|
||||
func InjectCASIntoTorrent(torrentData []byte, fileMD5 string, sliceMD5s []string, sliceSize int64) ([]byte, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("解析 torrent 失败: %w", err)
|
||||
}
|
||||
|
||||
// 计算 sliceMD5
|
||||
sliceMD5 := fileMD5
|
||||
if len(sliceMD5s) > 1 {
|
||||
joined := strings.Join(sliceMD5s, "\n")
|
||||
sliceMD5 = strings.ToUpper(torrent.GetMD5Str(joined))
|
||||
}
|
||||
|
||||
// 注入 CAS 信息
|
||||
t.SetCASInfo(&torrent.CASInfo{
|
||||
FileMD5: fileMD5,
|
||||
SliceMD5: sliceMD5,
|
||||
SliceMD5s: sliceMD5s,
|
||||
SliceSize: sliceSize,
|
||||
Cloud: "189",
|
||||
})
|
||||
|
||||
// 同时更新 info 中的 md5sum 字段
|
||||
if t.Info.MD5Sum == "" {
|
||||
t.Info.MD5Sum = fileMD5
|
||||
}
|
||||
|
||||
return t.Encode()
|
||||
}
|
||||
|
||||
// GetInfoHashHex 获取 torrent 的 info_hash(十六进制字符串)
|
||||
func GetInfoHashHex(torrentData []byte) (string, error) {
|
||||
t, err := torrent.Decode(torrentData)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(t.InfoHash), nil
|
||||
}
|
||||
|
||||
func isCASTorrentFile(fileName string) bool {
|
||||
return strings.HasSuffix(fileName, ".cas.torrent")
|
||||
}
|
||||
|
||||
// ComputeSliceMD5sFromReader 从 reader 中计算每个 10MB 分片的 MD5
|
||||
// 返回:整文件 MD5、分片 MD5 列表
|
||||
func ComputeSliceMD5sFromReader(reader io.Reader, sliceSize int64) (string, []string, error) {
|
||||
if sliceSize <= 0 {
|
||||
sliceSize = torrent.DefaultPieceSize
|
||||
}
|
||||
|
||||
fileMD5Hash := utils.MD5.NewFunc()
|
||||
sliceMD5s := make([]string, 0)
|
||||
|
||||
buf := make([]byte, sliceSize)
|
||||
for {
|
||||
n, err := io.ReadFull(reader, buf)
|
||||
if n > 0 {
|
||||
chunk := buf[:n]
|
||||
fileMD5Hash.Write(chunk)
|
||||
// 计算该分片的 MD5
|
||||
sliceMD5 := strings.ToUpper(utils.HashData(utils.MD5, chunk))
|
||||
sliceMD5s = append(sliceMD5s, sliceMD5)
|
||||
}
|
||||
if err == io.EOF || err == io.ErrUnexpectedEOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
}
|
||||
|
||||
fileMD5Hex := strings.ToUpper(hex.EncodeToString(fileMD5Hash.Sum(nil)))
|
||||
return fileMD5Hex, sliceMD5s, nil
|
||||
}
|
||||
|
||||
// torrentFollowCopy 跟随复制 torrent 文件(异步,不影响主操作)
|
||||
// srcFolderId: 源文件所在目录 ID
|
||||
// srcFileName: 源文件名
|
||||
// dstDir: 目标目录
|
||||
func (y *Cloud189PC) torrentFollowCopy(srcFolderId string, srcFileName string, dstDir model.Obj) {
|
||||
if !y.Addition.GenerateTorrent {
|
||||
return
|
||||
}
|
||||
if srcFolderId == "" {
|
||||
return
|
||||
}
|
||||
torrentName := srcFileName + ".cas.torrent"
|
||||
isFamily := y.isFamily()
|
||||
|
||||
go func() {
|
||||
torrentFile, err := y.findFileByName(context.Background(), torrentName, srcFolderId, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Debugf("查找 torrent 文件失败(跟随复制): %v", err)
|
||||
return
|
||||
}
|
||||
// 复制 torrent 文件到目标目录
|
||||
resp, copyErr := y.CreateBatchTask("COPY", IF(isFamily, y.FamilyID, ""), dstDir.GetID(),
|
||||
map[string]string{"targetFileName": dstDir.GetName()},
|
||||
BatchTaskInfo{
|
||||
FileId: torrentFile.GetID(),
|
||||
FileName: torrentFile.GetName(),
|
||||
IsFolder: 0,
|
||||
})
|
||||
if copyErr != nil {
|
||||
utils.Log.Warnf("跟随复制 torrent 文件失败: %v", copyErr)
|
||||
return
|
||||
}
|
||||
if err = y.WaitBatchTask("COPY", resp.TaskID, time.Second); err != nil {
|
||||
utils.Log.Warnf("等待跟随复制 torrent 文件失败: %v", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// torrentFollowMove 跟随移动 torrent 文件(异步,不影响主操作)
|
||||
// srcFolderId: 源文件所在目录 ID
|
||||
// srcFileName: 源文件名
|
||||
// dstDir: 目标目录
|
||||
func (y *Cloud189PC) torrentFollowMove(srcFolderId string, srcFileName string, dstDir model.Obj) {
|
||||
if !y.Addition.GenerateTorrent {
|
||||
return
|
||||
}
|
||||
if srcFolderId == "" {
|
||||
return
|
||||
}
|
||||
torrentName := srcFileName + ".cas.torrent"
|
||||
isFamily := y.isFamily()
|
||||
|
||||
go func() {
|
||||
torrentFile, err := y.findFileByName(context.Background(), torrentName, srcFolderId, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Debugf("查找 torrent 文件失败(跟随移动): %v", err)
|
||||
return
|
||||
}
|
||||
// 移动 torrent 文件到目标目录
|
||||
resp, moveErr := y.CreateBatchTask("MOVE", IF(isFamily, y.FamilyID, ""), dstDir.GetID(),
|
||||
map[string]string{"targetFileName": dstDir.GetName()},
|
||||
BatchTaskInfo{
|
||||
FileId: torrentFile.GetID(),
|
||||
FileName: torrentFile.GetName(),
|
||||
IsFolder: 0,
|
||||
})
|
||||
if moveErr != nil {
|
||||
utils.Log.Warnf("跟随移动 torrent 文件失败: %v", moveErr)
|
||||
return
|
||||
}
|
||||
if err = y.WaitBatchTask("MOVE", resp.TaskID, time.Millisecond*400); err != nil {
|
||||
utils.Log.Warnf("等待跟随移动 torrent 文件失败: %v", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// torrentFollowRename 跟随重命名 torrent 文件(异步,不影响主操作)
|
||||
// folderId: 文件所在目录 ID
|
||||
// oldFileName: 原文件名
|
||||
// newFileName: 新文件名
|
||||
func (y *Cloud189PC) torrentFollowRename(folderId string, oldFileName string, newFileName string) {
|
||||
if !y.Addition.GenerateTorrent {
|
||||
return
|
||||
}
|
||||
if folderId == "" {
|
||||
return
|
||||
}
|
||||
oldTorrentName := oldFileName + ".cas.torrent"
|
||||
newTorrentName := newFileName + ".cas.torrent"
|
||||
isFamily := y.isFamily()
|
||||
|
||||
go func() {
|
||||
torrentFile, err := y.findFileByName(context.Background(), oldTorrentName, folderId, isFamily)
|
||||
if err != nil {
|
||||
utils.Log.Debugf("查找 torrent 文件失败(跟随重命名): %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// 重命名 torrent 文件
|
||||
queryParam := make(map[string]string)
|
||||
fullUrl := API_URL
|
||||
method := "POST"
|
||||
if isFamily {
|
||||
fullUrl += "/family/file"
|
||||
method = "GET"
|
||||
queryParam["familyId"] = y.FamilyID
|
||||
}
|
||||
fullUrl += "/renameFile.action"
|
||||
queryParam["fileId"] = torrentFile.GetID()
|
||||
queryParam["destFileName"] = newTorrentName
|
||||
|
||||
_, renameErr := y.request(fullUrl, method, func(req *resty.Request) {
|
||||
req.SetContext(context.Background()).SetQueryParams(queryParam)
|
||||
}, nil, &RenameResp{}, isFamily)
|
||||
if renameErr != nil {
|
||||
utils.Log.Warnf("跟随重命名 torrent 文件失败: %v", renameErr)
|
||||
}
|
||||
}()
|
||||
}
|
||||
@@ -1,579 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/rand"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/fs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/sign"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
)
|
||||
|
||||
type Alias struct {
|
||||
model.Storage
|
||||
Addition
|
||||
rootOrder []string
|
||||
pathMap map[string][]string
|
||||
root model.Obj
|
||||
}
|
||||
|
||||
func (d *Alias) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Alias) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Alias) Init(ctx context.Context) error {
|
||||
paths := strings.Split(d.Paths, "\n")
|
||||
d.rootOrder = make([]string, 0, len(paths))
|
||||
d.pathMap = make(map[string][]string)
|
||||
for _, path := range paths {
|
||||
path = strings.TrimSpace(path)
|
||||
if path == "" {
|
||||
continue
|
||||
}
|
||||
k, v := getPair(path)
|
||||
temp, ok := d.pathMap[k]
|
||||
if !ok {
|
||||
d.rootOrder = append(d.rootOrder, k)
|
||||
}
|
||||
d.pathMap[k] = append(temp, v)
|
||||
}
|
||||
|
||||
switch len(d.rootOrder) {
|
||||
case 0:
|
||||
return errors.New("paths is required")
|
||||
case 1:
|
||||
paths := d.pathMap[d.rootOrder[0]]
|
||||
roots := make(BalancedObjs, 0, len(paths))
|
||||
roots = append(roots, &model.Object{
|
||||
Name: "root",
|
||||
Path: paths[0],
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Locked,
|
||||
})
|
||||
for _, path := range paths[1:] {
|
||||
roots = append(roots, &model.Object{
|
||||
Path: path,
|
||||
})
|
||||
}
|
||||
d.root = roots
|
||||
default:
|
||||
d.root = &model.Object{
|
||||
Name: "root",
|
||||
Path: "/",
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.ReadOnly,
|
||||
}
|
||||
}
|
||||
|
||||
if !utils.SliceContains(ValidReadConflictPolicy, d.ReadConflictPolicy) {
|
||||
d.ReadConflictPolicy = FirstRWP
|
||||
}
|
||||
if !utils.SliceContains(ValidWriteConflictPolicy, d.WriteConflictPolicy) {
|
||||
d.WriteConflictPolicy = DisabledWP
|
||||
}
|
||||
if !utils.SliceContains(ValidPutConflictPolicy, d.PutConflictPolicy) {
|
||||
d.PutConflictPolicy = DisabledWP
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Alias) Drop(ctx context.Context) error {
|
||||
d.rootOrder = nil
|
||||
d.pathMap = nil
|
||||
d.root = nil
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Alias) GetRoot(ctx context.Context) (model.Obj, error) {
|
||||
if d.root == nil {
|
||||
return nil, errs.StorageNotInit
|
||||
}
|
||||
return d.root, nil
|
||||
}
|
||||
|
||||
// 通过op.Get调用的话,path一定是子路径(/开头)
|
||||
func (d *Alias) Get(ctx context.Context, path string) (model.Obj, error) {
|
||||
roots, sub := d.getRootsAndPath(path)
|
||||
if len(roots) == 0 {
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
for idx, root := range roots {
|
||||
rawPath := stdpath.Join(root, sub)
|
||||
obj, err := fs.Get(ctx, rawPath, &fs.GetArgs{NoLog: true})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
mask := model.GetObjMask(obj) &^ model.Temp
|
||||
if sub == "" {
|
||||
// 根目录
|
||||
mask |= model.Locked | model.Virtual
|
||||
}
|
||||
ret := model.Object{
|
||||
Path: rawPath,
|
||||
Name: obj.GetName(),
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: obj.IsDir(),
|
||||
HashInfo: obj.GetHash(),
|
||||
Mask: mask,
|
||||
}
|
||||
obj = &ret
|
||||
if d.ProviderPassThrough && !obj.IsDir() {
|
||||
if storage, err := fs.GetStorage(rawPath, &fs.GetStoragesArgs{}); err == nil {
|
||||
obj = &model.ObjectProvider{
|
||||
Object: ret,
|
||||
Provider: model.Provider{
|
||||
Provider: storage.Config().Name,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
roots = roots[idx+1:]
|
||||
var objs BalancedObjs
|
||||
if idx > 0 {
|
||||
objs = make(BalancedObjs, 0, len(roots)+2)
|
||||
} else {
|
||||
objs = make(BalancedObjs, 0, len(roots)+1)
|
||||
}
|
||||
objs = append(objs, obj)
|
||||
if idx > 0 {
|
||||
objs = append(objs, nil)
|
||||
}
|
||||
for _, d := range roots {
|
||||
objs = append(objs, &tempObj{model.Object{
|
||||
Path: stdpath.Join(d, sub),
|
||||
}})
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
|
||||
func (d *Alias) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
dirs, ok := dir.(BalancedObjs)
|
||||
if !ok {
|
||||
return d.listRoot(ctx, args.WithStorageDetails && d.DetailsPassThrough, args.Refresh), nil
|
||||
}
|
||||
|
||||
// 因为alias是NoCache且Get方法不会返回NotSupport或NotImplement错误
|
||||
// 所以这里对象不会传回到alias,也就不需要返回BalancedObjs了
|
||||
objMap := make(map[string]model.Obj)
|
||||
for _, dir := range dirs {
|
||||
if dir == nil {
|
||||
continue
|
||||
}
|
||||
dirPath := dir.GetPath()
|
||||
tmp, err := fs.List(ctx, dirPath, &fs.ListArgs{
|
||||
NoLog: true,
|
||||
Refresh: args.Refresh,
|
||||
WithStorageDetails: args.WithStorageDetails && d.DetailsPassThrough,
|
||||
})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for _, obj := range tmp {
|
||||
name := obj.GetName()
|
||||
if _, exists := objMap[name]; exists {
|
||||
continue
|
||||
}
|
||||
mask := model.GetObjMask(obj) &^ model.Temp
|
||||
objRes := model.Object{
|
||||
Name: name,
|
||||
Path: stdpath.Join(dirPath, name),
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: obj.IsDir(),
|
||||
Mask: mask,
|
||||
}
|
||||
var objRet model.Obj
|
||||
if thumb, ok := model.GetThumb(obj); ok {
|
||||
objRet = &model.ObjThumb{
|
||||
Object: objRes,
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumb,
|
||||
},
|
||||
}
|
||||
} else {
|
||||
objRet = &objRes
|
||||
}
|
||||
if details, ok := model.GetStorageDetails(obj); ok {
|
||||
objRet = &model.ObjStorageDetails{
|
||||
Obj: objRet,
|
||||
StorageDetails: details,
|
||||
}
|
||||
}
|
||||
objMap[name] = objRet
|
||||
}
|
||||
}
|
||||
objs := make([]model.Obj, 0, len(objMap))
|
||||
for _, obj := range objMap {
|
||||
objs = append(objs, obj)
|
||||
}
|
||||
if d.OrderBy == "" {
|
||||
sort := getAllSort(dirs)
|
||||
if sort.OrderBy != "" {
|
||||
model.SortFiles(objs, sort.OrderBy, sort.OrderDirection)
|
||||
}
|
||||
if d.ExtractFolder == "" && sort.ExtractFolder != "" {
|
||||
model.ExtractFolder(objs, sort.ExtractFolder)
|
||||
}
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *Alias) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if d.ReadConflictPolicy == AllRWP && !args.Redirect {
|
||||
files, err := d.getAllObjs(ctx, file, getWriteAndPutFilterFunc(AllRWP))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
linkClosers := make([]io.Closer, 0, len(files))
|
||||
rrf := make([]model.RangeReaderIF, 0, len(files))
|
||||
requireReference := false
|
||||
for _, f := range files {
|
||||
link, fi, err := d.link(ctx, f.GetPath(), args)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if fi.GetSize() != files.GetSize() {
|
||||
_ = link.Close()
|
||||
continue
|
||||
}
|
||||
l := &model.Link{
|
||||
URL: link.URL,
|
||||
Header: link.Header,
|
||||
RangeReader: link.RangeReader,
|
||||
Concurrency: link.Concurrency,
|
||||
PartSize: link.PartSize,
|
||||
ContentLength: link.ContentLength,
|
||||
}
|
||||
if d.DownloadConcurrency > 0 {
|
||||
l.Concurrency = d.DownloadConcurrency
|
||||
}
|
||||
if d.DownloadPartSize > 0 {
|
||||
l.PartSize = d.DownloadPartSize * utils.KB
|
||||
}
|
||||
if l.ContentLength == 0 {
|
||||
l.ContentLength = fi.GetSize()
|
||||
}
|
||||
rr, err := stream.GetRangeReaderFromLink(l.ContentLength, l)
|
||||
if err != nil {
|
||||
_ = link.Close()
|
||||
continue
|
||||
}
|
||||
linkClosers = append(linkClosers, link)
|
||||
if link.RequireReference {
|
||||
requireReference = true
|
||||
}
|
||||
rrf = append(rrf, rr)
|
||||
}
|
||||
rr := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
|
||||
return rrf[rand.Intn(len(rrf))].RangeRead(ctx, httpRange)
|
||||
}
|
||||
return &model.Link{
|
||||
RangeReader: stream.RangeReaderFunc(rr),
|
||||
SyncClosers: utils.NewSyncClosers(linkClosers...),
|
||||
RequireReference: requireReference,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var link *model.Link
|
||||
var fi model.Obj
|
||||
var err error
|
||||
files := file.(BalancedObjs)
|
||||
if d.ReadConflictPolicy == RandomBalancedRP || d.ReadConflictPolicy == AllRWP {
|
||||
rand.Shuffle(len(files), func(i, j int) {
|
||||
files[i], files[j] = files[j], files[i]
|
||||
})
|
||||
}
|
||||
for _, f := range files {
|
||||
if f == nil {
|
||||
continue
|
||||
}
|
||||
link, fi, err = d.link(ctx, f.GetPath(), args)
|
||||
if err == nil {
|
||||
if link == nil {
|
||||
// 重定向且需要通过代理
|
||||
return &model.Link{
|
||||
URL: fmt.Sprintf("%s/p%s?sign=%s",
|
||||
common.GetApiUrl(ctx),
|
||||
utils.EncodePath(f.GetPath(), true),
|
||||
sign.Sign(f.GetPath())),
|
||||
}, nil
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resultLink := link.Clone() // 复制一份,避免修改到原始link
|
||||
resultLink.Expiration = nil
|
||||
if args.Redirect {
|
||||
return resultLink, nil
|
||||
}
|
||||
if d.DownloadConcurrency > 0 {
|
||||
resultLink.Concurrency = d.DownloadConcurrency
|
||||
}
|
||||
if d.DownloadPartSize > 0 {
|
||||
resultLink.PartSize = d.DownloadPartSize * utils.KB
|
||||
}
|
||||
if resultLink.ContentLength == 0 {
|
||||
resultLink.ContentLength = fi.GetSize()
|
||||
}
|
||||
return resultLink, nil
|
||||
}
|
||||
|
||||
func (d *Alias) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
|
||||
// Other 不应负载均衡,这是因为前端是否调用 /fs/other 的判断条件是返回的 provider 的值
|
||||
// 而 ProviderPassThrough 开启时,返回的 provider 固定为第一个 obj 的后端驱动
|
||||
storage, actualPath, err := op.GetStorageAndActualPath(args.Obj.GetPath())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return op.Other(ctx, storage, model.FsOtherArgs{
|
||||
Path: actualPath,
|
||||
Method: args.Method,
|
||||
Data: args.Data,
|
||||
})
|
||||
}
|
||||
|
||||
func (d *Alias) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
objs, err := d.getWriteObjs(ctx, parentDir)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.MakeDir(ctx, stdpath.Join(obj.GetPath(), dirName)))
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
srcs, dsts, err := d.getMoveObjs(ctx, srcObj, dstDir)
|
||||
if err == nil {
|
||||
for i, dst := range dsts {
|
||||
src := srcs[i]
|
||||
_, e := fs.Move(ctx, src.GetPath(), dst.GetPath())
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
srcs = srcs[len(dsts):]
|
||||
for _, src := range srcs {
|
||||
e := fs.Remove(ctx, src.GetPath())
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
objs, err := d.getWriteObjs(ctx, srcObj)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.Rename(ctx, obj.GetPath(), newName))
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
srcs, dsts, err := d.getCopyObjs(ctx, srcObj, dstDir)
|
||||
if err == nil {
|
||||
for i, src := range srcs {
|
||||
dst := dsts[i]
|
||||
_, e := fs.Copy(ctx, src.GetPath(), dst.GetPath())
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Remove(ctx context.Context, obj model.Obj) error {
|
||||
objs, err := d.getWriteObjs(ctx, obj)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.Remove(ctx, obj.GetPath()))
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) Put(ctx context.Context, dstDir model.Obj, s model.FileStreamer, up driver.UpdateProgress) error {
|
||||
objs, err := d.getPutObjs(ctx, dstDir)
|
||||
if err == nil {
|
||||
if len(objs) == 1 {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(objs.GetPath())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return op.Put(ctx, storage, reqActualPath, &stream.FileStream{
|
||||
Obj: s,
|
||||
Mimetype: s.GetMimetype(),
|
||||
Reader: s,
|
||||
}, up)
|
||||
} else {
|
||||
file, err := s.CacheFullAndWriter(nil, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
count := float64(len(objs) + 1)
|
||||
up(100 / count)
|
||||
for i, obj := range objs {
|
||||
err = errors.Join(err, fs.PutDirectly(ctx, obj.GetPath(), &stream.FileStream{
|
||||
Obj: s,
|
||||
Mimetype: s.GetMimetype(),
|
||||
Reader: file,
|
||||
}))
|
||||
up(float64(i+2) / float64(count) * 100)
|
||||
_, e := file.Seek(0, io.SeekStart)
|
||||
if e != nil {
|
||||
return errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) PutURL(ctx context.Context, dstDir model.Obj, name, url string) error {
|
||||
objs, err := d.getPutObjs(ctx, dstDir)
|
||||
if err == nil {
|
||||
for _, obj := range objs {
|
||||
err = errors.Join(err, fs.PutURL(ctx, obj.GetPath(), name, url))
|
||||
}
|
||||
return err
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
reqPath := d.getBalancedPath(ctx, obj)
|
||||
if reqPath == "" {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
meta, err := d.getArchiveMeta(ctx, reqPath, args)
|
||||
if err == nil {
|
||||
return meta, nil
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
reqPath := d.getBalancedPath(ctx, obj)
|
||||
if reqPath == "" {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
l, err := d.listArchive(ctx, reqPath, args)
|
||||
if err == nil {
|
||||
return l, nil
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
// alias的两个驱动,一个支持驱动提取,一个不支持,如何兼容?
|
||||
// 如果访问的是不支持驱动提取的驱动内的压缩文件,GetArchiveMeta就会返回errs.NotImplement,提取URL前缀就会是/ae,Extract就不会被调用
|
||||
// 如果访问的是支持驱动提取的驱动内的压缩文件,GetArchiveMeta就会返回有效值,提取URL前缀就会是/ad,Extract就会被调用
|
||||
reqPath := d.getBalancedPath(ctx, obj)
|
||||
if reqPath == "" {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
link, err := d.extract(ctx, reqPath, args)
|
||||
if err != nil {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if link == nil {
|
||||
return &model.Link{
|
||||
URL: fmt.Sprintf("%s/ap%s?inner=%s&pass=%s&sign=%s",
|
||||
common.GetApiUrl(ctx),
|
||||
utils.EncodePath(reqPath, true),
|
||||
utils.EncodePath(args.InnerPath, true),
|
||||
url.QueryEscape(args.Password),
|
||||
sign.SignArchive(reqPath)),
|
||||
}, nil
|
||||
}
|
||||
return link.Clone(), nil
|
||||
}
|
||||
|
||||
func (d *Alias) ArchiveDecompress(ctx context.Context, srcObj, dstDir model.Obj, args model.ArchiveDecompressArgs) error {
|
||||
srcs, dsts, err := d.getCopyObjs(ctx, srcObj, dstDir)
|
||||
if err == nil {
|
||||
for i, src := range srcs {
|
||||
dst := dsts[i]
|
||||
_, e := fs.ArchiveDecompress(ctx, src.GetPath(), dst.GetPath(), args)
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Alias) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
if !d.DetailsPassThrough {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if len(d.rootOrder) != 1 {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
backends := d.pathMap[d.rootOrder[0]]
|
||||
var storage driver.Driver
|
||||
for _, backend := range backends {
|
||||
s, err := fs.GetStorage(backend, &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if storage == nil {
|
||||
storage = s
|
||||
} else if storage.GetStorage().MountPath != s.GetStorage().MountPath {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
}
|
||||
if storage == nil { // should never access
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
return op.GetStorageDetails(ctx, storage)
|
||||
}
|
||||
|
||||
func (d *Alias) ResolveLinkCacheMode(path string) driver.LinkCacheMode {
|
||||
roots, sub := d.getRootsAndPath(path)
|
||||
if len(roots) == 0 {
|
||||
return 0
|
||||
}
|
||||
for _, root := range roots {
|
||||
storage, actualPath, err := op.GetStorageAndActualPath(stdpath.Join(root, sub))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if storage.Config().CheckStatus && storage.GetStorage().Status != op.WORK {
|
||||
continue
|
||||
}
|
||||
mode := storage.Config().LinkCacheMode
|
||||
if mode == -1 {
|
||||
return storage.(driver.LinkCacheModeResolver).ResolveLinkCacheMode(actualPath)
|
||||
} else {
|
||||
return mode
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Alias)(nil)
|
||||
@@ -1,34 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
Paths string `json:"paths" required:"true" type:"text"`
|
||||
ReadConflictPolicy string `json:"read_conflict_policy" type:"select" options:"first,random,all" default:"first"`
|
||||
WriteConflictPolicy string `json:"write_conflict_policy" type:"select" options:"disabled,first,deterministic,deterministic_or_all,all,all_strict" default:"disabled" help:"How the driver handles identical backend paths when renaming, removing, or making directories."`
|
||||
PutConflictPolicy string `json:"put_conflict_policy" type:"select" options:"disabled,first,deterministic,deterministic_or_all,all,all_strict,random,quota,quota_strict" default:"disabled" help:"How the driver handles identical backend paths when uploading, copying, moving, or decompressing."`
|
||||
FileConsistencyCheck bool `json:"file_consistency_check" type:"bool" default:"false"`
|
||||
DownloadConcurrency int `json:"download_concurrency" default:"0" required:"false" type:"number" help:"Need to enable proxy"`
|
||||
DownloadPartSize int `json:"download_part_size" default:"0" type:"number" required:"false" help:"Need to enable proxy. Unit: KB"`
|
||||
ProviderPassThrough bool `json:"provider_pass_through" type:"bool" default:"false"`
|
||||
DetailsPassThrough bool `json:"details_pass_through" type:"bool" default:"false"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Alias",
|
||||
LocalSort: true,
|
||||
NoCache: true,
|
||||
NoUpload: false,
|
||||
DefaultRoot: "/",
|
||||
ProxyRangeOption: true,
|
||||
LinkCacheMode: driver.LinkCacheAuto,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Alias{}
|
||||
})
|
||||
}
|
||||
@@ -1,78 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/pkg/errors"
|
||||
)
|
||||
|
||||
const (
|
||||
DisabledWP = "disabled"
|
||||
FirstRWP = "first"
|
||||
DeterministicWP = "deterministic"
|
||||
DeterministicOrAllWP = "deterministic_or_all"
|
||||
AllRWP = "all"
|
||||
AllStrictWP = "all_strict"
|
||||
RandomBalancedRP = "random"
|
||||
BalancedByQuotaP = "quota"
|
||||
BalancedByQuotaStrictP = "quota_strict"
|
||||
)
|
||||
|
||||
var (
|
||||
ValidReadConflictPolicy = []string{FirstRWP, RandomBalancedRP, AllRWP}
|
||||
ValidWriteConflictPolicy = []string{DisabledWP, FirstRWP, DeterministicWP, DeterministicOrAllWP, AllRWP,
|
||||
AllStrictWP}
|
||||
ValidPutConflictPolicy = []string{DisabledWP, FirstRWP, DeterministicWP, DeterministicOrAllWP, AllRWP,
|
||||
AllStrictWP, RandomBalancedRP, BalancedByQuotaP, BalancedByQuotaStrictP}
|
||||
)
|
||||
|
||||
var (
|
||||
ErrPathConflict = errors.New("path conflict")
|
||||
ErrSamePathLeak = errors.New("leak some of same-name dirs")
|
||||
ErrNoEnoughSpace = errors.New("none of same-name dirs has enough space")
|
||||
ErrNotEnoughSrcObjs = errors.New("cannot move fewer objs to more paths, please try copying")
|
||||
)
|
||||
|
||||
type BalancedObjs []model.Obj
|
||||
|
||||
func (b BalancedObjs) GetSize() int64 {
|
||||
return b[0].GetSize()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) ModTime() time.Time {
|
||||
return b[0].ModTime()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) CreateTime() time.Time {
|
||||
return b[0].CreateTime()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) IsDir() bool {
|
||||
return b[0].IsDir()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetHash() utils.HashInfo {
|
||||
return b[0].GetHash()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetName() string {
|
||||
return b[0].GetName()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetPath() string {
|
||||
return b[0].GetPath()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) GetID() string {
|
||||
return b[0].GetID()
|
||||
}
|
||||
|
||||
func (b BalancedObjs) Unwrap() model.Obj {
|
||||
return b[0]
|
||||
}
|
||||
|
||||
var _ model.Obj = (BalancedObjs)(nil)
|
||||
|
||||
type tempObj struct{ model.Object }
|
||||
@@ -1,532 +0,0 @@
|
||||
package alias
|
||||
|
||||
import (
|
||||
"context"
|
||||
"math/rand"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/fs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/pkg/errors"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
type detailWithIndex struct {
|
||||
idx int
|
||||
val *model.StorageDetails
|
||||
}
|
||||
|
||||
func (d *Alias) listRoot(ctx context.Context, withDetails, refresh bool) []model.Obj {
|
||||
var objs []model.Obj
|
||||
detailsChan := make(chan detailWithIndex, len(d.pathMap))
|
||||
workerCount := 0
|
||||
for _, k := range d.rootOrder {
|
||||
obj := &model.Object{
|
||||
Name: k,
|
||||
Path: "/" + k,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Locked | model.Virtual,
|
||||
}
|
||||
idx := len(objs)
|
||||
objs = append(objs, obj)
|
||||
v := d.pathMap[k]
|
||||
if !withDetails || len(v) != 1 {
|
||||
continue
|
||||
}
|
||||
remoteDriver, err := op.GetStorageByMountPath(v[0])
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
obj.Modified = remoteDriver.GetStorage().Modified
|
||||
_, ok := remoteDriver.(driver.WithDetails)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
objs[idx] = &model.ObjStorageDetails{
|
||||
Obj: objs[idx],
|
||||
StorageDetails: nil,
|
||||
}
|
||||
workerCount++
|
||||
go func(dri driver.Driver, i int) {
|
||||
details, e := op.GetStorageDetails(ctx, dri, refresh)
|
||||
if e != nil {
|
||||
if !errors.Is(e, errs.NotImplement) && !errors.Is(e, errs.StorageNotInit) {
|
||||
log.Errorf("failed get %s storage details: %+v", dri.GetStorage().MountPath, e)
|
||||
}
|
||||
}
|
||||
detailsChan <- detailWithIndex{idx: i, val: details}
|
||||
}(remoteDriver, idx)
|
||||
}
|
||||
for workerCount > 0 {
|
||||
select {
|
||||
case r := <-detailsChan:
|
||||
objs[r.idx].(*model.ObjStorageDetails).StorageDetails = r.val
|
||||
workerCount--
|
||||
case <-time.After(time.Second):
|
||||
workerCount = 0
|
||||
}
|
||||
}
|
||||
return objs
|
||||
}
|
||||
|
||||
// do others that not defined in Driver interface
|
||||
func getPair(path string) (string, string) {
|
||||
if name, path, ok := strings.Cut(path, ":"); ok && !strings.Contains(name, "/") {
|
||||
return name, path
|
||||
}
|
||||
return stdpath.Base(path), path
|
||||
}
|
||||
|
||||
func (d *Alias) getRootsAndPath(path string) (roots []string, sub string) {
|
||||
if len(d.rootOrder) == 1 {
|
||||
return d.pathMap[d.rootOrder[0]], path
|
||||
}
|
||||
path = strings.TrimPrefix(path, "/")
|
||||
before, after, ok := strings.Cut(path, "/")
|
||||
if !ok {
|
||||
return d.pathMap[path], ""
|
||||
}
|
||||
return d.pathMap[before], after
|
||||
}
|
||||
|
||||
func (d *Alias) link(ctx context.Context, reqPath string, args model.LinkArgs) (*model.Link, model.Obj, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if args.Redirect && common.ShouldProxy(storage, stdpath.Base(reqPath)) {
|
||||
return nil, nil, nil
|
||||
}
|
||||
return op.Link(ctx, storage, reqActualPath, args)
|
||||
}
|
||||
|
||||
func isConsistent(a, b model.Obj) bool {
|
||||
if a.GetSize() != b.GetSize() {
|
||||
return false
|
||||
}
|
||||
for ht, v := range a.GetHash().All() {
|
||||
ah := b.GetHash().GetHash(ht)
|
||||
if ah != "" && ah != v {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (d *Alias) getAllObjs(ctx context.Context, bObj model.Obj, ifContinue func(err error) (bool, error)) (BalancedObjs, error) {
|
||||
objs := bObj.(BalancedObjs)
|
||||
length := 0
|
||||
for _, o := range objs {
|
||||
var err error
|
||||
var obj model.Obj
|
||||
temp, isTemp := o.(*tempObj)
|
||||
if isTemp {
|
||||
obj, err = fs.Get(ctx, o.GetPath(), &fs.GetArgs{NoLog: true})
|
||||
if err == nil {
|
||||
if !bObj.IsDir() {
|
||||
if obj.IsDir() {
|
||||
err = errs.NotFile
|
||||
} else if d.FileConsistencyCheck && !isConsistent(bObj, obj) {
|
||||
err = errs.ObjectNotFound
|
||||
}
|
||||
} else if !obj.IsDir() {
|
||||
err = errs.NotFolder
|
||||
}
|
||||
}
|
||||
} else if o == nil {
|
||||
err = errs.ObjectNotFound
|
||||
}
|
||||
|
||||
cont, err := ifContinue(err)
|
||||
if err != nil {
|
||||
if cont {
|
||||
continue
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if isTemp {
|
||||
objRes := temp.Object
|
||||
// objRes.Name = obj.GetName()
|
||||
// objRes.Size = obj.GetSize()
|
||||
// objRes.Modified = obj.ModTime()
|
||||
// objRes.HashInfo = obj.GetHash()
|
||||
objs[length] = &objRes
|
||||
} else {
|
||||
objs[length] = o
|
||||
}
|
||||
length++
|
||||
if !cont {
|
||||
break
|
||||
}
|
||||
}
|
||||
if length == 0 {
|
||||
return nil, errs.ObjectNotFound
|
||||
}
|
||||
return objs[:length], nil
|
||||
}
|
||||
|
||||
func (d *Alias) getBalancedPath(ctx context.Context, file model.Obj) string {
|
||||
if d.ReadConflictPolicy == FirstRWP {
|
||||
return file.GetPath()
|
||||
}
|
||||
files := file.(BalancedObjs)
|
||||
if rand.Intn(len(files)) == 0 {
|
||||
return file.GetPath()
|
||||
}
|
||||
files, _ = d.getAllObjs(ctx, file, getWriteAndPutFilterFunc(AllRWP))
|
||||
return files[rand.Intn(len(files))].GetPath()
|
||||
}
|
||||
|
||||
func getWriteAndPutFilterFunc(policy string) func(error) (bool, error) {
|
||||
if policy == AllRWP {
|
||||
return func(err error) (bool, error) {
|
||||
return true, err
|
||||
}
|
||||
}
|
||||
all := true
|
||||
l := 0
|
||||
return func(err error) (bool, error) {
|
||||
if err != nil {
|
||||
switch policy {
|
||||
case AllStrictWP:
|
||||
return false, ErrSamePathLeak
|
||||
case DeterministicOrAllWP:
|
||||
if l >= 2 {
|
||||
return false, ErrSamePathLeak
|
||||
}
|
||||
}
|
||||
all = false
|
||||
} else {
|
||||
switch policy {
|
||||
case FirstRWP:
|
||||
return false, nil
|
||||
case DeterministicWP:
|
||||
if l > 0 {
|
||||
return false, ErrPathConflict
|
||||
}
|
||||
case DeterministicOrAllWP:
|
||||
if l > 0 && !all {
|
||||
return false, ErrSamePathLeak
|
||||
}
|
||||
}
|
||||
l += 1
|
||||
}
|
||||
return true, err
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Alias) getWriteObjs(ctx context.Context, obj model.Obj) (BalancedObjs, error) {
|
||||
if d.WriteConflictPolicy == DisabledWP {
|
||||
return nil, errs.PermissionDenied
|
||||
}
|
||||
return d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.WriteConflictPolicy))
|
||||
}
|
||||
|
||||
func (d *Alias) getPutObjs(ctx context.Context, obj model.Obj) (BalancedObjs, error) {
|
||||
if d.PutConflictPolicy == DisabledWP {
|
||||
return nil, errs.PermissionDenied
|
||||
}
|
||||
objs, err := d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.PutConflictPolicy))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
strict := false
|
||||
switch d.PutConflictPolicy {
|
||||
case RandomBalancedRP:
|
||||
ri := rand.Intn(len(objs))
|
||||
return objs[ri : ri+1], nil
|
||||
case BalancedByQuotaStrictP:
|
||||
strict = true
|
||||
fallthrough
|
||||
case BalancedByQuotaP:
|
||||
objs, ok := getRandomObjByQuotaBalanced(ctx, objs, strict, obj.GetSize())
|
||||
if !ok {
|
||||
return nil, ErrNoEnoughSpace
|
||||
}
|
||||
return objs, nil
|
||||
default:
|
||||
return objs, nil
|
||||
}
|
||||
}
|
||||
|
||||
func getRandomObjByQuotaBalanced(ctx context.Context, reqPath BalancedObjs, strict bool, objSize int64) (BalancedObjs, bool) {
|
||||
// Get all space
|
||||
details := make([]*model.StorageDetails, len(reqPath))
|
||||
detailsChan := make(chan detailWithIndex, len(reqPath))
|
||||
workerCount := 0
|
||||
for i, p := range reqPath {
|
||||
s, err := fs.GetStorage(p.GetPath(), &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if _, ok := s.(driver.WithDetails); !ok {
|
||||
continue
|
||||
}
|
||||
workerCount++
|
||||
go func(dri driver.Driver, i int) {
|
||||
d, e := op.GetStorageDetails(ctx, dri)
|
||||
if e != nil {
|
||||
if !errors.Is(e, errs.NotImplement) && !errors.Is(e, errs.StorageNotInit) {
|
||||
log.Errorf("failed get %s storage details: %+v", dri.GetStorage().MountPath, e)
|
||||
}
|
||||
}
|
||||
detailsChan <- detailWithIndex{idx: i, val: d}
|
||||
}(s, i)
|
||||
}
|
||||
for workerCount > 0 {
|
||||
select {
|
||||
case r := <-detailsChan:
|
||||
details[r.idx] = r.val
|
||||
workerCount--
|
||||
case <-time.After(time.Second):
|
||||
workerCount = 0
|
||||
}
|
||||
}
|
||||
|
||||
// Try select one that has space info
|
||||
selected, ok := selectRandom(details, func(d *model.StorageDetails) uint64 {
|
||||
if d == nil || d.FreeSpace() < objSize {
|
||||
return 0
|
||||
}
|
||||
return uint64(d.FreeSpace())
|
||||
})
|
||||
if !ok {
|
||||
if strict {
|
||||
return nil, false
|
||||
} else {
|
||||
// No strict mode, return any of non-details ones
|
||||
noDetails := make([]int, 0, len(details))
|
||||
for i, d := range details {
|
||||
if d == nil {
|
||||
noDetails = append(noDetails, i)
|
||||
}
|
||||
}
|
||||
if len(noDetails) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
selected = noDetails[rand.Intn(len(noDetails))]
|
||||
}
|
||||
}
|
||||
return reqPath[selected : selected+1], true
|
||||
}
|
||||
|
||||
func selectRandom[Item any](arr []Item, getWeight func(Item) uint64) (int, bool) {
|
||||
var totalWeight uint64 = 0
|
||||
for _, i := range arr {
|
||||
totalWeight += getWeight(i)
|
||||
}
|
||||
if totalWeight == 0 {
|
||||
return 0, false
|
||||
}
|
||||
r := rand.Uint64() % totalWeight
|
||||
for i, item := range arr {
|
||||
w := getWeight(item)
|
||||
if r < w {
|
||||
return i, true
|
||||
}
|
||||
r -= w
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
func (d *Alias) getCopyObjs(ctx context.Context, srcObj, dstDir model.Obj) (BalancedObjs, BalancedObjs, error) {
|
||||
if d.PutConflictPolicy == DisabledWP {
|
||||
return nil, nil, errs.PermissionDenied
|
||||
}
|
||||
dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
dstStorageMap := make(map[string][]model.Obj)
|
||||
allocatingDst := make(map[model.Obj]struct{})
|
||||
for _, o := range dstObjs {
|
||||
storage, e := fs.GetStorage(o.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
return nil, nil, errors.WithMessagef(e, "cannot copy to virtual path [%s]", o.GetPath())
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
dstStorageMap[mp] = append(dstStorageMap[mp], o)
|
||||
allocatingDst[o] = struct{}{}
|
||||
}
|
||||
tmpSrcObjs, err := d.getAllObjs(ctx, srcObj, getWriteAndPutFilterFunc(AllRWP))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
srcObjs := make(BalancedObjs, 0, len(dstObjs))
|
||||
for _, src := range tmpSrcObjs {
|
||||
storage, e := fs.GetStorage(src.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
continue
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
if tmp, ok := dstStorageMap[mp]; ok {
|
||||
for _, dst := range tmp {
|
||||
dstObjs[len(srcObjs)] = dst
|
||||
srcObjs = append(srcObjs, src)
|
||||
delete(allocatingDst, dst)
|
||||
}
|
||||
delete(dstStorageMap, mp)
|
||||
}
|
||||
}
|
||||
dstObjs = dstObjs[:len(srcObjs)]
|
||||
for dst := range allocatingDst {
|
||||
src := tmpSrcObjs[0]
|
||||
if d.ReadConflictPolicy == RandomBalancedRP || d.ReadConflictPolicy == AllRWP {
|
||||
src = tmpSrcObjs[rand.Intn(len(tmpSrcObjs))]
|
||||
}
|
||||
srcObjs = append(srcObjs, src)
|
||||
dstObjs = append(dstObjs, dst)
|
||||
}
|
||||
return srcObjs, dstObjs, nil
|
||||
}
|
||||
|
||||
func (d *Alias) getMoveObjs(ctx context.Context, srcObj, dstDir model.Obj) (BalancedObjs, BalancedObjs, error) {
|
||||
if d.PutConflictPolicy == DisabledWP {
|
||||
return nil, nil, errs.PermissionDenied
|
||||
}
|
||||
dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
tmpSrcObjs, err := d.getAllObjs(ctx, srcObj, getWriteAndPutFilterFunc(AllRWP))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if len(tmpSrcObjs) < len(dstObjs) {
|
||||
return nil, nil, ErrNotEnoughSrcObjs
|
||||
}
|
||||
dstStorageMap := make(map[string][]model.Obj)
|
||||
allocatingDst := make(map[model.Obj]struct{})
|
||||
for _, o := range dstObjs {
|
||||
storage, e := fs.GetStorage(o.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
return nil, nil, errors.WithMessagef(e, "cannot move to virtual path [%s]", o.GetPath())
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
dstStorageMap[mp] = append(dstStorageMap[mp], o)
|
||||
allocatingDst[o] = struct{}{}
|
||||
}
|
||||
srcObjs := make(BalancedObjs, 0, len(tmpSrcObjs))
|
||||
restSrcObjs := make(BalancedObjs, 0, len(tmpSrcObjs)-len(dstObjs))
|
||||
for _, src := range tmpSrcObjs {
|
||||
storage, e := fs.GetStorage(src.GetPath(), &fs.GetStoragesArgs{})
|
||||
if e != nil {
|
||||
continue
|
||||
}
|
||||
mp := storage.GetStorage().MountPath
|
||||
if tmp, ok := dstStorageMap[mp]; ok {
|
||||
dst := tmp[0]
|
||||
if len(tmp) == 1 {
|
||||
delete(dstStorageMap, mp)
|
||||
} else {
|
||||
dstStorageMap[mp] = tmp[1:]
|
||||
}
|
||||
dstObjs[len(srcObjs)] = dst
|
||||
srcObjs = append(srcObjs, src)
|
||||
delete(allocatingDst, dst)
|
||||
} else {
|
||||
restSrcObjs = append(restSrcObjs, src)
|
||||
}
|
||||
}
|
||||
dstObjs = dstObjs[:len(srcObjs)]
|
||||
// len(restSrcObjs) >= len(allocatingDst)
|
||||
srcObjs = append(srcObjs, restSrcObjs...)
|
||||
for dst := range allocatingDst {
|
||||
dstObjs = append(dstObjs, dst)
|
||||
}
|
||||
return srcObjs, dstObjs, nil
|
||||
}
|
||||
|
||||
func (d *Alias) getArchiveMeta(ctx context.Context, reqPath string, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := storage.(driver.ArchiveReader); ok {
|
||||
return op.GetArchiveMeta(ctx, storage, reqActualPath, model.ArchiveMetaArgs{
|
||||
ArchiveArgs: args,
|
||||
Refresh: true,
|
||||
})
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) listArchive(ctx context.Context, reqPath string, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := storage.(driver.ArchiveReader); ok {
|
||||
return op.ListArchive(ctx, storage, reqActualPath, model.ArchiveListArgs{
|
||||
ArchiveInnerArgs: args,
|
||||
Refresh: true,
|
||||
})
|
||||
}
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Alias) extract(ctx context.Context, reqPath string, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
storage, reqActualPath, err := op.GetStorageAndActualPath(reqPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := storage.(driver.ArchiveReader); !ok {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if args.Redirect && common.ShouldProxy(storage, stdpath.Base(reqPath)) {
|
||||
_, err := fs.Get(ctx, reqPath, &fs.GetArgs{NoLog: true})
|
||||
if err == nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
link, _, err := op.DriverExtract(ctx, storage, reqActualPath, args)
|
||||
return link, err
|
||||
}
|
||||
|
||||
func getAllSort(dirs []model.Obj) model.Sort {
|
||||
ret := model.Sort{}
|
||||
noSort := false
|
||||
noExtractFolder := false
|
||||
for _, dir := range dirs {
|
||||
if dir == nil {
|
||||
continue
|
||||
}
|
||||
storage, err := fs.GetStorage(dir.GetPath(), &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if !noSort && storage.GetStorage().OrderBy != "" {
|
||||
if ret.OrderBy == "" {
|
||||
ret.OrderBy = storage.GetStorage().OrderBy
|
||||
ret.OrderDirection = storage.GetStorage().OrderDirection
|
||||
if ret.OrderDirection == "" {
|
||||
ret.OrderDirection = "asc"
|
||||
}
|
||||
} else if ret.OrderBy != storage.GetStorage().OrderBy || ret.OrderDirection != storage.GetStorage().OrderDirection {
|
||||
ret.OrderBy = ""
|
||||
ret.OrderDirection = ""
|
||||
noSort = true
|
||||
}
|
||||
}
|
||||
if !noExtractFolder && storage.GetStorage().ExtractFolder != "" {
|
||||
if ret.ExtractFolder == "" {
|
||||
ret.ExtractFolder = storage.GetStorage().ExtractFolder
|
||||
} else if ret.ExtractFolder != storage.GetStorage().ExtractFolder {
|
||||
ret.ExtractFolder = ""
|
||||
noExtractFolder = true
|
||||
}
|
||||
}
|
||||
if noSort && noExtractFolder {
|
||||
break
|
||||
}
|
||||
}
|
||||
return ret
|
||||
}
|
||||
@@ -1,147 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type AliDoc struct {
|
||||
model.Storage
|
||||
Addition
|
||||
|
||||
client *resty.Client
|
||||
}
|
||||
|
||||
func (d *AliDoc) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *AliDoc) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *AliDoc) Init(ctx context.Context) error {
|
||||
d.Cookie = strings.TrimSpace(d.Cookie)
|
||||
d.RootFolderID = strings.TrimSpace(d.RootFolderID)
|
||||
if d.Cookie == "" {
|
||||
return fmt.Errorf("cookie is empty")
|
||||
}
|
||||
if d.RootFolderID == "" {
|
||||
return fmt.Errorf("root folder id is empty")
|
||||
}
|
||||
d.client = newClient()
|
||||
if err := d.checkCookie(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Drop(ctx context.Context) error {
|
||||
d.client = nil
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
items, err := d.list(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
objs := make([]model.Obj, 0, len(items))
|
||||
for _, item := range items {
|
||||
if strings.TrimSpace(item.DentryUUID) == "" || strings.TrimSpace(item.Name) == "" {
|
||||
continue
|
||||
}
|
||||
objs = append(objs, toObj(item))
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if file.IsDir() {
|
||||
return nil, fmt.Errorf("alidoc does not support directory links")
|
||||
}
|
||||
resp, err := d.download(ctx, file.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
url, err := firstDownloadURL(resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Link{
|
||||
URL: url,
|
||||
Header: http.Header{
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
"Referer": []string{apiBase + "/"},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
err := d.post(ctx, "/box/api/v2/dentry/createfolder", map[string]string{
|
||||
"dentryType": "folder",
|
||||
"name": dirName,
|
||||
"parentDentryUuid": parentDir.GetID(),
|
||||
"conflictHandleStrategy": "auto_rename",
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
err := d.post(ctx, "/box/api/v2/dentry/move", map[string]interface{}{
|
||||
"targetParentDentryUuid": dstDir.GetID(),
|
||||
"sourceDentryUuid": srcObj.GetID(),
|
||||
"operateFrom": 1,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
|
||||
err := d.post(ctx, "/box/api/v2/dentry/rename", map[string]string{
|
||||
"dentryUuid": srcObj.GetID(),
|
||||
"name": newName,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
srcObj.(*model.Object).Name = newName
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
return d.post(ctx, "/box/api/v2/dentry/copy", map[string]interface{}{
|
||||
"sourceDentryUuid": srcObj.GetID(),
|
||||
"targetParentDentryUuid": dstDir.GetID(),
|
||||
"operateFrom": 1,
|
||||
"onlyCopyMeta": false,
|
||||
})
|
||||
}
|
||||
|
||||
func (d *AliDoc) Remove(ctx context.Context, obj model.Obj) error {
|
||||
return d.post(ctx, "/box/api/v1/dentry/recycle", map[string]string{
|
||||
"dentryUuid": obj.GetID(),
|
||||
})
|
||||
}
|
||||
|
||||
var (
|
||||
_ driver.Driver = (*AliDoc)(nil)
|
||||
_ driver.MkdirResult = (*AliDoc)(nil)
|
||||
_ driver.MoveResult = (*AliDoc)(nil)
|
||||
_ driver.RenameResult = (*AliDoc)(nil)
|
||||
_ driver.Copy = (*AliDoc)(nil)
|
||||
_ driver.Remove = (*AliDoc)(nil)
|
||||
)
|
||||
@@ -1,23 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
Cookie string `json:"cookie" type:"text" required:"true" help:"钉钉文档网页 Cookie"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "AliDoc",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &AliDoc{}
|
||||
})
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
type apiResp struct {
|
||||
Status int `json:"status"`
|
||||
IsSuccess bool `json:"isSuccess"`
|
||||
Message string `json:"message"`
|
||||
Msg string `json:"msg"`
|
||||
}
|
||||
|
||||
func (r apiResp) ErrMessage() string {
|
||||
if r.Message != "" {
|
||||
return r.Message
|
||||
}
|
||||
if r.Msg != "" {
|
||||
return r.Msg
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type listResp struct {
|
||||
apiResp
|
||||
Data listData `json:"data"`
|
||||
}
|
||||
|
||||
type listData struct {
|
||||
Children []dentry `json:"children"`
|
||||
}
|
||||
|
||||
type dentry struct {
|
||||
DentryType string `json:"dentryType"`
|
||||
DentryUUID string `json:"dentryUuid"`
|
||||
ParentDentryUUID string `json:"parentDentryUuid"`
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
FileSize int64 `json:"fileSize"`
|
||||
CreatedTime int64 `json:"createdTime"`
|
||||
UpdatedTime int64 `json:"updatedTime"`
|
||||
ContentType string `json:"contentType"`
|
||||
Extension string `json:"extension"`
|
||||
DentryStatistic struct {
|
||||
ChildrenCount int `json:"childrenCount"`
|
||||
} `json:"dentryStatistic"`
|
||||
URL struct {
|
||||
PCChildAppPreviewURL string `json:"pcChildAppPreviewUrl"`
|
||||
PCChildAppURL string `json:"pcChildAppUrl"`
|
||||
} `json:"url"`
|
||||
}
|
||||
|
||||
type downloadResp struct {
|
||||
apiResp
|
||||
Data downloadData `json:"data"`
|
||||
}
|
||||
|
||||
type downloadData struct {
|
||||
OSSURLPreSignatureInfo struct {
|
||||
PreSignURLs []string `json:"preSignUrls"`
|
||||
} `json:"ossUrlPreSignatureInfo"`
|
||||
}
|
||||
|
||||
type uploadInfoResp struct {
|
||||
apiResp
|
||||
Data uploadInfoData `json:"data"`
|
||||
}
|
||||
|
||||
type uploadInfoData struct {
|
||||
CurrentTimestamp int64 `json:"currentTimestamp"`
|
||||
FileUploadProtocolConfig uploadProtocolConfig `json:"fileUploadProtocolConfig"`
|
||||
STSSignatureInfo uploadSTSSignatureInfo `json:"stsSignatureInfo"`
|
||||
UploadKey string `json:"uploadKey"`
|
||||
UploadType string `json:"uploadType"`
|
||||
}
|
||||
|
||||
type uploadProtocolConfig struct {
|
||||
MinPartSize int64 `json:"minPartSize"`
|
||||
}
|
||||
|
||||
type uploadSTSSignatureInfo struct {
|
||||
AccelerateCname string `json:"accelerateCname"`
|
||||
AccessKeyID string `json:"accessKeyId"`
|
||||
AccessKeySecret string `json:"accessKeySecret"`
|
||||
AccessToken string `json:"accessToken"`
|
||||
AccessTokenExpiration int64 `json:"accessTokenExpiration"`
|
||||
Bucket string `json:"bucket"`
|
||||
Cname string `json:"cname"`
|
||||
EndPoint string `json:"endPoint"`
|
||||
ObjectKey string `json:"objectKey"`
|
||||
}
|
||||
@@ -1,263 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
netutil "github.com/OpenListTeam/OpenList/v4/internal/net"
|
||||
streamPkg "github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/aliyun/aliyun-oss-go-sdk/oss"
|
||||
"github.com/avast/retry-go"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultAliDocMultipartThreshold = 16 * 1024 * 1024
|
||||
defaultAliDocPartSize = 100 * 1024
|
||||
maxAliDocMultipartParts = 10000
|
||||
)
|
||||
|
||||
func (d *AliDoc) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
size := file.GetSize()
|
||||
useMultipart := size > defaultAliDocMultipartThreshold
|
||||
info, err := d.getUploadInfo(ctx, dstDir.GetID(), file.GetName(), size, useMultipart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if size > 0 {
|
||||
partSize := calcAliDocPartSize(size, info.Data.FileUploadProtocolConfig.MinPartSize)
|
||||
if size > partSize && !useMultipart {
|
||||
useMultipart = true
|
||||
info, err = d.getUploadInfo(ctx, dstDir.GetID(), file.GetName(), size, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if useMultipart {
|
||||
err = d.multipartUpload(ctx, file, size, info, up)
|
||||
} else {
|
||||
err = d.singleUpload(ctx, file, size, info, up)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := d.commitUpload(ctx, dstDir.GetID(), file.GetName(), size, info.Data.UploadKey); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) getUploadInfo(ctx context.Context, parentDentryUUID, name string, fileSize int64, multipart bool) (uploadInfoResp, error) {
|
||||
var result uploadInfoResp
|
||||
body := map[string]interface{}{
|
||||
"uploadType": "STS_SIGNATURE",
|
||||
"supportUploadTypes": []string{"STS_SIGNATURE", "HTTP_TO_CENTER"},
|
||||
"parentDentryUuid": parentDentryUUID,
|
||||
"fileSize": fileSize,
|
||||
"name": name,
|
||||
"multipart": multipart,
|
||||
}
|
||||
resp, err := d.request(ctx).
|
||||
SetBody(body).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Post(apiBase + "/box/api/v2/file/uploadinfo")
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := checkResp(resp, result.apiResp); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if strings.TrimSpace(result.Data.STSSignatureInfo.Bucket) == "" {
|
||||
return result, fmt.Errorf("empty upload bucket")
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) commitUpload(ctx context.Context, parentDentryUUID, name string, fileSize int64, uploadKey string) error {
|
||||
uploadKey = strings.TrimSpace(uploadKey)
|
||||
if uploadKey == "" {
|
||||
return fmt.Errorf("empty upload key")
|
||||
}
|
||||
|
||||
var result apiResp
|
||||
body := map[string]interface{}{
|
||||
"parentDentryUuid": parentDentryUUID,
|
||||
"uploadKey": uploadKey,
|
||||
"fileSize": fileSize,
|
||||
"name": name,
|
||||
"toPrevDentryUuid": nil,
|
||||
"toNextDentryUuid": nil,
|
||||
"batchId": uuid.NewString(),
|
||||
"batchUploadType": 1,
|
||||
"batchParentDentryUuid": parentDentryUUID,
|
||||
}
|
||||
resp, err := d.request(ctx).
|
||||
SetBody(body).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Post(apiBase + "/box/api/v2/file/commit")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkResp(resp, result)
|
||||
}
|
||||
|
||||
func calcAliDocPartSize(fileSize, minPartSize int64) int64 {
|
||||
partSize := minPartSize
|
||||
if partSize <= 0 {
|
||||
partSize = defaultAliDocPartSize
|
||||
}
|
||||
if fileSize <= 0 {
|
||||
return partSize
|
||||
}
|
||||
minRequired := int64(math.Ceil(float64(fileSize) / maxAliDocMultipartParts))
|
||||
if minRequired > partSize {
|
||||
partSize = minRequired
|
||||
}
|
||||
return partSize
|
||||
}
|
||||
|
||||
func (d *AliDoc) singleUpload(ctx context.Context, src model.FileStreamer, size int64, info uploadInfoResp, up driver.UpdateProgress) error {
|
||||
bucket, objectKey, err := d.newOSSBucket(info)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = bucket.PutObject(
|
||||
objectKey,
|
||||
driver.NewLimitedUploadStream(ctx, io.TeeReader(src, driver.NewProgress(size, up))),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
up(100)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) multipartUpload(ctx context.Context, src model.FileStreamer, size int64, info uploadInfoResp, up driver.UpdateProgress) error {
|
||||
bucket, objectKey, err := d.newOSSBucket(info)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
imur, err := bucket.InitiateMultipartUpload(objectKey, oss.Sequential())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
partSize := calcAliDocPartSize(size, info.Data.FileUploadProtocolConfig.MinPartSize)
|
||||
partNum := int((size + partSize - 1) / partSize)
|
||||
parts := make([]oss.UploadPart, 0, partNum)
|
||||
ss, err := streamPkg.NewStreamSectionReader(src, int(partSize), &up)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var offset int64
|
||||
for partNumber := 1; partNumber <= partNum; partNumber++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
length := partSize
|
||||
if remain := size - offset; remain < length {
|
||||
length = remain
|
||||
}
|
||||
|
||||
reader, err := ss.GetSectionReader(offset, length)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var part oss.UploadPart
|
||||
var uploadErr error
|
||||
err = retry.Do(func() error {
|
||||
if _, err := reader.Seek(0, io.SeekStart); err != nil {
|
||||
return err
|
||||
}
|
||||
part, uploadErr = bucket.UploadPart(
|
||||
imur,
|
||||
driver.NewLimitedUploadStream(ctx, reader),
|
||||
length,
|
||||
partNumber,
|
||||
)
|
||||
return uploadErr
|
||||
},
|
||||
retry.Context(ctx),
|
||||
retry.Attempts(3),
|
||||
retry.DelayType(retry.BackOffDelay),
|
||||
retry.Delay(time.Second),
|
||||
)
|
||||
ss.FreeSectionReader(reader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
parts = append(parts, part)
|
||||
up(100 * float64(len(parts)) / float64(partNum+1))
|
||||
offset += length
|
||||
}
|
||||
|
||||
_, err = bucket.CompleteMultipartUpload(imur, parts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
up(100)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) newOSSBucket(info uploadInfoResp) (*oss.Bucket, string, error) {
|
||||
sts := info.Data.STSSignatureInfo
|
||||
objectKey := strings.TrimSpace(sts.ObjectKey)
|
||||
if objectKey == "" {
|
||||
objectKey = strings.TrimSpace(info.Data.UploadKey)
|
||||
}
|
||||
if objectKey == "" {
|
||||
return nil, "", fmt.Errorf("empty upload object key")
|
||||
}
|
||||
|
||||
endpoint, useCname := pickAliDocOSSEndpoint(sts)
|
||||
if endpoint == "" {
|
||||
return nil, "", fmt.Errorf("empty upload endpoint")
|
||||
}
|
||||
if !strings.HasPrefix(endpoint, "http://") && !strings.HasPrefix(endpoint, "https://") {
|
||||
endpoint = "https://" + endpoint
|
||||
}
|
||||
|
||||
options := []oss.ClientOption{oss.SecurityToken(sts.AccessToken)}
|
||||
if useCname {
|
||||
options = append(options, oss.UseCname(true))
|
||||
}
|
||||
client, err := netutil.NewOSSClient(
|
||||
endpoint,
|
||||
sts.AccessKeyID,
|
||||
sts.AccessKeySecret,
|
||||
options...,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
bucket, err := client.Bucket(sts.Bucket)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
return bucket, objectKey, nil
|
||||
}
|
||||
|
||||
func pickAliDocOSSEndpoint(sts uploadSTSSignatureInfo) (endpoint string, useCname bool) {
|
||||
if endpoint = strings.TrimSpace(sts.EndPoint); endpoint != "" {
|
||||
return endpoint, false
|
||||
}
|
||||
if endpoint = strings.TrimSpace(sts.Cname); endpoint != "" {
|
||||
return endpoint, true
|
||||
}
|
||||
if endpoint = strings.TrimSpace(sts.AccelerateCname); endpoint != "" {
|
||||
return endpoint, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
@@ -1,135 +0,0 @@
|
||||
package alidoc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
const apiBase = "https://alidocs.dingtalk.com"
|
||||
|
||||
func (d *AliDoc) request(ctx context.Context) *resty.Request {
|
||||
return d.client.R().
|
||||
SetContext(ctx).
|
||||
SetHeader("Cookie", d.Cookie).
|
||||
SetHeader("Accept", "application/json, text/plain, */*").
|
||||
SetHeader("Referer", apiBase+"/").
|
||||
SetHeader("Origin", apiBase)
|
||||
}
|
||||
|
||||
func msToTime(v int64) time.Time {
|
||||
if v <= 0 {
|
||||
return time.Time{}
|
||||
}
|
||||
return time.UnixMilli(v)
|
||||
}
|
||||
|
||||
func checkResp(resp *resty.Response, result apiResp) error {
|
||||
if resp != nil && resp.IsError() {
|
||||
if msg := result.ErrMessage(); msg != "" {
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
return fmt.Errorf("http error: %d", resp.StatusCode())
|
||||
}
|
||||
if !result.IsSuccess || result.Status != 200 {
|
||||
msg := result.ErrMessage()
|
||||
if msg == "" {
|
||||
msg = "request failed"
|
||||
}
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func toObj(item dentry) model.Obj {
|
||||
return &model.Object{
|
||||
ID: item.DentryUUID,
|
||||
Name: item.Name,
|
||||
Size: item.FileSize,
|
||||
Modified: msToTime(item.UpdatedTime),
|
||||
Ctime: msToTime(item.CreatedTime),
|
||||
IsFolder: item.DentryType == "folder",
|
||||
}
|
||||
}
|
||||
|
||||
func firstDownloadURL(resp downloadResp) (string, error) {
|
||||
if len(resp.Data.OSSURLPreSignatureInfo.PreSignURLs) == 0 {
|
||||
return "", fmt.Errorf("empty download url")
|
||||
}
|
||||
return resp.Data.OSSURLPreSignatureInfo.PreSignURLs[0], nil
|
||||
}
|
||||
|
||||
func newClient() *resty.Client {
|
||||
client := base.NewRestyClient()
|
||||
client.SetHeader("User-Agent", base.UserAgent)
|
||||
return client
|
||||
}
|
||||
|
||||
func (d *AliDoc) post(ctx context.Context, path string, body interface{}) error {
|
||||
var result apiResp
|
||||
resp, err := d.request(ctx).
|
||||
SetBody(body).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Post(apiBase + path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkResp(resp, result)
|
||||
}
|
||||
|
||||
func (d *AliDoc) checkCookie(ctx context.Context) error {
|
||||
var result apiResp
|
||||
resp, err := d.request(ctx).
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Get(apiBase + "/portal/api/v1/mine/info")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return checkResp(resp, result)
|
||||
}
|
||||
|
||||
func (d *AliDoc) list(ctx context.Context, dentryUUID string) ([]dentry, error) {
|
||||
var result listResp
|
||||
resp, err := d.request(ctx).
|
||||
SetQueryParam("dentryUuid", dentryUUID).
|
||||
SetQueryParam("withParentAncestors", "true").
|
||||
SetQueryParam("orderType", "SORT_KEY").
|
||||
SetQueryParam("sortType", "desc").
|
||||
SetQueryParam("listDentrySource", "2").
|
||||
SetQueryParam("pageSize", "1000").
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Get(apiBase + "/box/api/v2/dentry/list")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := checkResp(resp, result.apiResp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return result.Data.Children, nil
|
||||
}
|
||||
|
||||
func (d *AliDoc) download(ctx context.Context, dentryUUID string) (downloadResp, error) {
|
||||
var result downloadResp
|
||||
resp, err := d.request(ctx).
|
||||
SetQueryParam("dentryUuid", dentryUUID).
|
||||
SetQueryParam("version", "1").
|
||||
SetQueryParam("supportDownloadTypes", "URL_PRE_SIGNATURE,HTTP_TO_CENTER").
|
||||
SetQueryParam("downloadType", "URL_PRE_SIGNATURE").
|
||||
SetResult(&result).
|
||||
SetError(&result).
|
||||
Get(apiBase + "/box/api/v2/file/download")
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := checkResp(resp, result.apiResp); err != nil {
|
||||
return result, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
@@ -1,379 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
type AListV3 struct {
|
||||
model.Storage
|
||||
Addition
|
||||
}
|
||||
|
||||
func (d *AListV3) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *AListV3) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *AListV3) Init(ctx context.Context) error {
|
||||
d.Addition.Address = strings.TrimSuffix(d.Addition.Address, "/")
|
||||
var resp common.Resp[MeResp]
|
||||
_, _, err := d.request("/me", http.MethodGet, func(req *resty.Request) {
|
||||
req.SetResult(&resp)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// if the username is not empty and the username is not the same as the current username, then login again
|
||||
if d.Username != resp.Data.Username {
|
||||
err = d.login()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// re-get the user info
|
||||
_, _, err = d.request("/me", http.MethodGet, func(req *resty.Request) {
|
||||
req.SetResult(&resp)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if utils.SliceContains(resp.Data.Role, model.GUEST) {
|
||||
u := d.Address + "/api/public/settings"
|
||||
res, err := base.RestyClient.R().Get(u)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
allowMounted := utils.Json.Get(res.Body(), "data", conf.AllowMounted).ToString() == "true"
|
||||
if !allowMounted {
|
||||
return fmt.Errorf("the site does not allow mounted")
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AListV3) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
var resp common.Resp[FsListResp]
|
||||
_, _, err := d.request("/fs/list", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ListReq{
|
||||
PageReq: model.PageReq{
|
||||
Page: 1,
|
||||
PerPage: 0,
|
||||
},
|
||||
Path: dir.GetPath(),
|
||||
Password: d.MetaPassword,
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []model.Obj
|
||||
for _, f := range resp.Data.Content {
|
||||
file := model.ObjThumb{
|
||||
Object: model.Object{
|
||||
Name: f.Name,
|
||||
Modified: f.Modified,
|
||||
Ctime: f.Created,
|
||||
Size: f.Size,
|
||||
IsFolder: f.IsDir,
|
||||
HashInfo: utils.FromString(f.HashInfo),
|
||||
},
|
||||
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
|
||||
}
|
||||
files = append(files, &file)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
var resp common.Resp[FsGetResp]
|
||||
headers := map[string]string{
|
||||
"User-Agent": base.UserAgent,
|
||||
}
|
||||
// if PassUAToUpsteam is true, then pass the user-agent to the upstream
|
||||
if d.PassUAToUpsteam {
|
||||
userAgent := args.Header.Get("user-agent")
|
||||
if userAgent != "" {
|
||||
headers["User-Agent"] = userAgent
|
||||
}
|
||||
}
|
||||
// if PassIPToUpsteam is true, then pass the ip address to the upstream
|
||||
if d.PassIPToUpsteam {
|
||||
ip := args.IP
|
||||
if ip != "" {
|
||||
headers["X-Forwarded-For"] = ip
|
||||
headers["X-Real-Ip"] = ip
|
||||
}
|
||||
}
|
||||
_, _, err := d.request("/fs/get", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(FsGetReq{
|
||||
Path: file.GetPath(),
|
||||
Password: d.MetaPassword,
|
||||
}).SetHeaders(headers)
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Link{
|
||||
URL: resp.Data.RawURL,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
_, _, err := d.request("/fs/mkdir", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(MkdirOrLinkReq{
|
||||
Path: path.Join(parentDir.GetPath(), dirName),
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
_, _, err := d.request("/fs/move", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(MoveCopyReq{
|
||||
SrcDir: path.Dir(srcObj.GetPath()),
|
||||
DstDir: dstDir.GetPath(),
|
||||
Names: []string{srcObj.GetName()},
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
_, _, err := d.request("/fs/rename", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(RenameReq{
|
||||
Path: srcObj.GetPath(),
|
||||
Name: newName,
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
_, _, err := d.request("/fs/copy", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(MoveCopyReq{
|
||||
SrcDir: path.Dir(srcObj.GetPath()),
|
||||
DstDir: dstDir.GetPath(),
|
||||
Names: []string{srcObj.GetName()},
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Remove(ctx context.Context, obj model.Obj) error {
|
||||
_, _, err := d.request("/fs/remove", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(RemoveReq{
|
||||
Dir: path.Dir(obj.GetPath()),
|
||||
Names: []string{obj.GetName()},
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) Put(ctx context.Context, dstDir model.Obj, s model.FileStreamer, up driver.UpdateProgress) error {
|
||||
reader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: s,
|
||||
UpdateProgress: up,
|
||||
})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, d.Address+"/api/fs/put", reader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", d.Token)
|
||||
req.Header.Set("File-Path", path.Join(dstDir.GetPath(), s.GetName()))
|
||||
req.Header.Set("Password", d.MetaPassword)
|
||||
if md5 := s.GetHash().GetHash(utils.MD5); len(md5) > 0 {
|
||||
req.Header.Set("X-File-Md5", md5)
|
||||
}
|
||||
if sha1 := s.GetHash().GetHash(utils.SHA1); len(sha1) > 0 {
|
||||
req.Header.Set("X-File-Sha1", sha1)
|
||||
}
|
||||
if sha256 := s.GetHash().GetHash(utils.SHA256); len(sha256) > 0 {
|
||||
req.Header.Set("X-File-Sha256", sha256)
|
||||
}
|
||||
|
||||
req.ContentLength = s.GetSize()
|
||||
// client := base.NewHttpClient()
|
||||
// client.Timeout = time.Hour * 6
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
bytes, err := io.ReadAll(res.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
log.Debugf("[openlist] response body: %s", string(bytes))
|
||||
if res.StatusCode >= 400 {
|
||||
return fmt.Errorf("request failed, status: %s", res.Status)
|
||||
}
|
||||
code := utils.Json.Get(bytes, "code").ToInt()
|
||||
if code != 200 {
|
||||
if code == 401 || code == 403 {
|
||||
err = d.login()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("request failed,code: %d, message: %s", code, utils.Json.Get(bytes, "message").ToString())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AListV3) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
if !d.ForwardArchiveReq {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
var resp common.Resp[ArchiveMetaResp]
|
||||
_, code, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Password: d.MetaPassword,
|
||||
Path: obj.GetPath(),
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if code == 202 {
|
||||
return nil, errs.WrongArchivePassword
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tree []model.ObjTree
|
||||
if resp.Data.Content != nil {
|
||||
tree = make([]model.ObjTree, 0, len(resp.Data.Content))
|
||||
for _, content := range resp.Data.Content {
|
||||
tree = append(tree, &content)
|
||||
}
|
||||
}
|
||||
return &model.ArchiveMetaInfo{
|
||||
Comment: resp.Data.Comment,
|
||||
Encrypted: resp.Data.Encrypted,
|
||||
Tree: tree,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
if !d.ForwardArchiveReq {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
var resp common.Resp[ArchiveListResp]
|
||||
_, code, err := d.request("/fs/archive/list", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveListReq{
|
||||
ArchiveMetaReq: ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Password: d.MetaPassword,
|
||||
Path: obj.GetPath(),
|
||||
Refresh: false,
|
||||
},
|
||||
PageReq: model.PageReq{
|
||||
Page: 1,
|
||||
PerPage: 0,
|
||||
},
|
||||
InnerPath: args.InnerPath,
|
||||
})
|
||||
})
|
||||
if code == 202 {
|
||||
return nil, errs.WrongArchivePassword
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []model.Obj
|
||||
for _, f := range resp.Data.Content {
|
||||
file := model.ObjThumb{
|
||||
Object: model.Object{
|
||||
Name: f.Name,
|
||||
Modified: f.Modified,
|
||||
Ctime: f.Created,
|
||||
Size: f.Size,
|
||||
IsFolder: f.IsDir,
|
||||
HashInfo: utils.FromString(f.HashInfo),
|
||||
},
|
||||
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
|
||||
}
|
||||
files = append(files, &file)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
if !d.ForwardArchiveReq {
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
var resp common.Resp[ArchiveMetaResp]
|
||||
_, _, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Password: d.MetaPassword,
|
||||
Path: obj.GetPath(),
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.Link{
|
||||
URL: fmt.Sprintf("%s?inner=%s&pass=%s&sign=%s",
|
||||
resp.Data.RawURL,
|
||||
utils.EncodePath(args.InnerPath, true),
|
||||
url.QueryEscape(args.Password),
|
||||
resp.Data.Sign),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AListV3) ArchiveDecompress(ctx context.Context, srcObj, dstDir model.Obj, args model.ArchiveDecompressArgs) error {
|
||||
if !d.ForwardArchiveReq {
|
||||
return errs.NotImplement
|
||||
}
|
||||
dir, name := path.Split(srcObj.GetPath())
|
||||
_, _, err := d.request("/fs/archive/decompress", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetBody(DecompressReq{
|
||||
ArchivePass: args.Password,
|
||||
CacheFull: args.CacheFull,
|
||||
DstDir: dstDir.GetPath(),
|
||||
InnerPath: args.InnerPath,
|
||||
Name: []string{name},
|
||||
PutIntoNewDir: args.PutIntoNewDir,
|
||||
SrcDir: dir,
|
||||
})
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *AListV3) ResolveLinkCacheMode(_ string) driver.LinkCacheMode {
|
||||
var mode driver.LinkCacheMode
|
||||
if d.PassIPToUpsteam {
|
||||
mode |= driver.LinkCacheIP
|
||||
}
|
||||
if d.PassUAToUpsteam {
|
||||
mode |= driver.LinkCacheUA
|
||||
}
|
||||
return mode
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*AListV3)(nil)
|
||||
@@ -1,182 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type ListReq struct {
|
||||
model.PageReq
|
||||
Path string `json:"path" form:"path"`
|
||||
Password string `json:"password" form:"password"`
|
||||
Refresh bool `json:"refresh"`
|
||||
}
|
||||
|
||||
type ObjResp struct {
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
IsDir bool `json:"is_dir"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Created time.Time `json:"created"`
|
||||
Sign string `json:"sign"`
|
||||
Thumb string `json:"thumb"`
|
||||
Type int `json:"type"`
|
||||
HashInfo string `json:"hashinfo"`
|
||||
}
|
||||
|
||||
type FsListResp struct {
|
||||
Content []ObjResp `json:"content"`
|
||||
Total int64 `json:"total"`
|
||||
Readme string `json:"readme"`
|
||||
Write bool `json:"write"`
|
||||
Provider string `json:"provider"`
|
||||
}
|
||||
|
||||
type FsGetReq struct {
|
||||
Path string `json:"path" form:"path"`
|
||||
Password string `json:"password" form:"password"`
|
||||
}
|
||||
|
||||
type FsGetResp struct {
|
||||
ObjResp
|
||||
RawURL string `json:"raw_url"`
|
||||
Readme string `json:"readme"`
|
||||
Provider string `json:"provider"`
|
||||
Related []ObjResp `json:"related"`
|
||||
}
|
||||
|
||||
type MkdirOrLinkReq struct {
|
||||
Path string `json:"path" form:"path"`
|
||||
}
|
||||
|
||||
type MoveCopyReq struct {
|
||||
SrcDir string `json:"src_dir"`
|
||||
DstDir string `json:"dst_dir"`
|
||||
Names []string `json:"names"`
|
||||
}
|
||||
|
||||
type RenameReq struct {
|
||||
Path string `json:"path"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type RemoveReq struct {
|
||||
Dir string `json:"dir"`
|
||||
Names []string `json:"names"`
|
||||
}
|
||||
|
||||
type LoginResp struct {
|
||||
Token string `json:"token"`
|
||||
}
|
||||
|
||||
type MeResp struct {
|
||||
Id int `json:"id"`
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
BasePath string `json:"base_path"`
|
||||
Role IntSlice `json:"role"`
|
||||
Disabled bool `json:"disabled"`
|
||||
Permission int `json:"permission"`
|
||||
SsoId string `json:"sso_id"`
|
||||
Otp bool `json:"otp"`
|
||||
}
|
||||
|
||||
type IntSlice []int
|
||||
|
||||
func (s *IntSlice) UnmarshalJSON(b []byte) error {
|
||||
var i int
|
||||
if json.Unmarshal(b, &i) == nil {
|
||||
*s = []int{i}
|
||||
return nil
|
||||
}
|
||||
return json.Unmarshal(b, (*[]int)(s))
|
||||
}
|
||||
|
||||
type ArchiveMetaReq struct {
|
||||
ArchivePass string `json:"archive_pass"`
|
||||
Password string `json:"password"`
|
||||
Path string `json:"path"`
|
||||
Refresh bool `json:"refresh"`
|
||||
}
|
||||
|
||||
type TreeResp struct {
|
||||
ObjResp
|
||||
Children []TreeResp `json:"children"`
|
||||
hashCache *utils.HashInfo
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetSize() int64 {
|
||||
return t.Size
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetName() string {
|
||||
return t.Name
|
||||
}
|
||||
|
||||
func (t *TreeResp) ModTime() time.Time {
|
||||
return t.Modified
|
||||
}
|
||||
|
||||
func (t *TreeResp) CreateTime() time.Time {
|
||||
return t.Created
|
||||
}
|
||||
|
||||
func (t *TreeResp) IsDir() bool {
|
||||
return t.ObjResp.IsDir
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetHash() utils.HashInfo {
|
||||
return utils.FromString(t.HashInfo)
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetID() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetPath() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetChildren() []model.ObjTree {
|
||||
ret := make([]model.ObjTree, 0, len(t.Children))
|
||||
for _, child := range t.Children {
|
||||
ret = append(ret, &child)
|
||||
}
|
||||
return ret
|
||||
}
|
||||
|
||||
func (t *TreeResp) Thumb() string {
|
||||
return t.ObjResp.Thumb
|
||||
}
|
||||
|
||||
type ArchiveMetaResp struct {
|
||||
Comment string `json:"comment"`
|
||||
Encrypted bool `json:"encrypted"`
|
||||
Content []TreeResp `json:"content"`
|
||||
RawURL string `json:"raw_url"`
|
||||
Sign string `json:"sign"`
|
||||
}
|
||||
|
||||
type ArchiveListReq struct {
|
||||
model.PageReq
|
||||
ArchiveMetaReq
|
||||
InnerPath string `json:"inner_path"`
|
||||
}
|
||||
|
||||
type ArchiveListResp struct {
|
||||
Content []ObjResp `json:"content"`
|
||||
Total int64 `json:"total"`
|
||||
}
|
||||
|
||||
type DecompressReq struct {
|
||||
ArchivePass string `json:"archive_pass"`
|
||||
CacheFull bool `json:"cache_full"`
|
||||
DstDir string `json:"dst_dir"`
|
||||
InnerPath string `json:"inner_path"`
|
||||
Name []string `json:"name"`
|
||||
PutIntoNewDir bool `json:"put_into_new_dir"`
|
||||
SrcDir string `json:"src_dir"`
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
package alist_v3
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func (d *AListV3) login() error {
|
||||
if d.Username == "" {
|
||||
return nil
|
||||
}
|
||||
var resp common.Resp[LoginResp]
|
||||
_, _, err := d.request("/auth/login", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(base.Json{
|
||||
"username": d.Username,
|
||||
"password": d.Password,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.Token = resp.Data.Token
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AListV3) request(api, method string, callback base.ReqCallback, retry ...bool) ([]byte, int, error) {
|
||||
url := d.Address + "/api" + api
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeader("Authorization", d.Token)
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
res, err := req.Execute(method, url)
|
||||
if err != nil {
|
||||
code := 0
|
||||
if res != nil {
|
||||
code = res.StatusCode()
|
||||
}
|
||||
return nil, code, err
|
||||
}
|
||||
log.Debugf("[openlist] response body: %s", res.String())
|
||||
if res.StatusCode() >= 400 {
|
||||
return nil, res.StatusCode(), fmt.Errorf("request failed, status: %s", res.Status())
|
||||
}
|
||||
code := utils.Json.Get(res.Body(), "code").ToInt()
|
||||
if code != 200 {
|
||||
if (code == 401 || code == 403) && !utils.IsBool(retry...) {
|
||||
err = d.login()
|
||||
if err != nil {
|
||||
return nil, code, err
|
||||
}
|
||||
return d.request(api, method, callback, true)
|
||||
}
|
||||
return nil, code, fmt.Errorf("request failed,code: %d, message: %s", code, utils.Json.Get(res.Body(), "message").ToString())
|
||||
}
|
||||
return res.Body(), 200, nil
|
||||
}
|
||||
@@ -1,169 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/antchfx/htmlquery"
|
||||
"github.com/antchfx/xpath"
|
||||
"github.com/pkg/errors"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
type AutoIndex struct {
|
||||
model.Storage
|
||||
Addition
|
||||
itemXPath *xpath.Expr
|
||||
nameXPath *xpath.Expr
|
||||
modifiedXPath *xpath.Expr
|
||||
sizeXPath *xpath.Expr
|
||||
ignores map[string]any
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *AutoIndex) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Init(ctx context.Context) error {
|
||||
var err error
|
||||
d.itemXPath, err = xpath.Compile(d.ItemXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Item XPath")
|
||||
}
|
||||
d.nameXPath, err = xpath.Compile(d.NameXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Name XPath")
|
||||
}
|
||||
if len(d.ModifiedXPath) > 0 {
|
||||
d.modifiedXPath, err = xpath.Compile(d.ModifiedXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Modified XPath")
|
||||
}
|
||||
}
|
||||
if len(d.SizeXPath) > 0 {
|
||||
d.sizeXPath, err = xpath.Compile(d.SizeXPath)
|
||||
if err != nil {
|
||||
return errors.WithMessage(err, "failed to compile Size XPath")
|
||||
}
|
||||
}
|
||||
ignores := strings.Split(d.IgnoreFileNames, "\n")
|
||||
d.ignores = make(map[string]any, len(ignores))
|
||||
for _, i := range ignores {
|
||||
i = strings.TrimSpace(i)
|
||||
if len(i) == 0 {
|
||||
continue
|
||||
}
|
||||
d.ignores[i] = struct{}{}
|
||||
}
|
||||
hasScheme := strings.Contains(d.URL, "://")
|
||||
hasSuffix := strings.HasSuffix(d.URL, "/")
|
||||
if !hasScheme || !hasSuffix {
|
||||
if !hasSuffix {
|
||||
d.URL = d.URL + "/"
|
||||
}
|
||||
if !hasScheme {
|
||||
d.URL = "https://" + d.URL
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) GetRoot(ctx context.Context) (model.Obj, error) {
|
||||
return &model.Object{
|
||||
Name: op.RootName,
|
||||
Path: d.URL,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Locked,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
res, err := base.RestyClient.R().
|
||||
SetContext(ctx).
|
||||
SetDoNotParseResponse(true).
|
||||
Get(dir.GetPath())
|
||||
if err != nil {
|
||||
return nil, errors.WithMessagef(err, "failed to get url [%s]", dir.GetPath())
|
||||
}
|
||||
defer res.RawResponse.Body.Close()
|
||||
doc, err := htmlquery.Parse(res.RawBody())
|
||||
if err != nil {
|
||||
return nil, errors.WithMessagef(err, "failed to parse [%s]", dir.GetPath())
|
||||
}
|
||||
itemsIter := d.itemXPath.Select(htmlquery.CreateXPathNavigator(doc))
|
||||
var objs []model.Obj
|
||||
for itemsIter.MoveNext() {
|
||||
nameFull, err := parseString(d.nameXPath.Evaluate(itemsIter.Current().Copy()))
|
||||
if err != nil {
|
||||
log.Warnf("skip invalid name evaluating result: %v", err)
|
||||
continue
|
||||
}
|
||||
nameFull = strings.TrimSpace(nameFull)
|
||||
name, isDir := strings.CutSuffix(nameFull, "/")
|
||||
if _, ok := d.ignores[name]; ok {
|
||||
continue
|
||||
}
|
||||
var size int64 = 0
|
||||
exact := false
|
||||
modified := time.Now()
|
||||
if d.sizeXPath != nil {
|
||||
size, exact, err = parseSize(d.sizeXPath.Evaluate(itemsIter.Current().Copy()))
|
||||
if err != nil {
|
||||
log.Errorf("failed to parse size of %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
if d.modifiedXPath != nil {
|
||||
modified, err = parseTime(d.modifiedXPath.Evaluate(itemsIter.Current().Copy()), d.ModifiedTimeFormat)
|
||||
if err != nil {
|
||||
log.Errorf("failed to parse modified time of %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
var o model.Obj = &model.Object{
|
||||
Name: name,
|
||||
IsFolder: isDir,
|
||||
Path: dir.GetPath() + nameFull,
|
||||
Modified: modified,
|
||||
Size: size,
|
||||
}
|
||||
if exact {
|
||||
o = &exactSizeObj{Obj: o}
|
||||
}
|
||||
objs = append(objs, o)
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *AutoIndex) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if _, ok := file.(*exactSizeObj); ok || args.Redirect {
|
||||
return &model.Link{URL: file.GetPath()}, nil
|
||||
}
|
||||
res, err := base.RestyClient.R().
|
||||
SetContext(ctx).
|
||||
SetDoNotParseResponse(true).
|
||||
Head(file.GetPath())
|
||||
if err != nil {
|
||||
return nil, errors.WithMessagef(err, "failed to head [%s]", file.GetPath())
|
||||
}
|
||||
_ = res.RawResponse.Body.Close()
|
||||
return &model.Link{
|
||||
URL: file.GetPath(),
|
||||
ContentLength: res.RawResponse.ContentLength,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*AutoIndex)(nil)
|
||||
@@ -1,29 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
URL string `json:"url" required:"true"`
|
||||
ItemXPath string `json:"item_xpath" required:"true"`
|
||||
NameXPath string `json:"name_xpath" required:"true"`
|
||||
ModifiedXPath string `json:"modified_xpath"`
|
||||
SizeXPath string `json:"size_xpath"`
|
||||
IgnoreFileNames string `json:"ignore_file_names" type:"text" default:".\n..\nParent Directory\nUp"`
|
||||
ModifiedTimeFormat string `json:"modified_time_format" default:"02-Jan-2006 15:04" help:"Must be based on the time point Mon Jan 2 15:04:05 -0700 MST 2006"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "AutoIndex",
|
||||
LocalSort: true,
|
||||
CheckStatus: true,
|
||||
NoUpload: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &AutoIndex{}
|
||||
})
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
var (
|
||||
errEmptyEvaluateResult = fmt.Errorf("empty result")
|
||||
)
|
||||
|
||||
type exactSizeObj struct{ model.Obj }
|
||||
@@ -1,116 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/antchfx/xpath"
|
||||
"github.com/pkg/errors"
|
||||
)
|
||||
|
||||
var units = map[string]int64{
|
||||
"": 1,
|
||||
"b": 1,
|
||||
"byte": 1,
|
||||
"bytes": 1,
|
||||
"k": 1 << 10,
|
||||
"kb": 1 << 10,
|
||||
"kib": 1 << 10,
|
||||
"m": 1 << 20,
|
||||
"mb": 1 << 20,
|
||||
"mib": 1 << 20,
|
||||
"g": 1 << 30,
|
||||
"gb": 1 << 30,
|
||||
"gib": 1 << 30,
|
||||
"t": 1 << 40,
|
||||
"tb": 1 << 40,
|
||||
"tib": 1 << 40,
|
||||
"p": 1 << 50,
|
||||
"pb": 1 << 50,
|
||||
"pib": 1 << 50,
|
||||
}
|
||||
|
||||
func splitUnit(s string) (string, string) {
|
||||
for i := len(s) - 1; i >= 0; i-- {
|
||||
if s[i] >= '0' && s[i] <= '9' {
|
||||
return strings.TrimSpace(s[:i+1]), strings.TrimSpace(s[i+1:])
|
||||
}
|
||||
}
|
||||
return "", s
|
||||
}
|
||||
|
||||
func parseSize(a any) (int64, bool, error) {
|
||||
// 第二个返回值exact表示大小是否精确
|
||||
if f, ok := a.(float64); ok {
|
||||
return int64(f), false, nil
|
||||
}
|
||||
s, err := parseString(a)
|
||||
if errors.Is(err, errEmptyEvaluateResult) {
|
||||
// 可能是错误,也可能确实大小为0
|
||||
// 如果确实大小为0,大概率不会下载,exact返回false也不会有什么性能损失
|
||||
// 如果是错误,exact返回true会导致本地代理出错,综合来看返回false更好
|
||||
return 0, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return 0, false, err
|
||||
}
|
||||
s = strings.TrimSpace(s)
|
||||
if s == "-" {
|
||||
return 0, false, nil
|
||||
}
|
||||
nbs, unit := splitUnit(s)
|
||||
mul, ok := units[strings.ToLower(unit)]
|
||||
exact := mul == 1
|
||||
if !ok {
|
||||
mul = 1
|
||||
// 推测无单位,exact应为false
|
||||
}
|
||||
nb, err := strconv.ParseInt(nbs, 10, 64)
|
||||
if err != nil {
|
||||
fnb, err := strconv.ParseFloat(nbs, 64)
|
||||
if err != nil {
|
||||
return 0, false, fmt.Errorf("failed to convert %s to number", nbs)
|
||||
}
|
||||
nb = int64(fnb * float64(mul))
|
||||
exact = false
|
||||
} else {
|
||||
nb = nb * mul
|
||||
}
|
||||
return nb, exact, nil
|
||||
}
|
||||
|
||||
func parseString(res any) (string, error) {
|
||||
if r, ok := res.(string); ok {
|
||||
if len(r) == 0 {
|
||||
return "", errEmptyEvaluateResult
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
n, ok := res.(*xpath.NodeIterator)
|
||||
if !ok {
|
||||
return "", fmt.Errorf("unsupported evaluating result")
|
||||
}
|
||||
if !n.MoveNext() {
|
||||
return "", fmt.Errorf("no matched nodes")
|
||||
}
|
||||
ns := n.Current().Value()
|
||||
if len(ns) == 0 {
|
||||
return "", errEmptyEvaluateResult
|
||||
}
|
||||
return ns, nil
|
||||
}
|
||||
|
||||
func parseTime(res any, format string) (time.Time, error) {
|
||||
s, err := parseString(res)
|
||||
if err != nil {
|
||||
return time.Now(), err
|
||||
}
|
||||
s = strings.TrimSpace(s)
|
||||
t, err := time.Parse(format, s)
|
||||
if err != nil {
|
||||
return time.Now(), errors.WithMessagef(err, "failed to convert %s to time", s)
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
package autoindex
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
type wantType struct {
|
||||
v int64
|
||||
exact bool
|
||||
error bool
|
||||
}
|
||||
|
||||
func TestParseSize(t *testing.T) {
|
||||
tests := []struct {
|
||||
input string
|
||||
want wantType
|
||||
}{
|
||||
{"100", wantType{100, true, false}},
|
||||
{"1k", wantType{1024, false, false}},
|
||||
{"1kb", wantType{1024, false, false}},
|
||||
{"1K", wantType{1024, false, false}}, // case insensitive
|
||||
{"1.5m", wantType{1572864, false, false}}, // 1.5 * 1024^2
|
||||
{"500 bytes", wantType{500, true, false}},
|
||||
{"-", wantType{0, false, false}},
|
||||
{"", wantType{0, false, false}},
|
||||
{"abc", wantType{0, false, true}},
|
||||
{"1.5GB", wantType{1610612736, false, false}}, // 1.5 * 1024^3
|
||||
{"2t", wantType{2199023255552, false, false}}, // 2 * 1024^4
|
||||
{"1p", wantType{1125899906842624, false, false}}, // 1 * 1024^5
|
||||
{"0", wantType{0, true, false}},
|
||||
{" 100 ", wantType{100, true, false}}, // trimmed
|
||||
{"100b", wantType{100, true, false}},
|
||||
{"1gib", wantType{1073741824, false, false}}, // 1024^3
|
||||
{"1z", wantType{1, false, false}}, // invalid unit, mul=1
|
||||
{"1.5", wantType{1, false, false}}, // float without unit, truncated
|
||||
{"2.7k", wantType{2764, false, false}}, // 2.7 * 1024 truncated
|
||||
{"1.0g", wantType{1073741824, false, false}}, // 1.0 * 1024^3
|
||||
{"invalid", wantType{0, false, true}},
|
||||
{"123xyz", wantType{123, false, false}}, // unit not found, mul=1
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.input, func(t *testing.T) {
|
||||
got, exact, err := parseSize(tt.input)
|
||||
if got != tt.want.v || exact != tt.want.exact || (err != nil) != tt.want.error {
|
||||
t.Errorf("ParseSize(%q) = (%d, %t, %t), want (%d, %t, %t)", tt.input, got, exact, err != nil, tt.want.v, tt.want.exact, tt.want.error)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,545 +0,0 @@
|
||||
package chunk
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
stdpath "path"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/conf"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/fs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/sign"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/avast/retry-go"
|
||||
)
|
||||
|
||||
type Chunk struct {
|
||||
model.Storage
|
||||
Addition
|
||||
}
|
||||
|
||||
func (d *Chunk) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Chunk) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Chunk) Init(ctx context.Context) error {
|
||||
if d.PartSize <= 0 {
|
||||
return errors.New("part size must be positive")
|
||||
}
|
||||
if len(d.ChunkPrefix) <= 0 {
|
||||
return errors.New("chunk folder prefix must not be empty")
|
||||
}
|
||||
d.RemotePath = utils.FixAndCleanPath(d.RemotePath)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Chunk) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (*Chunk) GetRootPath() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (d *Chunk) Get(ctx context.Context, path string) (model.Obj, error) {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
remoteActualPath = stdpath.Join(remoteActualPath, path)
|
||||
if remoteObj, err := op.Get(ctx, remoteStorage, remoteActualPath); err == nil {
|
||||
return &model.Object{
|
||||
Path: path,
|
||||
Name: remoteObj.GetName(),
|
||||
Size: remoteObj.GetSize(),
|
||||
Modified: remoteObj.ModTime(),
|
||||
IsFolder: remoteObj.IsDir(),
|
||||
HashInfo: remoteObj.GetHash(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
remoteActualDir, name := stdpath.Split(remoteActualPath)
|
||||
chunkName := d.ChunkPrefix + name
|
||||
chunkObjs, err := op.List(ctx, remoteStorage, stdpath.Join(remoteActualDir, chunkName), model.ListArgs{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var totalSize int64
|
||||
// 0号块默认为-1 以支持空文件
|
||||
chunkSizes := []int64{-1}
|
||||
h := make(map[*utils.HashType]string)
|
||||
var first model.Obj
|
||||
for _, o := range chunkObjs {
|
||||
if o.IsDir() {
|
||||
continue
|
||||
}
|
||||
if after, ok := strings.CutPrefix(o.GetName(), "hash_"); ok {
|
||||
hn, value, ok := strings.Cut(strings.TrimSuffix(after, d.CustomExt), "_")
|
||||
if ok {
|
||||
ht, ok := utils.GetHashByName(hn)
|
||||
if ok {
|
||||
h[ht] = value
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
idx, err := strconv.Atoi(strings.TrimSuffix(o.GetName(), d.CustomExt))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
totalSize += o.GetSize()
|
||||
if len(chunkSizes) > idx {
|
||||
if idx == 0 {
|
||||
first = o
|
||||
}
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
} else if len(chunkSizes) == idx {
|
||||
chunkSizes = append(chunkSizes, o.GetSize())
|
||||
} else {
|
||||
newChunkSizes := make([]int64, idx+1)
|
||||
copy(newChunkSizes, chunkSizes)
|
||||
chunkSizes = newChunkSizes
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
}
|
||||
}
|
||||
reqDir, _ := stdpath.Split(path)
|
||||
// 文件块不完整时,返回文件夹对象
|
||||
if chunkSizes[0] == -1 {
|
||||
return &model.Object{
|
||||
Path: stdpath.Join(reqDir, chunkName),
|
||||
Name: chunkName,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
}, nil
|
||||
}
|
||||
for i, l := 1, len(chunkSizes)-1; i < l; i++ {
|
||||
if chunkSizes[i] == 0 {
|
||||
return &model.Object{
|
||||
Path: stdpath.Join(reqDir, chunkName),
|
||||
Name: chunkName,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
objRes := chunkObject{
|
||||
Object: model.Object{
|
||||
Path: stdpath.Join(reqDir, chunkName),
|
||||
Name: name,
|
||||
Size: totalSize,
|
||||
Modified: first.ModTime(),
|
||||
Ctime: first.CreateTime(),
|
||||
},
|
||||
chunkSizes: chunkSizes,
|
||||
}
|
||||
if len(h) > 0 {
|
||||
objRes.HashInfo = utils.NewHashInfoByMap(h)
|
||||
}
|
||||
return &objRes, nil
|
||||
}
|
||||
|
||||
func (d *Chunk) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
remoteActualDir := stdpath.Join(remoteActualPath, dir.GetPath())
|
||||
remoteObjs, err := op.List(ctx, remoteStorage, remoteActualDir, model.ListArgs{
|
||||
ReqPath: args.ReqPath,
|
||||
Refresh: args.Refresh,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := make([]model.Obj, 0, len(remoteObjs))
|
||||
listG, listCtx := errgroup.NewGroupWithContext(ctx, d.NumListWorkers, retry.Attempts(3))
|
||||
for _, obj := range remoteObjs {
|
||||
if utils.IsCanceled(listCtx) {
|
||||
break
|
||||
}
|
||||
rawName := obj.GetName()
|
||||
if obj.IsDir() {
|
||||
if name, ok := strings.CutPrefix(rawName, d.ChunkPrefix); ok {
|
||||
resultIdx := len(result)
|
||||
result = append(result, nil)
|
||||
listG.Go(func(ctx context.Context) error {
|
||||
chunkObjs, err := op.List(ctx, remoteStorage, stdpath.Join(remoteActualDir, rawName), model.ListArgs{
|
||||
Refresh: args.Refresh,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var totalSize int64
|
||||
// 0号块默认为-1 以支持空文件
|
||||
chunkSizes := []int64{-1}
|
||||
h := make(map[*utils.HashType]string)
|
||||
var first model.Obj
|
||||
for _, o := range chunkObjs {
|
||||
if o.IsDir() {
|
||||
continue
|
||||
}
|
||||
if after, ok := strings.CutPrefix(o.GetName(), "hash_"); ok {
|
||||
hn, value, ok := strings.Cut(strings.TrimSuffix(after, d.CustomExt), "_")
|
||||
if ok {
|
||||
ht, ok := utils.GetHashByName(hn)
|
||||
if ok {
|
||||
h[ht] = value
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
idx, err := strconv.Atoi(strings.TrimSuffix(o.GetName(), d.CustomExt))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
totalSize += o.GetSize()
|
||||
if len(chunkSizes) > idx {
|
||||
if idx == 0 {
|
||||
first = o
|
||||
}
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
} else if len(chunkSizes) == idx {
|
||||
chunkSizes = append(chunkSizes, o.GetSize())
|
||||
} else {
|
||||
newChunkSizes := make([]int64, idx+1)
|
||||
copy(newChunkSizes, chunkSizes)
|
||||
chunkSizes = newChunkSizes
|
||||
chunkSizes[idx] = o.GetSize()
|
||||
}
|
||||
}
|
||||
// 文件块不完整时,返回文件夹对象
|
||||
if chunkSizes[0] == -1 {
|
||||
result[resultIdx] = &model.Object{
|
||||
Name: rawName,
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: true,
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for i, l := 1, len(chunkSizes)-1; i < l; i++ {
|
||||
if chunkSizes[i] == 0 {
|
||||
result[resultIdx] = &model.Object{
|
||||
Name: rawName,
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: true,
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
objRes := model.Object{
|
||||
Name: name,
|
||||
Size: totalSize,
|
||||
Modified: first.ModTime(),
|
||||
Ctime: first.CreateTime(),
|
||||
}
|
||||
if len(h) > 0 {
|
||||
objRes.HashInfo = utils.NewHashInfoByMap(h)
|
||||
}
|
||||
if !d.Thumbnail {
|
||||
result[resultIdx] = &objRes
|
||||
} else {
|
||||
thumbPath := stdpath.Join(args.ReqPath, ".thumbnails", name+".webp")
|
||||
thumb := fmt.Sprintf("%s/d%s?sign=%s",
|
||||
common.GetApiUrl(ctx),
|
||||
utils.EncodePath(thumbPath, true),
|
||||
sign.Sign(thumbPath))
|
||||
result[resultIdx] = &model.ObjThumb{
|
||||
Object: objRes,
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumb,
|
||||
},
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
if !d.ShowHidden && strings.HasPrefix(rawName, ".") {
|
||||
continue
|
||||
}
|
||||
thumb, ok := model.GetThumb(obj)
|
||||
objRes := model.Object{
|
||||
Name: rawName,
|
||||
Size: obj.GetSize(),
|
||||
Modified: obj.ModTime(),
|
||||
IsFolder: obj.IsDir(),
|
||||
HashInfo: obj.GetHash(),
|
||||
}
|
||||
if !ok {
|
||||
result = append(result, &objRes)
|
||||
} else {
|
||||
result = append(result, &model.ObjThumb{
|
||||
Object: objRes,
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumb,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
if err = listG.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (d *Chunk) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
chunkFile, ok := file.(*chunkObject)
|
||||
remoteActualPath = stdpath.Join(remoteActualPath, file.GetPath())
|
||||
if !ok {
|
||||
l, _, err := op.Link(ctx, remoteStorage, remoteActualPath, args)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return l.Clone(), nil
|
||||
}
|
||||
// 检查0号块不等于-1 以支持空文件
|
||||
// 如果块数量大于1 最后一块不可能为0
|
||||
// 只检查中间块是否有0
|
||||
if chunkFile.chunkSizes[0] == -1 {
|
||||
return nil, fmt.Errorf("chunk part[%d] are missing", 0)
|
||||
}
|
||||
for i, l := 1, len(chunkFile.chunkSizes)-1; i < l; i++ {
|
||||
if chunkFile.chunkSizes[i] == 0 {
|
||||
return nil, fmt.Errorf("chunk part[%d] are missing", i)
|
||||
}
|
||||
}
|
||||
fileSize := chunkFile.GetSize()
|
||||
mergedRrf := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
|
||||
start := httpRange.Start
|
||||
length := httpRange.Length
|
||||
if length < 0 || start+length > fileSize {
|
||||
length = fileSize - start
|
||||
}
|
||||
if length == 0 {
|
||||
return io.NopCloser(strings.NewReader("")), nil
|
||||
}
|
||||
rs := make([]io.Reader, 0)
|
||||
cs := make(utils.Closers, 0)
|
||||
var (
|
||||
rc io.ReadCloser
|
||||
readFrom bool
|
||||
)
|
||||
for idx, chunkSize := range chunkFile.chunkSizes {
|
||||
if readFrom {
|
||||
l, o, err := op.Link(ctx, remoteStorage, stdpath.Join(remoteActualPath, d.getPartName(idx)), args)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
cs = append(cs, l)
|
||||
chunkSize2 := l.ContentLength
|
||||
if chunkSize2 <= 0 {
|
||||
chunkSize2 = o.GetSize()
|
||||
}
|
||||
if chunkSize2 != chunkSize {
|
||||
_ = cs.Close()
|
||||
return nil, fmt.Errorf("chunk part[%d] size not match", idx)
|
||||
}
|
||||
rrf, err := stream.GetRangeReaderFromLink(chunkSize2, l)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
newLength := length - chunkSize2
|
||||
if newLength >= 0 {
|
||||
length = newLength
|
||||
rc, err = rrf.RangeRead(ctx, http_range.Range{Length: -1})
|
||||
} else {
|
||||
rc, err = rrf.RangeRead(ctx, http_range.Range{Length: length})
|
||||
}
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
rs = append(rs, rc)
|
||||
cs = append(cs, rc)
|
||||
if newLength <= 0 {
|
||||
return utils.ReadCloser{
|
||||
Reader: io.MultiReader(rs...),
|
||||
Closer: &cs,
|
||||
}, nil
|
||||
}
|
||||
} else if newStart := start - chunkSize; newStart >= 0 {
|
||||
start = newStart
|
||||
} else {
|
||||
l, o, err := op.Link(ctx, remoteStorage, stdpath.Join(remoteActualPath, d.getPartName(idx)), args)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
cs = append(cs, l)
|
||||
chunkSize2 := l.ContentLength
|
||||
if chunkSize2 <= 0 {
|
||||
chunkSize2 = o.GetSize()
|
||||
}
|
||||
if chunkSize2 != chunkSize {
|
||||
_ = cs.Close()
|
||||
return nil, fmt.Errorf("chunk part[%d] size not match", idx)
|
||||
}
|
||||
rrf, err := stream.GetRangeReaderFromLink(chunkSize2, l)
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
rc, err = rrf.RangeRead(ctx, http_range.Range{Start: start, Length: -1})
|
||||
if err != nil {
|
||||
_ = cs.Close()
|
||||
return nil, err
|
||||
}
|
||||
length -= chunkSize2 - start
|
||||
cs = append(cs, rc)
|
||||
if length <= 0 {
|
||||
return utils.ReadCloser{
|
||||
Reader: rc,
|
||||
Closer: &cs,
|
||||
}, nil
|
||||
}
|
||||
rs = append(rs, rc)
|
||||
readFrom = true
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("invalid range: start=%d,length=%d,fileSize=%d", httpRange.Start, httpRange.Length, fileSize)
|
||||
}
|
||||
return &model.Link{
|
||||
RangeReader: stream.RangeReaderFunc(mergedRrf),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *Chunk) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
path := stdpath.Join(d.RemotePath, parentDir.GetPath(), dirName)
|
||||
return fs.MakeDir(ctx, path)
|
||||
}
|
||||
|
||||
func (d *Chunk) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
src := stdpath.Join(d.RemotePath, srcObj.GetPath())
|
||||
dst := stdpath.Join(d.RemotePath, dstDir.GetPath())
|
||||
_, err := fs.Move(ctx, src, dst)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Chunk) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
if _, ok := srcObj.(*chunkObject); ok {
|
||||
newName = d.ChunkPrefix + newName
|
||||
}
|
||||
return fs.Rename(ctx, stdpath.Join(d.RemotePath, srcObj.GetPath()), newName)
|
||||
}
|
||||
|
||||
func (d *Chunk) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
dst := stdpath.Join(d.RemotePath, dstDir.GetPath())
|
||||
src := stdpath.Join(d.RemotePath, srcObj.GetPath())
|
||||
_, err := fs.Copy(ctx, src, dst)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Chunk) Remove(ctx context.Context, obj model.Obj) error {
|
||||
return fs.Remove(ctx, stdpath.Join(d.RemotePath, obj.GetPath()))
|
||||
}
|
||||
|
||||
func (d *Chunk) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
remoteStorage, remoteActualPath, err := op.GetStorageAndActualPath(d.RemotePath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if (d.Thumbnail && dstDir.GetName() == ".thumbnails") || (d.ChunkLargeFileOnly && file.GetSize() <= d.PartSize) {
|
||||
return op.Put(ctx, remoteStorage, stdpath.Join(remoteActualPath, dstDir.GetPath()), file, up)
|
||||
}
|
||||
upReader := &driver.ReaderUpdatingProgress{
|
||||
Reader: file,
|
||||
UpdateProgress: up,
|
||||
}
|
||||
dst := stdpath.Join(remoteActualPath, dstDir.GetPath(), d.ChunkPrefix+file.GetName())
|
||||
skipHookCtx := context.WithValue(ctx, conf.SkipHookKey, struct{}{})
|
||||
if d.StoreHash {
|
||||
for ht, value := range file.GetHash().All() {
|
||||
_ = op.Put(skipHookCtx, remoteStorage, dst, &stream.FileStream{
|
||||
Obj: &model.Object{
|
||||
Name: fmt.Sprintf("hash_%s_%s%s", ht.Name, value, d.CustomExt),
|
||||
Size: 1,
|
||||
Modified: file.ModTime(),
|
||||
},
|
||||
Mimetype: "application/octet-stream",
|
||||
Reader: bytes.NewReader([]byte{0}), // 兼容不支持空文件的驱动
|
||||
}, nil)
|
||||
}
|
||||
}
|
||||
fullPartCount := int(file.GetSize() / d.PartSize)
|
||||
tailSize := file.GetSize() % d.PartSize
|
||||
if tailSize == 0 && fullPartCount > 0 {
|
||||
fullPartCount--
|
||||
tailSize = d.PartSize
|
||||
}
|
||||
partIndex := 0
|
||||
for partIndex < fullPartCount {
|
||||
err = op.Put(skipHookCtx, remoteStorage, dst, &stream.FileStream{
|
||||
Obj: &model.Object{
|
||||
Name: d.getPartName(partIndex),
|
||||
Size: d.PartSize,
|
||||
Modified: file.ModTime(),
|
||||
},
|
||||
Mimetype: file.GetMimetype(),
|
||||
Reader: io.LimitReader(upReader, d.PartSize),
|
||||
}, nil)
|
||||
if err != nil {
|
||||
_ = op.Remove(ctx, remoteStorage, dst)
|
||||
return err
|
||||
}
|
||||
partIndex++
|
||||
}
|
||||
err = op.Put(ctx, remoteStorage, dst, &stream.FileStream{
|
||||
Obj: &model.Object{
|
||||
Name: d.getPartName(fullPartCount),
|
||||
Size: tailSize,
|
||||
Modified: file.ModTime(),
|
||||
},
|
||||
Mimetype: file.GetMimetype(),
|
||||
Reader: upReader,
|
||||
}, nil)
|
||||
if err != nil {
|
||||
_ = op.Remove(ctx, remoteStorage, dst)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Chunk) getPartName(part int) string {
|
||||
return fmt.Sprintf("%d%s", part, d.CustomExt)
|
||||
}
|
||||
|
||||
func (d *Chunk) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
remoteStorage, err := fs.GetStorage(d.RemotePath, &fs.GetStoragesArgs{})
|
||||
if err != nil {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
remoteDetails, err := op.GetStorageDetails(ctx, remoteStorage)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: remoteDetails.DiskUsage,
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Chunk)(nil)
|
||||
@@ -1,39 +0,0 @@
|
||||
package chunk
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
RemotePath string `json:"remote_path" required:"true"`
|
||||
PartSize int64 `json:"part_size" required:"true" type:"number" help:"bytes"`
|
||||
ChunkLargeFileOnly bool `json:"chunk_large_file_only" default:"false" help:"chunk only if file size > part_size"`
|
||||
ChunkPrefix string `json:"chunk_prefix" type:"string" default:"[openlist_chunk]" help:"the prefix of chunk folder"`
|
||||
CustomExt string `json:"custom_ext" type:"string"`
|
||||
StoreHash bool `json:"store_hash" type:"bool" default:"true"`
|
||||
NumListWorkers int `json:"num_list_workers" required:"true" type:"number" default:"5"`
|
||||
|
||||
Thumbnail bool `json:"thumbnail" required:"true" default:"false" help:"enable thumbnail which pre-generated under .thumbnails folder"`
|
||||
ShowHidden bool `json:"show_hidden" default:"true" required:"false" help:"show hidden directories and files"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Chunk",
|
||||
LocalSort: true,
|
||||
OnlyProxy: true,
|
||||
NoCache: true,
|
||||
DefaultRoot: "/",
|
||||
NoLinkURL: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Chunk{
|
||||
Addition: Addition{
|
||||
ChunkPrefix: "[openlist_chunk]",
|
||||
NumListWorkers: 5,
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
package chunk
|
||||
|
||||
import "github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
|
||||
type chunkObject struct {
|
||||
model.Object
|
||||
chunkSizes []int64
|
||||
}
|
||||
@@ -1,168 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/cache"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type CFImgBed struct {
|
||||
model.Storage
|
||||
Addition
|
||||
client *resty.Client
|
||||
virtualDir *cache.WeakCacheMap[string, model.Object]
|
||||
publicUrlPrefix string
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Config() driver.Config { return config }
|
||||
func (d *CFImgBed) GetAddition() driver.Additional { return &d.Addition }
|
||||
|
||||
func (d *CFImgBed) Init(ctx context.Context) error {
|
||||
d.UploadThread = min(d.UploadThread, 32)
|
||||
if d.UploadThread < 1 {
|
||||
d.UploadThread = 3
|
||||
}
|
||||
d.Address = strings.TrimRight(d.Address, "/")
|
||||
|
||||
d.client = base.NewRestyClient().
|
||||
SetBaseURL(d.Address).
|
||||
SetHeader("Authorization", "Bearer "+d.Token).
|
||||
SetDebug(false)
|
||||
|
||||
// 连通性测试:尝试获取根目录单条数据
|
||||
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"start": "0",
|
||||
"count": "1",
|
||||
"dir": "/",
|
||||
})
|
||||
}, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("init verification failed: %w", err)
|
||||
}
|
||||
d.virtualDir = cache.NewWeakCacheMap[string, model.Object]()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Drop(ctx context.Context) error {
|
||||
if d.virtualDir != nil {
|
||||
d.virtualDir.Clear()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
// if !args.Refresh && model.ObjHasMask(dir, model.Virtual) {
|
||||
// if _, ok := d.virtualDir.Load(dir.GetPath()); ok {
|
||||
// return nil, nil
|
||||
// }
|
||||
// }
|
||||
|
||||
var dirSeen map[string]bool
|
||||
var fileSeen map[string]bool
|
||||
var objs []model.Obj
|
||||
|
||||
start := 0
|
||||
for {
|
||||
var resp ListResponse
|
||||
_, err := d.doRequest(ctx, http.MethodGet, listApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"dir": dir.GetPath(),
|
||||
"start": fmt.Sprintf("%d", start),
|
||||
"count": fmt.Sprintf("%d", listPageSize),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(resp.Files) == 0 && len(resp.Directories) == 0 {
|
||||
break
|
||||
}
|
||||
|
||||
if start == 0 {
|
||||
dirSeen = make(map[string]bool, len(resp.Directories))
|
||||
fileSeen = make(map[string]bool, len(resp.Files))
|
||||
objs = make([]model.Obj, 0, len(resp.Directories)+len(resp.Files))
|
||||
}
|
||||
|
||||
for _, rawDir := range resp.Directories {
|
||||
rawDir = "/" + strings.TrimRight(rawDir, "/")
|
||||
if !dirSeen[rawDir] {
|
||||
dirSeen[rawDir] = true
|
||||
objs = append(objs, &model.Object{
|
||||
Path: rawDir,
|
||||
Name: path.Base(rawDir),
|
||||
Modified: d.Modified,
|
||||
IsFolder: true,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
for _, item := range resp.Files {
|
||||
if !fileSeen[item.Name] {
|
||||
fileSeen[item.Name] = true
|
||||
objs = append(objs, parseFile(item))
|
||||
}
|
||||
}
|
||||
|
||||
// 如果当前获取的数量少于分页大小,说明已加载完毕
|
||||
if len(resp.Files)+len(resp.Directories) < listPageSize {
|
||||
break
|
||||
}
|
||||
start += listPageSize
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if d.publicUrlPrefix != "" {
|
||||
return &model.Link{URL: d.publicUrlPrefix + utils.EncodePath(file.GetPath())}, nil
|
||||
}
|
||||
return &model.Link{URL: d.Address + "/file" + utils.EncodePath(file.GetPath())}, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Get(ctx context.Context, pathStr string) (model.Obj, error) {
|
||||
fullPath := path.Join(d.RootFolderPath, pathStr)
|
||||
if obj, found := d.virtualDir.Load(fullPath); found {
|
||||
return obj, nil
|
||||
}
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
|
||||
// MakeDir 在图床中通常是虚拟的,此处返回虚拟目录对象以支持上传时的路径展示
|
||||
func (d *CFImgBed) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
fullPath := path.Join(parentDir.GetPath(), dirName)
|
||||
temp := &model.Object{
|
||||
Path: fullPath,
|
||||
Name: dirName,
|
||||
IsFolder: true,
|
||||
Modified: d.Modified,
|
||||
Mask: model.Virtual,
|
||||
}
|
||||
d.virtualDir.Store(fullPath, temp)
|
||||
return temp, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) Remove(ctx context.Context, obj model.Obj) error {
|
||||
reqPath := obj.GetPath()
|
||||
if model.ObjHasMask(obj, model.Virtual) {
|
||||
d.virtualDir.Delete(reqPath)
|
||||
return nil
|
||||
}
|
||||
_, err := d.doRequest(ctx, http.MethodPost, deleteApi+utils.EncodePath(reqPath), func(req *resty.Request) {
|
||||
req.SetQueryParam("folder", fmt.Sprintf("%t", obj.IsDir()))
|
||||
}, nil)
|
||||
return err
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*CFImgBed)(nil)
|
||||
@@ -1,27 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Address string `json:"address" required:"true" help:"Backend API address of the image hosting service, e.g., https://img.example.com"`
|
||||
Token string `json:"token" required:"true" help:"Authentication Token"`
|
||||
SmallChannelName string `json:"smallChannelName" help:"Channel name for regular files (typically <20MB)"`
|
||||
LargeChannelName string `json:"largeChannelName" help:"Channel name for large files"`
|
||||
LargeChannelType string `json:"largeChannelType" type:"select" options:",huggingface,telegram,cfr2,s3,discord" help:"Large File Channel Type: Hugging Face (Direct Upload)、telegram/cfr2/s3/discord(Multipart Upload)"`
|
||||
UploadThread int `json:"uploadThread" type:"number" default:"3" help:"Concurrent thread count for HuggingFace chunked direct upload"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "cloudflare_imgbed",
|
||||
LocalSort: true,
|
||||
NoUpload: false,
|
||||
DefaultRoot: "/",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver { return &CFImgBed{} })
|
||||
}
|
||||
@@ -1,117 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
const listPageSize = 1000
|
||||
|
||||
// ListResponse 列表接口响应
|
||||
type ListResponse struct {
|
||||
Files []FileItem `json:"files"`
|
||||
Directories []string `json:"directories"`
|
||||
}
|
||||
|
||||
type FileItem struct {
|
||||
Name string `json:"name"`
|
||||
Metadata map[string]interface{} `json:"metadata"` // 存储文件大小、哈希、时间戳等
|
||||
}
|
||||
|
||||
type apiError struct {
|
||||
Error string `json:"error"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// standardUploadResp 标准上传成功返回的数组
|
||||
type standardUploadResp []struct {
|
||||
Src string `json:"src"`
|
||||
PublicUrl string `json:"publicUrl"`
|
||||
}
|
||||
|
||||
// hfGetUrlResp 获取 HF 直传授权地址的响应
|
||||
type hfGetUrlResp struct {
|
||||
Success bool `json:"success"`
|
||||
FullID string `json:"fullId"`
|
||||
FilePath string `json:"filePath"`
|
||||
ChannelName string `json:"channelName"`
|
||||
Repo string `json:"repo"`
|
||||
NeedsLfs bool `json:"needsLfs"` // 是否需要进行 LFS 物理上传
|
||||
AlreadyExists bool `json:"alreadyExists"` // 是否秒传成功
|
||||
Oid string `json:"oid"` // Git LFS 对象 ID (SHA256)
|
||||
UploadAction *UploadAction `json:"uploadAction"`
|
||||
}
|
||||
|
||||
type UploadAction struct {
|
||||
Href string `json:"href"`
|
||||
Header map[string]string `json:"header"`
|
||||
}
|
||||
|
||||
type hfCommitResp struct {
|
||||
Success bool `json:"success"`
|
||||
Src string `json:"src"`
|
||||
PublicUrl string `json:"publicUrl"`
|
||||
FileUrl string `json:"fileUrl"`
|
||||
FullID string `json:"fullId"`
|
||||
}
|
||||
|
||||
// 辅助函数:从 map 中安全提取字符串/数值
|
||||
func getString(m map[string]interface{}, keys ...string) string {
|
||||
for _, k := range keys {
|
||||
if v, ok := m[k]; ok {
|
||||
switch val := v.(type) {
|
||||
case string:
|
||||
return val
|
||||
case float64:
|
||||
return strconv.FormatInt(int64(val), 10)
|
||||
default:
|
||||
return fmt.Sprintf("%v", val)
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func getInt64(m map[string]interface{}, keys ...string) int64 {
|
||||
for _, k := range keys {
|
||||
if v, ok := m[k]; ok {
|
||||
switch val := v.(type) {
|
||||
case string:
|
||||
n, _ := strconv.ParseInt(val, 10, 64)
|
||||
return n
|
||||
case float64:
|
||||
return int64(val)
|
||||
case int64:
|
||||
return val
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func parseFile(item FileItem) *model.Object {
|
||||
name := path.Base(item.Name)
|
||||
var size int64
|
||||
var modTime time.Time
|
||||
|
||||
if item.Metadata != nil {
|
||||
size = getInt64(item.Metadata, "FileSizeBytes", "File-Size")
|
||||
ts := getInt64(item.Metadata, "TimeStamp")
|
||||
if ts > 0 {
|
||||
modTime = time.UnixMilli(ts)
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: "/" + strings.TrimRight(item.Name, "/"),
|
||||
Name: name,
|
||||
Size: size,
|
||||
Modified: modTime,
|
||||
IsFolder: false,
|
||||
}
|
||||
}
|
||||
@@ -1,560 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/avast/retry-go"
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func (d *CFImgBed) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (newObj model.Obj, err error) {
|
||||
if file.GetSize() < hfDirectThreshold {
|
||||
newObj, err = d.standardUpload(ctx, dstDir, file, up)
|
||||
} else {
|
||||
switch d.LargeChannelType {
|
||||
case "huggingface":
|
||||
newObj, err = d.hfDirectUpload(ctx, dstDir, file, up)
|
||||
case "telegram", "cfr2", "s3", "discord":
|
||||
newObj, err = d.chunkedUpload(ctx, dstDir, file, up, d.LargeChannelType, d.LargeChannelName)
|
||||
default:
|
||||
newObj, err = d.standardUpload(ctx, dstDir, file, up)
|
||||
}
|
||||
}
|
||||
if newObj != nil && model.ObjHasMask(dstDir, model.Virtual) {
|
||||
key := dstDir.GetPath()
|
||||
for d.virtualDir.Delete(key) {
|
||||
key = path.Dir(key)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// standardUpload 通过普通 multipart 表单上传。
|
||||
// 使用 io.MultiReader 实现虚拟拼接,避免将整个大文件读入内存构建表单。
|
||||
func (d *CFImgBed) standardUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
|
||||
channelName := d.SmallChannelName
|
||||
if file.GetSize() >= hfDirectThreshold {
|
||||
channelName = d.LargeChannelName
|
||||
log.WithField("size", file.GetSize()).Warn("large file falls back to standard upload, consider configuring LargeChannelType")
|
||||
}
|
||||
if channelName == "" {
|
||||
return nil, fmt.Errorf("channel name not configured")
|
||||
}
|
||||
|
||||
// 1. 将参数放入 Query String
|
||||
reqUrl, err := url.Parse(d.Address + uploadApi)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
q := reqUrl.Query()
|
||||
q.Set("returnFormat", "default")
|
||||
q.Set("channelName", channelName)
|
||||
q.Set("uploadFolder", dstDir.GetPath())
|
||||
q.Set("autoRetry", "true")
|
||||
reqUrl.RawQuery = q.Encode()
|
||||
|
||||
// 2. 构建 multipart 表单的头部
|
||||
b := bytes.NewBuffer(make([]byte, 0, 164+len(file.GetName()))) // 预估头部大小,避免频繁扩容
|
||||
w := multipart.NewWriter(b)
|
||||
_, err = w.CreateFormFile("file", file.GetName())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
headSize := b.Len()
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
|
||||
// 3. 将 [表单头 + 文件流 + 表单尾] 组合成单一 Reader
|
||||
rateLimitedReader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: &driver.SimpleReaderWithSize{
|
||||
Reader: io.MultiReader(head, file, tail),
|
||||
Size: int64(b.Len()) + file.GetSize(),
|
||||
},
|
||||
UpdateProgress: up,
|
||||
})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, reqUrl.String(), rateLimitedReader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+d.Token)
|
||||
req.ContentLength = int64(b.Len()) + file.GetSize()
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
|
||||
b.Reset()
|
||||
_, err = b.ReadFrom(res.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("upload failed %d: %s", res.StatusCode, b.String())
|
||||
}
|
||||
|
||||
var resp standardUploadResp
|
||||
if err := json.Unmarshal(b.Bytes(), &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(resp) == 0 || resp[0].Src == "" {
|
||||
return nil, fmt.Errorf("no src returned")
|
||||
}
|
||||
|
||||
srcPath := strings.TrimPrefix(resp[0].Src, "/file/")
|
||||
srcPath = strings.TrimPrefix(srcPath, "/")
|
||||
|
||||
if resp[0].PublicUrl != "" {
|
||||
if u, err := url.Parse(resp[0].PublicUrl); err == nil {
|
||||
d.publicUrlPrefix = u.Scheme + "://" + u.Host
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: srcPath,
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
Modified: file.ModTime(),
|
||||
IsFolder: false,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *CFImgBed) chunkedUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress, channelType, channelName string) (model.Obj, error) {
|
||||
if channelName == "" {
|
||||
return nil, fmt.Errorf("channel name not configured for chunked upload")
|
||||
}
|
||||
|
||||
fileSize := file.GetSize()
|
||||
fileMime := file.GetMimetype()
|
||||
fileName := file.GetName()
|
||||
|
||||
var chunkSizeMap = map[string]int64{
|
||||
|
||||
}
|
||||
chunkSize := chunkSizeMap[channelType]
|
||||
if chunkSize == 0 {
|
||||
chunkSize = 5 * 1024 * 1024
|
||||
}
|
||||
totalChunks := int((fileSize + chunkSize - 1) / chunkSize)
|
||||
|
||||
// 第一步:initChunked
|
||||
var initResp struct {
|
||||
Success bool `json:"success"`
|
||||
UploadId string `json:"uploadId"`
|
||||
}
|
||||
_, err := d.doRequest(ctx, http.MethodPost, uploadApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"initChunked": "true",
|
||||
"uploadChannel": channelType,
|
||||
"channelName": channelName,
|
||||
})
|
||||
req.SetFormData(map[string]string{
|
||||
"originalFileName": fileName,
|
||||
"originalFileType": fileMime,
|
||||
"totalChunks": strconv.Itoa(totalChunks),
|
||||
})
|
||||
}, &initResp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("initChunked failed: %w", err)
|
||||
}
|
||||
if !initResp.Success || initResp.UploadId == "" {
|
||||
return nil, fmt.Errorf("initChunked returned no uploadId")
|
||||
}
|
||||
uploadId := initResp.UploadId
|
||||
|
||||
// 第二步:逐块上传
|
||||
ss, err := stream.NewStreamSectionReader(file, int(chunkSize), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
reqUrl := d.Address + uploadApi + "?chunked=true&uploadChannel=" + channelType + "&channelName=" + channelName
|
||||
b := bytes.NewBuffer(make([]byte, 0, 2048))
|
||||
|
||||
g, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, min(d.UploadThread, totalChunks),
|
||||
retry.Attempts(3),
|
||||
retry.Delay(time.Second),
|
||||
retry.DelayType(retry.BackOffDelay))
|
||||
|
||||
for i := 0; i < totalChunks; i++ {
|
||||
if utils.IsCanceled(uploadCtx) {
|
||||
break
|
||||
}
|
||||
chunkIndex := i
|
||||
offset := int64(chunkIndex) * chunkSize
|
||||
sizeToRead := chunkSize
|
||||
if offset+sizeToRead > fileSize {
|
||||
sizeToRead = fileSize - offset
|
||||
}
|
||||
|
||||
var reader io.ReadSeeker
|
||||
g.GoWithLifecycle(errgroup.Lifecycle{
|
||||
Before: func(ctx context.Context) (err error) {
|
||||
reader, err = ss.GetSectionReader(offset, sizeToRead)
|
||||
return
|
||||
},
|
||||
After: func(err error) {
|
||||
ss.FreeSectionReader(reader)
|
||||
},
|
||||
Do: func(ctx context.Context) (err error) {
|
||||
_, err = reader.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
b.Reset()
|
||||
w := multipart.NewWriter(b)
|
||||
_ = w.WriteField("uploadId", uploadId)
|
||||
_ = w.WriteField("chunkIndex", strconv.Itoa(chunkIndex))
|
||||
_ = w.WriteField("totalChunks", strconv.Itoa(totalChunks))
|
||||
_ = w.WriteField("originalFileName", fileName)
|
||||
_ = w.WriteField("originalFileType", fileMime)
|
||||
_, _ = w.CreateFormFile("file", fileName)
|
||||
headSize := b.Len()
|
||||
_ = w.Close()
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
|
||||
rateLimitedRd := driver.NewLimitedUploadStream(ctx, io.MultiReader(head, reader, tail))
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, reqUrl, rateLimitedRd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+d.Token)
|
||||
req.ContentLength = int64(headSize) + sizeToRead + int64(b.Len()-headSize)
|
||||
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("chunk %d upload failed: %d", chunkIndex, res.StatusCode)
|
||||
}
|
||||
|
||||
up(90 * float64(chunkIndex+1) / float64(totalChunks))
|
||||
return nil
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 第三步:merge
|
||||
var mergeResp standardUploadResp
|
||||
_, err = d.doRequest(ctx, http.MethodPost, uploadApi, func(req *resty.Request) {
|
||||
req.SetQueryParams(map[string]string{
|
||||
"chunked": "true",
|
||||
"merge": "true",
|
||||
"uploadChannel": channelType,
|
||||
"channelName": channelName,
|
||||
"returnFormat": "default",
|
||||
"uploadFolder": dstDir.GetPath(),
|
||||
})
|
||||
req.SetFormData(map[string]string{
|
||||
"uploadId": uploadId,
|
||||
"totalChunks": strconv.Itoa(totalChunks),
|
||||
"originalFileName": fileName,
|
||||
"originalFileType": fileMime,
|
||||
})
|
||||
}, &mergeResp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("merge failed: %w", err)
|
||||
}
|
||||
if len(mergeResp) == 0 || mergeResp[0].Src == "" {
|
||||
return nil, fmt.Errorf("merge returned no src")
|
||||
}
|
||||
|
||||
up(95)
|
||||
|
||||
srcPath := strings.TrimPrefix(mergeResp[0].Src, "/file/")
|
||||
srcPath = strings.TrimPrefix(srcPath, "/")
|
||||
|
||||
if mergeResp[0].PublicUrl != "" {
|
||||
if u, err := url.Parse(mergeResp[0].PublicUrl); err == nil {
|
||||
d.publicUrlPrefix = u.Scheme + "://" + u.Host
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: srcPath,
|
||||
Name: fileName,
|
||||
Size: fileSize,
|
||||
Modified: file.ModTime(),
|
||||
IsFolder: false,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// hfDirectUpload 处理 HuggingFace 的 LFS 直传逻辑(申请授权 -> 物理上传 -> 后端 Commit)
|
||||
func (d *CFImgBed) hfDirectUpload(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
channelName := d.LargeChannelName
|
||||
if channelName == "" {
|
||||
return nil, errors.New("LargeChannelName not configured")
|
||||
}
|
||||
|
||||
sha256Hash := file.GetHash().GetHash(utils.SHA256)
|
||||
if len(sha256Hash) != utils.SHA256.Width {
|
||||
var err error
|
||||
_, sha256Hash, err = stream.CacheFullAndHash(file, &up, utils.SHA256)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
fileSize := file.GetSize()
|
||||
sampleSize := min(fileSize, fileSampleSize)
|
||||
sampleRd, err := file.RangeRead(http_range.Range{Start: 0, Length: sampleSize})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sampleBuf := make([]byte, sampleSize)
|
||||
_, err = io.ReadFull(sampleRd, sampleBuf)
|
||||
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
|
||||
return nil, err
|
||||
}
|
||||
fileSample := base64.StdEncoding.EncodeToString(sampleBuf)
|
||||
|
||||
fileMime := file.GetMimetype()
|
||||
// 1. 请求图床后端获取 HF 授权地址
|
||||
reqBody := map[string]interface{}{
|
||||
"fileName": file.GetName(),
|
||||
"fileType": fileMime,
|
||||
"fileSize": fileSize,
|
||||
"sha256": sha256Hash,
|
||||
"fileSample": fileSample,
|
||||
"channelName": channelName,
|
||||
"uploadFolder": dstDir.GetPath(),
|
||||
}
|
||||
|
||||
var getUrlResp hfGetUrlResp
|
||||
_, err = d.doRequest(ctx, http.MethodPost, hfGetUrlApi, func(req *resty.Request) {
|
||||
req.SetBody(reqBody)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
}, &getUrlResp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 秒传逻辑
|
||||
if getUrlResp.AlreadyExists || !getUrlResp.NeedsLfs {
|
||||
up(100)
|
||||
return d.hfCommit(ctx, getUrlResp, file.GetName(), fileSize, fileMime, file.ModTime())
|
||||
}
|
||||
|
||||
if getUrlResp.UploadAction == nil {
|
||||
return nil, fmt.Errorf("HF upload action is nil")
|
||||
}
|
||||
|
||||
headers := getUrlResp.UploadAction.Header
|
||||
href := getUrlResp.UploadAction.Href
|
||||
|
||||
// 2. 根据响应判断是执行分片上传还是单文件上传
|
||||
chunkSizeStr, needChunk := headers["chunk_size"]
|
||||
if needChunk {
|
||||
// 分片直传 (AWS S3 Multipart 风格)
|
||||
chunkSize, _ := strconv.ParseInt(chunkSizeStr, 10, 64)
|
||||
if chunkSize <= 0 {
|
||||
chunkSize = 20 * 1024 * 1024
|
||||
}
|
||||
|
||||
partUrls := make(map[int]string)
|
||||
for k, v := range headers {
|
||||
if k != "chunk_size" {
|
||||
if idx, err := strconv.Atoi(k); err == nil {
|
||||
partUrls[idx] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
totalParts := len(partUrls)
|
||||
|
||||
ss, err := stream.NewStreamSectionReader(file, int(chunkSize), &up)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
g, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, min(d.UploadThread, totalParts),
|
||||
retry.Attempts(3),
|
||||
retry.Delay(time.Second),
|
||||
retry.DelayType(retry.BackOffDelay))
|
||||
|
||||
parts := make([]map[string]any, totalParts)
|
||||
|
||||
for partNumber := range partUrls {
|
||||
if utils.IsCanceled(uploadCtx) {
|
||||
break
|
||||
}
|
||||
partUrl := partUrls[partNumber]
|
||||
offset := int64(partNumber-1) * chunkSize
|
||||
sizeToRead := chunkSize
|
||||
if offset+sizeToRead > fileSize {
|
||||
sizeToRead = fileSize - offset
|
||||
}
|
||||
|
||||
var reader io.ReadSeeker
|
||||
g.GoWithLifecycle(errgroup.Lifecycle{
|
||||
Before: func(ctx context.Context) (err error) {
|
||||
reader, err = ss.GetSectionReader(offset, sizeToRead)
|
||||
return
|
||||
},
|
||||
After: func(err error) {
|
||||
ss.FreeSectionReader(reader)
|
||||
},
|
||||
Do: func(ctx context.Context) (err error) {
|
||||
_, err = reader.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
limitedReader := driver.NewLimitedUploadStream(ctx, reader)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, partUrl, limitedReader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
req.ContentLength = sizeToRead
|
||||
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("chunk %d failed: %d", partNumber, res.StatusCode)
|
||||
}
|
||||
|
||||
etag := res.Header.Get("ETag")
|
||||
parts[partNumber-1] = map[string]any{"partNumber": partNumber, "etag": etag}
|
||||
|
||||
up(95 * float64(g.Success()+1) / float64(totalParts))
|
||||
return nil
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
if err := g.Wait(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 合并分片
|
||||
// sort.Slice(parts, func(i, j int) bool { return parts[i]["partNumber"].(int) < parts[j]["partNumber"].(int) })
|
||||
mergeBody, _ := json.Marshal(map[string]any{"oid": getUrlResp.Oid, "parts": parts})
|
||||
mergeReq, err := http.NewRequestWithContext(ctx, http.MethodPost, href, bytes.NewReader(mergeBody))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mergeReq.Header.Set("Content-Type", "application/vnd.git-lfs+json")
|
||||
if d.Token != "" {
|
||||
mergeReq.Header.Set("Authorization", "Bearer "+d.Token)
|
||||
}
|
||||
|
||||
res, err := base.HttpClient.Do(mergeReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
body, _ := io.ReadAll(res.Body) // 读取 HF 返回的错误详情
|
||||
if res.StatusCode != http.StatusOK {
|
||||
log.WithField("status", res.StatusCode).WithField("response", string(body)).Error("HF merge chunks failed")
|
||||
return nil, fmt.Errorf("merge chunks failed: %s", string(body))
|
||||
}
|
||||
up(97)
|
||||
|
||||
} else {
|
||||
// 单文件直传 (PUT)
|
||||
limitedReader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: file,
|
||||
UpdateProgress: model.UpdateProgressWithRange(up, 0, 97),
|
||||
})
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, href, limitedReader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.ContentLength = fileSize
|
||||
for k, v := range headers {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("direct upload failed")
|
||||
}
|
||||
}
|
||||
|
||||
defer up(100)
|
||||
|
||||
// 3. 通知图床后端完成文件登记
|
||||
return d.hfCommit(ctx, getUrlResp, file.GetName(), fileSize, fileMime, file.ModTime())
|
||||
}
|
||||
|
||||
func (d *CFImgBed) hfCommit(ctx context.Context, getUrlResp hfGetUrlResp, fileName string, fileSize int64, fileMime string, modTime time.Time) (model.Obj, error) {
|
||||
commitBody := map[string]interface{}{
|
||||
"fullId": getUrlResp.FullID,
|
||||
"filePath": getUrlResp.FilePath,
|
||||
"sha256": getUrlResp.Oid,
|
||||
"fileSize": fileSize,
|
||||
"fileName": fileName,
|
||||
"fileType": fileMime,
|
||||
"channelName": getUrlResp.ChannelName,
|
||||
}
|
||||
var commitResp hfCommitResp
|
||||
_, err := d.doRequest(ctx, http.MethodPost, hfCommitApi, func(req *resty.Request) {
|
||||
req.SetBody(commitBody)
|
||||
}, &commitResp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("HF commit request failed: %w", err)
|
||||
}
|
||||
if !commitResp.Success {
|
||||
return nil, fmt.Errorf("HF commit failed: success=false")
|
||||
}
|
||||
|
||||
srcPath := strings.TrimPrefix(commitResp.Src, "/file/")
|
||||
srcPath = strings.TrimPrefix(srcPath, "/")
|
||||
|
||||
if commitResp.PublicUrl != "" {
|
||||
if u, err := url.Parse(commitResp.PublicUrl); err == nil {
|
||||
d.publicUrlPrefix = u.Scheme + "://" + u.Host
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
Path: srcPath,
|
||||
Name: fileName,
|
||||
Size: fileSize,
|
||||
Modified: modTime,
|
||||
IsFolder: false,
|
||||
}, nil
|
||||
}
|
||||
@@ -1,70 +0,0 @@
|
||||
package cloudflare_imgbed
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/go-resty/resty/v2"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
listApi = "/api/manage/list"
|
||||
deleteApi = "/api/manage/delete"
|
||||
uploadApi = "/upload"
|
||||
hfGetUrlApi = "/upload/huggingface/getUploadUrl"
|
||||
hfCommitApi = "/upload/huggingface/commitUpload"
|
||||
hfDirectThreshold int64 = 20 * 1024 * 1024
|
||||
fileSampleSize = 512 // HF 申请上传地址时需提供文件前 512 字节的 Sample
|
||||
)
|
||||
|
||||
// doRequest 通用请求封装,包含重试和 API 错误解析
|
||||
func (d *CFImgBed) doRequest(ctx context.Context, method, urlPath string, callback func(*resty.Request), resp interface{}) ([]byte, error) {
|
||||
maxRetries := 3
|
||||
for i := 0; i < maxRetries; i++ {
|
||||
req := d.client.R()
|
||||
req.SetContext(ctx)
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
if resp != nil {
|
||||
req.SetResult(resp)
|
||||
}
|
||||
|
||||
res, err := req.Execute(method, urlPath)
|
||||
if err != nil {
|
||||
log.WithError(err).Warnf("request %s %s failed, attempt %d/%d", method, urlPath, i+1, maxRetries)
|
||||
if i < maxRetries-1 {
|
||||
time.Sleep(time.Duration(i+1) * time.Second)
|
||||
continue
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Retry on rate limit before attempting to interpret the body as an API error.
|
||||
if res.StatusCode() == 429 {
|
||||
time.Sleep(time.Duration(i+1) * 2 * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
body := res.Body()
|
||||
var apiErr apiError
|
||||
if err := json.Unmarshal(body, &apiErr); err == nil {
|
||||
if apiErr.Error != "" || apiErr.Message != "" {
|
||||
msg := apiErr.Error
|
||||
if msg == "" {
|
||||
msg = apiErr.Message
|
||||
}
|
||||
return nil, fmt.Errorf("API error: %s", msg)
|
||||
}
|
||||
}
|
||||
|
||||
if res.IsError() {
|
||||
return nil, fmt.Errorf("HTTP %d", res.StatusCode())
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
return nil, fmt.Errorf("max retries exceeded")
|
||||
}
|
||||
@@ -1,232 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type CnbReleases struct {
|
||||
model.Storage
|
||||
Addition
|
||||
ref *CnbReleases
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *CnbReleases) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Init(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) InitReference(storage driver.Driver) error {
|
||||
refStorage, ok := storage.(*CnbReleases)
|
||||
if ok {
|
||||
d.ref = refStorage
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("ref: storage is not CnbReleases")
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Drop(ctx context.Context) error {
|
||||
d.ref = nil
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
dirID := dir.GetID()
|
||||
if dirID == "" {
|
||||
// get all releases for root dir
|
||||
var resp ReleaseList
|
||||
|
||||
err := d.Request(http.MethodGet, "/{repo}/-/releases", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return utils.SliceConvert(resp, func(src Release) (model.Obj, error) {
|
||||
name := src.Name
|
||||
if d.UseTagName {
|
||||
name = src.TagName
|
||||
}
|
||||
return &model.Object{
|
||||
ID: src.ID,
|
||||
Name: name,
|
||||
Size: d.sumAssetsSize(src.Assets),
|
||||
Ctime: src.CreatedAt,
|
||||
Modified: src.UpdatedAt,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
})
|
||||
}
|
||||
|
||||
var resp Release
|
||||
err := d.Request(http.MethodGet, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", dirID)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return utils.SliceConvert(resp.Assets, func(src ReleaseAsset) (model.Obj, error) {
|
||||
return &Object{
|
||||
Object: model.Object{
|
||||
ID: src.ID,
|
||||
Path: src.Path,
|
||||
Name: src.Name,
|
||||
Size: src.Size,
|
||||
Ctime: src.CreatedAt,
|
||||
Modified: src.UpdatedAt,
|
||||
IsFolder: false,
|
||||
},
|
||||
ParentID: dirID,
|
||||
}, nil
|
||||
})
|
||||
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
return &model.Link{
|
||||
URL: "https://cnb.cool" + file.GetPath(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
if parentDir.GetPath() == "/" {
|
||||
// create a new release
|
||||
branch := d.DefaultBranch
|
||||
if branch == "" {
|
||||
branch = "main" // fallback to "main" if not set
|
||||
}
|
||||
return d.Request(http.MethodPost, "/{repo}/-/releases", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetBody(base.Json{
|
||||
"name": dirName,
|
||||
"tag_name": dirName,
|
||||
"target_commitish": branch,
|
||||
})
|
||||
}, nil)
|
||||
}
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
if srcObj.IsDir() && !d.UseTagName {
|
||||
return d.Request(http.MethodPatch, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", srcObj.GetID())
|
||||
req.SetFormData(map[string]string{
|
||||
"name": newName,
|
||||
})
|
||||
}, nil)
|
||||
}
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Remove(ctx context.Context, obj model.Obj) error {
|
||||
if obj.IsDir() {
|
||||
return d.Request(http.MethodDelete, "/{repo}/-/releases/{release_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", obj.GetID())
|
||||
}, nil)
|
||||
}
|
||||
if o, ok := obj.(*Object); ok {
|
||||
return d.Request(http.MethodDelete, "/{repo}/-/releases/{release_id}/assets/{asset_id}", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", o.ParentID)
|
||||
req.SetPathParam("asset_id", obj.GetID())
|
||||
}, nil)
|
||||
} else {
|
||||
return fmt.Errorf("unable to get release ID")
|
||||
}
|
||||
}
|
||||
|
||||
func (d *CnbReleases) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
// 1. get upload info
|
||||
var resp ReleaseAssetUploadURL
|
||||
err := d.Request(http.MethodPost, "/{repo}/-/releases/{release_id}/asset-upload-url", func(req *resty.Request) {
|
||||
req.SetPathParam("repo", d.Repo)
|
||||
req.SetPathParam("release_id", dstDir.GetID())
|
||||
req.SetBody(base.Json{
|
||||
"asset_name": file.GetName(),
|
||||
"overwrite": true,
|
||||
"size": file.GetSize(),
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 2. upload file
|
||||
// use multipart to create form file
|
||||
var b bytes.Buffer
|
||||
w := multipart.NewWriter(&b)
|
||||
_, err = w.CreateFormFile("file", file.GetName())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
headSize := b.Len()
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
r := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: &driver.SimpleReaderWithSize{
|
||||
Reader: io.MultiReader(head, file, tail),
|
||||
Size: int64(b.Len()) + file.GetSize(),
|
||||
},
|
||||
UpdateProgress: up,
|
||||
})
|
||||
|
||||
// use net/http to upload file
|
||||
ctxWithTimeout, cancel := context.WithTimeout(ctx, time.Duration(resp.ExpiresInSec+1)*time.Second)
|
||||
defer cancel()
|
||||
req, err := http.NewRequestWithContext(ctxWithTimeout, http.MethodPost, resp.UploadURL, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
req.Header.Set("User-Agent", base.UserAgent)
|
||||
httpResp, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer httpResp.Body.Close()
|
||||
if httpResp.StatusCode != http.StatusNoContent {
|
||||
return fmt.Errorf("upload file failed: %s", httpResp.Status)
|
||||
}
|
||||
|
||||
// 3. verify upload
|
||||
return d.Request(http.MethodPost, resp.VerifyURL, nil, nil)
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*CnbReleases)(nil)
|
||||
@@ -1,25 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
Repo string `json:"repo" type:"string" required:"true"`
|
||||
Token string `json:"token" type:"string" required:"true"`
|
||||
UseTagName bool `json:"use_tag_name" type:"bool" default:"false" help:"Use tag name instead of release name"`
|
||||
DefaultBranch string `json:"default_branch" type:"string" default:"main" help:"Default branch for new releases"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "CNB Releases",
|
||||
LocalSort: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &CnbReleases{}
|
||||
})
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
type Object struct {
|
||||
model.Object
|
||||
ParentID string
|
||||
}
|
||||
|
||||
type TagList []Tag
|
||||
|
||||
type Tag struct {
|
||||
Commit struct {
|
||||
Author UserInfo `json:"author"`
|
||||
Commit CommitObject `json:"commit"`
|
||||
Committer UserInfo `json:"committer"`
|
||||
Parents []CommitParent `json:"parents"`
|
||||
Sha string `json:"sha"`
|
||||
} `json:"commit"`
|
||||
Name string `json:"name"`
|
||||
Target string `json:"target"`
|
||||
TargetType string `json:"target_type"`
|
||||
Verification TagObjectVerification `json:"verification"`
|
||||
}
|
||||
|
||||
type UserInfo struct {
|
||||
Freeze bool `json:"freeze"`
|
||||
Nickname string `json:"nickname"`
|
||||
Username string `json:"username"`
|
||||
}
|
||||
|
||||
type CommitObject struct {
|
||||
Author Signature `json:"author"`
|
||||
CommentCount int `json:"comment_count"`
|
||||
Committer Signature `json:"committer"`
|
||||
Message string `json:"message"`
|
||||
Tree CommitObjectTree `json:"tree"`
|
||||
Verification CommitObjectVerification `json:"verification"`
|
||||
}
|
||||
|
||||
type Signature struct {
|
||||
Date time.Time `json:"date"`
|
||||
Email string `json:"email"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type CommitObjectTree struct {
|
||||
Sha string `json:"sha"`
|
||||
}
|
||||
|
||||
type CommitObjectVerification struct {
|
||||
Payload string `json:"payload"`
|
||||
Reason string `json:"reason"`
|
||||
Signature string `json:"signature"`
|
||||
Verified bool `json:"verified"`
|
||||
VerifiedAt string `json:"verified_at"`
|
||||
}
|
||||
|
||||
type CommitParent = CommitObjectTree
|
||||
|
||||
type TagObjectVerification = CommitObjectVerification
|
||||
|
||||
type ReleaseList []Release
|
||||
|
||||
type Release struct {
|
||||
Assets []ReleaseAsset `json:"assets"`
|
||||
Author UserInfo `json:"author"`
|
||||
Body string `json:"body"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
Draft bool `json:"draft"`
|
||||
ID string `json:"id"`
|
||||
IsLatest bool `json:"is_latest"`
|
||||
Name string `json:"name"`
|
||||
Prerelease bool `json:"prerelease"`
|
||||
PublishedAt time.Time `json:"published_at"`
|
||||
TagCommitish string `json:"tag_commitish"`
|
||||
TagName string `json:"tag_name"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type ReleaseAsset struct {
|
||||
ContentType string `json:"content_type"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Size int64 `json:"size"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Uploader UserInfo `json:"uploader"`
|
||||
}
|
||||
|
||||
type ReleaseAssetUploadURL struct {
|
||||
UploadURL string `json:"upload_url"`
|
||||
ExpiresInSec int `json:"expires_in_sec"`
|
||||
VerifyURL string `json:"verify_url"`
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
package cnb_releases
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
// do others that not defined in Driver interface
|
||||
|
||||
func (d *CnbReleases) Request(method string, path string, callback base.ReqCallback, resp any) error {
|
||||
if d.ref != nil {
|
||||
return d.ref.Request(method, path, callback, resp)
|
||||
}
|
||||
var url string
|
||||
if strings.HasPrefix(path, "http") {
|
||||
url = path
|
||||
} else {
|
||||
url = "https://api.cnb.cool" + path
|
||||
}
|
||||
req := base.RestyClient.R()
|
||||
req.SetHeader("Accept", "application/json")
|
||||
req.SetAuthScheme("Bearer")
|
||||
req.SetAuthToken(d.Token)
|
||||
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
res, err := req.Execute(method, url)
|
||||
log.Debugln(res.String())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if res.StatusCode() != http.StatusOK && res.StatusCode() != http.StatusCreated && res.StatusCode() != http.StatusNoContent {
|
||||
return fmt.Errorf("failed to request %s, status code: %d, message: %s", url, res.StatusCode(), res.String())
|
||||
}
|
||||
|
||||
if resp != nil {
|
||||
err = json.Unmarshal(res.Body(), resp)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *CnbReleases) sumAssetsSize(assets []ReleaseAsset) int64 {
|
||||
var size int64
|
||||
for _, asset := range assets {
|
||||
size += asset.Size
|
||||
}
|
||||
return size
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
package crypt
|
||||
|
||||
import (
|
||||
stdpath "path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// will give the best guessing based on the path
|
||||
func guessPath(path string) (isFolder, secondTry bool) {
|
||||
if strings.HasSuffix(path, "/") {
|
||||
//confirmed a folder
|
||||
return true, false
|
||||
}
|
||||
lastSlash := strings.LastIndex(path, "/")
|
||||
if !strings.Contains(path[lastSlash:], ".") {
|
||||
//no dot, try folder then try file
|
||||
return true, true
|
||||
}
|
||||
return false, true
|
||||
}
|
||||
|
||||
func (d *Crypt) encryptPath(path string, isFolder bool) string {
|
||||
if isFolder {
|
||||
return d.cipher.EncryptDirName(path)
|
||||
}
|
||||
dir, fileName := filepath.Split(path)
|
||||
return stdpath.Join(d.cipher.EncryptDirName(dir), d.cipher.EncryptFileName(fileName))
|
||||
}
|
||||
@@ -1,224 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type Degoo struct {
|
||||
model.Storage
|
||||
Addition
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func (d *Degoo) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Degoo) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Degoo) Init(ctx context.Context) error {
|
||||
|
||||
d.client = base.HttpClient
|
||||
|
||||
// Ensure we have a valid token (will login if needed or refresh if expired)
|
||||
if err := d.ensureValidToken(ctx); err != nil {
|
||||
return fmt.Errorf("failed to initialize token: %w", err)
|
||||
}
|
||||
|
||||
return d.getDevices(ctx)
|
||||
}
|
||||
|
||||
func (d *Degoo) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
items, err := d.getAllFileChildren5(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return utils.MustSliceConvert(items, func(s DegooFileItem) model.Obj {
|
||||
isFolder := s.Category == 2 || s.Category == 1 || s.Category == 10
|
||||
|
||||
createTime, modTime, _ := humanReadableTimes(s.CreationTime, s.LastModificationTime, s.LastUploadTime)
|
||||
|
||||
size, err := strconv.ParseInt(s.Size, 10, 64)
|
||||
if err != nil {
|
||||
size = 0 // Default to 0 if size parsing fails
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
ID: s.ID,
|
||||
Path: s.FilePath,
|
||||
Name: s.Name,
|
||||
Size: size,
|
||||
Modified: modTime,
|
||||
Ctime: createTime,
|
||||
IsFolder: isFolder,
|
||||
}
|
||||
}), nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
item, err := d.getOverlay4(ctx, file.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &model.Link{URL: item.URL}, nil
|
||||
}
|
||||
|
||||
func (d *Degoo) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
// This is done by calling the setUploadFile3 API with a special checksum and size.
|
||||
const query = `mutation SetUploadFile3($Token: String!, $FileInfos: [FileInfoUpload3]!) { setUploadFile3(Token: $Token, FileInfos: $FileInfos) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"FileInfos": []map[string]interface{}{
|
||||
{
|
||||
"Checksum": folderChecksum,
|
||||
"Name": dirName,
|
||||
"CreationTime": time.Now().UnixMilli(),
|
||||
"ParentID": parentDir.GetID(),
|
||||
"Size": 0,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetUploadFile3", query, variables)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
const query = `mutation SetMoveFile($Token: String!, $Copy: Boolean, $NewParentID: String!, $FileIDs: [String]!) { setMoveFile(Token: $Token, Copy: $Copy, NewParentID: $NewParentID, FileIDs: $FileIDs) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"Copy": false,
|
||||
"NewParentID": dstDir.GetID(),
|
||||
"FileIDs": []string{srcObj.GetID()},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetMoveFile", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
const query = `mutation SetRenameFile($Token: String!, $FileRenames: [FileRenameInfo]!) { setRenameFile(Token: $Token, FileRenames: $FileRenames) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"FileRenames": []DegooFileRenameInfo{
|
||||
{
|
||||
ID: srcObj.GetID(),
|
||||
NewName: newName,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetRenameFile", query, variables)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
// Copy is not implemented, Degoo API does not support direct copy.
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *Degoo) Remove(ctx context.Context, obj model.Obj) error {
|
||||
// Remove deletes a file or folder (moves to trash).
|
||||
const query = `mutation SetDeleteFile5($Token: String!, $IsInRecycleBin: Boolean!, $IDs: [IDType]!) { setDeleteFile5(Token: $Token, IsInRecycleBin: $IsInRecycleBin, IDs: $IDs) }`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"IsInRecycleBin": false,
|
||||
"IDs": []map[string]string{{"FileID": obj.GetID()}},
|
||||
}
|
||||
|
||||
_, err := d.apiCall(ctx, "SetDeleteFile5", query, variables)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *Degoo) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
tmpF, err := file.CacheFullAndWriter(&up, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
parentID := dstDir.GetID()
|
||||
|
||||
// Calculate the checksum for the file.
|
||||
checksum, err := d.checkSum(tmpF)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 1. Get upload authorization via getBucketWriteAuth4.
|
||||
auths, err := d.getBucketWriteAuth4(ctx, file, parentID, checksum)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 2. Upload file.
|
||||
// support rapid upload
|
||||
if auths.GetBucketWriteAuth4[0].Error != "Already exist!" {
|
||||
err = d.uploadS3(ctx, auths, tmpF, file, checksum)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Register metadata with setUploadFile3.
|
||||
data, err := d.SetUploadFile3(ctx, file, parentID, checksum)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !data.SetUploadFile3 {
|
||||
return fmt.Errorf("setUploadFile3 failed: %v", data)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Degoo) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
quota, err := d.getUserInfo(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
used, err := strconv.ParseInt(quota.GetUserInfo3.UsedQuota, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse used quota: %v", err)
|
||||
}
|
||||
total, err := strconv.ParseInt(quota.GetUserInfo3.TotalQuota, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse total quota: %v", err)
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: total,
|
||||
UsedSpace: used,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
Username string `json:"username" help:"Your Degoo account email"`
|
||||
Password string `json:"password" help:"Your Degoo account password"`
|
||||
RefreshToken string `json:"refresh_token" help:"Refresh token for automatic token renewal, obtained automatically"`
|
||||
AccessToken string `json:"access_token" help:"Access token for Degoo API, obtained automatically"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Degoo",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "0",
|
||||
NoOverwriteUpload: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Degoo{}
|
||||
})
|
||||
}
|
||||
@@ -1,133 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
)
|
||||
|
||||
// DegooLoginRequest represents the login request body.
|
||||
type DegooLoginRequest struct {
|
||||
GenerateToken bool `json:"GenerateToken"`
|
||||
Username string `json:"Username"`
|
||||
Password string `json:"Password"`
|
||||
}
|
||||
|
||||
// DegooLoginResponse represents a successful login response.
|
||||
type DegooLoginResponse struct {
|
||||
Token string `json:"Token"`
|
||||
RefreshToken string `json:"RefreshToken"`
|
||||
}
|
||||
|
||||
// DegooAccessTokenRequest represents the token refresh request body.
|
||||
type DegooAccessTokenRequest struct {
|
||||
RefreshToken string `json:"RefreshToken"`
|
||||
}
|
||||
|
||||
// DegooAccessTokenResponse represents the token refresh response.
|
||||
type DegooAccessTokenResponse struct {
|
||||
AccessToken string `json:"AccessToken"`
|
||||
}
|
||||
|
||||
// DegooFileItem represents a Degoo file or folder.
|
||||
type DegooFileItem struct {
|
||||
ID string `json:"ID"`
|
||||
ParentID string `json:"ParentID"`
|
||||
Name string `json:"Name"`
|
||||
Category int `json:"Category"`
|
||||
Size string `json:"Size"`
|
||||
URL string `json:"URL"`
|
||||
CreationTime string `json:"CreationTime"`
|
||||
LastModificationTime string `json:"LastModificationTime"`
|
||||
LastUploadTime string `json:"LastUploadTime"`
|
||||
MetadataID string `json:"MetadataID"`
|
||||
DeviceID int64 `json:"DeviceID"`
|
||||
FilePath string `json:"FilePath"`
|
||||
IsInRecycleBin bool `json:"IsInRecycleBin"`
|
||||
}
|
||||
|
||||
type DegooErrors struct {
|
||||
Path []string `json:"path"`
|
||||
Data interface{} `json:"data"`
|
||||
ErrorType string `json:"errorType"`
|
||||
ErrorInfo interface{} `json:"errorInfo"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// DegooGraphqlResponse is the common structure for GraphQL API responses.
|
||||
type DegooGraphqlResponse struct {
|
||||
Data json.RawMessage `json:"data"`
|
||||
Errors []DegooErrors `json:"errors,omitempty"`
|
||||
}
|
||||
|
||||
// DegooGetChildren5Data is the data field for getFileChildren5.
|
||||
type DegooGetChildren5Data struct {
|
||||
GetFileChildren5 struct {
|
||||
Items []DegooFileItem `json:"Items"`
|
||||
NextToken string `json:"NextToken"`
|
||||
} `json:"getFileChildren5"`
|
||||
}
|
||||
|
||||
// DegooGetOverlay4Data is the data field for getOverlay4.
|
||||
type DegooGetOverlay4Data struct {
|
||||
GetOverlay4 DegooFileItem `json:"getOverlay4"`
|
||||
}
|
||||
|
||||
// DegooFileRenameInfo represents a file rename operation.
|
||||
type DegooFileRenameInfo struct {
|
||||
ID string `json:"ID"`
|
||||
NewName string `json:"NewName"`
|
||||
}
|
||||
|
||||
// DegooFileIDs represents a list of file IDs for move operations.
|
||||
type DegooFileIDs struct {
|
||||
FileIDs []string `json:"FileIDs"`
|
||||
}
|
||||
|
||||
// DegooGetBucketWriteAuth4Data is the data field for GetBucketWriteAuth4.
|
||||
type DegooGetBucketWriteAuth4Data struct {
|
||||
GetBucketWriteAuth4 []struct {
|
||||
AuthData struct {
|
||||
PolicyBase64 string `json:"PolicyBase64"`
|
||||
Signature string `json:"Signature"`
|
||||
BaseURL string `json:"BaseURL"`
|
||||
KeyPrefix string `json:"KeyPrefix"`
|
||||
AccessKey struct {
|
||||
Key string `json:"Key"`
|
||||
Value string `json:"Value"`
|
||||
} `json:"AccessKey"`
|
||||
ACL string `json:"ACL"`
|
||||
AdditionalBody []struct {
|
||||
Key string `json:"Key"`
|
||||
Value string `json:"Value"`
|
||||
} `json:"AdditionalBody"`
|
||||
} `json:"AuthData"`
|
||||
Error interface{} `json:"Error"`
|
||||
} `json:"getBucketWriteAuth4"`
|
||||
}
|
||||
|
||||
// DegooSetUploadFile3Data is the data field for SetUploadFile3.
|
||||
type DegooSetUploadFile3Data struct {
|
||||
SetUploadFile3 bool `json:"setUploadFile3"`
|
||||
}
|
||||
|
||||
type DegooGetUserInfo3Data struct {
|
||||
GetUserInfo3 struct {
|
||||
// ID string
|
||||
// FirstName string
|
||||
// LastName string
|
||||
// Email string
|
||||
// AvatarURL string
|
||||
// CountryCode string = CN
|
||||
// LanguageCode string = zh-cn
|
||||
// Phone string
|
||||
// AccountType int
|
||||
UsedQuota string `json:"UsedQuota"`
|
||||
TotalQuota string `json:"TotalQuota"`
|
||||
// OAuth2Provider
|
||||
// GPMigrationStatus int
|
||||
// FeatureNoAds bool
|
||||
// FeatureTopSecret bool
|
||||
// FeatureDownsampling bool
|
||||
// FeatureAutomaticVideoUploads bool
|
||||
// FileSizeLimit string
|
||||
} `json:"getUserInfo3"`
|
||||
}
|
||||
@@ -1,198 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
func (d *Degoo) getBucketWriteAuth4(ctx context.Context, file model.FileStreamer, parentID string, checksum string) (*DegooGetBucketWriteAuth4Data, error) {
|
||||
const query = `query GetBucketWriteAuth4(
|
||||
$Token: String!
|
||||
$ParentID: String!
|
||||
$StorageUploadInfos: [StorageUploadInfo2]
|
||||
) {
|
||||
getBucketWriteAuth4(
|
||||
Token: $Token
|
||||
ParentID: $ParentID
|
||||
StorageUploadInfos: $StorageUploadInfos
|
||||
) {
|
||||
AuthData {
|
||||
PolicyBase64
|
||||
Signature
|
||||
BaseURL
|
||||
KeyPrefix
|
||||
AccessKey {
|
||||
Key
|
||||
Value
|
||||
}
|
||||
ACL
|
||||
AdditionalBody {
|
||||
Key
|
||||
Value
|
||||
}
|
||||
}
|
||||
Error
|
||||
}
|
||||
}`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ParentID": parentID,
|
||||
"StorageUploadInfos": []map[string]string{{
|
||||
"FileName": file.GetName(),
|
||||
"Checksum": checksum,
|
||||
"Size": strconv.FormatInt(file.GetSize(), 10),
|
||||
}}}
|
||||
|
||||
data, err := d.apiCall(ctx, "GetBucketWriteAuth4", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp DegooGetBucketWriteAuth4Data
|
||||
err = json.Unmarshal(data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
// checkSum calculates the SHA1-based checksum for Degoo upload API.
|
||||
func (d *Degoo) checkSum(file io.Reader) (string, error) {
|
||||
seed := []byte{13, 7, 2, 2, 15, 40, 75, 117, 13, 10, 19, 16, 29, 23, 3, 36}
|
||||
hasher := sha1.New()
|
||||
hasher.Write(seed)
|
||||
|
||||
if _, err := utils.CopyWithBuffer(hasher, file); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
cs := hasher.Sum(nil)
|
||||
|
||||
csBytes := []byte{10, byte(len(cs))}
|
||||
csBytes = append(csBytes, cs...)
|
||||
csBytes = append(csBytes, 16, 0)
|
||||
|
||||
return strings.ReplaceAll(base64.StdEncoding.EncodeToString(csBytes), "/", "_"), nil
|
||||
}
|
||||
|
||||
func (d *Degoo) uploadS3(ctx context.Context, auths *DegooGetBucketWriteAuth4Data, tmpF model.File, file model.FileStreamer, checksum string) error {
|
||||
a := auths.GetBucketWriteAuth4[0].AuthData
|
||||
|
||||
_, err := tmpF.Seek(0, io.SeekStart)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ext := utils.Ext(file.GetName())
|
||||
key := fmt.Sprintf("%s%s/%s.%s", a.KeyPrefix, ext, checksum, ext)
|
||||
|
||||
var b bytes.Buffer
|
||||
w := multipart.NewWriter(&b)
|
||||
err = w.WriteField("key", key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField("acl", a.ACL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField("policy", a.PolicyBase64)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField("signature", a.Signature)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = w.WriteField(a.AccessKey.Key, a.AccessKey.Value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, additional := range a.AdditionalBody {
|
||||
err = w.WriteField(additional.Key, additional.Value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
err = w.WriteField("Content-Type", "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = w.CreateFormFile("file", key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
headSize := b.Len()
|
||||
err = w.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
head := bytes.NewReader(b.Bytes()[:headSize])
|
||||
tail := bytes.NewReader(b.Bytes()[headSize:])
|
||||
|
||||
rateLimitedRd := driver.NewLimitedUploadStream(ctx, io.MultiReader(head, tmpF, tail))
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.BaseURL, rateLimitedRd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Add("ngsw-bypass", "1")
|
||||
req.Header.Add("Content-Type", w.FormDataContentType())
|
||||
|
||||
res, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusNoContent {
|
||||
return fmt.Errorf("upload failed with status code %d", res.StatusCode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Degoo)(nil)
|
||||
|
||||
func (d *Degoo) SetUploadFile3(ctx context.Context, file model.FileStreamer, parentID string, checksum string) (*DegooSetUploadFile3Data, error) {
|
||||
const query = `mutation SetUploadFile3($Token: String!, $FileInfos: [FileInfoUpload3]!) {
|
||||
setUploadFile3(Token: $Token, FileInfos: $FileInfos)
|
||||
}`
|
||||
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"FileInfos": []map[string]string{{
|
||||
"Checksum": checksum,
|
||||
"CreationTime": strconv.FormatInt(file.CreateTime().UnixMilli(), 10),
|
||||
"Name": file.GetName(),
|
||||
"ParentID": parentID,
|
||||
"Size": strconv.FormatInt(file.GetSize(), 10),
|
||||
}}}
|
||||
|
||||
data, err := d.apiCall(ctx, "SetUploadFile3", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var resp DegooSetUploadFile3Data
|
||||
err = json.Unmarshal(data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
@@ -1,478 +0,0 @@
|
||||
package degoo
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
// Thanks to https://github.com/bernd-wechner/Degoo for API research.
|
||||
|
||||
const (
|
||||
// API endpoints
|
||||
loginURL = "https://rest-api.degoo.com/login"
|
||||
accessTokenURL = "https://rest-api.degoo.com/access-token/v2"
|
||||
apiURL = "https://production-appsync.degoo.com/graphql"
|
||||
|
||||
// API configuration
|
||||
apiKey = "da2-vs6twz5vnjdavpqndtbzg3prra"
|
||||
folderChecksum = "CgAQAg"
|
||||
|
||||
// Token management
|
||||
tokenRefreshThreshold = 5 * time.Minute
|
||||
|
||||
// Rate limiting
|
||||
minRequestInterval = 1 * time.Second
|
||||
|
||||
// Error messages
|
||||
errRateLimited = "rate limited (429), please try again later"
|
||||
errUnauthorized = "unauthorized access"
|
||||
)
|
||||
|
||||
var (
|
||||
// Global rate limiting - protects against concurrent API calls
|
||||
lastRequestTime time.Time
|
||||
requestMutex sync.Mutex
|
||||
)
|
||||
|
||||
// JWT payload structure for token expiration checking
|
||||
type JWTPayload struct {
|
||||
UserID string `json:"userID"`
|
||||
Exp int64 `json:"exp"`
|
||||
Iat int64 `json:"iat"`
|
||||
}
|
||||
|
||||
// Rate limiting helper functions
|
||||
|
||||
// applyRateLimit ensures minimum interval between API requests
|
||||
func applyRateLimit() {
|
||||
requestMutex.Lock()
|
||||
defer requestMutex.Unlock()
|
||||
|
||||
if !lastRequestTime.IsZero() {
|
||||
if elapsed := time.Since(lastRequestTime); elapsed < minRequestInterval {
|
||||
time.Sleep(minRequestInterval - elapsed)
|
||||
}
|
||||
}
|
||||
lastRequestTime = time.Now()
|
||||
}
|
||||
|
||||
// HTTP request helper functions
|
||||
|
||||
// createJSONRequest creates a new HTTP request with JSON body
|
||||
func createJSONRequest(ctx context.Context, method, url string, body interface{}) (*http.Request, error) {
|
||||
jsonBody, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to marshal request body: %w", err)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, method, url, bytes.NewBuffer(jsonBody))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to create request: %w", err)
|
||||
}
|
||||
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", base.UserAgent)
|
||||
return req, nil
|
||||
}
|
||||
|
||||
// checkHTTPResponse checks for common HTTP error conditions
|
||||
func checkHTTPResponse(resp *http.Response, operation string) error {
|
||||
if resp.StatusCode == http.StatusTooManyRequests {
|
||||
return fmt.Errorf("%s %s", operation, errRateLimited)
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("%s failed: %s", operation, resp.Status)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// isTokenExpired checks if the JWT token is expired or will expire soon
|
||||
func (d *Degoo) isTokenExpired() bool {
|
||||
if d.AccessToken == "" {
|
||||
return true
|
||||
}
|
||||
|
||||
payload, err := extractJWTPayload(d.AccessToken)
|
||||
if err != nil {
|
||||
return true // Invalid token format
|
||||
}
|
||||
|
||||
// Check if token expires within the threshold
|
||||
expireTime := time.Unix(payload.Exp, 0)
|
||||
return time.Now().Add(tokenRefreshThreshold).After(expireTime)
|
||||
}
|
||||
|
||||
// extractJWTPayload extracts and parses JWT payload
|
||||
func extractJWTPayload(token string) (*JWTPayload, error) {
|
||||
parts := strings.Split(token, ".")
|
||||
if len(parts) != 3 {
|
||||
return nil, fmt.Errorf("invalid JWT format")
|
||||
}
|
||||
|
||||
// Decode the payload (second part)
|
||||
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to decode JWT payload: %w", err)
|
||||
}
|
||||
|
||||
var jwtPayload JWTPayload
|
||||
if err := json.Unmarshal(payload, &jwtPayload); err != nil {
|
||||
return nil, fmt.Errorf("failed to parse JWT payload: %w", err)
|
||||
}
|
||||
|
||||
return &jwtPayload, nil
|
||||
}
|
||||
|
||||
// refreshToken attempts to refresh the access token using the refresh token
|
||||
func (d *Degoo) refreshToken(ctx context.Context) error {
|
||||
if d.RefreshToken == "" {
|
||||
return fmt.Errorf("no refresh token available")
|
||||
}
|
||||
|
||||
// Create request
|
||||
tokenReq := DegooAccessTokenRequest{RefreshToken: d.RefreshToken}
|
||||
req, err := createJSONRequest(ctx, "POST", accessTokenURL, tokenReq)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create refresh token request: %w", err)
|
||||
}
|
||||
|
||||
// Execute request
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("refresh token request failed: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Check response
|
||||
if err := checkHTTPResponse(resp, "refresh token"); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var accessTokenResp DegooAccessTokenResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&accessTokenResp); err != nil {
|
||||
return fmt.Errorf("failed to parse access token response: %w", err)
|
||||
}
|
||||
|
||||
if accessTokenResp.AccessToken == "" {
|
||||
return fmt.Errorf("empty access token received")
|
||||
}
|
||||
|
||||
d.AccessToken = accessTokenResp.AccessToken
|
||||
// Save the updated token to storage
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ensureValidToken ensures we have a valid, non-expired token
|
||||
func (d *Degoo) ensureValidToken(ctx context.Context) error {
|
||||
// Check if token is expired or will expire soon
|
||||
if d.isTokenExpired() {
|
||||
// Try to refresh token first if we have a refresh token
|
||||
if d.RefreshToken != "" {
|
||||
if refreshErr := d.refreshToken(ctx); refreshErr == nil {
|
||||
return nil // Successfully refreshed
|
||||
} else {
|
||||
// If refresh failed, fall back to full login
|
||||
fmt.Printf("Token refresh failed, falling back to full login: %v\n", refreshErr)
|
||||
}
|
||||
}
|
||||
|
||||
// Perform full login
|
||||
if d.Username != "" && d.Password != "" {
|
||||
return d.login(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// login performs the login process and retrieves the access token.
|
||||
func (d *Degoo) login(ctx context.Context) error {
|
||||
if d.Username == "" || d.Password == "" {
|
||||
return fmt.Errorf("username or password not provided")
|
||||
}
|
||||
|
||||
creds := DegooLoginRequest{
|
||||
GenerateToken: true,
|
||||
Username: d.Username,
|
||||
Password: d.Password,
|
||||
}
|
||||
|
||||
jsonCreds, err := json.Marshal(creds)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to serialize login credentials: %w", err)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, "POST", loginURL, bytes.NewBuffer(jsonCreds))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create login request: %w", err)
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("User-Agent", base.UserAgent)
|
||||
req.Header.Set("Origin", "https://app.degoo.com")
|
||||
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("login request failed: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Handle rate limiting (429 Too Many Requests)
|
||||
if resp.StatusCode == http.StatusTooManyRequests {
|
||||
return fmt.Errorf("login rate limited (429), please try again later")
|
||||
}
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("login failed: %s", resp.Status)
|
||||
}
|
||||
|
||||
var loginResp DegooLoginResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(&loginResp); err != nil {
|
||||
return fmt.Errorf("failed to parse login response: %w", err)
|
||||
}
|
||||
|
||||
if loginResp.RefreshToken != "" {
|
||||
tokenReq := DegooAccessTokenRequest{RefreshToken: loginResp.RefreshToken}
|
||||
jsonTokenReq, err := json.Marshal(tokenReq)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to serialize access token request: %w", err)
|
||||
}
|
||||
|
||||
tokenReqHTTP, err := http.NewRequestWithContext(ctx, "POST", accessTokenURL, bytes.NewBuffer(jsonTokenReq))
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create access token request: %w", err)
|
||||
}
|
||||
|
||||
tokenReqHTTP.Header.Set("User-Agent", base.UserAgent)
|
||||
|
||||
tokenResp, err := d.client.Do(tokenReqHTTP)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get access token: %w", err)
|
||||
}
|
||||
defer tokenResp.Body.Close()
|
||||
|
||||
var accessTokenResp DegooAccessTokenResponse
|
||||
if err := json.NewDecoder(tokenResp.Body).Decode(&accessTokenResp); err != nil {
|
||||
return fmt.Errorf("failed to parse access token response: %w", err)
|
||||
}
|
||||
d.AccessToken = accessTokenResp.AccessToken
|
||||
d.RefreshToken = loginResp.RefreshToken // Save refresh token
|
||||
} else if loginResp.Token != "" {
|
||||
d.AccessToken = loginResp.Token
|
||||
d.RefreshToken = "" // Direct token, no refresh token available
|
||||
} else {
|
||||
return fmt.Errorf("login failed, no valid token returned")
|
||||
}
|
||||
|
||||
// Save the updated tokens to storage
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// apiCall performs a Degoo GraphQL API request.
|
||||
func (d *Degoo) apiCall(ctx context.Context, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
|
||||
// Apply rate limiting
|
||||
applyRateLimit()
|
||||
|
||||
// Ensure we have a valid token before making the API call
|
||||
if err := d.ensureValidToken(ctx); err != nil {
|
||||
return nil, fmt.Errorf("failed to ensure valid token: %w", err)
|
||||
}
|
||||
|
||||
// Update the Token in variables if it exists (after potential refresh)
|
||||
d.updateTokenInVariables(variables)
|
||||
|
||||
return d.executeGraphQLRequest(ctx, operationName, query, variables)
|
||||
}
|
||||
|
||||
// updateTokenInVariables updates the Token field in GraphQL variables
|
||||
func (d *Degoo) updateTokenInVariables(variables map[string]interface{}) {
|
||||
if variables != nil {
|
||||
if _, hasToken := variables["Token"]; hasToken {
|
||||
variables["Token"] = d.AccessToken
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// executeGraphQLRequest executes a GraphQL request with retry logic
|
||||
func (d *Degoo) executeGraphQLRequest(ctx context.Context, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
|
||||
reqBody := map[string]interface{}{
|
||||
"operationName": operationName,
|
||||
"query": query,
|
||||
"variables": variables,
|
||||
}
|
||||
|
||||
// Create and configure request
|
||||
req, err := createJSONRequest(ctx, "POST", apiURL, reqBody)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set Degoo-specific headers
|
||||
req.Header.Set("x-api-key", apiKey)
|
||||
if d.AccessToken != "" {
|
||||
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", d.AccessToken))
|
||||
}
|
||||
|
||||
// Execute request
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("GraphQL API request failed: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Check for HTTP errors
|
||||
if err := checkHTTPResponse(resp, "GraphQL API"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Parse GraphQL response
|
||||
var degooResp DegooGraphqlResponse
|
||||
if err := json.NewDecoder(resp.Body).Decode(°ooResp); err != nil {
|
||||
return nil, fmt.Errorf("failed to decode GraphQL response: %w", err)
|
||||
}
|
||||
|
||||
// Handle GraphQL errors
|
||||
if len(degooResp.Errors) > 0 {
|
||||
return d.handleGraphQLError(ctx, degooResp.Errors[0], operationName, query, variables)
|
||||
}
|
||||
|
||||
return degooResp.Data, nil
|
||||
}
|
||||
|
||||
// handleGraphQLError handles GraphQL-level errors with retry logic
|
||||
func (d *Degoo) handleGraphQLError(ctx context.Context, gqlError DegooErrors, operationName, query string, variables map[string]interface{}) (json.RawMessage, error) {
|
||||
if gqlError.ErrorType == "Unauthorized" {
|
||||
// Re-login and retry
|
||||
if err := d.login(ctx); err != nil {
|
||||
return nil, fmt.Errorf("%s, login failed: %w", errUnauthorized, err)
|
||||
}
|
||||
|
||||
// Update token in variables and retry
|
||||
d.updateTokenInVariables(variables)
|
||||
return d.apiCall(ctx, operationName, query, variables)
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("GraphQL API error: %s", gqlError.Message)
|
||||
}
|
||||
|
||||
// humanReadableTimes converts Degoo timestamps to Go time.Time.
|
||||
func humanReadableTimes(creation, modification, upload string) (cTime, mTime, uTime time.Time) {
|
||||
cTime, _ = time.Parse(time.RFC3339, creation)
|
||||
if modification != "" {
|
||||
modMillis, _ := strconv.ParseInt(modification, 10, 64)
|
||||
mTime = time.Unix(0, modMillis*int64(time.Millisecond))
|
||||
}
|
||||
if upload != "" {
|
||||
upMillis, _ := strconv.ParseInt(upload, 10, 64)
|
||||
uTime = time.Unix(0, upMillis*int64(time.Millisecond))
|
||||
}
|
||||
return cTime, mTime, uTime
|
||||
}
|
||||
|
||||
// getDevices fetches and caches top-level devices and folders.
|
||||
func (d *Degoo) getDevices(ctx context.Context) error {
|
||||
const query = `query GetFileChildren5($Token: String! $ParentID: String $AllParentIDs: [String] $Limit: Int! $Order: Int! $NextToken: String ) { getFileChildren5(Token: $Token ParentID: $ParentID AllParentIDs: $AllParentIDs Limit: $Limit Order: $Order NextToken: $NextToken) { Items { ParentID } NextToken } }`
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ParentID": "0",
|
||||
"Limit": 10,
|
||||
"Order": 3,
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetFileChildren5", query, variables)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp DegooGetChildren5Data
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return fmt.Errorf("failed to parse device list: %w", err)
|
||||
}
|
||||
if d.RootFolderID == "0" {
|
||||
if len(resp.GetFileChildren5.Items) > 0 {
|
||||
d.RootFolderID = resp.GetFileChildren5.Items[0].ParentID
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// getAllFileChildren5 fetches all children of a directory with pagination.
|
||||
func (d *Degoo) getAllFileChildren5(ctx context.Context, parentID string) ([]DegooFileItem, error) {
|
||||
const query = `query GetFileChildren5($Token: String! $ParentID: String $AllParentIDs: [String] $Limit: Int! $Order: Int! $NextToken: String ) { getFileChildren5(Token: $Token ParentID: $ParentID AllParentIDs: $AllParentIDs Limit: $Limit Order: $Order NextToken: $NextToken) { Items { ID ParentID Name Category Size CreationTime LastModificationTime LastUploadTime FilePath IsInRecycleBin DeviceID MetadataID } NextToken } }`
|
||||
var allItems []DegooFileItem
|
||||
nextToken := ""
|
||||
for {
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ParentID": parentID,
|
||||
"Limit": 1000,
|
||||
"Order": 3,
|
||||
}
|
||||
if nextToken != "" {
|
||||
variables["NextToken"] = nextToken
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetFileChildren5", query, variables)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var resp DegooGetChildren5Data
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
allItems = append(allItems, resp.GetFileChildren5.Items...)
|
||||
if resp.GetFileChildren5.NextToken == "" {
|
||||
break
|
||||
}
|
||||
nextToken = resp.GetFileChildren5.NextToken
|
||||
}
|
||||
return allItems, nil
|
||||
}
|
||||
|
||||
// getOverlay4 fetches metadata for a single item by ID.
|
||||
func (d *Degoo) getOverlay4(ctx context.Context, id string) (DegooFileItem, error) {
|
||||
const query = `query GetOverlay4($Token: String!, $ID: IDType!) { getOverlay4(Token: $Token, ID: $ID) { ID ParentID Name Category Size CreationTime LastModificationTime LastUploadTime URL FilePath IsInRecycleBin DeviceID MetadataID } }`
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
"ID": map[string]string{
|
||||
"FileID": id,
|
||||
},
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetOverlay4", query, variables)
|
||||
if err != nil {
|
||||
return DegooFileItem{}, err
|
||||
}
|
||||
var resp DegooGetOverlay4Data
|
||||
if err := json.Unmarshal(data, &resp); err != nil {
|
||||
return DegooFileItem{}, fmt.Errorf("failed to parse item metadata: %w", err)
|
||||
}
|
||||
return resp.GetOverlay4, nil
|
||||
}
|
||||
|
||||
func (d *Degoo) getUserInfo(ctx context.Context) (DegooGetUserInfo3Data, error) {
|
||||
const query = "query GetUserInfo3($Token: String!) { getUserInfo3(Token: $Token) { UsedQuota TotalQuota } }"
|
||||
variables := map[string]interface{}{
|
||||
"Token": d.AccessToken,
|
||||
}
|
||||
data, err := d.apiCall(ctx, "GetUserInfo3", query, variables)
|
||||
var resp DegooGetUserInfo3Data
|
||||
if err != nil {
|
||||
return resp, err
|
||||
}
|
||||
if err = json.Unmarshal(data, &resp); err != nil {
|
||||
return resp, fmt.Errorf("failed to parse user info: %w", err)
|
||||
}
|
||||
return resp, nil
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
package doubao
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
// Usually one of two
|
||||
// driver.RootPath
|
||||
driver.RootID
|
||||
// define other
|
||||
Cookie string `json:"cookie" type:"text"`
|
||||
UploadThread string `json:"upload_thread" default:"3"`
|
||||
DownloadApi string `json:"download_api" type:"select" options:"get_file_url,get_download_info" default:"get_file_url"`
|
||||
LimitRate float64 `json:"limit_rate" type:"float" default:"2" help:"limit all api request rate ([limit]r/1s)"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Doubao",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "0",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Doubao{
|
||||
Addition: Addition{
|
||||
LimitRate: 2,
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,597 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/big"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
|
||||
"github.com/go-resty/resty/v2"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/crypto/pbkdf2"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultAuthRefreshAheadSeconds = int64(120)
|
||||
defaultDpopRefreshAheadSeconds = int64(5)
|
||||
)
|
||||
|
||||
type Clock interface {
|
||||
Now() (int64, error)
|
||||
}
|
||||
|
||||
type SystemClock struct{}
|
||||
|
||||
func (SystemClock) Now() (int64, error) { return time.Now().Unix(), nil }
|
||||
|
||||
type DPoPTokenInput struct {
|
||||
KeyPair *ecdsa.PrivateKey
|
||||
ExpiresIn int64 // 默认 15
|
||||
|
||||
JTI string
|
||||
HTM string
|
||||
HTU string
|
||||
IAT int64
|
||||
Nonce string
|
||||
Clock Clock
|
||||
}
|
||||
|
||||
type DPoPTokenOutput struct {
|
||||
DPoPToken string `json:"dpopToken"`
|
||||
ExpiredTime int64 `json:"expiredTime"`
|
||||
ExpiresIn int64 `json:"expiresIn"`
|
||||
}
|
||||
|
||||
type JWTPayload struct {
|
||||
Exp int64 `json:"exp,omitempty"`
|
||||
Iat int64 `json:"iat,omitempty"`
|
||||
Nbf int64 `json:"nbf,omitempty"`
|
||||
Jti string `json:"jti,omitempty"`
|
||||
Htm string `json:"htm,omitempty"`
|
||||
Htu string `json:"htu,omitempty"`
|
||||
Nonce string `json:"nonce,omitempty"`
|
||||
Sub string `json:"sub,omitempty"`
|
||||
}
|
||||
|
||||
type jwkECPrivateKey struct {
|
||||
Kty string `json:"kty"`
|
||||
Crv string `json:"crv"`
|
||||
X string `json:"x"`
|
||||
Y string `json:"y"`
|
||||
D string `json:"d"`
|
||||
}
|
||||
|
||||
type dpopKeyPairEnvelope struct {
|
||||
PrivateKey *jwkECPrivateKey `json:"privateKey"`
|
||||
KeyPair *jwkECPrivateKey `json:"keyPair"`
|
||||
JWK *jwkECPrivateKey `json:"jwk"`
|
||||
}
|
||||
|
||||
type encryptedDpopKeyPair struct {
|
||||
Data string `json:"data"`
|
||||
Ciphertext string `json:"ciphertext"`
|
||||
Encrypted string `json:"encrypted"`
|
||||
Secret string `json:"secret"`
|
||||
Password string `json:"password"`
|
||||
Passphrase string `json:"passphrase"`
|
||||
}
|
||||
|
||||
func GenerateDPoPToken(in DPoPTokenInput) (*DPoPTokenOutput, error) {
|
||||
if in.KeyPair == nil {
|
||||
return nil, errors.New("keyPair required")
|
||||
}
|
||||
if in.KeyPair.Curve != elliptic.P256() {
|
||||
return nil, errors.New("ES256 requires P-256 key")
|
||||
}
|
||||
if in.Clock == nil {
|
||||
in.Clock = SystemClock{}
|
||||
}
|
||||
if in.ExpiresIn <= 0 {
|
||||
in.ExpiresIn = 15
|
||||
}
|
||||
|
||||
now, err := in.Clock.Now()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
payload := map[string]any{
|
||||
"jti": pickStr(in.JTI, uuid.NewString()),
|
||||
"htm": pickStr(in.HTM, ""),
|
||||
"htu": pickStr(in.HTU, ""),
|
||||
"iat": pickI64(in.IAT, now),
|
||||
"nonce": pickStr(in.Nonce, uuid.NewString()),
|
||||
}
|
||||
if in.ExpiresIn > 0 {
|
||||
payload["exp"] = payload["iat"].(int64) + in.ExpiresIn
|
||||
}
|
||||
|
||||
pub := in.KeyPair.PublicKey
|
||||
header := map[string]any{
|
||||
"typ": "dpop+jwt",
|
||||
"alg": "ES256",
|
||||
"jwk": map[string]string{
|
||||
"kty": "EC",
|
||||
"crv": "P-256",
|
||||
"x": b64url(pad32(pub.X.Bytes())),
|
||||
"y": b64url(pad32(pub.Y.Bytes())),
|
||||
},
|
||||
}
|
||||
|
||||
hb, err := json.Marshal(header)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pb, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
hEnc := b64url(hb)
|
||||
pEnc := b64url(pb)
|
||||
signingInput := hEnc + "." + pEnc
|
||||
|
||||
sum := sha256.Sum256([]byte(signingInput))
|
||||
r, s, err := ecdsa.Sign(rand.Reader, in.KeyPair, sum[:])
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
sig := append(pad32(r.Bytes()), pad32(s.Bytes())...)
|
||||
token := signingInput + "." + b64url(sig)
|
||||
|
||||
iat := payload["iat"].(int64)
|
||||
return &DPoPTokenOutput{
|
||||
DPoPToken: token,
|
||||
ExpiredTime: iat + in.ExpiresIn,
|
||||
ExpiresIn: in.ExpiresIn,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func GenerateDPoPKeyPair() (*ecdsa.PrivateKey, error) {
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return validateP256Key(key)
|
||||
}
|
||||
|
||||
func ParseJWTPayload(token string, out any) error {
|
||||
token = strings.TrimSpace(trimTokenScheme(token))
|
||||
parts := strings.Split(token, ".")
|
||||
if len(parts) < 2 {
|
||||
return fmt.Errorf("invalid JWT format")
|
||||
}
|
||||
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to decode JWT payload: %w", err)
|
||||
}
|
||||
if err := json.Unmarshal(payload, out); err != nil {
|
||||
return fmt.Errorf("failed to parse JWT payload: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func parseECPrivateKeyJWK(raw string) (*ecdsa.PrivateKey, error) {
|
||||
var jwk jwkECPrivateKey
|
||||
if err := json.Unmarshal([]byte(raw), &jwk); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if jwk.D == "" || jwk.X == "" || jwk.Y == "" {
|
||||
var env dpopKeyPairEnvelope
|
||||
if err := json.Unmarshal([]byte(raw), &env); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch {
|
||||
case env.PrivateKey != nil:
|
||||
jwk = *env.PrivateKey
|
||||
case env.KeyPair != nil:
|
||||
jwk = *env.KeyPair
|
||||
case env.JWK != nil:
|
||||
jwk = *env.JWK
|
||||
default:
|
||||
return nil, errors.New("missing private key JWK")
|
||||
}
|
||||
}
|
||||
|
||||
if jwk.Kty != "" && jwk.Kty != "EC" {
|
||||
return nil, errors.New("unsupported JWK kty")
|
||||
}
|
||||
if jwk.Crv != "" && jwk.Crv != "P-256" {
|
||||
return nil, errors.New("unsupported JWK curve")
|
||||
}
|
||||
if jwk.D == "" || jwk.X == "" || jwk.Y == "" {
|
||||
return nil, errors.New("incomplete JWK")
|
||||
}
|
||||
|
||||
xBytes, err := base64.RawURLEncoding.DecodeString(jwk.X)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid jwk x: %w", err)
|
||||
}
|
||||
yBytes, err := base64.RawURLEncoding.DecodeString(jwk.Y)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid jwk y: %w", err)
|
||||
}
|
||||
dBytes, err := base64.RawURLEncoding.DecodeString(jwk.D)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid jwk d: %w", err)
|
||||
}
|
||||
|
||||
key := &ecdsa.PrivateKey{
|
||||
PublicKey: ecdsa.PublicKey{
|
||||
Curve: elliptic.P256(),
|
||||
X: new(big.Int).SetBytes(xBytes),
|
||||
Y: new(big.Int).SetBytes(yBytes),
|
||||
},
|
||||
D: new(big.Int).SetBytes(dBytes),
|
||||
}
|
||||
return validateP256Key(key)
|
||||
}
|
||||
|
||||
func parseEncryptedDPoPKeyPair(raw, secret string) (*ecdsa.PrivateKey, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if raw == "" {
|
||||
return nil, errors.New("empty encrypted key pair")
|
||||
}
|
||||
|
||||
var payload encryptedDpopKeyPair
|
||||
ciphertext := raw
|
||||
if strings.HasPrefix(raw, "{") {
|
||||
if err := json.Unmarshal([]byte(raw), &payload); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch {
|
||||
case strings.TrimSpace(payload.Data) != "":
|
||||
ciphertext = strings.TrimSpace(payload.Data)
|
||||
case strings.TrimSpace(payload.Ciphertext) != "":
|
||||
ciphertext = strings.TrimSpace(payload.Ciphertext)
|
||||
case strings.TrimSpace(payload.Encrypted) != "":
|
||||
ciphertext = strings.TrimSpace(payload.Encrypted)
|
||||
default:
|
||||
return nil, errors.New("missing encrypted dpop payload")
|
||||
}
|
||||
}
|
||||
|
||||
decoded, err := decodeBase64Loose(ciphertext)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(decoded) <= 12 {
|
||||
return nil, errors.New("encrypted dpop payload too short")
|
||||
}
|
||||
|
||||
plain, err := decryptDoubaoKeyPair(decoded, secret)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to decrypt with secret: %w", err)
|
||||
}
|
||||
return parseECPrivateKeyJWK(string(plain))
|
||||
}
|
||||
|
||||
func decryptDoubaoKeyPair(ciphertext []byte, secret string) ([]byte, error) {
|
||||
key := pbkdf2.Key([]byte(secret), []byte("fixed-salt"), 100000, 32, sha256.New)
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
aead, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
nonceSize := aead.NonceSize()
|
||||
if len(ciphertext) <= nonceSize {
|
||||
return nil, errors.New("ciphertext too short")
|
||||
}
|
||||
nonce := ciphertext[:nonceSize]
|
||||
enc := ciphertext[nonceSize:]
|
||||
return aead.Open(nil, nonce, enc, nil)
|
||||
}
|
||||
|
||||
func decodeBase64Loose(raw string) ([]byte, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
raw = strings.ReplaceAll(raw, "\n", "")
|
||||
raw = strings.ReplaceAll(raw, "\r", "")
|
||||
raw = strings.ReplaceAll(raw, "\t", "")
|
||||
raw = strings.ReplaceAll(raw, " ", "")
|
||||
|
||||
encodings := []*base64.Encoding{
|
||||
base64.StdEncoding,
|
||||
base64.RawStdEncoding,
|
||||
base64.URLEncoding,
|
||||
base64.RawURLEncoding,
|
||||
}
|
||||
var lastErr error
|
||||
for _, enc := range encodings {
|
||||
decoded, err := enc.DecodeString(raw)
|
||||
if err == nil {
|
||||
return decoded, nil
|
||||
}
|
||||
lastErr = err
|
||||
}
|
||||
if lastErr == nil {
|
||||
lastErr = errors.New("invalid base64")
|
||||
}
|
||||
return nil, lastErr
|
||||
}
|
||||
|
||||
func validateP256Key(key *ecdsa.PrivateKey) (*ecdsa.PrivateKey, error) {
|
||||
if key == nil {
|
||||
return nil, errors.New("nil private key")
|
||||
}
|
||||
if key.Curve != elliptic.P256() {
|
||||
return nil, errors.New("ES256 requires P-256 key")
|
||||
}
|
||||
if key.PublicKey.X == nil || key.PublicKey.Y == nil || key.D == nil {
|
||||
return nil, errors.New("invalid private key")
|
||||
}
|
||||
if !key.Curve.IsOnCurve(key.PublicKey.X, key.PublicKey.Y) {
|
||||
return nil, errors.New("public key is not on P-256 curve")
|
||||
}
|
||||
return key, nil
|
||||
}
|
||||
|
||||
func trimTokenScheme(token string) string {
|
||||
token = strings.TrimSpace(token)
|
||||
if i := strings.IndexByte(token, ' '); i > 0 {
|
||||
scheme := strings.ToLower(strings.TrimSpace(token[:i]))
|
||||
if scheme == "bearer" || scheme == "dpop" {
|
||||
return strings.TrimSpace(token[i+1:])
|
||||
}
|
||||
}
|
||||
return token
|
||||
}
|
||||
|
||||
func b64url(b []byte) string {
|
||||
return base64.RawURLEncoding.EncodeToString(b)
|
||||
}
|
||||
|
||||
func pad32(b []byte) []byte {
|
||||
if len(b) >= 32 {
|
||||
return b[len(b)-32:]
|
||||
}
|
||||
out := make([]byte, 32)
|
||||
copy(out[32-len(b):], b)
|
||||
return out
|
||||
}
|
||||
|
||||
func pickStr(v, def string) string {
|
||||
if v != "" {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
func pickI64(v, def int64) int64 {
|
||||
if v != 0 {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveAuthorization() string {
|
||||
auth := trimTokenScheme(d.Authorization)
|
||||
if auth == "" {
|
||||
return ""
|
||||
}
|
||||
return "DPoP " + auth
|
||||
}
|
||||
|
||||
func shouldRefreshJWT(token string) bool {
|
||||
if token == "" {
|
||||
return true
|
||||
}
|
||||
var payload JWTPayload
|
||||
if err := ParseJWTPayload(token, &payload); err != nil {
|
||||
return true
|
||||
}
|
||||
if payload.Exp <= 0 {
|
||||
return false
|
||||
}
|
||||
return payload.Exp <= time.Now().Unix()+defaultAuthRefreshAheadSeconds
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) fetchBizAuth(dpop string, public bool) (string, error) {
|
||||
var reqUrl string
|
||||
client := base.RestyClient.Clone()
|
||||
req := client.R()
|
||||
req.SetHeader("accept", "application/json, text/javascript")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("content-type", "application/x-www-form-urlencoded")
|
||||
if public {
|
||||
reqUrl = DoubaoURL + "/passport/anonymity_user/biz_auth/"
|
||||
} else {
|
||||
reqUrl = DoubaoURL + "/passport/user/biz_auth/"
|
||||
if d.Cookie != "" {
|
||||
req.SetHeader("cookie", d.Cookie)
|
||||
if csrf := strings.TrimSpace(cookie.GetStr(d.Cookie, "passport_csrf_token")); csrf != "" {
|
||||
req.SetHeader("x-tt-passport-csrf-token", csrf)
|
||||
}
|
||||
}
|
||||
if oldAuth := d.resolveAuthorization(); oldAuth != "" {
|
||||
req.SetHeader("authorization", oldAuth)
|
||||
}
|
||||
}
|
||||
if dpop != "" {
|
||||
req.SetHeader("dpop", dpop)
|
||||
}
|
||||
values := url.Values{}
|
||||
values.Set("client_id", d.AuthClientID)
|
||||
values.Set("client_type", d.AuthClientType)
|
||||
values.Set("scope", d.AuthScope)
|
||||
values.Set("d_pop", dpop)
|
||||
req.SetBody(values.Encode())
|
||||
req.SetQueryParam("aid", d.AppID)
|
||||
req.SetQueryParam("account_sdk_source", d.AuthSDKSource)
|
||||
req.SetQueryParam("sdk_version", d.AuthSDKVersion)
|
||||
|
||||
res, err := req.Post(reqUrl)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
var resp bizAuthResp
|
||||
if err = json.Unmarshal(res.Body(), &resp); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if resp.Message != "success" || resp.Data.AccessToken == "" {
|
||||
return "", fmt.Errorf("[doubao_new] %s: %s", resp.Message, resp.Data.Description)
|
||||
}
|
||||
return resp.Data.AccessToken, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) refreshAuthorizationWithDPoP(dpop string) (string, error) {
|
||||
token, err := d.fetchBizAuth(dpop, false)
|
||||
if err == nil && token != "" {
|
||||
return token, nil
|
||||
}
|
||||
if err == nil {
|
||||
err = errors.New("biz auth refresh failed")
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveDpopForRequest(method, rawURL string) (string, error) {
|
||||
if d.DPoPKeyPair != nil {
|
||||
proof, err := GenerateDPoPToken(DPoPTokenInput{
|
||||
KeyPair: d.DPoPKeyPair,
|
||||
HTM: strings.ToUpper(strings.TrimSpace(method)),
|
||||
HTU: normalizeDPoPURL(rawURL),
|
||||
})
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return proof.DPoPToken, nil
|
||||
}
|
||||
|
||||
static := d.DPoP
|
||||
if static == "" {
|
||||
return "", nil
|
||||
}
|
||||
if !d.IgnoreJWTCheck {
|
||||
if payload, err := parseDPoPPayload(static); err == nil && payload.Exp > 0 {
|
||||
now := time.Now().Unix()
|
||||
if payload.Exp <= now+defaultDpopRefreshAheadSeconds {
|
||||
return "", errors.New("static dpop token expired or near expiry; configure dpop_key_pair for automatic refresh")
|
||||
}
|
||||
}
|
||||
}
|
||||
return static, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) ensureAuthAdditons() bool {
|
||||
return d.DPoPKeySecret != "" && d.AuthClientID != "" && d.AuthClientType != "" &&
|
||||
d.AuthScope != "" && d.AuthSDKSource != "" && d.AuthSDKVersion != ""
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveAuthorizationForRequest(method, rawURL string) (string, error) {
|
||||
if !shouldRefreshJWT(d.Authorization) {
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
if d.DPoPKeyPair == nil || strings.TrimSpace(d.Cookie) == "" || !d.ensureAuthAdditons() {
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
d.authRefreshMu.Lock()
|
||||
defer d.authRefreshMu.Unlock()
|
||||
|
||||
if !shouldRefreshJWT(d.Authorization) {
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
refreshDpop, err := d.resolveDpopForRequest(method, rawURL)
|
||||
if err != nil || refreshDpop == "" {
|
||||
return "", err
|
||||
}
|
||||
|
||||
newToken, err := d.refreshAuthorizationWithDPoP(refreshDpop)
|
||||
if err != nil {
|
||||
if auth := d.resolveAuthorization(); auth != "" {
|
||||
return auth, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
d.Authorization = trimTokenScheme(newToken)
|
||||
return d.resolveAuthorization(), nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) resolveAuthorizationForPublic() (dpop string, auth string, err error) {
|
||||
if d.DPoPPublic != "" && !shouldRefreshJWT(d.AuthorizationPublic) {
|
||||
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
|
||||
}
|
||||
|
||||
if !d.ensureAuthAdditons() {
|
||||
return "", "", fmt.Errorf("[doubao_new] missing auth additions, please fill them all")
|
||||
}
|
||||
|
||||
d.authRefreshPublicMu.Lock()
|
||||
defer d.authRefreshPublicMu.Unlock()
|
||||
|
||||
if d.DPoPPublic != "" && !shouldRefreshJWT(d.AuthorizationPublic) {
|
||||
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
|
||||
}
|
||||
|
||||
// generate new public dpop
|
||||
keypair, err := GenerateDPoPKeyPair()
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
proof, err := GenerateDPoPToken(DPoPTokenInput{
|
||||
KeyPair: keypair,
|
||||
})
|
||||
d.DPoPPublic = proof.DPoPToken
|
||||
|
||||
// get authorization token
|
||||
d.AuthorizationPublic, err = d.fetchBizAuth(proof.DPoPToken, true)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
|
||||
return d.DPoPPublic, "DPoP " + d.AuthorizationPublic, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) applyAuthHeaders(req *resty.Request, method, rawURL string) error {
|
||||
auth, err := d.resolveAuthorizationForRequest(method, rawURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if auth != "" {
|
||||
req.SetHeader("authorization", auth)
|
||||
}
|
||||
dpop, err := d.resolveDpopForRequest(method, rawURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if dpop != "" {
|
||||
req.SetHeader("dpop", dpop)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func normalizeDPoPURL(rawURL string) string {
|
||||
u, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return rawURL
|
||||
}
|
||||
u.Fragment = ""
|
||||
return u.String()
|
||||
}
|
||||
|
||||
func parseDPoPPayload(token string) (*JWTPayload, error) {
|
||||
var payload JWTPayload
|
||||
if err := ParseJWTPayload(token, &payload); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &payload, nil
|
||||
}
|
||||
@@ -1,533 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/ecdsa"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type DoubaoNew struct {
|
||||
model.Storage
|
||||
Addition
|
||||
TtLogid string
|
||||
|
||||
// DPoP access token (Authorization header value, without DPoP prefix)
|
||||
Authorization string
|
||||
AuthorizationPublic string
|
||||
// DPoP header value
|
||||
DPoP string
|
||||
DPoPPublic string
|
||||
// DPoP key pair for generating DPoP
|
||||
DPoPKeyPairStr string
|
||||
DPoPKeyPair *ecdsa.PrivateKey
|
||||
|
||||
authRefreshMu sync.Mutex
|
||||
authRefreshPublicMu sync.Mutex
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Init(ctx context.Context) error {
|
||||
if cookieStr := strings.TrimSpace(d.Cookie); cookieStr != "" {
|
||||
d.Cookie = cookieStr
|
||||
auth := trimTokenScheme(cookie.GetStr(d.Cookie, "LARK_SUITE_ACCESS_TOKEN"))
|
||||
if auth != "" {
|
||||
d.Authorization = auth
|
||||
}
|
||||
dpop := strings.TrimSpace(cookie.GetStr(d.Cookie, "LARK_SUITE_DPOP"))
|
||||
if dpop != "" {
|
||||
d.DPoP = dpop
|
||||
}
|
||||
keypair := strings.TrimSpace(cookie.GetStr(d.Cookie, "feishu_dpop_keypair"))
|
||||
if keypair != "" && d.DPoPKeySecret != "" {
|
||||
d.DPoPKeyPairStr = keypair
|
||||
d.DPoPKeyPair, _ = parseEncryptedDPoPKeyPair(keypair, d.DPoPKeySecret)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Drop(ctx context.Context) error {
|
||||
if d.Authorization != "" {
|
||||
d.Cookie = cookie.SetStr(d.Cookie, "LARK_SUITE_ACCESS_TOKEN", d.Authorization)
|
||||
}
|
||||
if d.DPoP != "" {
|
||||
d.Cookie = cookie.SetStr(d.Cookie, "LARK_SUITE_DPOP", d.DPoP)
|
||||
}
|
||||
if d.DPoPKeyPairStr != "" {
|
||||
d.Cookie = cookie.SetStr(d.Cookie, "feishu_dpop_keypair", d.DPoPKeyPairStr)
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
nodes, err := d.listAllChildren(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
objs := make([]model.Obj, 0, len(nodes))
|
||||
for _, node := range nodes {
|
||||
if node.NodeToken == "" || node.ObjToken == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
size := parseSize(node.Extra.Size)
|
||||
isFolder := node.Type == 0
|
||||
if isFolder && node.NodeToken == dir.GetID() {
|
||||
continue
|
||||
}
|
||||
|
||||
obj := &Object{
|
||||
Object: model.Object{
|
||||
ID: node.NodeToken,
|
||||
Path: dir.GetID(),
|
||||
Name: node.Name,
|
||||
Size: size,
|
||||
Modified: time.Unix(node.EditTime, 0),
|
||||
Ctime: time.Unix(node.CreateTime, 0),
|
||||
IsFolder: isFolder,
|
||||
},
|
||||
ObjToken: node.ObjToken,
|
||||
NodeType: node.NodeType,
|
||||
ObjType: node.Type,
|
||||
URL: node.URL,
|
||||
}
|
||||
objs = append(objs, obj)
|
||||
}
|
||||
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
obj, ok := file.(*Object)
|
||||
if !ok {
|
||||
return nil, errors.New("unsupported object type")
|
||||
}
|
||||
if obj.IsFolder {
|
||||
return nil, fmt.Errorf("link is directory")
|
||||
}
|
||||
var (
|
||||
err error
|
||||
auth, dpop string
|
||||
)
|
||||
if d.ShareLink {
|
||||
err := d.createShare(ctx, obj)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
dpop, auth, err = d.resolveAuthorizationForPublic()
|
||||
} else {
|
||||
// TODO: append previewLink() with auth args to support ShareLink
|
||||
if args.Type == "preview" || args.Type == "thumb" {
|
||||
if link, err := d.previewLink(ctx, obj, args); err == nil {
|
||||
return link, nil
|
||||
}
|
||||
}
|
||||
auth = d.resolveAuthorization()
|
||||
dpop, err = d.resolveDpopForRequest(http.MethodGet, DownloadBaseURL+"/space/api/box/stream/download/all/"+obj.ObjToken+"/")
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if auth == "" || dpop == "" {
|
||||
return nil, errors.New("missing authorization or dpop")
|
||||
}
|
||||
if obj.ObjToken == "" {
|
||||
return nil, errors.New("missing obj_token")
|
||||
}
|
||||
|
||||
query := url.Values{}
|
||||
query.Set("authorization", auth)
|
||||
query.Set("dpop", dpop)
|
||||
|
||||
downloadURL := DownloadBaseURL + "/space/api/box/stream/download/all/" + obj.ObjToken + "/?" + query.Encode()
|
||||
|
||||
headers := http.Header{
|
||||
"Referer": []string{DoubaoURL + "/"},
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
URL: downloadURL,
|
||||
Header: headers,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
node, err := d.createFolder(ctx, parentDir.GetID(), dirName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Object{
|
||||
Object: model.Object{
|
||||
ID: node.NodeToken,
|
||||
Path: parentDir.GetID(),
|
||||
Name: node.Name,
|
||||
Size: parseSize(node.Extra.Size),
|
||||
Modified: time.Unix(node.EditTime, 0),
|
||||
Ctime: time.Unix(node.CreateTime, 0),
|
||||
IsFolder: true,
|
||||
},
|
||||
ObjToken: node.ObjToken,
|
||||
NodeType: node.NodeType,
|
||||
ObjType: node.Type,
|
||||
URL: node.URL,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
if srcObj == nil {
|
||||
return nil, errors.New("nil source object")
|
||||
}
|
||||
if dstDir == nil {
|
||||
return nil, errors.New("nil destination dir")
|
||||
}
|
||||
srcToken := srcObj.GetID()
|
||||
if srcToken == "" {
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
srcToken = obj.ObjToken
|
||||
}
|
||||
}
|
||||
if srcToken == "" {
|
||||
return nil, errors.New("missing source token")
|
||||
}
|
||||
if err := d.moveObj(ctx, srcToken, dstDir.GetID()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
clone := *obj
|
||||
clone.Path = dstDir.GetID()
|
||||
return &clone, nil
|
||||
}
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
|
||||
if srcObj == nil {
|
||||
return nil, errors.New("nil source object")
|
||||
}
|
||||
if srcObj.IsDir() {
|
||||
if err := d.renameFolder(ctx, srcObj.GetID(), newName); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
fileToken := ""
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
fileToken = obj.ObjToken
|
||||
}
|
||||
if fileToken == "" {
|
||||
fileToken = srcObj.GetID()
|
||||
}
|
||||
if err := d.renameFile(ctx, fileToken, newName); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if obj, ok := srcObj.(*Object); ok {
|
||||
clone := *obj
|
||||
clone.Name = newName
|
||||
return &clone, nil
|
||||
}
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
// TODO copy obj, optional
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Remove(ctx context.Context, obj model.Obj) error {
|
||||
if obj == nil {
|
||||
return errors.New("nil object")
|
||||
}
|
||||
token := obj.GetID()
|
||||
if token == "" {
|
||||
if o, ok := obj.(*Object); ok {
|
||||
token = o.ObjToken
|
||||
}
|
||||
}
|
||||
if token == "" {
|
||||
return errors.New("missing object token")
|
||||
}
|
||||
return d.removeObj(ctx, []string{token})
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
if file == nil {
|
||||
return nil, errors.New("nil file")
|
||||
}
|
||||
if file.GetSize() <= 0 {
|
||||
return nil, errors.New("invalid file size")
|
||||
}
|
||||
|
||||
uploadPrep, err := d.prepareUpload(ctx, file.GetName(), file.GetSize(), dstDir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if uploadPrep.BlockSize <= 0 {
|
||||
return nil, errors.New("invalid block size from prepare")
|
||||
}
|
||||
|
||||
tmpFile, err := utils.CreateTempFile(file, file.GetSize())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() {
|
||||
_ = tmpFile.Close()
|
||||
_ = os.Remove(tmpFile.Name())
|
||||
}()
|
||||
|
||||
blockSize := uploadPrep.BlockSize
|
||||
totalSize := file.GetSize()
|
||||
numBlocks := int((totalSize + blockSize - 1) / blockSize)
|
||||
blocks := make([]UploadBlock, 0, numBlocks)
|
||||
blockMeta := make(map[int]UploadBlock, numBlocks)
|
||||
|
||||
for seq := 0; seq < numBlocks; seq++ {
|
||||
offset := int64(seq) * blockSize
|
||||
length := blockSize
|
||||
if remain := totalSize - offset; remain < length {
|
||||
length = remain
|
||||
}
|
||||
buf := make([]byte, int(length))
|
||||
n, err := tmpFile.ReadAt(buf, offset)
|
||||
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
|
||||
return nil, err
|
||||
}
|
||||
buf = buf[:n]
|
||||
sum := sha256.Sum256(buf)
|
||||
hash := base64.StdEncoding.EncodeToString(sum[:])
|
||||
checksum := adler32String(buf)
|
||||
|
||||
block := UploadBlock{
|
||||
Hash: hash,
|
||||
Seq: seq,
|
||||
Size: int64(n),
|
||||
Checksum: checksum,
|
||||
IsUploaded: true,
|
||||
}
|
||||
blocks = append(blocks, block)
|
||||
blockMeta[seq] = block
|
||||
}
|
||||
|
||||
needed, err := d.uploadBlocks(ctx, uploadPrep.UploadID, blocks, "explorer")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(needed.NeededUploadBlocks) > 0 {
|
||||
sort.Slice(needed.NeededUploadBlocks, func(i, j int) bool {
|
||||
return needed.NeededUploadBlocks[i].Seq < needed.NeededUploadBlocks[j].Seq
|
||||
})
|
||||
const maxMergeBlockCount = 20
|
||||
var (
|
||||
groupSeqs []int
|
||||
groupChecksums []string
|
||||
groupSizes []int64
|
||||
groupRealSize int64
|
||||
groupExpectSum int64
|
||||
groupBuf bytes.Buffer
|
||||
uploadedBytes int64
|
||||
)
|
||||
|
||||
flushGroup := func() error {
|
||||
if len(groupSeqs) == 0 {
|
||||
return nil
|
||||
}
|
||||
data := groupBuf.Bytes()
|
||||
expectLen := groupExpectSum
|
||||
if int64(len(data)) != expectLen {
|
||||
return fmt.Errorf("[doubao_new] merge blocks invalid body len: got=%d expect=%d seqs=%v", len(data), expectLen, groupSeqs)
|
||||
}
|
||||
mergeResp, err := d.mergeUploadBlocks(ctx, uploadPrep.UploadID, groupSeqs, groupChecksums, groupSizes, blockSize, data)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(mergeResp.SuccessSeqList) != len(groupSeqs) {
|
||||
return fmt.Errorf("[doubao_new] merge blocks incomplete: %v", mergeResp.SuccessSeqList)
|
||||
}
|
||||
success := make(map[int]bool, len(mergeResp.SuccessSeqList))
|
||||
for _, seq := range mergeResp.SuccessSeqList {
|
||||
success[seq] = true
|
||||
}
|
||||
for _, seq := range groupSeqs {
|
||||
if !success[seq] {
|
||||
return fmt.Errorf("[doubao_new] merge blocks missing seq %d", seq)
|
||||
}
|
||||
}
|
||||
|
||||
uploadedBytes += groupRealSize
|
||||
groupSeqs = groupSeqs[:0]
|
||||
groupChecksums = groupChecksums[:0]
|
||||
groupSizes = groupSizes[:0]
|
||||
groupRealSize = 0
|
||||
groupExpectSum = 0
|
||||
groupBuf.Reset()
|
||||
if up != nil {
|
||||
percent := float64(uploadedBytes) / float64(totalSize) * 100
|
||||
up(percent)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
for _, item := range needed.NeededUploadBlocks {
|
||||
if _, ok := blockMeta[item.Seq]; !ok {
|
||||
return nil, fmt.Errorf("[doubao_new] missing block meta for seq %d", item.Seq)
|
||||
}
|
||||
if item.Size <= 0 {
|
||||
return nil, fmt.Errorf("[doubao_new] invalid block size from needed list: seq=%d size=%d", item.Seq, item.Size)
|
||||
}
|
||||
offset := int64(item.Seq) * blockSize
|
||||
buf := make([]byte, int(item.Size))
|
||||
n, err := tmpFile.ReadAt(buf, offset)
|
||||
if err != nil && err != io.EOF && err != io.ErrUnexpectedEOF {
|
||||
return nil, err
|
||||
}
|
||||
if n != len(buf) {
|
||||
return nil, fmt.Errorf("[doubao_new] short read: seq=%d want=%d got=%d", item.Seq, len(buf), n)
|
||||
}
|
||||
buf = buf[:n]
|
||||
realAdler := adler32String(buf)
|
||||
if realAdler != item.Checksum {
|
||||
return nil, fmt.Errorf("[doubao_new] block checksum mismatch: seq=%d offset=%d adler32=%s step2=%s", item.Seq, offset, realAdler, item.Checksum)
|
||||
}
|
||||
payloadStart := groupBuf.Len()
|
||||
groupBuf.Write(buf)
|
||||
payloadEnd := groupBuf.Len()
|
||||
payloadAdler := adler32String(groupBuf.Bytes()[payloadStart:payloadEnd])
|
||||
if payloadAdler != item.Checksum {
|
||||
return nil, fmt.Errorf("[doubao_new] payload checksum mismatch: seq=%d start=%d end=%d adler32=%s step2=%s", item.Seq, payloadStart, payloadEnd, payloadAdler, item.Checksum)
|
||||
}
|
||||
groupSeqs = append(groupSeqs, item.Seq)
|
||||
groupChecksums = append(groupChecksums, item.Checksum)
|
||||
groupSizes = append(groupSizes, item.Size)
|
||||
groupRealSize += int64(n)
|
||||
groupExpectSum += item.Size
|
||||
if len(groupSeqs) >= maxMergeBlockCount {
|
||||
if err := flushGroup(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if err := flushGroup(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if up != nil {
|
||||
up(100)
|
||||
}
|
||||
} else if up != nil {
|
||||
up(100)
|
||||
}
|
||||
|
||||
numBlocksFinish := uploadPrep.NumBlocks
|
||||
if numBlocksFinish <= 0 {
|
||||
numBlocksFinish = numBlocks
|
||||
}
|
||||
finish, err := d.finishUpload(ctx, uploadPrep.UploadID, numBlocksFinish, "explorer")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
nodeToken := finish.Extra.NodeToken
|
||||
if nodeToken == "" {
|
||||
nodeToken = finish.FileToken
|
||||
}
|
||||
now := time.Now()
|
||||
return &Object{
|
||||
Object: model.Object{
|
||||
ID: nodeToken,
|
||||
Path: dstDir.GetID(),
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
Modified: now,
|
||||
Ctime: now,
|
||||
IsFolder: false,
|
||||
},
|
||||
ObjToken: finish.FileToken,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
data, err := d.getUserStorage(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: data.TotalSizeLimitBytes,
|
||||
UsedSpace: data.UsedSizeBytes,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
|
||||
switch args.Method {
|
||||
case "doubao_preview", "preview":
|
||||
obj, ok := args.Obj.(*Object)
|
||||
if !ok {
|
||||
return nil, errors.New("unsupported object type")
|
||||
}
|
||||
info, err := d.getFileInfo(ctx, obj.ObjToken)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entry, ok := info.PreviewMeta.Data["22"]
|
||||
if !ok || entry.Status != 0 {
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
|
||||
imgExt := ".webp"
|
||||
pageNums := 1
|
||||
if entry.Extra != "" {
|
||||
var extra PreviewImageExtra
|
||||
if err := json.Unmarshal([]byte(entry.Extra), &extra); err == nil {
|
||||
if extra.ImgExt != "" {
|
||||
imgExt = extra.ImgExt
|
||||
}
|
||||
if extra.PageNums > 0 {
|
||||
pageNums = extra.PageNums
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return base.Json{
|
||||
"version": info.Version,
|
||||
"img_ext": imgExt,
|
||||
"page_nums": pageNums,
|
||||
}, nil
|
||||
default:
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*DoubaoNew)(nil)
|
||||
@@ -1,39 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
// Usually one of two
|
||||
driver.RootID
|
||||
// define other
|
||||
Cookie string `json:"cookie" required:"true" help:"Web Cookie"`
|
||||
AppID string `json:"app_id" required:"true" default:"497858" help:"Doubao App ID"`
|
||||
DPoPKeySecret string `json:"dpop_key_secret" help:"DPoP Key Secret for generating DPoP token"`
|
||||
AuthClientID string `json:"auth_client_id" help:"Doubao Biz Auth Client ID"`
|
||||
AuthClientType string `json:"auth_client_type" help:"Doubao Biz Auth Client Type"`
|
||||
AuthScope string `json:"auth_scope" help:"Doubao Biz Auth Scope"`
|
||||
AuthSDKSource string `json:"auth_sdk_source" help:"Doubao Biz Auth SDK Source"`
|
||||
AuthSDKVersion string `json:"auth_sdk_version" help:"Doubao Biz Auth SDK Version"`
|
||||
ShareLink bool `json:"share_link" help:"Whether to use share link for download"`
|
||||
IgnoreJWTCheck bool `json:"ignore_jwt_check" help:"Whether to ignore JWT check to prevent time issue"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "DoubaoNew",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "",
|
||||
Alert: `danger|Do not use 302 if the storage is public accessible.
|
||||
Otherwise, the download link may leak sensitive information such as access token or signature.
|
||||
Others may use the leaked link to access all your files.`,
|
||||
NoOverwriteUpload: false,
|
||||
PreferProxy: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &DoubaoNew{}
|
||||
})
|
||||
}
|
||||
@@ -1,192 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import "github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
|
||||
type BaseResp struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg,omitempty"`
|
||||
Message string `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
type ListResp struct {
|
||||
BaseResp
|
||||
Data ListData `json:"data"`
|
||||
}
|
||||
|
||||
type ListData struct {
|
||||
HasMore bool `json:"has_more"`
|
||||
LastLabel string `json:"last_label"`
|
||||
NodeList []string `json:"node_list"`
|
||||
Entities struct {
|
||||
Nodes map[string]Node `json:"nodes"`
|
||||
Users map[string]User `json:"users"`
|
||||
} `json:"entities"`
|
||||
}
|
||||
|
||||
type Node struct {
|
||||
Token string `json:"token"`
|
||||
NodeToken string `json:"node_token"`
|
||||
ObjToken string `json:"obj_token"`
|
||||
Name string `json:"name"`
|
||||
Type int `json:"type"`
|
||||
NodeType int `json:"node_type"`
|
||||
OwnerID string `json:"owner_id"`
|
||||
EditUID string `json:"edit_uid"`
|
||||
CreateTime int64 `json:"create_time"`
|
||||
EditTime int64 `json:"edit_time"`
|
||||
URL string `json:"url"`
|
||||
Extra struct {
|
||||
Size string `json:"size"`
|
||||
} `json:"extra"`
|
||||
}
|
||||
|
||||
type User struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
type Object struct {
|
||||
model.Object
|
||||
ObjToken string
|
||||
NodeType int
|
||||
ObjType int
|
||||
URL string
|
||||
}
|
||||
|
||||
type CreateFolderResp struct {
|
||||
BaseResp
|
||||
Data struct {
|
||||
Entities struct {
|
||||
Nodes map[string]Node `json:"nodes"`
|
||||
} `json:"entities"`
|
||||
NodeList []string `json:"node_list"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type FileInfoResp struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Data FileInfo `json:"data"`
|
||||
}
|
||||
|
||||
type FileInfo struct {
|
||||
Name string `json:"name"`
|
||||
NumBlocks int `json:"num_blocks"`
|
||||
Version string `json:"version"`
|
||||
MimeType string `json:"mime_type"`
|
||||
MountPoint string `json:"mount_point"`
|
||||
PreviewMeta PreviewMeta `json:"preview_meta"`
|
||||
}
|
||||
|
||||
type PreviewMeta struct {
|
||||
Data map[string]PreviewMetaEntry `json:"data"`
|
||||
}
|
||||
|
||||
type PreviewMetaEntry struct {
|
||||
Status int `json:"status"`
|
||||
Extra string `json:"extra"`
|
||||
PreviewFileSize int64 `json:"preview_file_size"`
|
||||
}
|
||||
|
||||
type PreviewImageExtra struct {
|
||||
ImgExt string `json:"img_ext"`
|
||||
PageNums int `json:"page_nums"`
|
||||
}
|
||||
|
||||
type UserStorageResp struct {
|
||||
BaseResp
|
||||
Data UserStorageData `json:"data"`
|
||||
}
|
||||
|
||||
type UserStorageData struct {
|
||||
ShowSizeLimit bool `json:"show_size_limit"`
|
||||
TotalSizeLimitBytes int64 `json:"total_size_limit_bytes"`
|
||||
UsedSizeBytes int64 `json:"used_size_bytes"`
|
||||
}
|
||||
|
||||
type UploadPrepareResp struct {
|
||||
BaseResp
|
||||
Data UploadPrepareData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadPrepareData struct {
|
||||
BlockSize int64 `json:"block_size"`
|
||||
NumBlocks int `json:"num_blocks"`
|
||||
OptionBlockSize int64 `json:"option_block_size"`
|
||||
DedupeSupport bool `json:"dedupe_support"`
|
||||
UploadID string `json:"upload_id"`
|
||||
}
|
||||
|
||||
type UploadBlock struct {
|
||||
Hash string `json:"hash"`
|
||||
Seq int `json:"seq"`
|
||||
Size int64 `json:"size"`
|
||||
Checksum string `json:"checksum"`
|
||||
IsUploaded bool `json:"isUploaded"`
|
||||
}
|
||||
|
||||
type UploadBlocksResp struct {
|
||||
BaseResp
|
||||
Data UploadBlocksData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadBlocksData struct {
|
||||
NeededUploadBlocks []UploadBlockNeed `json:"needed_upload_blocks"`
|
||||
}
|
||||
|
||||
type UploadBlockNeed struct {
|
||||
Seq int `json:"seq"`
|
||||
Size int64 `json:"size"`
|
||||
Checksum string `json:"checksum"`
|
||||
Hash string `json:"hash"`
|
||||
}
|
||||
|
||||
type UploadMergeResp struct {
|
||||
BaseResp
|
||||
Data UploadMergeData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadMergeData struct {
|
||||
SuccessSeqList []int `json:"success_seq_list"`
|
||||
}
|
||||
|
||||
type UploadFinishResp struct {
|
||||
BaseResp
|
||||
Data UploadFinishData `json:"data"`
|
||||
}
|
||||
|
||||
type UploadFinishData struct {
|
||||
Version string `json:"version"`
|
||||
DataVersion string `json:"data_version"`
|
||||
Extra struct {
|
||||
NodeToken string `json:"node_token"`
|
||||
} `json:"extra"`
|
||||
FileToken string `json:"file_token"`
|
||||
}
|
||||
|
||||
type RemoveResp struct {
|
||||
BaseResp
|
||||
Data struct {
|
||||
TaskID string `json:"task_id"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
type TaskStatusResp struct {
|
||||
BaseResp
|
||||
Data TaskStatusData `json:"data"`
|
||||
}
|
||||
|
||||
type TaskStatusData struct {
|
||||
IsFinish bool `json:"is_finish"`
|
||||
IsFail bool `json:"is_fail"`
|
||||
}
|
||||
|
||||
type bizAuthResp struct {
|
||||
Data struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
AuthScheme string `json:"auth_scheme"`
|
||||
ExpiresIn int64 `json:"expires_in"`
|
||||
Description string `json:"description,omitempty"`
|
||||
} `json:"data"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
@@ -1,283 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
func (d *DoubaoNew) prepareUpload(ctx context.Context, name string, size int64, mountNodeToken string) (UploadPrepareData, error) {
|
||||
var resp UploadPrepareResp
|
||||
_, err := d.request(ctx, "/space/api/box/upload/prepare/", http.MethodPost, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
req.SetQueryParamsFromValues(values)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetHeader("x-command", "space.api.box.upload.prepare")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("cache-control", "no-cache")
|
||||
req.SetHeader("pragma", "no-cache")
|
||||
body := base.Json{
|
||||
"mount_point": "explorer",
|
||||
"mount_node_token": "",
|
||||
"name": name,
|
||||
"size": size,
|
||||
"size_checker": true,
|
||||
}
|
||||
if mountNodeToken != "" {
|
||||
body["mount_node_token"] = mountNodeToken
|
||||
}
|
||||
req.SetBody(body)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return UploadPrepareData{}, err
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) uploadBlocks(ctx context.Context, uploadID string, blocks []UploadBlock, mountPoint string) (UploadBlocksData, error) {
|
||||
if uploadID == "" {
|
||||
return UploadBlocksData{}, fmt.Errorf("[doubao_new] upload blocks missing upload_id")
|
||||
}
|
||||
if mountPoint == "" {
|
||||
mountPoint = "explorer"
|
||||
}
|
||||
var resp UploadBlocksResp
|
||||
_, err := d.request(ctx, "/space/api/box/upload/blocks/", http.MethodPost, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
req.SetQueryParamsFromValues(values)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetHeader("x-command", "space.api.box.upload.blocks")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("cache-control", "no-cache")
|
||||
req.SetHeader("pragma", "no-cache")
|
||||
req.SetBody(base.Json{
|
||||
"blocks": blocks,
|
||||
"upload_id": uploadID,
|
||||
"mount_point": mountPoint,
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return UploadBlocksData{}, err
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) mergeUploadBlocks(ctx context.Context, uploadID string, seqList []int, checksumList []string, sizeList []int64, blockOriginSize int64, data []byte) (UploadMergeData, error) {
|
||||
if uploadID == "" {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks missing upload_id")
|
||||
}
|
||||
if len(seqList) == 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty seq list")
|
||||
}
|
||||
if len(checksumList) == 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty checksum list")
|
||||
}
|
||||
if len(sizeList) != len(seqList) {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks size list mismatch")
|
||||
}
|
||||
if blockOriginSize <= 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks invalid block origin size")
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks empty data")
|
||||
}
|
||||
|
||||
seqHeader := joinIntComma(seqList)
|
||||
checksumHeader := buildCommaHeader(checksumList)
|
||||
|
||||
client := base.NewRestyClient()
|
||||
client.SetCookieJar(nil)
|
||||
req := client.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("content-type", "application/octet-stream")
|
||||
req.SetHeader("x-block-list-checksum", checksumHeader)
|
||||
req.SetHeader("x-seq-list", seqHeader)
|
||||
req.SetHeader("x-block-origin-size", strconv.FormatInt(blockOriginSize, 10))
|
||||
req.SetHeader("x-command", "space.api.box.stream.upload.merge_block")
|
||||
req.SetHeader("x-csrftoken", "")
|
||||
reqID := ""
|
||||
if buf := make([]byte, 16); true {
|
||||
if _, err := rand.Read(buf); err == nil {
|
||||
reqID = hex.EncodeToString(buf)
|
||||
}
|
||||
}
|
||||
if reqID != "" {
|
||||
req.SetHeader("x-request-id", reqID)
|
||||
}
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("upload_id", uploadID)
|
||||
values.Set("mount_point", "explorer")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
urlStr := DownloadBaseURL + "/space/api/box/stream/upload/merge_block/?" + values.Encode()
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, urlStr); err != nil {
|
||||
return UploadMergeData{}, err
|
||||
}
|
||||
req.Header.Del("cookie")
|
||||
if req.Header.Get("x-command") == "" {
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] merge blocks missing x-command header")
|
||||
}
|
||||
req.SetBody(data)
|
||||
|
||||
res, err := req.Execute(http.MethodPost, urlStr)
|
||||
if err != nil {
|
||||
return UploadMergeData{}, err
|
||||
}
|
||||
if v := res.Header().Get("X-Tt-Logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
} else if v := res.Header().Get("x-tt-logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
}
|
||||
body := res.Body()
|
||||
var resp UploadMergeResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return UploadMergeData{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
if res != nil && res.StatusCode() == http.StatusBadRequest && resp.Code == 2 {
|
||||
success := make([]int, 0, len(seqList))
|
||||
offset := 0
|
||||
for i, seq := range seqList {
|
||||
size := sizeList[i]
|
||||
if size <= 0 {
|
||||
return UploadMergeData{SuccessSeqList: success}, fmt.Errorf("[doubao_new] v3 fallback invalid size: seq=%d size=%d", seq, size)
|
||||
}
|
||||
if offset+int(size) > len(data) {
|
||||
return UploadMergeData{SuccessSeqList: success}, fmt.Errorf("[doubao_new] v3 fallback payload out of range: seq=%d offset=%d size=%d total=%d", seq, offset, size, len(data))
|
||||
}
|
||||
payload := data[offset : offset+int(size)]
|
||||
block := UploadBlockNeed{
|
||||
Seq: seq,
|
||||
Size: size,
|
||||
Checksum: checksumList[i],
|
||||
}
|
||||
if err := d.uploadBlockV3(ctx, uploadID, block, payload); err != nil {
|
||||
return UploadMergeData{SuccessSeqList: success}, err
|
||||
}
|
||||
success = append(success, seq)
|
||||
offset += int(size)
|
||||
}
|
||||
return UploadMergeData{SuccessSeqList: success}, nil
|
||||
}
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return UploadMergeData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) uploadBlockV3(ctx context.Context, uploadID string, block UploadBlockNeed, data []byte) error {
|
||||
if uploadID == "" {
|
||||
return fmt.Errorf("[doubao_new] upload v3 block missing upload_id")
|
||||
}
|
||||
if block.Seq < 0 {
|
||||
return fmt.Errorf("[doubao_new] upload v3 block invalid seq")
|
||||
}
|
||||
if len(data) == 0 {
|
||||
return fmt.Errorf("[doubao_new] upload v3 block empty data")
|
||||
}
|
||||
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("x-block-seq", strconv.Itoa(block.Seq))
|
||||
req.SetHeader("x-block-checksum", block.Checksum)
|
||||
req.SetMultipartFormData(map[string]string{
|
||||
"upload_id": uploadID,
|
||||
"size": strconv.FormatInt(int64(len(data)), 10),
|
||||
})
|
||||
req.SetMultipartField("file", "blob", "application/octet-stream", bytes.NewReader(data))
|
||||
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("upload_id", uploadID)
|
||||
values.Set("seq", strconv.Itoa(block.Seq))
|
||||
values.Set("size", strconv.FormatInt(int64(len(data)), 10))
|
||||
values.Set("checksum", block.Checksum)
|
||||
values.Set("mount_point", "explorer")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
urlStr := DownloadBaseURL + "/space/api/box/stream/upload/v3/block/?" + values.Encode()
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, urlStr); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
res, err := req.Execute(http.MethodPost, urlStr)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body := res.Body()
|
||||
if err := decodeBaseResp(body, res); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) finishUpload(ctx context.Context, uploadID string, numBlocks int, mountPoint string) (UploadFinishData, error) {
|
||||
if uploadID == "" {
|
||||
return UploadFinishData{}, fmt.Errorf("[doubao_new] finish upload missing upload_id")
|
||||
}
|
||||
if numBlocks <= 0 {
|
||||
return UploadFinishData{}, fmt.Errorf("[doubao_new] finish upload invalid num_blocks")
|
||||
}
|
||||
if mountPoint == "" {
|
||||
mountPoint = "explorer"
|
||||
}
|
||||
var resp UploadFinishResp
|
||||
_, err := d.request(ctx, "/space/api/box/upload/finish/", http.MethodPost, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
values.Set("shouldBypassScsDialog", "true")
|
||||
values.Set("doubao_storage", "imagex_other")
|
||||
values.Set("doubao_app_id", d.AppID)
|
||||
req.SetQueryParamsFromValues(values)
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetHeader("x-command", "space.api.box.upload.finish")
|
||||
req.SetHeader("rpc-persist-doubao-pan", "true")
|
||||
req.SetHeader("cache-control", "no-cache")
|
||||
req.SetHeader("pragma", "no-cache")
|
||||
req.SetHeader("biz-scene", "file_upload")
|
||||
req.SetHeader("biz-ua-type", "Web")
|
||||
req.SetBody(base.Json{
|
||||
"upload_id": uploadID,
|
||||
"num_blocks": numBlocks,
|
||||
"mount_point": mountPoint,
|
||||
"push_open_history_record": 1,
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return UploadFinishData{}, err
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
@@ -1,713 +0,0 @@
|
||||
package doubao_new
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"hash/adler32"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cookie"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
const (
|
||||
BaseURL = "https://my.feishu.cn"
|
||||
DownloadBaseURL = "https://internal-api-drive-stream.feishu.cn"
|
||||
DoubaoURL = "https://www.doubao.com"
|
||||
)
|
||||
|
||||
var defaultObjTypes = []string{"124", "0", "12", "30", "123", "22"}
|
||||
|
||||
func (d *DoubaoNew) request(ctx context.Context, path string, method string, callback base.ReqCallback, resp interface{}) ([]byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, method, BaseURL+path); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if callback != nil {
|
||||
callback(req)
|
||||
}
|
||||
|
||||
res, err := req.Execute(method, BaseURL+path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if res != nil {
|
||||
if v := res.Header().Get("X-Tt-Logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
} else if v := res.Header().Get("x-tt-logid"); v != "" {
|
||||
d.TtLogid = v
|
||||
}
|
||||
}
|
||||
|
||||
body := res.Body()
|
||||
var common BaseResp
|
||||
if err = json.Unmarshal(body, &common); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return body, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if common.Code != 0 {
|
||||
errMsg := common.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = common.Message
|
||||
}
|
||||
return body, fmt.Errorf("[doubao_new] API error (code: %d): %s", common.Code, errMsg)
|
||||
}
|
||||
if resp != nil {
|
||||
if err = json.Unmarshal(body, resp); err != nil {
|
||||
return body, err
|
||||
}
|
||||
}
|
||||
|
||||
return body, nil
|
||||
}
|
||||
|
||||
func adler32String(data []byte) string {
|
||||
sum := adler32.Checksum(data)
|
||||
return strconv.FormatUint(uint64(sum), 10)
|
||||
}
|
||||
|
||||
func buildCommaHeader(items []string) string {
|
||||
return strings.Join(items, ",")
|
||||
}
|
||||
|
||||
func joinIntComma(items []int) string {
|
||||
if len(items) == 0 {
|
||||
return ""
|
||||
}
|
||||
var sb strings.Builder
|
||||
for i, v := range items {
|
||||
if i > 0 {
|
||||
sb.WriteByte(',')
|
||||
}
|
||||
sb.WriteString(strconv.Itoa(v))
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
func previewList(items []string, n int) string {
|
||||
if n <= 0 || len(items) == 0 {
|
||||
return ""
|
||||
}
|
||||
if len(items) < n {
|
||||
n = len(items)
|
||||
}
|
||||
return strings.Join(items[:n], ",")
|
||||
}
|
||||
|
||||
func parseSize(size string) int64 {
|
||||
if size == "" {
|
||||
return 0
|
||||
}
|
||||
val, err := strconv.ParseInt(size, 10, 64)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) listChildren(ctx context.Context, parentToken string, lastLabel string, length int) (ListData, error) {
|
||||
var resp ListResp
|
||||
_, err := d.request(ctx, "/space/api/explorer/doubao/children/list/", http.MethodGet, func(req *resty.Request) {
|
||||
values := url.Values{}
|
||||
for _, t := range defaultObjTypes {
|
||||
values.Add("obj_type", t)
|
||||
}
|
||||
values.Set("length", strconv.Itoa(length))
|
||||
values.Set("rank", "0")
|
||||
values.Set("asc", "0")
|
||||
values.Set("min_length", "40")
|
||||
values.Set("thumbnail_width", "1028")
|
||||
values.Set("thumbnail_height", "1028")
|
||||
values.Set("thumbnail_policy", "4")
|
||||
if parentToken != "" {
|
||||
values.Set("token", parentToken)
|
||||
}
|
||||
if lastLabel != "" {
|
||||
values.Set("last_label", lastLabel)
|
||||
}
|
||||
req.SetQueryParamsFromValues(values)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return ListData{}, err
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) listAllChildren(ctx context.Context, parentToken string) ([]Node, error) {
|
||||
length := 50
|
||||
nodes := make([]Node, 0, length)
|
||||
lastLabel := ""
|
||||
for range 100 {
|
||||
data, err := d.listChildren(ctx, parentToken, lastLabel, length)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if len(data.NodeList) > 0 {
|
||||
for _, token := range data.NodeList {
|
||||
node, ok := data.Entities.Nodes[token]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
nodes = append(nodes, node)
|
||||
}
|
||||
} else {
|
||||
for _, node := range data.Entities.Nodes {
|
||||
nodes = append(nodes, node)
|
||||
}
|
||||
}
|
||||
|
||||
if !data.HasMore || data.LastLabel == "" || data.LastLabel == lastLabel {
|
||||
break
|
||||
}
|
||||
lastLabel = data.LastLabel
|
||||
}
|
||||
|
||||
if len(nodes) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
return nodes, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) getFileInfo(ctx context.Context, fileToken string) (FileInfo, error) {
|
||||
var resp FileInfoResp
|
||||
_, err := d.request(ctx, "/space/api/box/file/info/", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"caller": "explorer",
|
||||
"file_token": fileToken,
|
||||
"mount_point": "explorer",
|
||||
"option_params": []string{"preview_meta", "check_cipher"},
|
||||
})
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
return FileInfo{}, err
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) previewLink(ctx context.Context, obj *Object, args model.LinkArgs) (*model.Link, error) {
|
||||
auth := d.resolveAuthorization()
|
||||
dpop, err := d.resolveDpopForRequest(http.MethodGet, fmt.Sprintf("%s/space/api/box/stream/download/preview_sub/%s", BaseURL, obj.ObjToken))
|
||||
if auth == "" || dpop == "" {
|
||||
return nil, errors.New("missing authorization or dpop")
|
||||
}
|
||||
if obj.ObjToken == "" {
|
||||
return nil, errors.New("missing obj_token")
|
||||
}
|
||||
info, err := d.getFileInfo(ctx, obj.ObjToken)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
entry, ok := info.PreviewMeta.Data["22"]
|
||||
if !ok || entry.Status != 0 {
|
||||
return nil, errors.New("preview not available")
|
||||
}
|
||||
|
||||
subID := ""
|
||||
pageIndex := 0
|
||||
|
||||
if subID == "" {
|
||||
imgExt := ".webp"
|
||||
pageNums := 0
|
||||
if entry.Extra != "" {
|
||||
var extra PreviewImageExtra
|
||||
if err := json.Unmarshal([]byte(entry.Extra), &extra); err == nil {
|
||||
if extra.ImgExt != "" {
|
||||
imgExt = extra.ImgExt
|
||||
}
|
||||
pageNums = extra.PageNums
|
||||
}
|
||||
}
|
||||
if pageNums > 0 && pageIndex >= pageNums {
|
||||
pageIndex = pageNums - 1
|
||||
}
|
||||
subID = fmt.Sprintf("img_%d%s", pageIndex, imgExt)
|
||||
}
|
||||
|
||||
query := url.Values{}
|
||||
query.Set("preview_type", "22")
|
||||
query.Set("sub_id", subID)
|
||||
if info.Version != "" {
|
||||
query.Set("version", info.Version)
|
||||
}
|
||||
previewURL := fmt.Sprintf("%s/space/api/box/stream/download/preview_sub/%s?%s", BaseURL, obj.ObjToken, query.Encode())
|
||||
|
||||
headers := http.Header{
|
||||
"Referer": []string{DoubaoURL + "/"},
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
"Authorization": []string{auth},
|
||||
"Dpop": []string{dpop},
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
URL: previewURL,
|
||||
Header: headers,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) createShare(ctx context.Context, obj *Object) error {
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/suite/permission/public/update.v5/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"external_access_entity": 1,
|
||||
"link_share_entity": 4,
|
||||
"token": obj.ObjToken,
|
||||
"type": obj.ObjType,
|
||||
})
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/suite/permission/public/update.v5/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := decodeBaseResp(body, res); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) createFolder(ctx context.Context, parentToken, name string) (Node, error) {
|
||||
data := url.Values{}
|
||||
data.Set("name", name)
|
||||
data.Set("source", "0")
|
||||
if parentToken != "" {
|
||||
data.Set("parent_token", parentToken)
|
||||
}
|
||||
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/create/folder/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.SetBody(data.Encode())
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/create/folder/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return Node{}, err
|
||||
}
|
||||
if err := decodeBaseResp(body, res); err != nil {
|
||||
return Node{}, err
|
||||
}
|
||||
|
||||
var resp CreateFolderResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return Node{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
|
||||
var node Node
|
||||
if len(resp.Data.NodeList) > 0 {
|
||||
if n, ok := resp.Data.Entities.Nodes[resp.Data.NodeList[0]]; ok {
|
||||
node = n
|
||||
}
|
||||
}
|
||||
if node.Token == "" {
|
||||
for _, n := range resp.Data.Entities.Nodes {
|
||||
node = n
|
||||
break
|
||||
}
|
||||
}
|
||||
if node.Token == "" && node.ObjToken == "" && node.NodeToken == "" {
|
||||
return Node{}, fmt.Errorf("[doubao_new] create folder failed: empty response")
|
||||
}
|
||||
if node.NodeToken == "" {
|
||||
if node.Token != "" {
|
||||
node.NodeToken = node.Token
|
||||
} else if node.ObjToken != "" {
|
||||
node.NodeToken = node.ObjToken
|
||||
}
|
||||
}
|
||||
if node.ObjToken == "" && node.Token != "" {
|
||||
node.ObjToken = node.Token
|
||||
}
|
||||
return node, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) renameFolder(ctx context.Context, token, name string) error {
|
||||
if token == "" {
|
||||
return fmt.Errorf("[doubao_new] rename folder missing token")
|
||||
}
|
||||
data := url.Values{}
|
||||
data.Set("token", token)
|
||||
data.Set("name", name)
|
||||
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/rename/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.SetBody(data.Encode())
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/rename/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return decodeBaseResp(body, res)
|
||||
}
|
||||
|
||||
func isCsrfTokenError(body []byte, res *resty.Response) bool {
|
||||
if len(body) == 0 {
|
||||
return false
|
||||
}
|
||||
if strings.Contains(strings.ToLower(string(body)), "csrf token error") {
|
||||
return true
|
||||
}
|
||||
if res != nil && res.StatusCode() == http.StatusForbidden {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func doRequestWithCsrf(doRequest func(csrfToken string) (*resty.Response, []byte, error)) (*resty.Response, []byte, error) {
|
||||
res, body, err := doRequest("")
|
||||
if err != nil {
|
||||
return res, body, err
|
||||
}
|
||||
if isCsrfTokenError(body, res) {
|
||||
csrfToken := extractCsrfTokenFromResponse(res)
|
||||
if csrfToken != "" {
|
||||
return doRequest(csrfToken)
|
||||
}
|
||||
}
|
||||
return res, body, err
|
||||
}
|
||||
|
||||
func extractCsrfTokenFromResponse(res *resty.Response) string {
|
||||
if res == nil || res.Request == nil {
|
||||
return ""
|
||||
}
|
||||
if res.Request.RawRequest != nil {
|
||||
if csrf := cookie.GetStr(res.Request.RawRequest.Header.Get("Cookie"), "_csrf_token"); csrf != "" {
|
||||
return csrf
|
||||
}
|
||||
}
|
||||
if csrf := cookie.GetStr(res.Request.Header.Get("Cookie"), "_csrf_token"); csrf != "" {
|
||||
return csrf
|
||||
}
|
||||
for _, c := range res.Cookies() {
|
||||
if c.Name == "_csrf_token" {
|
||||
return c.Value
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func decodeBaseResp(body []byte, res *resty.Response) error {
|
||||
var common BaseResp
|
||||
if err := json.Unmarshal(body, &common); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
if common.Code != 0 {
|
||||
errMsg := common.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = common.Message
|
||||
}
|
||||
return fmt.Errorf("[doubao_new] API error (code: %d): %s", common.Code, errMsg)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) renameFile(ctx context.Context, fileToken, name string) error {
|
||||
if fileToken == "" {
|
||||
return fmt.Errorf("[doubao_new] rename file missing file token")
|
||||
}
|
||||
_, err := d.request(ctx, "/space/api/box/file/update_info/", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"file_token": fileToken,
|
||||
"name": name,
|
||||
})
|
||||
}, nil)
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) moveObj(ctx context.Context, srcToken, destToken string) error {
|
||||
if srcToken == "" {
|
||||
return fmt.Errorf("[doubao_new] move missing src token")
|
||||
}
|
||||
data := url.Values{}
|
||||
data.Set("src_token", srcToken)
|
||||
if destToken != "" {
|
||||
data.Set("dest_token", destToken)
|
||||
}
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v2/move/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.SetBody(data.Encode())
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v2/move/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return decodeBaseResp(body, res)
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) removeObj(ctx context.Context, tokens []string) error {
|
||||
if len(tokens) == 0 {
|
||||
return fmt.Errorf("[doubao_new] remove missing tokens")
|
||||
}
|
||||
doRequest := func(csrfToken string) (*resty.Response, []byte, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, BaseURL+"/space/api/explorer/v3/remove/"); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if csrfToken != "" {
|
||||
req.SetHeader("x-csrftoken", csrfToken)
|
||||
}
|
||||
req.SetHeader("Content-Type", "application/json")
|
||||
req.SetBody(base.Json{
|
||||
"tokens": tokens,
|
||||
"apply": 1,
|
||||
})
|
||||
res, err := req.Execute(http.MethodPost, BaseURL+"/space/api/explorer/v3/remove/")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return res, res.Body(), nil
|
||||
}
|
||||
|
||||
res, body, err := doRequestWithCsrf(doRequest)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp RemoveResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
if resp.Data.TaskID == "" {
|
||||
return nil
|
||||
}
|
||||
return d.waitTask(ctx, resp.Data.TaskID)
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) getUserStorage(ctx context.Context) (UserStorageData, error) {
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "*/*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
req.SetHeader("agw-js-conv", "str")
|
||||
req.SetHeader("content-type", "application/json")
|
||||
if err := d.applyAuthHeaders(req, http.MethodPost, DoubaoURL+"/alice/aispace/facade/get_user_storage"); err != nil {
|
||||
return UserStorageData{}, err
|
||||
}
|
||||
if d.Cookie != "" {
|
||||
req.SetHeader("cookie", d.Cookie)
|
||||
}
|
||||
req.SetBody(base.Json{})
|
||||
|
||||
res, err := req.Execute(http.MethodPost, DoubaoURL+"/alice/aispace/facade/get_user_storage")
|
||||
if err != nil {
|
||||
return UserStorageData{}, err
|
||||
}
|
||||
|
||||
body := res.Body()
|
||||
var resp UserStorageResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return UserStorageData{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return UserStorageData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) waitTask(ctx context.Context, taskID string) error {
|
||||
const (
|
||||
taskPollInterval = time.Second
|
||||
taskPollMaxAttempts = 120
|
||||
)
|
||||
var lastErr error
|
||||
for attempt := 0; attempt < taskPollMaxAttempts; attempt++ {
|
||||
if attempt > 0 {
|
||||
if err := waitWithContext(ctx, taskPollInterval); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
status, err := d.getTaskStatus(ctx, taskID)
|
||||
if err != nil {
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
if status.IsFail {
|
||||
return fmt.Errorf("[doubao_new] remove task failed: %s", taskID)
|
||||
}
|
||||
if status.IsFinish {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if lastErr != nil {
|
||||
return lastErr
|
||||
}
|
||||
return fmt.Errorf("[doubao_new] remove task timed out: %s", taskID)
|
||||
}
|
||||
|
||||
func (d *DoubaoNew) getTaskStatus(ctx context.Context, taskID string) (TaskStatusData, error) {
|
||||
if taskID == "" {
|
||||
return TaskStatusData{}, fmt.Errorf("[doubao_new] task status missing task_id")
|
||||
}
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
req.SetHeader("accept", "application/json, text/plain, */*")
|
||||
req.SetHeader("origin", DoubaoURL)
|
||||
req.SetHeader("referer", DoubaoURL+"/")
|
||||
if err := d.applyAuthHeaders(req, http.MethodGet, BaseURL+"/space/api/explorer/v2/task/"); err != nil {
|
||||
return TaskStatusData{}, err
|
||||
}
|
||||
req.SetQueryParam("task_id", taskID)
|
||||
res, err := req.Execute(http.MethodGet, BaseURL+"/space/api/explorer/v2/task/")
|
||||
if err != nil {
|
||||
return TaskStatusData{}, err
|
||||
}
|
||||
body := res.Body()
|
||||
var resp TaskStatusResp
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
msg := fmt.Sprintf("[doubao_new] decode response failed (status: %s, content-type: %s, body: %s): %v",
|
||||
res.Status(),
|
||||
res.Header().Get("Content-Type"),
|
||||
string(body),
|
||||
err,
|
||||
)
|
||||
return TaskStatusData{}, fmt.Errorf("%s", msg)
|
||||
}
|
||||
if resp.Code != 0 {
|
||||
errMsg := resp.Msg
|
||||
if errMsg == "" {
|
||||
errMsg = resp.Message
|
||||
}
|
||||
return TaskStatusData{}, fmt.Errorf("[doubao_new] API error (code: %d): %s", resp.Code, errMsg)
|
||||
}
|
||||
return resp.Data, nil
|
||||
}
|
||||
|
||||
func waitWithContext(ctx context.Context, d time.Duration) error {
|
||||
timer := time.NewTimer(d)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@@ -1,261 +0,0 @@
|
||||
package emby
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Emby struct {
|
||||
model.Storage
|
||||
Addition
|
||||
|
||||
client *http.Client
|
||||
token string
|
||||
userID string
|
||||
}
|
||||
|
||||
func (d *Emby) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Emby) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *Emby) Init(ctx context.Context) error {
|
||||
d.URL = strings.TrimRight(strings.TrimSpace(d.URL), "/")
|
||||
if d.URL == "" {
|
||||
return fmt.Errorf("url is required")
|
||||
}
|
||||
|
||||
if strings.TrimSpace(d.RootFolderID) == "" {
|
||||
d.RootFolderID = "1"
|
||||
}
|
||||
|
||||
d.client = base.HttpClient
|
||||
d.token = strings.TrimSpace(d.ApiKey)
|
||||
d.userID = strings.TrimSpace(d.UserID)
|
||||
|
||||
if d.token != "" {
|
||||
if d.userID == "" {
|
||||
return fmt.Errorf("user_id is required when api_key is set")
|
||||
}
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
if strings.TrimSpace(d.Username) == "" || strings.TrimSpace(d.Password) == "" {
|
||||
return fmt.Errorf("please provide api_key+user_id or username+password")
|
||||
}
|
||||
|
||||
if err := d.login(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
d.ApiKey = d.token
|
||||
d.UserID = d.userID
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Emby) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Emby) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
parentID := strings.TrimSpace(d.RootFolderID)
|
||||
if dir != nil && strings.TrimSpace(dir.GetID()) != "" {
|
||||
parentID = strings.TrimSpace(dir.GetID())
|
||||
}
|
||||
|
||||
items, err := d.getItems(ctx, parentID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
parentPath := "/"
|
||||
if dir != nil && strings.TrimSpace(dir.GetPath()) != "" {
|
||||
parentPath = dir.GetPath()
|
||||
}
|
||||
|
||||
objs := make([]model.Obj, 0, len(items.Items))
|
||||
for _, it := range items.Items {
|
||||
modified := time.Now()
|
||||
if it.DateCreated != "" {
|
||||
if t, parseErr := time.Parse(time.RFC3339Nano, it.DateCreated); parseErr == nil {
|
||||
modified = t
|
||||
}
|
||||
}
|
||||
|
||||
name := strings.TrimSpace(it.Name)
|
||||
id := strings.TrimSpace(it.ID)
|
||||
displayName := name
|
||||
if name != "" && id != "" {
|
||||
if it.IsFolder {
|
||||
displayName = fmt.Sprintf("%s (ID%s)", name, id)
|
||||
} else {
|
||||
ext := path.Ext(strings.TrimSpace(it.Path))
|
||||
if ext == "" {
|
||||
ext = path.Ext(name)
|
||||
}
|
||||
|
||||
base := strings.TrimSpace(strings.TrimSuffix(name, ext))
|
||||
episodeCode := ""
|
||||
if m := episodeCodeRegexp.FindString(base); m != "" {
|
||||
episodeCode = strings.ToUpper(m)
|
||||
} else if it.ParentIndex > 0 && it.IndexNumber > 0 {
|
||||
episodeCode = fmt.Sprintf("S%02dE%02d", it.ParentIndex, it.IndexNumber)
|
||||
}
|
||||
|
||||
title := strings.TrimSpace(base)
|
||||
if episodeCode != "" {
|
||||
title = strings.TrimSpace(episodeCodeRegexp.ReplaceAllString(title, ""))
|
||||
title = strings.TrimSpace(strings.Trim(title, "-_:[]() "))
|
||||
}
|
||||
|
||||
series := strings.TrimSpace(it.SeriesName)
|
||||
if series == "" && episodeCode != "" {
|
||||
if idx := strings.Index(title, " - "); idx > 0 {
|
||||
series = strings.TrimSpace(title[:idx])
|
||||
title = strings.TrimSpace(title[idx+3:])
|
||||
}
|
||||
}
|
||||
|
||||
core := title
|
||||
if series != "" {
|
||||
if title == "" || strings.EqualFold(series, title) {
|
||||
core = series
|
||||
} else {
|
||||
core = series + " " + title
|
||||
}
|
||||
}
|
||||
if core == "" {
|
||||
core = base
|
||||
}
|
||||
|
||||
if episodeCode != "" {
|
||||
core = fmt.Sprintf("%s - [%s]", core, episodeCode)
|
||||
}
|
||||
if ext == "" {
|
||||
displayName = fmt.Sprintf("%s (ID%s)", core, id)
|
||||
} else {
|
||||
displayName = fmt.Sprintf("%s (ID%s)%s", core, id, ext)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
obj := &model.Object{
|
||||
ID: id,
|
||||
Name: displayName,
|
||||
Path: path.Join(parentPath, displayName),
|
||||
Size: it.Size,
|
||||
Modified: modified,
|
||||
IsFolder: it.IsFolder,
|
||||
}
|
||||
if it.IsFolder {
|
||||
obj.Size = 0
|
||||
}
|
||||
objs = append(objs, obj)
|
||||
}
|
||||
return objs, nil
|
||||
}
|
||||
|
||||
func (d *Emby) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if file.IsDir() {
|
||||
return nil, errs.NotFile
|
||||
}
|
||||
fileID := strings.TrimSpace(file.GetID())
|
||||
if fileID == "" {
|
||||
return nil, fmt.Errorf("invalid file id")
|
||||
}
|
||||
|
||||
u, err := url.Parse(d.URL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
linkMethod := strings.ToLower(strings.TrimSpace(d.LinkMethod))
|
||||
useDownload := linkMethod == "download"
|
||||
|
||||
mediaSourceID := ""
|
||||
mediaContainer := ""
|
||||
if !useDownload {
|
||||
detailURL, parseErr := url.Parse(d.URL + "/Users/" + d.userID + "/Items/" + fileID)
|
||||
if parseErr == nil {
|
||||
q := detailURL.Query()
|
||||
q.Set("Fields", "MediaSources")
|
||||
q.Set("api_key", d.token)
|
||||
detailURL.RawQuery = q.Encode()
|
||||
|
||||
req, reqErr := http.NewRequestWithContext(ctx, http.MethodGet, detailURL.String(), nil)
|
||||
if reqErr == nil {
|
||||
resp, doErr := d.client.Do(req)
|
||||
if doErr == nil {
|
||||
func() {
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return
|
||||
}
|
||||
var detail itemDetailResp
|
||||
if decodeErr := json.NewDecoder(resp.Body).Decode(&detail); decodeErr != nil || len(detail.MediaSources) == 0 {
|
||||
return
|
||||
}
|
||||
for i := range detail.MediaSources {
|
||||
if strings.TrimSpace(detail.MediaSources[i].ID) != "" && detail.MediaSources[i].SupportsDirectStream {
|
||||
mediaSourceID = strings.TrimSpace(detail.MediaSources[i].ID)
|
||||
mediaContainer = strings.TrimSpace(detail.MediaSources[i].Container)
|
||||
return
|
||||
}
|
||||
}
|
||||
for i := range detail.MediaSources {
|
||||
if strings.TrimSpace(detail.MediaSources[i].ID) != "" {
|
||||
mediaSourceID = strings.TrimSpace(detail.MediaSources[i].ID)
|
||||
mediaContainer = strings.TrimSpace(detail.MediaSources[i].Container)
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if useDownload {
|
||||
u.Path = path.Join(u.Path, "/Items", fileID, "Download")
|
||||
} else {
|
||||
if mediaContainer != "" {
|
||||
u.Path = path.Join(u.Path, "/Videos", fileID, "stream."+mediaContainer)
|
||||
} else {
|
||||
u.Path = path.Join(u.Path, "/Videos", fileID, "stream")
|
||||
}
|
||||
}
|
||||
q := u.Query()
|
||||
q.Set("api_key", d.token)
|
||||
if mediaSourceID != "" {
|
||||
q.Set("MediaSourceId", mediaSourceID)
|
||||
}
|
||||
if !useDownload {
|
||||
q.Set("Static", "true")
|
||||
}
|
||||
u.RawQuery = q.Encode()
|
||||
|
||||
return &model.Link{
|
||||
URL: u.String(),
|
||||
Header: http.Header{
|
||||
"User-Agent": []string{base.UserAgent},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Emby)(nil)
|
||||
@@ -1,30 +0,0 @@
|
||||
package emby
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
URL string `json:"url" required:"true"`
|
||||
ApiKey string `json:"api_key"`
|
||||
UserID string `json:"user_id"`
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
LinkMethod string `json:"link_method" type:"select" options:"stream,download" default:"stream"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Emby",
|
||||
LocalSort: true,
|
||||
NoUpload: true,
|
||||
DefaultRoot: "1",
|
||||
CheckStatus: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Emby{}
|
||||
})
|
||||
}
|
||||
@@ -1,41 +0,0 @@
|
||||
package emby
|
||||
|
||||
type authReq struct {
|
||||
Username string `json:"Username"`
|
||||
Pw string `json:"Pw"`
|
||||
}
|
||||
|
||||
type authResp struct {
|
||||
AccessToken string `json:"AccessToken"`
|
||||
User struct {
|
||||
ID string `json:"Id"`
|
||||
} `json:"User"`
|
||||
}
|
||||
|
||||
type listResp struct {
|
||||
Items []embyItem `json:"Items"`
|
||||
TotalRecordCount int `json:"TotalRecordCount"`
|
||||
}
|
||||
|
||||
type embyItem struct {
|
||||
Name string `json:"Name"`
|
||||
ID string `json:"Id"`
|
||||
Type string `json:"Type"`
|
||||
Path string `json:"Path"`
|
||||
SeriesName string `json:"SeriesName"`
|
||||
IndexNumber int `json:"IndexNumber"`
|
||||
ParentIndex int `json:"ParentIndexNumber"`
|
||||
IsFolder bool `json:"IsFolder"`
|
||||
Size int64 `json:"Size"`
|
||||
DateCreated string `json:"DateCreated"`
|
||||
}
|
||||
|
||||
type itemDetailResp struct {
|
||||
MediaSources []embyMediaSource `json:"MediaSources"`
|
||||
}
|
||||
|
||||
type embyMediaSource struct {
|
||||
ID string `json:"Id"`
|
||||
Container string `json:"Container"`
|
||||
SupportsDirectStream bool `json:"SupportsDirectStream"`
|
||||
}
|
||||
@@ -1,91 +0,0 @@
|
||||
package emby
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var episodeCodeRegexp = regexp.MustCompile(`(?i)\bS\d{1,2}E\d{1,2}\b`)
|
||||
|
||||
func (d *Emby) login(ctx context.Context) error {
|
||||
payload, err := json.Marshal(authReq{
|
||||
Username: d.Username,
|
||||
Pw: d.Password,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
endpoint := d.URL + "/Users/AuthenticateByName"
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("X-Emby-Authorization", `MediaBrowser Client="OpenList", Device="OpenList", DeviceId="openlist-emby", Version="1.0.0"`)
|
||||
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return fmt.Errorf("emby auth failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(body)))
|
||||
}
|
||||
|
||||
var data authResp
|
||||
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(data.AccessToken) == "" || strings.TrimSpace(data.User.ID) == "" {
|
||||
return fmt.Errorf("emby auth response missing access token or user id")
|
||||
}
|
||||
|
||||
d.token = data.AccessToken
|
||||
d.userID = data.User.ID
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Emby) getItems(ctx context.Context, parentID string) (*listResp, error) {
|
||||
u, err := url.Parse(d.URL + "/Users/" + d.userID + "/Items")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
q := u.Query()
|
||||
q.Set("ParentId", parentID)
|
||||
q.Set("Recursive", "false")
|
||||
q.Set("Fields", "Path,Size,DateCreated,SeriesName,IndexNumber,ParentIndexNumber")
|
||||
q.Set("api_key", d.token)
|
||||
u.RawQuery = q.Encode()
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := d.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return nil, fmt.Errorf("emby list failed: status=%d body=%s", resp.StatusCode, strings.TrimSpace(string(body)))
|
||||
}
|
||||
|
||||
var data listResp
|
||||
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &data, nil
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package google_drive
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootID
|
||||
RefreshToken string `json:"refresh_token" required:"true"`
|
||||
OrderBy string `json:"order_by" type:"string" help:"such as: folder,name,modifiedTime"`
|
||||
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc"`
|
||||
UseOnlineAPI bool `json:"use_online_api" default:"true"`
|
||||
APIAddress string `json:"api_url_address" default:"https://api.oplist.org/googleui/renewapi"`
|
||||
ClientID string `json:"client_id"`
|
||||
ClientSecret string `json:"client_secret"`
|
||||
ChunkSize int64 `json:"chunk_size" type:"number" default:"5" help:"chunk size while uploading (unit: MB)"`
|
||||
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "GoogleDrive",
|
||||
OnlyProxy: true,
|
||||
DefaultRoot: "root",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &GoogleDrive{}
|
||||
})
|
||||
}
|
||||
@@ -1,111 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
sdkUser "github.com/halalcloud/golang-sdk-lite/halalcloud/services/user"
|
||||
)
|
||||
|
||||
var (
|
||||
slicePostErrorRetryInterval = time.Second * 120
|
||||
retryTimes = 5
|
||||
)
|
||||
|
||||
type halalCommon struct {
|
||||
// *AuthService // 登录信息
|
||||
UserInfo *sdkUser.User // 用户信息
|
||||
refreshTokenFunc func(token string) error
|
||||
// serv *AuthService
|
||||
configs sync.Map
|
||||
}
|
||||
|
||||
func (m *halalCommon) GetAccessToken() (string, error) {
|
||||
value, exists := m.configs.Load("access_token")
|
||||
if !exists {
|
||||
return "", nil // 如果不存在,返回空字符串
|
||||
}
|
||||
return value.(string), nil // 返回配置项的值
|
||||
}
|
||||
|
||||
// GetRefreshToken implements ConfigStore.
|
||||
func (m *halalCommon) GetRefreshToken() (string, error) {
|
||||
value, exists := m.configs.Load("refresh_token")
|
||||
if !exists {
|
||||
return "", nil // 如果不存在,返回空字符串
|
||||
}
|
||||
return value.(string), nil // 返回配置项的值
|
||||
}
|
||||
|
||||
// SetAccessToken implements ConfigStore.
|
||||
func (m *halalCommon) SetAccessToken(token string) error {
|
||||
m.configs.Store("access_token", token)
|
||||
return nil
|
||||
}
|
||||
|
||||
// SetRefreshToken implements ConfigStore.
|
||||
func (m *halalCommon) SetRefreshToken(token string) error {
|
||||
m.configs.Store("refresh_token", token)
|
||||
if m.refreshTokenFunc != nil {
|
||||
return m.refreshTokenFunc(token)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SetToken implements ConfigStore.
|
||||
func (m *halalCommon) SetToken(accessToken string, refreshToken string, expiresIn int64) error {
|
||||
m.configs.Store("access_token", accessToken)
|
||||
m.configs.Store("refresh_token", refreshToken)
|
||||
m.configs.Store("expires_in", expiresIn)
|
||||
if m.refreshTokenFunc != nil {
|
||||
return m.refreshTokenFunc(refreshToken)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ClearConfigs implements ConfigStore.
|
||||
func (m *halalCommon) ClearConfigs() error {
|
||||
m.configs = sync.Map{} // 清空map
|
||||
return nil
|
||||
}
|
||||
|
||||
// DeleteConfig implements ConfigStore.
|
||||
func (m *halalCommon) DeleteConfig(key string) error {
|
||||
_, exists := m.configs.Load(key)
|
||||
if !exists {
|
||||
return nil // 如果不存在,直接返回
|
||||
}
|
||||
m.configs.Delete(key) // 删除指定的配置项
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetConfig implements ConfigStore.
|
||||
func (m *halalCommon) GetConfig(key string) (string, error) {
|
||||
value, exists := m.configs.Load(key)
|
||||
if !exists {
|
||||
return "", nil // 如果不存在,返回空字符串
|
||||
}
|
||||
return value.(string), nil // 返回配置项的值
|
||||
}
|
||||
|
||||
// ListConfigs implements ConfigStore.
|
||||
func (m *halalCommon) ListConfigs() (map[string]string, error) {
|
||||
configs := make(map[string]string)
|
||||
m.configs.Range(func(key, value interface{}) bool {
|
||||
configs[key.(string)] = value.(string) // 将每个配置项添加到map中
|
||||
return true // 继续遍历
|
||||
})
|
||||
return configs, nil // 返回所有配置项
|
||||
}
|
||||
|
||||
// SetConfig implements ConfigStore.
|
||||
func (m *halalCommon) SetConfig(key string, value string) error {
|
||||
m.configs.Store(key, value) // 使用Store方法设置或更新配置项
|
||||
return nil // 成功设置配置项后返回nil
|
||||
}
|
||||
|
||||
func NewHalalCommon() *halalCommon {
|
||||
return &halalCommon{
|
||||
configs: sync.Map{},
|
||||
}
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
sdkClient "github.com/halalcloud/golang-sdk-lite/halalcloud/apiclient"
|
||||
sdkUser "github.com/halalcloud/golang-sdk-lite/halalcloud/services/user"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
)
|
||||
|
||||
type HalalCloudOpen struct {
|
||||
*halalCommon
|
||||
model.Storage
|
||||
Addition
|
||||
sdkClient *sdkClient.Client
|
||||
sdkUserFileService *sdkUserFile.UserFileService
|
||||
sdkUserService *sdkUser.UserService
|
||||
uploadThread int
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*HalalCloudOpen)(nil)
|
||||
@@ -1,127 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
sdkModel "github.com/halalcloud/golang-sdk-lite/halalcloud/model"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
)
|
||||
|
||||
func (d *HalalCloudOpen) getFiles(ctx context.Context, dir model.Obj) ([]model.Obj, error) {
|
||||
|
||||
files := make([]model.Obj, 0)
|
||||
limit := int64(100)
|
||||
token := ""
|
||||
|
||||
for {
|
||||
result, err := d.sdkUserFileService.List(ctx, &sdkUserFile.FileListRequest{
|
||||
Parent: &sdkUserFile.File{Path: dir.GetPath()},
|
||||
ListInfo: &sdkModel.ScanListRequest{
|
||||
Limit: limit,
|
||||
Token: token,
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for i := 0; len(result.Files) > i; i++ {
|
||||
files = append(files, NewObjFile(result.Files[i]))
|
||||
}
|
||||
|
||||
if result.ListInfo == nil || result.ListInfo.Token == "" {
|
||||
break
|
||||
}
|
||||
token = result.ListInfo.Token
|
||||
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) makeDir(ctx context.Context, dir model.Obj, name string) (model.Obj, error) {
|
||||
_, err := d.sdkUserFileService.Create(ctx, &sdkUserFile.File{
|
||||
Path: dir.GetPath(),
|
||||
Name: name,
|
||||
})
|
||||
return nil, err
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) move(ctx context.Context, obj model.Obj, dir model.Obj) (model.Obj, error) {
|
||||
oldDir := obj.GetPath()
|
||||
newDir := dir.GetPath()
|
||||
_, err := d.sdkUserFileService.Move(ctx, &sdkUserFile.BatchOperationRequest{
|
||||
Source: []*sdkUserFile.File{
|
||||
{
|
||||
Path: oldDir,
|
||||
},
|
||||
},
|
||||
Dest: &sdkUserFile.File{
|
||||
Path: newDir,
|
||||
},
|
||||
})
|
||||
return nil, err
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) rename(ctx context.Context, obj model.Obj, name string) (model.Obj, error) {
|
||||
|
||||
_, err := d.sdkUserFileService.Rename(ctx, &sdkUserFile.File{
|
||||
Path: obj.GetPath(),
|
||||
Name: name,
|
||||
})
|
||||
return nil, err
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) copy(ctx context.Context, obj model.Obj, dir model.Obj) (model.Obj, error) {
|
||||
id := obj.GetID()
|
||||
sourcePath := obj.GetPath()
|
||||
if len(id) > 0 {
|
||||
sourcePath = ""
|
||||
}
|
||||
|
||||
destID := dir.GetID()
|
||||
destPath := dir.GetPath()
|
||||
if len(destID) > 0 {
|
||||
destPath = ""
|
||||
}
|
||||
dest := &sdkUserFile.File{
|
||||
Path: destPath,
|
||||
Identity: destID,
|
||||
}
|
||||
_, err := d.sdkUserFileService.Copy(ctx, &sdkUserFile.BatchOperationRequest{
|
||||
Source: []*sdkUserFile.File{
|
||||
{
|
||||
Path: sourcePath,
|
||||
Identity: id,
|
||||
},
|
||||
},
|
||||
Dest: dest,
|
||||
})
|
||||
return nil, err
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) remove(ctx context.Context, obj model.Obj) error {
|
||||
id := obj.GetID()
|
||||
_, err := d.sdkUserFileService.Delete(ctx, &sdkUserFile.BatchOperationRequest{
|
||||
Source: []*sdkUserFile.File{
|
||||
{
|
||||
Identity: id,
|
||||
Path: obj.GetPath(),
|
||||
},
|
||||
},
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) details(ctx context.Context) (*model.StorageDetails, error) {
|
||||
ret, err := d.sdkUserService.GetStatisticsAndQuota(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: ret.DiskStatisticsQuota.BytesQuota,
|
||||
UsedSpace: ret.DiskStatisticsQuota.BytesUsed,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
@@ -1,108 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"io"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
"github.com/rclone/rclone/lib/readers"
|
||||
)
|
||||
|
||||
func (d *HalalCloudOpen) getLink(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
if args.Redirect {
|
||||
// return nil, model.ErrUnsupported
|
||||
fid := file.GetID()
|
||||
fpath := file.GetPath()
|
||||
if fid != "" {
|
||||
fpath = ""
|
||||
}
|
||||
fi, err := d.sdkUserFileService.GetDirectDownloadAddress(ctx, &sdkUserFile.DirectDownloadRequest{
|
||||
Identity: fid,
|
||||
Path: fpath,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
expireAt := fi.ExpireAt
|
||||
duration := time.Until(time.UnixMilli(expireAt))
|
||||
return &model.Link{
|
||||
URL: fi.DownloadAddress,
|
||||
Expiration: &duration,
|
||||
}, nil
|
||||
}
|
||||
result, err := d.sdkUserFileService.ParseFileSlice(ctx, &sdkUserFile.File{
|
||||
Identity: file.GetID(),
|
||||
Path: file.GetPath(),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
fileAddrs := []*sdkUserFile.SliceDownloadInfo{}
|
||||
var addressDuration int64
|
||||
|
||||
nodesNumber := len(result.RawNodes)
|
||||
nodesIndex := nodesNumber - 1
|
||||
startIndex, endIndex := 0, nodesIndex
|
||||
for nodesIndex >= 0 {
|
||||
if nodesIndex >= 200 {
|
||||
endIndex = 200
|
||||
} else {
|
||||
endIndex = nodesNumber
|
||||
}
|
||||
for ; endIndex <= nodesNumber; endIndex += 200 {
|
||||
if endIndex == 0 {
|
||||
endIndex = 1
|
||||
}
|
||||
sliceAddress, err := d.sdkUserFileService.GetSliceDownloadAddress(ctx, &sdkUserFile.SliceDownloadAddressRequest{
|
||||
Identity: result.RawNodes[startIndex:endIndex],
|
||||
Version: 1,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
addressDuration, _ = strconv.ParseInt(sliceAddress.ExpireAt, 10, 64)
|
||||
fileAddrs = append(fileAddrs, sliceAddress.Addresses...)
|
||||
startIndex = endIndex
|
||||
nodesIndex -= 200
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
size, _ := strconv.ParseInt(result.FileSize, 10, 64)
|
||||
chunks := getChunkSizes(result.Sizes)
|
||||
resultRangeReader := func(ctx context.Context, httpRange http_range.Range) (io.ReadCloser, error) {
|
||||
length := httpRange.Length
|
||||
if httpRange.Length < 0 || httpRange.Start+httpRange.Length >= size {
|
||||
length = size - httpRange.Start
|
||||
}
|
||||
oo := &openObject{
|
||||
ctx: ctx,
|
||||
d: fileAddrs,
|
||||
chunk: []byte{},
|
||||
chunks: chunks,
|
||||
skip: httpRange.Start,
|
||||
sha: result.Sha1,
|
||||
shaTemp: sha1.New(),
|
||||
}
|
||||
|
||||
return readers.NewLimitedReadCloser(oo, length), nil
|
||||
}
|
||||
|
||||
var duration time.Duration
|
||||
if addressDuration != 0 {
|
||||
duration = time.Until(time.UnixMilli(addressDuration))
|
||||
} else {
|
||||
duration = time.Until(time.Now().Add(time.Hour))
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
RangeReader: stream.RateLimitRangeReaderFunc(resultRangeReader),
|
||||
Expiration: &duration,
|
||||
}, nil
|
||||
}
|
||||
@@ -1,50 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/halalcloud/golang-sdk-lite/halalcloud/apiclient"
|
||||
sdkUser "github.com/halalcloud/golang-sdk-lite/halalcloud/services/user"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
)
|
||||
|
||||
func (d *HalalCloudOpen) Init(ctx context.Context) error {
|
||||
if d.uploadThread < 1 || d.uploadThread > 32 {
|
||||
d.uploadThread, d.UploadThread = 3, 3
|
||||
}
|
||||
if d.halalCommon == nil {
|
||||
d.halalCommon = &halalCommon{
|
||||
UserInfo: &sdkUser.User{},
|
||||
refreshTokenFunc: func(token string) error {
|
||||
d.Addition.RefreshToken = token
|
||||
op.MustSaveDriverStorage(d)
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
if d.Addition.RefreshToken != "" {
|
||||
d.halalCommon.SetRefreshToken(d.Addition.RefreshToken)
|
||||
}
|
||||
timeout := d.Addition.TimeOut
|
||||
if timeout <= 0 {
|
||||
timeout = 60
|
||||
}
|
||||
host := d.Addition.Host
|
||||
if host == "" {
|
||||
host = "openapi.2dland.cn"
|
||||
}
|
||||
|
||||
client := apiclient.NewClient(nil, host, d.Addition.ClientID, d.Addition.ClientSecret, d.halalCommon, apiclient.WithTimeout(time.Second*time.Duration(timeout)))
|
||||
d.sdkClient = client
|
||||
d.sdkUserFileService = sdkUserFile.NewUserFileService(client)
|
||||
d.sdkUserService = sdkUser.NewUserService(client)
|
||||
userInfo, err := d.sdkUserService.Get(ctx, &sdkUser.User{})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.halalCommon.UserInfo = userInfo
|
||||
// 能够获取到用户信息,已经检查了 RefreshToken 的有效性,无需再次检查
|
||||
return nil
|
||||
}
|
||||
@@ -1,48 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
func (d *HalalCloudOpen) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
return d.getFiles(ctx, dir)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
return d.getLink(ctx, file, args)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
return d.makeDir(ctx, parentDir, dirName)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
return d.move(ctx, srcObj, dstDir)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
|
||||
return d.rename(ctx, srcObj, newName)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
return d.copy(ctx, srcObj, dstDir)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Remove(ctx context.Context, obj model.Obj) error {
|
||||
return d.remove(ctx, obj)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) Put(ctx context.Context, dstDir model.Obj, stream model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
return d.put(ctx, dstDir, stream, up)
|
||||
}
|
||||
|
||||
func (d *HalalCloudOpen) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
return d.details(ctx)
|
||||
}
|
||||
@@ -1,258 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
"github.com/ipfs/go-cid"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
func (d *HalalCloudOpen) put(ctx context.Context, dstDir model.Obj, fileStream model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
|
||||
newPath := path.Join(dstDir.GetPath(), fileStream.GetName())
|
||||
|
||||
uploadTask, err := d.sdkUserFileService.CreateUploadTask(ctx, &sdkUserFile.File{
|
||||
Path: newPath,
|
||||
Size: fileStream.GetSize(),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if uploadTask.Created {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
slicesList := make([]string, 0)
|
||||
codec := uint64(0x55)
|
||||
if uploadTask.BlockCodec > 0 {
|
||||
codec = uint64(uploadTask.BlockCodec)
|
||||
}
|
||||
blockHashType := uploadTask.BlockHashType
|
||||
mhType := uint64(0x12)
|
||||
if blockHashType > 0 {
|
||||
mhType = uint64(blockHashType)
|
||||
}
|
||||
prefix := cid.Prefix{
|
||||
Codec: codec,
|
||||
MhLength: -1,
|
||||
MhType: mhType,
|
||||
Version: 1,
|
||||
}
|
||||
blockSize := uploadTask.BlockSize
|
||||
//
|
||||
// Not sure whether FileStream supports concurrent read and write operations, so currently using single-threaded upload to ensure safety.
|
||||
// read file
|
||||
bufferSize := int(blockSize)
|
||||
buffer := make([]byte, bufferSize)
|
||||
offset := 0
|
||||
teeReader := io.TeeReader(fileStream, driver.NewProgress(fileStream.GetSize(), up))
|
||||
for {
|
||||
n, err := teeReader.Read(buffer[offset:]) // 这里 len(buf[offset:]) <= 4MB
|
||||
if n > 0 {
|
||||
offset += n
|
||||
if offset == int(blockSize) {
|
||||
uploadCid, err := postFileSlice(ctx, buffer, uploadTask.Task, uploadTask.UploadAddress, prefix, retryTimes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
slicesList = append(slicesList, uploadCid.String())
|
||||
offset = 0
|
||||
}
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
if err == io.EOF {
|
||||
if offset > 0 {
|
||||
uploadCid, err := postFileSlice(ctx, buffer[:offset], uploadTask.Task, uploadTask.UploadAddress, prefix, retryTimes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
slicesList = append(slicesList, uploadCid.String())
|
||||
}
|
||||
break
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
newFile, err := makeFile(ctx, slicesList, uploadTask.Task, uploadTask.UploadAddress, retryTimes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return NewObjFile(newFile), nil
|
||||
|
||||
}
|
||||
|
||||
func makeFile(ctx context.Context, fileSlice []string, taskID string, uploadAddress string, retry int) (*sdkUserFile.File, error) {
|
||||
var lastError error = nil
|
||||
for range retry {
|
||||
newFile, err := doMakeFile(fileSlice, taskID, uploadAddress)
|
||||
if err == nil {
|
||||
return newFile, nil
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return nil, err
|
||||
}
|
||||
log.Errorf("make file slice failed, retrying... error: %s", err.Error())
|
||||
if strings.Contains(err.Error(), "not found") {
|
||||
return nil, err
|
||||
}
|
||||
lastError = err
|
||||
time.Sleep(slicePostErrorRetryInterval)
|
||||
}
|
||||
return nil, fmt.Errorf("mk file slice failed after %d times, error: %s", retry, lastError.Error())
|
||||
}
|
||||
|
||||
func doMakeFile(fileSlice []string, taskID string, uploadAddress string) (*sdkUserFile.File, error) {
|
||||
accessUrl := uploadAddress + "/" + taskID
|
||||
getTimeOut := time.Minute * 2
|
||||
u, err := url.Parse(accessUrl)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n, _ := json.Marshal(fileSlice)
|
||||
httpRequest := http.Request{
|
||||
Method: http.MethodPost,
|
||||
URL: u,
|
||||
Header: map[string][]string{
|
||||
"Accept": {"application/json"},
|
||||
"Content-Type": {"application/json"},
|
||||
//"Content-Length": {strconv.Itoa(len(n))},
|
||||
},
|
||||
Body: io.NopCloser(bytes.NewReader(n)),
|
||||
}
|
||||
httpClient := http.Client{
|
||||
Timeout: getTimeOut,
|
||||
}
|
||||
httpResponse, err := httpClient.Do(&httpRequest)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer httpResponse.Body.Close()
|
||||
if httpResponse.StatusCode != http.StatusOK && httpResponse.StatusCode != http.StatusCreated {
|
||||
b, _ := io.ReadAll(httpResponse.Body)
|
||||
message := string(b)
|
||||
log.Errorf("make file failed, status code: %d, message: %s", httpResponse.StatusCode, message)
|
||||
|
||||
return nil, fmt.Errorf("mk file slice failed, status code: %d, message: %s", httpResponse.StatusCode, message)
|
||||
}
|
||||
b, _ := io.ReadAll(httpResponse.Body)
|
||||
var result *UploadedFile
|
||||
err = json.Unmarshal(b, &result)
|
||||
if err != nil {
|
||||
log.Errorf("make file failed from response, status code: %d, message: %s", httpResponse.StatusCode, string(b))
|
||||
return nil, err
|
||||
}
|
||||
return &sdkUserFile.File{
|
||||
Identity: result.Identity,
|
||||
Path: result.Path,
|
||||
Size: result.Size,
|
||||
ContentIdentity: result.ContentIdentity,
|
||||
}, nil
|
||||
}
|
||||
func postFileSlice(ctx context.Context, fileSlice []byte, taskID string, uploadAddress string, preix cid.Prefix, retry int) (cid.Cid, error) {
|
||||
var lastError error = nil
|
||||
for range retry {
|
||||
newCid, err := doPostFileSlice(fileSlice, taskID, uploadAddress, preix)
|
||||
if err == nil {
|
||||
return newCid, nil
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return cid.Undef, err
|
||||
}
|
||||
time.Sleep(slicePostErrorRetryInterval)
|
||||
lastError = err
|
||||
}
|
||||
return cid.Undef, fmt.Errorf("upload file slice failed after %d times, error: %s", retry, lastError.Error())
|
||||
}
|
||||
func doPostFileSlice(fileSlice []byte, taskID string, uploadAddress string, preix cid.Prefix) (cid.Cid, error) {
|
||||
// 1. sum file slice
|
||||
newCid, err := preix.Sum(fileSlice)
|
||||
if err != nil {
|
||||
return cid.Undef, err
|
||||
}
|
||||
// 2. post file slice
|
||||
sliceCidString := newCid.String()
|
||||
// /{taskID}/{sliceID}
|
||||
accessUrl := uploadAddress + "/" + taskID + "/" + sliceCidString
|
||||
getTimeOut := time.Second * 30
|
||||
// get {accessUrl} in {getTimeOut}
|
||||
u, err := url.Parse(accessUrl)
|
||||
if err != nil {
|
||||
return cid.Undef, err
|
||||
}
|
||||
// header: accept: application/json
|
||||
// header: content-type: application/octet-stream
|
||||
// header: content-length: {fileSlice.length}
|
||||
// header: x-content-cid: {sliceCidString}
|
||||
// header: x-task-id: {taskID}
|
||||
httpRequest := http.Request{
|
||||
Method: http.MethodGet,
|
||||
URL: u,
|
||||
Header: map[string][]string{
|
||||
"Accept": {"application/json"},
|
||||
},
|
||||
}
|
||||
httpClient := http.Client{
|
||||
Timeout: getTimeOut,
|
||||
}
|
||||
httpResponse, err := httpClient.Do(&httpRequest)
|
||||
if err != nil {
|
||||
log.Errorf("access %s failed, method: %s", accessUrl, http.MethodGet)
|
||||
return cid.Undef, err
|
||||
}
|
||||
if httpResponse.StatusCode != http.StatusOK {
|
||||
log.Errorf("access %s failed, method: %s, status code: %d", accessUrl, http.MethodGet, httpResponse.StatusCode)
|
||||
return cid.Undef, fmt.Errorf("upload file slice failed, status code: %d", httpResponse.StatusCode)
|
||||
}
|
||||
var result bool
|
||||
b, err := io.ReadAll(httpResponse.Body)
|
||||
if err != nil {
|
||||
return cid.Undef, err
|
||||
}
|
||||
err = json.Unmarshal(b, &result)
|
||||
if err != nil {
|
||||
return cid.Undef, err
|
||||
}
|
||||
if result {
|
||||
return newCid, nil
|
||||
}
|
||||
|
||||
httpRequest = http.Request{
|
||||
Method: http.MethodPost,
|
||||
URL: u,
|
||||
Header: map[string][]string{
|
||||
"Accept": {"application/json"},
|
||||
"Content-Type": {"application/octet-stream"},
|
||||
// "Content-Length": {strconv.Itoa(len(fileSlice))},
|
||||
},
|
||||
Body: io.NopCloser(bytes.NewReader(fileSlice)),
|
||||
}
|
||||
httpResponse, err = httpClient.Do(&httpRequest)
|
||||
if err != nil {
|
||||
return cid.Undef, err
|
||||
}
|
||||
defer httpResponse.Body.Close()
|
||||
if httpResponse.StatusCode != http.StatusOK && httpResponse.StatusCode != http.StatusCreated {
|
||||
b, _ := io.ReadAll(httpResponse.Body)
|
||||
message := string(b)
|
||||
log.Errorf("upload file slice failed, status code: %d, message: %s", httpResponse.StatusCode, message)
|
||||
return cid.Undef, fmt.Errorf("upload file slice failed, status code: %d, message: %s", httpResponse.StatusCode, message)
|
||||
}
|
||||
//
|
||||
|
||||
return newCid, nil
|
||||
}
|
||||
@@ -1,40 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
// Usually one of two
|
||||
driver.RootPath
|
||||
// define other
|
||||
RefreshToken string `json:"refresh_token" required:"false" help:"If using a personal API approach, the RefreshToken is not required."`
|
||||
UploadThread int `json:"upload_thread" type:"number" default:"3" help:"1 <= thread <= 32"`
|
||||
|
||||
ClientID string `json:"client_id" required:"true" default:""`
|
||||
ClientSecret string `json:"client_secret" required:"true" default:""`
|
||||
Host string `json:"host" required:"false" default:"openapi.2dland.cn"`
|
||||
TimeOut int `json:"timeout" type:"number" default:"60" help:"timeout in seconds"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "HalalCloudOpen",
|
||||
OnlyProxy: false,
|
||||
DefaultRoot: "/",
|
||||
NoLinkURL: false,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &HalalCloudOpen{}
|
||||
})
|
||||
}
|
||||
|
||||
type UploadedFile struct {
|
||||
Identity string `json:"identity"`
|
||||
UserIdentity string `json:"user_identity"`
|
||||
Path string `json:"path"`
|
||||
Size int64 `json:"size"`
|
||||
ContentIdentity string `json:"content_identity"`
|
||||
}
|
||||
@@ -1,60 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
)
|
||||
|
||||
type ObjFile struct {
|
||||
sdkFile *sdkUserFile.File
|
||||
fileSize int64
|
||||
modTime time.Time
|
||||
createTime time.Time
|
||||
}
|
||||
|
||||
func NewObjFile(f *sdkUserFile.File) model.Obj {
|
||||
ofile := &ObjFile{sdkFile: f}
|
||||
ofile.fileSize = f.Size
|
||||
modTimeTs := f.UpdateTs
|
||||
ofile.modTime = time.UnixMilli(modTimeTs)
|
||||
createTimeTs := f.CreateTs
|
||||
ofile.createTime = time.UnixMilli(createTimeTs)
|
||||
return ofile
|
||||
}
|
||||
|
||||
func (f *ObjFile) GetSize() int64 {
|
||||
return f.fileSize
|
||||
}
|
||||
|
||||
func (f *ObjFile) GetName() string {
|
||||
return f.sdkFile.Name
|
||||
}
|
||||
|
||||
func (f *ObjFile) ModTime() time.Time {
|
||||
return f.modTime
|
||||
}
|
||||
|
||||
func (f *ObjFile) IsDir() bool {
|
||||
return f.sdkFile.Dir
|
||||
}
|
||||
|
||||
func (f *ObjFile) GetHash() utils.HashInfo {
|
||||
return utils.HashInfo{
|
||||
// TODO: support more hash types
|
||||
}
|
||||
}
|
||||
|
||||
func (f *ObjFile) GetID() string {
|
||||
return f.sdkFile.Identity
|
||||
}
|
||||
|
||||
func (f *ObjFile) GetPath() string {
|
||||
return f.sdkFile.Path
|
||||
}
|
||||
|
||||
func (f *ObjFile) CreateTime() time.Time {
|
||||
return f.createTime
|
||||
}
|
||||
@@ -1,185 +0,0 @@
|
||||
package halalcloudopen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/md5"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"hash"
|
||||
"io"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
sdkUserFile "github.com/halalcloud/golang-sdk-lite/halalcloud/services/userfile"
|
||||
"github.com/ipfs/go-cid"
|
||||
)
|
||||
|
||||
// get the next chunk
|
||||
func (oo *openObject) getChunk(_ context.Context) (err error) {
|
||||
if oo.id >= len(oo.chunks) {
|
||||
return io.EOF
|
||||
}
|
||||
var chunk []byte
|
||||
err = utils.Retry(3, time.Second, func() (err error) {
|
||||
chunk, err = getRawFiles(oo.d[oo.id])
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
oo.id++
|
||||
oo.chunk = chunk
|
||||
return nil
|
||||
}
|
||||
|
||||
// Read reads up to len(p) bytes into p.
|
||||
func (oo *openObject) Read(p []byte) (n int, err error) {
|
||||
oo.mu.Lock()
|
||||
defer oo.mu.Unlock()
|
||||
if oo.closed {
|
||||
return 0, fmt.Errorf("read on closed file")
|
||||
}
|
||||
// Skip data at the start if requested
|
||||
for oo.skip > 0 {
|
||||
//size := 1024 * 1024
|
||||
_, size, err := oo.ChunkLocation(oo.id)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if oo.skip < int64(size) {
|
||||
break
|
||||
}
|
||||
oo.id++
|
||||
oo.skip -= int64(size)
|
||||
}
|
||||
if len(oo.chunk) == 0 {
|
||||
err = oo.getChunk(oo.ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if oo.skip > 0 {
|
||||
oo.chunk = (oo.chunk)[oo.skip:]
|
||||
oo.skip = 0
|
||||
}
|
||||
}
|
||||
n = copy(p, oo.chunk)
|
||||
oo.shaTemp.Write(p[:n])
|
||||
oo.chunk = (oo.chunk)[n:]
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// Close closed the file - MAC errors are reported here
|
||||
func (oo *openObject) Close() (err error) {
|
||||
oo.mu.Lock()
|
||||
defer oo.mu.Unlock()
|
||||
if oo.closed {
|
||||
return nil
|
||||
}
|
||||
// 校验Sha1
|
||||
if string(oo.shaTemp.Sum(nil)) != oo.sha {
|
||||
return fmt.Errorf("failed to finish download: SHA mismatch")
|
||||
}
|
||||
|
||||
oo.closed = true
|
||||
return nil
|
||||
}
|
||||
|
||||
func GetMD5Hash(text string) string {
|
||||
tHash := md5.Sum([]byte(text))
|
||||
return hex.EncodeToString(tHash[:])
|
||||
}
|
||||
|
||||
type chunkSize struct {
|
||||
position int64
|
||||
size int
|
||||
}
|
||||
|
||||
type openObject struct {
|
||||
ctx context.Context
|
||||
mu sync.Mutex
|
||||
d []*sdkUserFile.SliceDownloadInfo
|
||||
id int
|
||||
skip int64
|
||||
chunk []byte
|
||||
chunks []chunkSize
|
||||
closed bool
|
||||
sha string
|
||||
shaTemp hash.Hash
|
||||
}
|
||||
|
||||
func getChunkSizes(sliceSize []*sdkUserFile.SliceSize) (chunks []chunkSize) {
|
||||
chunks = make([]chunkSize, 0)
|
||||
for _, s := range sliceSize {
|
||||
// 对最后一个做特殊处理
|
||||
endIndex := s.EndIndex
|
||||
startIndex := s.StartIndex
|
||||
if endIndex == 0 {
|
||||
endIndex = startIndex
|
||||
}
|
||||
for j := startIndex; j <= endIndex; j++ {
|
||||
size := s.Size
|
||||
chunks = append(chunks, chunkSize{position: j, size: int(size)})
|
||||
}
|
||||
}
|
||||
return chunks
|
||||
}
|
||||
|
||||
func (oo *openObject) ChunkLocation(id int) (position int64, size int, err error) {
|
||||
if id < 0 || id >= len(oo.chunks) {
|
||||
return 0, 0, errors.New("invalid arguments")
|
||||
}
|
||||
|
||||
return (oo.chunks)[id].position, (oo.chunks)[id].size, nil
|
||||
}
|
||||
|
||||
func getRawFiles(addr *sdkUserFile.SliceDownloadInfo) ([]byte, error) {
|
||||
|
||||
if addr == nil {
|
||||
return nil, errors.New("addr is nil")
|
||||
}
|
||||
|
||||
client := http.Client{
|
||||
Timeout: time.Duration(60 * time.Second), // Set timeout to 60 seconds
|
||||
}
|
||||
resp, err := client.Get(addr.DownloadAddress)
|
||||
if err != nil {
|
||||
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("bad status: %s, body: %s", resp.Status, body)
|
||||
}
|
||||
|
||||
if addr.Encrypt > 0 {
|
||||
cd := uint8(addr.Encrypt)
|
||||
for idx := 0; idx < len(body); idx++ {
|
||||
body[idx] = body[idx] ^ cd
|
||||
}
|
||||
}
|
||||
storeType := addr.StoreType
|
||||
if storeType != 10 {
|
||||
|
||||
sourceCid, err := cid.Decode(addr.Identity)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
checkCid, err := sourceCid.Prefix().Sum(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !checkCid.Equals(sourceCid) {
|
||||
return nil, fmt.Errorf("bad cid: %s, body: %s", checkCid.String(), body)
|
||||
}
|
||||
}
|
||||
|
||||
return body, nil
|
||||
|
||||
}
|
||||
@@ -1,16 +0,0 @@
|
||||
//go:build !windows && !plan9 && !netbsd && !aix && !illumos && !solaris && !js
|
||||
|
||||
package local
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func copyNamedPipe(dstPath string, mode os.FileMode, dirMode os.FileMode) error {
|
||||
if err := os.MkdirAll(filepath.Dir(dstPath), dirMode); err != nil {
|
||||
return err
|
||||
}
|
||||
return syscall.Mkfifo(dstPath, uint32(mode))
|
||||
}
|
||||
@@ -1,9 +0,0 @@
|
||||
//go:build windows || plan9 || netbsd || aix || illumos || solaris || js
|
||||
|
||||
package local
|
||||
|
||||
import "os"
|
||||
|
||||
func copyNamedPipe(_ string, _, _ os.FileMode) error {
|
||||
return nil
|
||||
}
|
||||
@@ -1,44 +0,0 @@
|
||||
package local
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
)
|
||||
|
||||
func TestRemoveDeletesThumbCache(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
cacheDir := filepath.Join(root, "thumbs")
|
||||
if err := os.Mkdir(cacheDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
filePath := filepath.Join(root, "photo.jpg")
|
||||
if err := os.WriteFile(filePath, []byte("image"), 0o666); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
driver := &Local{
|
||||
Addition: Addition{
|
||||
ThumbCacheFolder: cacheDir,
|
||||
},
|
||||
}
|
||||
thumbPath := driver.thumbCachePath(filePath)
|
||||
if err := os.WriteFile(thumbPath, []byte("thumb"), 0o666); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
err := driver.Remove(context.Background(), &model.Object{
|
||||
Path: filePath,
|
||||
Name: filepath.Base(filePath),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if _, err = os.Stat(thumbPath); !os.IsNotExist(err) {
|
||||
t.Fatalf("expected thumb cache to be removed, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
//go:build !windows
|
||||
|
||||
package local
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
func isHidden(f fs.FileInfo, _ string) bool {
|
||||
return strings.HasPrefix(f.Name(), ".")
|
||||
}
|
||||
|
||||
func getDiskUsage(path string) (model.DiskUsage, error) {
|
||||
var stat syscall.Statfs_t
|
||||
err := syscall.Statfs(path, &stat)
|
||||
if err != nil {
|
||||
return model.DiskUsage{}, err
|
||||
}
|
||||
total := int64(stat.Blocks) * int64(stat.Bsize)
|
||||
free := int64(stat.Bfree) * int64(stat.Bsize)
|
||||
return model.DiskUsage{
|
||||
TotalSpace: total,
|
||||
UsedSpace: total - free,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func isCrossDeviceError(err error) bool {
|
||||
return errors.Is(err, unix.EXDEV)
|
||||
}
|
||||
@@ -1,55 +0,0 @@
|
||||
//go:build windows
|
||||
|
||||
package local
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"golang.org/x/sys/windows"
|
||||
)
|
||||
|
||||
func isHidden(f fs.FileInfo, fullPath string) bool {
|
||||
filePath := filepath.Join(fullPath, f.Name())
|
||||
namePtr, err := syscall.UTF16PtrFromString(filePath)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
attrs, err := syscall.GetFileAttributes(namePtr)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return attrs&syscall.FILE_ATTRIBUTE_HIDDEN != 0
|
||||
}
|
||||
|
||||
func getDiskUsage(path string) (model.DiskUsage, error) {
|
||||
abs, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return model.DiskUsage{}, err
|
||||
}
|
||||
root := filepath.VolumeName(abs)
|
||||
if len(root) != 2 || root[1] != ':' {
|
||||
return model.DiskUsage{}, errors.New("cannot get disk label")
|
||||
}
|
||||
var freeBytes, totalBytes, totalFreeBytes uint64
|
||||
err = windows.GetDiskFreeSpaceEx(
|
||||
windows.StringToUTF16Ptr(root),
|
||||
&freeBytes,
|
||||
&totalBytes,
|
||||
&totalFreeBytes,
|
||||
)
|
||||
if err != nil {
|
||||
return model.DiskUsage{}, err
|
||||
}
|
||||
return model.DiskUsage{
|
||||
TotalSpace: int64(totalBytes),
|
||||
UsedSpace: int64(totalBytes - freeBytes),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func isCrossDeviceError(err error) bool {
|
||||
return errors.Is(err, windows.ERROR_NOT_SAME_DEVICE)
|
||||
}
|
||||
@@ -1,435 +0,0 @@
|
||||
package mediafire
|
||||
|
||||
/*
|
||||
Package mediafire
|
||||
Author: Da3zKi7<da3zki7@duck.com>
|
||||
Date: 2025-09-11
|
||||
|
||||
D@' 3z K!7 - The King Of Cracking
|
||||
|
||||
Modifications by ILoveScratch2<ilovescratch@foxmail.com>
|
||||
Date: 2025-09-21
|
||||
|
||||
Date: 2025-09-26
|
||||
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
|
||||
*/
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cron"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"golang.org/x/time/rate"
|
||||
)
|
||||
|
||||
type Mediafire struct {
|
||||
model.Storage
|
||||
Addition
|
||||
|
||||
cron *cron.Cron
|
||||
|
||||
actionToken string
|
||||
limiter *rate.Limiter
|
||||
|
||||
appBase string
|
||||
apiBase string
|
||||
hostBase string
|
||||
maxRetries int
|
||||
|
||||
secChUa string
|
||||
secChUaPlatform string
|
||||
userAgent string
|
||||
}
|
||||
|
||||
func (d *Mediafire) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *Mediafire) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
// Init initializes the MediaFire driver with session token and cookie validation
|
||||
func (d *Mediafire) Init(ctx context.Context) error {
|
||||
if d.Cookie == "" {
|
||||
return fmt.Errorf("Init :: [MediaFire] {critical} missing Cookie")
|
||||
}
|
||||
|
||||
// If SessionToken is empty, try to get it from cookie
|
||||
if d.SessionToken == "" {
|
||||
if _, err := d.getSessionToken(ctx); err != nil {
|
||||
return fmt.Errorf("Init :: [MediaFire] {critical} failed to get session token from cookie: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Setup rate limiter if rate limit is configured
|
||||
if d.LimitRate > 0 {
|
||||
d.limiter = rate.NewLimiter(rate.Limit(d.LimitRate), 1)
|
||||
}
|
||||
|
||||
// Validate and refresh session token if needed
|
||||
if _, err := d.getSessionToken(ctx); err != nil {
|
||||
d.renewToken(ctx)
|
||||
|
||||
// Avoids 10 mins token expiry (6- 9)
|
||||
num := rand.Intn(4) + 6
|
||||
|
||||
d.cron = cron.NewCron(time.Minute * time.Duration(num))
|
||||
d.cron.Do(func() {
|
||||
// Crazy, but working way to refresh session token
|
||||
d.renewToken(ctx)
|
||||
})
|
||||
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Drop cleans up driver resources
|
||||
func (d *Mediafire) Drop(ctx context.Context) error {
|
||||
// Clear cached resources
|
||||
d.actionToken = ""
|
||||
if d.cron != nil {
|
||||
d.cron.Stop()
|
||||
d.cron = nil
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// List retrieves files and folders from the specified directory
|
||||
func (d *Mediafire) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
files, err := d.getFiles(ctx, dir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return utils.SliceConvert(files, func(src File) (model.Obj, error) {
|
||||
return d.fileToObj(src), nil
|
||||
})
|
||||
}
|
||||
|
||||
// Link generates a direct download link for the specified file
|
||||
func (d *Mediafire) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
downloadUrl, err := d.getDirectDownloadLink(ctx, file.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
res, err := base.NoRedirectClient.R().SetDoNotParseResponse(true).SetContext(ctx).Head(downloadUrl)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() {
|
||||
_ = res.RawBody().Close()
|
||||
}()
|
||||
|
||||
if res.StatusCode() == 302 {
|
||||
downloadUrl = res.Header().Get("location")
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
URL: downloadUrl,
|
||||
Header: http.Header{
|
||||
"Origin": []string{d.appBase},
|
||||
"Referer": []string{d.appBase + "/"},
|
||||
"sec-ch-ua": []string{d.secChUa},
|
||||
"sec-ch-ua-platform": []string{d.secChUaPlatform},
|
||||
"User-Agent": []string{d.userAgent},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
// MakeDir creates a new folder in the specified parent directory
|
||||
func (d *Mediafire) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) (model.Obj, error) {
|
||||
data := map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"parent_key": parentDir.GetID(),
|
||||
"foldername": dirName,
|
||||
}
|
||||
|
||||
var resp MediafireFolderCreateResponse
|
||||
_, err := d.postForm(ctx, "/folder/create.php", data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
created, _ := time.Parse("2006-01-02T15:04:05Z", resp.Response.CreatedUTC)
|
||||
|
||||
return &model.Object{
|
||||
ID: resp.Response.FolderKey,
|
||||
Name: resp.Response.Name,
|
||||
Size: 0,
|
||||
Modified: created,
|
||||
Ctime: created,
|
||||
IsFolder: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Move relocates a file or folder to a different parent directory
|
||||
func (d *Mediafire) Move(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
var data map[string]string
|
||||
var endpoint string
|
||||
|
||||
if srcObj.IsDir() {
|
||||
|
||||
endpoint = "/folder/move.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"folder_key_src": srcObj.GetID(),
|
||||
"folder_key_dst": dstDir.GetID(),
|
||||
}
|
||||
} else {
|
||||
|
||||
endpoint = "/file/move.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"quick_key": srcObj.GetID(),
|
||||
"folder_key": dstDir.GetID(),
|
||||
}
|
||||
}
|
||||
|
||||
var resp MediafireMoveResponse
|
||||
_, err := d.postForm(ctx, endpoint, data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return srcObj, nil
|
||||
}
|
||||
|
||||
// Rename changes the name of a file or folder
|
||||
func (d *Mediafire) Rename(ctx context.Context, srcObj model.Obj, newName string) (model.Obj, error) {
|
||||
var data map[string]string
|
||||
var endpoint string
|
||||
|
||||
if srcObj.IsDir() {
|
||||
|
||||
endpoint = "/folder/update.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"folder_key": srcObj.GetID(),
|
||||
"foldername": newName,
|
||||
}
|
||||
} else {
|
||||
|
||||
endpoint = "/file/update.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"quick_key": srcObj.GetID(),
|
||||
"filename": newName,
|
||||
}
|
||||
}
|
||||
|
||||
var resp MediafireRenameResponse
|
||||
_, err := d.postForm(ctx, endpoint, data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
ID: srcObj.GetID(),
|
||||
Name: newName,
|
||||
Size: srcObj.GetSize(),
|
||||
Modified: srcObj.ModTime(),
|
||||
Ctime: srcObj.CreateTime(),
|
||||
IsFolder: srcObj.IsDir(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Copy creates a duplicate of a file or folder in the specified destination directory
|
||||
func (d *Mediafire) Copy(ctx context.Context, srcObj, dstDir model.Obj) (model.Obj, error) {
|
||||
var data map[string]string
|
||||
var endpoint string
|
||||
|
||||
if srcObj.IsDir() {
|
||||
|
||||
endpoint = "/folder/copy.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"folder_key_src": srcObj.GetID(),
|
||||
"folder_key_dst": dstDir.GetID(),
|
||||
}
|
||||
} else {
|
||||
|
||||
endpoint = "/file/copy.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"quick_key": srcObj.GetID(),
|
||||
"folder_key": dstDir.GetID(),
|
||||
}
|
||||
}
|
||||
|
||||
var resp MediafireCopyResponse
|
||||
_, err := d.postForm(ctx, endpoint, data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var newID string
|
||||
if srcObj.IsDir() {
|
||||
if len(resp.Response.NewFolderKeys) > 0 {
|
||||
newID = resp.Response.NewFolderKeys[0]
|
||||
}
|
||||
} else {
|
||||
if len(resp.Response.NewQuickKeys) > 0 {
|
||||
newID = resp.Response.NewQuickKeys[0]
|
||||
}
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
ID: newID,
|
||||
Name: srcObj.GetName(),
|
||||
Size: srcObj.GetSize(),
|
||||
Modified: srcObj.ModTime(),
|
||||
Ctime: srcObj.CreateTime(),
|
||||
IsFolder: srcObj.IsDir(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Remove deletes a file or folder permanently
|
||||
func (d *Mediafire) Remove(ctx context.Context, obj model.Obj) error {
|
||||
var data map[string]string
|
||||
var endpoint string
|
||||
|
||||
if obj.IsDir() {
|
||||
|
||||
endpoint = "/folder/delete.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"folder_key": obj.GetID(),
|
||||
}
|
||||
} else {
|
||||
|
||||
endpoint = "/file/delete.php"
|
||||
data = map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"quick_key": obj.GetID(),
|
||||
}
|
||||
}
|
||||
|
||||
var resp MediafireRemoveResponse
|
||||
_, err := d.postForm(ctx, endpoint, data, &resp)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return checkAPIResult(resp.Response.Result)
|
||||
}
|
||||
|
||||
// Put uploads a file to the specified directory with support for resumable upload and quick upload
|
||||
func (d *Mediafire) Put(ctx context.Context, dstDir model.Obj, file model.FileStreamer, up driver.UpdateProgress) (model.Obj, error) {
|
||||
fileHash := file.GetHash().GetHash(utils.SHA256)
|
||||
var err error
|
||||
|
||||
// Try to use existing hash first, cache only if necessary
|
||||
if len(fileHash) != utils.SHA256.Width {
|
||||
_, fileHash, err = stream.CacheFullAndHash(file, &up, utils.SHA256)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
checkResp, err := d.uploadCheck(ctx, file.GetName(), file.GetSize(), fileHash, dstDir.GetID())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if checkResp.Response.HashExists == "yes" && checkResp.Response.InAccount == "yes" {
|
||||
up(100.0)
|
||||
existingFile, err := d.getExistingFileInfo(ctx, fileHash, file.GetName(), dstDir.GetID())
|
||||
if err == nil && existingFile != nil {
|
||||
// File exists, return existing file info
|
||||
return &model.Object{
|
||||
ID: existingFile.GetID(),
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
}, nil
|
||||
}
|
||||
// If getExistingFileInfo fails, log and continue with normal upload
|
||||
// This ensures upload doesn't fail due to search issues
|
||||
}
|
||||
|
||||
var pollKey string
|
||||
|
||||
if checkResp.Response.ResumableUpload.AllUnitsReady != "yes" {
|
||||
pollKey, err = d.uploadUnits(ctx, file, checkResp, file.GetName(), fileHash, dstDir.GetID(), up)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
pollKey = checkResp.Response.ResumableUpload.UploadKey
|
||||
}
|
||||
defer up(100.0)
|
||||
|
||||
pollResp, err := d.pollUpload(ctx, pollKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &model.Object{
|
||||
ID: pollResp.Response.Doupload.QuickKey,
|
||||
Name: file.GetName(),
|
||||
Size: file.GetSize(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
data := map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
}
|
||||
var resp MediafireUserInfoResponse
|
||||
_, err := d.postForm(ctx, "/user/get_info.php", data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
used, err := strconv.ParseInt(resp.Response.UserInfo.UsedStorageSize, 10, 64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
total, err := strconv.ParseInt(resp.Response.UserInfo.StorageLimit, 10, 64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: total,
|
||||
UsedSpace: used,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Mediafire)(nil)
|
||||
@@ -1,60 +0,0 @@
|
||||
package mediafire
|
||||
|
||||
/*
|
||||
Package mediafire
|
||||
Author: Da3zKi7<da3zki7@duck.com>
|
||||
Date: 2025-09-11
|
||||
|
||||
D@' 3z K!7 - The King Of Cracking
|
||||
|
||||
Modifications by ILoveScratch2<ilovescratch@foxmail.com>
|
||||
Date: 2025-09-21
|
||||
|
||||
Date: 2025-09-26
|
||||
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
|
||||
*/
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
//driver.RootID
|
||||
|
||||
SessionToken string `json:"session_token" required:"false" type:"string" help:"Optional for MediaFire API, can be auto-acquired from cookie"`
|
||||
Cookie string `json:"cookie" required:"true" type:"string" help:"Required for MediaFire API authentication"`
|
||||
|
||||
OrderBy string `json:"order_by" type:"select" options:"name,time,size" default:"name"`
|
||||
OrderDirection string `json:"order_direction" type:"select" options:"asc,desc" default:"asc"`
|
||||
ChunkSize int64 `json:"chunk_size" type:"number" default:"100"`
|
||||
UploadThreads int `json:"upload_threads" type:"number" default:"3" help:"concurrent upload threads"`
|
||||
LimitRate float64 `json:"limit_rate" type:"float" default:"2" help:"limit all api request rate ([limit]r/1s)"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "MediaFire",
|
||||
LocalSort: false,
|
||||
OnlyProxy: false,
|
||||
NoCache: false,
|
||||
NoUpload: false,
|
||||
NeedMs: false,
|
||||
DefaultRoot: "/",
|
||||
CheckStatus: false,
|
||||
Alert: "",
|
||||
NoOverwriteUpload: true,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Mediafire{
|
||||
appBase: "https://app.mediafire.com",
|
||||
apiBase: "https://www.mediafire.com/api/1.5",
|
||||
hostBase: "https://www.mediafire.com",
|
||||
maxRetries: 3,
|
||||
userAgent: base.UserAgent,
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,246 +0,0 @@
|
||||
package mediafire
|
||||
|
||||
/*
|
||||
Package mediafire
|
||||
Author: Da3zKi7<da3zki7@duck.com>
|
||||
Date: 2025-09-11
|
||||
|
||||
D@' 3z K!7 - The King Of Cracking
|
||||
*/
|
||||
|
||||
type MediafireRenewTokenResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
SessionToken string `json:"session_token"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
FolderContent struct {
|
||||
ChunkSize string `json:"chunk_size"`
|
||||
ContentType string `json:"content_type"`
|
||||
ChunkNumber string `json:"chunk_number"`
|
||||
FolderKey string `json:"folderkey"`
|
||||
Folders []MediafireFolder `json:"folders,omitempty"`
|
||||
Files []MediafireFile `json:"files,omitempty"`
|
||||
MoreChunks string `json:"more_chunks"`
|
||||
} `json:"folder_content"`
|
||||
Result string `json:"result"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireFolder struct {
|
||||
FolderKey string `json:"folderkey"`
|
||||
Name string `json:"name"`
|
||||
Created string `json:"created"`
|
||||
CreatedUTC string `json:"created_utc"`
|
||||
}
|
||||
|
||||
type MediafireFile struct {
|
||||
QuickKey string `json:"quickkey"`
|
||||
Filename string `json:"filename"`
|
||||
Size string `json:"size"`
|
||||
Created string `json:"created"`
|
||||
CreatedUTC string `json:"created_utc"`
|
||||
MimeType string `json:"mimetype"`
|
||||
}
|
||||
|
||||
type File struct {
|
||||
ID string
|
||||
Name string
|
||||
Size int64
|
||||
CreatedUTC string
|
||||
IsFolder bool
|
||||
}
|
||||
|
||||
type FolderContentResponse struct {
|
||||
Folders []MediafireFolder
|
||||
Files []MediafireFile
|
||||
MoreChunks bool
|
||||
}
|
||||
|
||||
type MediafireLinksResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Links []struct {
|
||||
QuickKey string `json:"quickkey"`
|
||||
View string `json:"view"`
|
||||
NormalDownload string `json:"normal_download"`
|
||||
OneTime struct {
|
||||
Download string `json:"download"`
|
||||
View string `json:"view"`
|
||||
} `json:"one_time"`
|
||||
} `json:"links"`
|
||||
OneTimeKeyRequestCount string `json:"one_time_key_request_count"`
|
||||
OneTimeKeyRequestMaxCount string `json:"one_time_key_request_max_count"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireDirectDownloadResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Links []struct {
|
||||
QuickKey string `json:"quickkey"`
|
||||
DirectDownload string `json:"direct_download"`
|
||||
} `json:"links"`
|
||||
DirectDownloadFreeBandwidth string `json:"direct_download_free_bandwidth"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireFolderCreateResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
FolderKey string `json:"folder_key"`
|
||||
UploadKey string `json:"upload_key"`
|
||||
ParentFolderKey string `json:"parent_folderkey"`
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
Created string `json:"created"`
|
||||
CreatedUTC string `json:"created_utc"`
|
||||
Privacy string `json:"privacy"`
|
||||
FileCount string `json:"file_count"`
|
||||
FolderCount string `json:"folder_count"`
|
||||
Revision string `json:"revision"`
|
||||
DropboxEnabled string `json:"dropbox_enabled"`
|
||||
Flag string `json:"flag"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
NewDeviceRevision int `json:"new_device_revision"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireMoveResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Asynchronous string `json:"asynchronous,omitempty"`
|
||||
NewNames []string `json:"new_names"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
NewDeviceRevision int `json:"new_device_revision"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireRenameResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Asynchronous string `json:"asynchronous,omitempty"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
NewDeviceRevision int `json:"new_device_revision"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireCopyResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Asynchronous string `json:"asynchronous,omitempty"`
|
||||
NewQuickKeys []string `json:"new_quickkeys,omitempty"`
|
||||
NewFolderKeys []string `json:"new_folderkeys,omitempty"`
|
||||
SkippedCount string `json:"skipped_count,omitempty"`
|
||||
OtherCount string `json:"other_count,omitempty"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
NewDeviceRevision int `json:"new_device_revision"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireRemoveResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Asynchronous string `json:"asynchronous,omitempty"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
NewDeviceRevision int `json:"new_device_revision"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireCheckResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
HashExists string `json:"hash_exists"`
|
||||
InAccount string `json:"in_account"`
|
||||
InFolder string `json:"in_folder"`
|
||||
FileExists string `json:"file_exists"`
|
||||
ResumableUpload struct {
|
||||
AllUnitsReady string `json:"all_units_ready"`
|
||||
NumberOfUnits string `json:"number_of_units"`
|
||||
UnitSize string `json:"unit_size"`
|
||||
Bitmap struct {
|
||||
Count string `json:"count"`
|
||||
Words []string `json:"words"`
|
||||
} `json:"bitmap"`
|
||||
UploadKey string `json:"upload_key"`
|
||||
} `json:"resumable_upload"`
|
||||
AvailableSpace string `json:"available_space"`
|
||||
UsedStorageSize string `json:"used_storage_size"`
|
||||
StorageLimit string `json:"storage_limit"`
|
||||
StorageLimitExceeded string `json:"storage_limit_exceeded"`
|
||||
UploadURL struct {
|
||||
Simple string `json:"simple"`
|
||||
SimpleFallback string `json:"simple_fallback"`
|
||||
Resumable string `json:"resumable"`
|
||||
ResumableFallback string `json:"resumable_fallback"`
|
||||
} `json:"upload_url"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
type MediafireActionTokenResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
ActionToken string `json:"action_token"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafirePollResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
Doupload struct {
|
||||
Result string `json:"result"`
|
||||
Status string `json:"status"`
|
||||
Description string `json:"description"`
|
||||
QuickKey string `json:"quickkey"`
|
||||
Hash string `json:"hash"`
|
||||
Filename string `json:"filename"`
|
||||
Size string `json:"size"`
|
||||
Created string `json:"created"`
|
||||
CreatedUTC string `json:"created_utc"`
|
||||
Revision string `json:"revision"`
|
||||
} `json:"doupload"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireFileSearchResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
FileInfo []File `json:"file_info"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
type MediafireUserInfoResponse struct {
|
||||
Response struct {
|
||||
Action string `json:"action"`
|
||||
UserInfo struct {
|
||||
Email string `json:"string"`
|
||||
DisplayName string `json:"display_name"`
|
||||
UsedStorageSize string `json:"used_storage_size"`
|
||||
StorageLimit string `json:"storage_limit"`
|
||||
} `json:"user_info"`
|
||||
Result string `json:"result"`
|
||||
CurrentAPIVersion string `json:"current_api_version"`
|
||||
} `json:"response"`
|
||||
}
|
||||
@@ -1,734 +0,0 @@
|
||||
package mediafire
|
||||
|
||||
/*
|
||||
Package mediafire
|
||||
Author: Da3zKi7<da3zki7@duck.com>
|
||||
Date: 2025-09-11
|
||||
|
||||
D@' 3z K!7 - The King Of Cracking
|
||||
|
||||
Modifications by ILoveScratch2<ilovescratch@foxmail.com>
|
||||
Date: 2025-09-21
|
||||
|
||||
Date: 2025-09-26
|
||||
Final opts by @Suyunjing @j2rong4cn @KirCute @Da3zKi7
|
||||
*/
|
||||
|
||||
import (
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/errgroup"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/avast/retry-go"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
// checkAPIResult validates MediaFire API response result and returns error if not successful
|
||||
func checkAPIResult(result string) error {
|
||||
if result != "Success" {
|
||||
return fmt.Errorf("MediaFire API error: %s", result)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// getSessionToken retrieves and validates session token from MediaFire
|
||||
func (d *Mediafire) getSessionToken(ctx context.Context) (string, error) {
|
||||
if d.limiter != nil {
|
||||
if err := d.limiter.Wait(ctx); err != nil {
|
||||
return "", fmt.Errorf("rate limit wait failed: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
tokenURL := d.hostBase + "/application/get_session_token.php"
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, tokenURL, nil)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
req.Header.Set("Accept", "*/*")
|
||||
req.Header.Set("Accept-Encoding", "gzip")
|
||||
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
|
||||
req.Header.Set("Content-Length", "0")
|
||||
req.Header.Set("Cookie", d.Cookie)
|
||||
req.Header.Set("DNT", "1")
|
||||
req.Header.Set("Origin", d.hostBase)
|
||||
req.Header.Set("Priority", "u=1, i")
|
||||
req.Header.Set("Referer", (d.hostBase + "/"))
|
||||
req.Header.Set("Sec-Ch-Ua", d.secChUa)
|
||||
req.Header.Set("Sec-Ch-Ua-Mobile", "?0")
|
||||
req.Header.Set("Sec-Ch-Ua-Platform", d.secChUaPlatform)
|
||||
req.Header.Set("Sec-Fetch-Dest", "empty")
|
||||
req.Header.Set("Sec-Fetch-Mode", "cors")
|
||||
req.Header.Set("Sec-Fetch-Site", "same-site")
|
||||
req.Header.Set("User-Agent", d.userAgent)
|
||||
// req.Header.Set("Connection", "keep-alive")
|
||||
|
||||
resp, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var body []byte
|
||||
// Handle gzip decompression if needed
|
||||
if resp.Header.Get("Content-Encoding") == "gzip" {
|
||||
gzipReader, err := gzip.NewReader(resp.Body)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to create gzip reader: %w", err)
|
||||
}
|
||||
defer gzipReader.Close()
|
||||
body, _ = io.ReadAll(gzipReader)
|
||||
} else {
|
||||
body, err = io.ReadAll(resp.Body)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// fmt.Printf("getSessionToken :: Raw response: %s\n", string(body))
|
||||
// fmt.Printf("getSessionToken :: Parsed response: %+v\n", resp)
|
||||
|
||||
var tokenResp struct {
|
||||
Response struct {
|
||||
SessionToken string `json:"session_token"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
if resp.StatusCode == 200 {
|
||||
if err := json.Unmarshal(body, &tokenResp); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
if tokenResp.Response.SessionToken == "" {
|
||||
return "", fmt.Errorf("empty session token received")
|
||||
}
|
||||
|
||||
cookieMap := make(map[string]string)
|
||||
for _, cookie := range resp.Cookies() {
|
||||
cookieMap[cookie.Name] = cookie.Value
|
||||
}
|
||||
|
||||
if len(cookieMap) > 0 {
|
||||
|
||||
var cookies []string
|
||||
for name, value := range cookieMap {
|
||||
cookies = append(cookies, fmt.Sprintf("%s=%s", name, value))
|
||||
}
|
||||
d.Cookie = strings.Join(cookies, "; ")
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
// fmt.Printf("getSessionToken :: Captured cookies: %s\n", d.Cookie)
|
||||
}
|
||||
|
||||
} else {
|
||||
return "", fmt.Errorf("getSessionToken :: failed to get session token, status code: %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
d.SessionToken = tokenResp.Response.SessionToken
|
||||
|
||||
// fmt.Printf("Init :: Obtain Session Token %v", d.SessionToken)
|
||||
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
return d.SessionToken, nil
|
||||
}
|
||||
|
||||
// renewToken refreshes the current session token when expired
|
||||
func (d *Mediafire) renewToken(ctx context.Context) error {
|
||||
query := map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
}
|
||||
|
||||
var resp MediafireRenewTokenResponse
|
||||
_, err := d.postForm(ctx, "/user/renew_session_token.php", query, &resp)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to renew token: %w", err)
|
||||
}
|
||||
|
||||
// fmt.Printf("getInfo :: Raw response: %s\n", string(body))
|
||||
// fmt.Printf("getInfo :: Parsed response: %+v\n", resp)
|
||||
|
||||
if resp.Response.Result != "Success" {
|
||||
return fmt.Errorf("MediaFire token renewal failed: %s", resp.Response.Result)
|
||||
}
|
||||
|
||||
d.SessionToken = resp.Response.SessionToken
|
||||
|
||||
// fmt.Printf("Init :: Renew Session Token: %s", resp.Response.Result)
|
||||
|
||||
op.MustSaveDriverStorage(d)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) getFiles(ctx context.Context, folderKey string) ([]File, error) {
|
||||
// Pre-allocate slice with reasonable capacity to reduce memory allocations
|
||||
files := make([]File, 0, d.ChunkSize*2) // Estimate: ChunkSize for files + folders
|
||||
hasMore := true
|
||||
chunkNumber := 1
|
||||
|
||||
for hasMore {
|
||||
resp, err := d.getFolderContent(ctx, folderKey, chunkNumber)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Process folders and files in single loop to improve cache locality
|
||||
totalItems := len(resp.Folders) + len(resp.Files)
|
||||
if cap(files)-len(files) < totalItems {
|
||||
// Grow slice if needed
|
||||
newFiles := make([]File, len(files), len(files)+totalItems+int(d.ChunkSize))
|
||||
copy(newFiles, files)
|
||||
files = newFiles
|
||||
}
|
||||
|
||||
for _, folder := range resp.Folders {
|
||||
files = append(files, File{
|
||||
ID: folder.FolderKey,
|
||||
Name: folder.Name,
|
||||
Size: 0,
|
||||
CreatedUTC: folder.CreatedUTC,
|
||||
IsFolder: true,
|
||||
})
|
||||
}
|
||||
|
||||
for _, file := range resp.Files {
|
||||
size, _ := strconv.ParseInt(file.Size, 10, 64)
|
||||
files = append(files, File{
|
||||
ID: file.QuickKey,
|
||||
Name: file.Filename,
|
||||
Size: size,
|
||||
CreatedUTC: file.CreatedUTC,
|
||||
IsFolder: false,
|
||||
})
|
||||
}
|
||||
|
||||
hasMore = resp.MoreChunks
|
||||
chunkNumber++
|
||||
}
|
||||
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) getFolderContent(ctx context.Context, folderKey string, chunkNumber int) (*FolderContentResponse, error) {
|
||||
foldersResp, err := d.getFolderContentByType(ctx, folderKey, "folders", chunkNumber)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
filesResp, err := d.getFolderContentByType(ctx, folderKey, "files", chunkNumber)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &FolderContentResponse{
|
||||
Folders: foldersResp.Response.FolderContent.Folders,
|
||||
Files: filesResp.Response.FolderContent.Files,
|
||||
MoreChunks: foldersResp.Response.FolderContent.MoreChunks == "yes" || filesResp.Response.FolderContent.MoreChunks == "yes",
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) getFolderContentByType(ctx context.Context, folderKey, contentType string, chunkNumber int) (*MediafireResponse, error) {
|
||||
data := map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"folder_key": folderKey,
|
||||
"content_type": contentType,
|
||||
"chunk": strconv.Itoa(chunkNumber),
|
||||
"chunk_size": strconv.FormatInt(d.ChunkSize, 10),
|
||||
"details": "yes",
|
||||
"order_direction": d.OrderDirection,
|
||||
"order_by": d.OrderBy,
|
||||
"filter": "",
|
||||
}
|
||||
|
||||
var resp MediafireResponse
|
||||
_, err := d.postForm(ctx, "/folder/get_content.php", data, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
// fileToObj converts MediaFire file data to model.ObjThumb with thumbnail support
|
||||
func (d *Mediafire) fileToObj(f File) *model.ObjThumb {
|
||||
created, _ := time.Parse("2006-01-02T15:04:05Z", f.CreatedUTC)
|
||||
|
||||
var thumbnailURL string
|
||||
if !f.IsFolder && f.ID != "" {
|
||||
thumbnailURL = d.hostBase + "/convkey/acaa/" + f.ID + "3g.jpg"
|
||||
}
|
||||
|
||||
return &model.ObjThumb{
|
||||
Object: model.Object{
|
||||
ID: f.ID,
|
||||
// Path: "",
|
||||
Name: f.Name,
|
||||
Size: f.Size,
|
||||
Modified: created,
|
||||
Ctime: created,
|
||||
IsFolder: f.IsFolder,
|
||||
},
|
||||
Thumbnail: model.Thumbnail{
|
||||
Thumbnail: thumbnailURL,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Mediafire) setCommonHeaders(req *resty.Request) {
|
||||
req.SetHeaders(map[string]string{
|
||||
"Cookie": d.Cookie,
|
||||
"User-Agent": d.userAgent,
|
||||
"Origin": d.appBase,
|
||||
"Referer": d.appBase + "/",
|
||||
})
|
||||
}
|
||||
|
||||
// apiRequest performs HTTP request to MediaFire API with rate limiting and common headers
|
||||
func (d *Mediafire) apiRequest(ctx context.Context, method, endpoint string, queryParams, formData map[string]string, resp interface{}) ([]byte, error) {
|
||||
if d.limiter != nil {
|
||||
if err := d.limiter.Wait(ctx); err != nil {
|
||||
return nil, fmt.Errorf("rate limit wait failed: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
req := base.RestyClient.R()
|
||||
req.SetContext(ctx)
|
||||
d.setCommonHeaders(req)
|
||||
|
||||
// Set query parameters for GET requests
|
||||
if queryParams != nil {
|
||||
req.SetQueryParams(queryParams)
|
||||
}
|
||||
|
||||
// Set form data for POST requests
|
||||
if formData != nil {
|
||||
req.SetFormData(formData)
|
||||
req.SetHeader("Content-Type", "application/x-www-form-urlencoded")
|
||||
}
|
||||
|
||||
// Set response object if provided
|
||||
if resp != nil {
|
||||
req.SetResult(resp)
|
||||
}
|
||||
|
||||
var res *resty.Response
|
||||
var err error
|
||||
|
||||
// Execute request based on method
|
||||
switch method {
|
||||
case "GET":
|
||||
res, err = req.Get(d.apiBase + endpoint)
|
||||
case "POST":
|
||||
res, err = req.Post(d.apiBase + endpoint)
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported HTTP method: %s", method)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return res.Body(), nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) getForm(ctx context.Context, endpoint string, query map[string]string, resp interface{}) ([]byte, error) {
|
||||
return d.apiRequest(ctx, "GET", endpoint, query, nil, resp)
|
||||
}
|
||||
|
||||
func (d *Mediafire) postForm(ctx context.Context, endpoint string, data map[string]string, resp interface{}) ([]byte, error) {
|
||||
return d.apiRequest(ctx, "POST", endpoint, nil, data, resp)
|
||||
}
|
||||
|
||||
func (d *Mediafire) getDirectDownloadLink(ctx context.Context, fileID string) (string, error) {
|
||||
data := map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"quick_key": fileID,
|
||||
"link_type": "direct_download",
|
||||
"response_format": "json",
|
||||
}
|
||||
|
||||
var resp MediafireDirectDownloadResponse
|
||||
_, err := d.getForm(ctx, "/file/get_links.php", data, &resp)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
if len(resp.Response.Links) == 0 {
|
||||
return "", fmt.Errorf("no download links found")
|
||||
}
|
||||
|
||||
return resp.Response.Links[0].DirectDownload, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) uploadCheck(ctx context.Context, filename string, filesize int64, filehash, folderKey string) (*MediafireCheckResponse, error) {
|
||||
actionToken, err := d.getActionToken(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get action token: %w", err)
|
||||
}
|
||||
|
||||
query := map[string]string{
|
||||
"session_token": actionToken, /* d.SessionToken */
|
||||
"filename": filename,
|
||||
"size": strconv.FormatInt(filesize, 10),
|
||||
"hash": filehash,
|
||||
"folder_key": folderKey,
|
||||
"resumable": "yes",
|
||||
"response_format": "json",
|
||||
}
|
||||
|
||||
var resp MediafireCheckResponse
|
||||
_, err = d.postForm(ctx, "/upload/check.php", query, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// fmt.Printf("uploadCheck :: Raw response: %s\n", string(body))
|
||||
// fmt.Printf("uploadCheck :: Parsed response: %+v\n", resp)
|
||||
|
||||
// fmt.Printf("uploadCheck :: ResumableUpload section: %+v\n", resp.Response.ResumableUpload)
|
||||
// fmt.Printf("uploadCheck :: Upload key specifically: '%s'\n", resp.Response.ResumableUpload.UploadKey)
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) uploadUnits(ctx context.Context, file model.FileStreamer, checkResp *MediafireCheckResponse, filename, fileHash, folderKey string, up driver.UpdateProgress) (string, error) {
|
||||
unitSize, _ := strconv.ParseInt(checkResp.Response.ResumableUpload.UnitSize, 10, 64)
|
||||
numUnits, _ := strconv.Atoi(checkResp.Response.ResumableUpload.NumberOfUnits)
|
||||
uploadKey := checkResp.Response.ResumableUpload.UploadKey
|
||||
|
||||
stringWords := checkResp.Response.ResumableUpload.Bitmap.Words
|
||||
intWords := make([]int, 0, len(stringWords))
|
||||
for _, word := range stringWords {
|
||||
if intWord, err := strconv.Atoi(word); err == nil {
|
||||
intWords = append(intWords, intWord)
|
||||
}
|
||||
}
|
||||
|
||||
// Intelligent buffer sizing for large files
|
||||
bufferSize := int(unitSize)
|
||||
fileSize := file.GetSize()
|
||||
|
||||
// Split in chunks
|
||||
if fileSize > d.ChunkSize*1024*1024 {
|
||||
|
||||
// Large, use ChunkSize (default = 100MB)
|
||||
bufferSize = min(int(fileSize), int(d.ChunkSize)*1024*1024)
|
||||
} else if fileSize > 10*1024*1024 {
|
||||
// Medium, use full file size for concurrent access
|
||||
bufferSize = int(fileSize)
|
||||
}
|
||||
|
||||
// Create stream section reader for efficient chunking
|
||||
ss, err := stream.NewStreamSectionReader(file, bufferSize, &up)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// Cal minimal parallel upload threads, allows MediaFire resumable upload to rule it over custom value
|
||||
// If file is big, likely will respect d.UploadThreads instead of MediaFire's suggestion i.e. 5 threads
|
||||
thread := min(numUnits, d.UploadThreads)
|
||||
|
||||
// Create ordered group for sequential upload processing with retry logic
|
||||
threadG, uploadCtx := errgroup.NewOrderedGroupWithContext(ctx, thread,
|
||||
retry.Attempts(3),
|
||||
retry.Delay(time.Second),
|
||||
retry.DelayType(retry.BackOffDelay))
|
||||
|
||||
var finalUploadKey string
|
||||
var keyMutex sync.Mutex
|
||||
|
||||
fileSize = file.GetSize()
|
||||
for unitID := range numUnits {
|
||||
if utils.IsCanceled(uploadCtx) {
|
||||
break
|
||||
}
|
||||
|
||||
start := int64(unitID) * unitSize
|
||||
size := unitSize
|
||||
if start+size > fileSize {
|
||||
size = fileSize - start
|
||||
}
|
||||
|
||||
var reader io.ReadSeeker
|
||||
var unitHash string
|
||||
|
||||
// Use lifecycle pattern for proper resource management
|
||||
threadG.GoWithLifecycle(errgroup.Lifecycle{
|
||||
Before: func(ctx context.Context) (err error) {
|
||||
// Skip already uploaded units
|
||||
if d.isUnitUploaded(intWords, unitID) {
|
||||
return ss.DiscardSection(start, size)
|
||||
}
|
||||
reader, err = ss.GetSectionReader(start, size)
|
||||
return
|
||||
},
|
||||
Do: func(ctx context.Context) (err error) {
|
||||
if reader == nil {
|
||||
return nil // Skip if reader is not initialized (already uploaded)
|
||||
}
|
||||
reader.Seek(0, io.SeekStart)
|
||||
|
||||
if unitHash == "" {
|
||||
var err error
|
||||
unitHash, err = utils.HashReader(utils.SHA256, reader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
reader.Seek(0, io.SeekStart)
|
||||
}
|
||||
|
||||
// Perform upload
|
||||
|
||||
actionToken, err := d.getActionToken(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.limiter != nil {
|
||||
if err := d.limiter.Wait(ctx); err != nil {
|
||||
return fmt.Errorf("rate limit wait failed: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
url := d.apiBase + "/upload/resumable.php"
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, driver.NewLimitedUploadStream(ctx, reader))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
q := req.URL.Query()
|
||||
q.Add("folder_key", folderKey)
|
||||
q.Add("response_format", "json")
|
||||
q.Add("session_token", actionToken)
|
||||
q.Add("key", uploadKey)
|
||||
req.URL.RawQuery = q.Encode()
|
||||
|
||||
req.Header.Set("x-filehash", fileHash)
|
||||
req.Header.Set("x-filesize", strconv.FormatInt(fileSize, 10))
|
||||
req.Header.Set("x-unit-id", strconv.Itoa(unitID))
|
||||
req.Header.Set("x-unit-size", strconv.FormatInt(size, 10))
|
||||
req.Header.Set("x-unit-hash", unitHash)
|
||||
req.Header.Set("x-filename", filename)
|
||||
req.Header.Set("Content-Type", "application/octet-stream")
|
||||
req.ContentLength = size
|
||||
|
||||
/* fmt.Printf("Debug resumable upload request:\n")
|
||||
fmt.Printf(" URL: %s\n", req.URL.String())
|
||||
fmt.Printf(" Headers: %+v\n", req.Header)
|
||||
fmt.Printf(" Unit ID: %d\n", unitID)
|
||||
fmt.Printf(" Unit Size: %d\n", len(unitData))
|
||||
fmt.Printf(" Upload Key: %s\n", uploadKey)
|
||||
fmt.Printf(" Action Token: %s\n", actionToken) */
|
||||
|
||||
res, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
|
||||
body, err := io.ReadAll(res.Body)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to read response body: %v", err)
|
||||
}
|
||||
|
||||
// fmt.Printf("MediaFire resumable upload response (status %d): %s\n", res.StatusCode, string(body))
|
||||
|
||||
var uploadResp struct {
|
||||
Response struct {
|
||||
Doupload struct {
|
||||
Key string `json:"key"`
|
||||
} `json:"doupload"`
|
||||
Result string `json:"result"`
|
||||
} `json:"response"`
|
||||
}
|
||||
|
||||
if err := json.Unmarshal(body, &uploadResp); err != nil {
|
||||
return fmt.Errorf("failed to parse response: %v", err)
|
||||
}
|
||||
|
||||
if res.StatusCode != 200 {
|
||||
return fmt.Errorf("resumable upload failed with status %d", res.StatusCode)
|
||||
}
|
||||
|
||||
// Thread-safe update of final upload key
|
||||
keyMutex.Lock()
|
||||
finalUploadKey = uploadResp.Response.Doupload.Key
|
||||
keyMutex.Unlock()
|
||||
|
||||
up(float64(threadG.Success()+1) * 100 / float64(numUnits+1))
|
||||
return nil
|
||||
},
|
||||
After: func(err error) {
|
||||
if reader != nil {
|
||||
// Cleanup resources
|
||||
ss.FreeSectionReader(reader)
|
||||
}
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
if err := threadG.Wait(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return finalUploadKey, nil
|
||||
}
|
||||
|
||||
/*func (d *Mediafire) uploadSingleUnit(ctx context.Context, file model.FileStreamer, unitID int, unitSize int64, fileHash, filename, uploadKey, folderKey string, fileSize int64) (string, error) {
|
||||
start := int64(unitID) * unitSize
|
||||
size := unitSize
|
||||
|
||||
if start+size > fileSize {
|
||||
size = fileSize - start
|
||||
}
|
||||
|
||||
unitData := make([]byte, size)
|
||||
_, err := file.Read(unitData)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
return d.resumableUpload(ctx, folderKey, uploadKey, unitData, unitID, fileHash, filename, fileSize)
|
||||
}*/
|
||||
|
||||
func (d *Mediafire) getActionToken(ctx context.Context) (string, error) {
|
||||
if d.actionToken != "" {
|
||||
return d.actionToken, nil
|
||||
}
|
||||
|
||||
data := map[string]string{
|
||||
"type": "upload",
|
||||
"lifespan": "1440",
|
||||
"response_format": "json",
|
||||
"session_token": d.SessionToken,
|
||||
}
|
||||
|
||||
var resp MediafireActionTokenResponse
|
||||
_, err := d.postForm(ctx, "/user/get_action_token.php", data, &resp)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
if resp.Response.Result != "Success" {
|
||||
return "", fmt.Errorf("MediaFire action token failed: %s", resp.Response.Result)
|
||||
}
|
||||
|
||||
return resp.Response.ActionToken, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) pollUpload(ctx context.Context, key string) (*MediafirePollResponse, error) {
|
||||
actionToken, err := d.getActionToken(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get action token: %w", err)
|
||||
}
|
||||
|
||||
// fmt.Printf("Debug Key: %+v\n", key)
|
||||
|
||||
query := map[string]string{
|
||||
"key": key,
|
||||
"response_format": "json",
|
||||
"session_token": actionToken, /* d.SessionToken */
|
||||
}
|
||||
|
||||
var resp MediafirePollResponse
|
||||
_, err = d.postForm(ctx, "/upload/poll_upload.php", query, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// fmt.Printf("pollUpload :: Raw response: %s\n", string(body))
|
||||
// fmt.Printf("pollUpload :: Parsed response: %+v\n", resp)
|
||||
|
||||
// fmt.Printf("pollUpload :: Debug Result: %+v\n", resp.Response.Result)
|
||||
|
||||
if err := checkAPIResult(resp.Response.Result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (d *Mediafire) isUnitUploaded(words []int, unitID int) bool {
|
||||
wordIndex := unitID / 16
|
||||
bitIndex := unitID % 16
|
||||
if wordIndex >= len(words) {
|
||||
return false
|
||||
}
|
||||
return (words[wordIndex]>>bitIndex)&1 == 1
|
||||
}
|
||||
|
||||
func (d *Mediafire) getExistingFileInfo(ctx context.Context, fileHash, filename, folderKey string) (*model.ObjThumb, error) {
|
||||
// First try to find by hash directly (most efficient)
|
||||
if fileInfo, err := d.getFileByHash(ctx, fileHash); err == nil && fileInfo != nil {
|
||||
return fileInfo, nil
|
||||
}
|
||||
|
||||
// If hash search fails, search in the target folder
|
||||
// This is a fallback method in case the file exists but hash search doesn't work
|
||||
files, err := d.getFiles(ctx, folderKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, file := range files {
|
||||
if file.Name == filename && !file.IsFolder {
|
||||
return d.fileToObj(file), nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("existing file not found")
|
||||
}
|
||||
|
||||
func (d *Mediafire) getFileByHash(ctx context.Context, hash string) (*model.ObjThumb, error) {
|
||||
query := map[string]string{
|
||||
"session_token": d.SessionToken,
|
||||
"response_format": "json",
|
||||
"hash": hash,
|
||||
}
|
||||
|
||||
var resp MediafireFileSearchResponse
|
||||
_, err := d.postForm(ctx, "/file/get_info.php", query, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if resp.Response.Result != "Success" {
|
||||
return nil, fmt.Errorf("MediaFire file search failed: %s", resp.Response.Result)
|
||||
}
|
||||
|
||||
if len(resp.Response.FileInfo) == 0 {
|
||||
return nil, fmt.Errorf("file not found by hash")
|
||||
}
|
||||
|
||||
file := resp.Response.FileInfo[0]
|
||||
return d.fileToObj(file), nil
|
||||
}
|
||||
@@ -1,35 +0,0 @@
|
||||
package onedrive
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Region string `json:"region" type:"select" required:"true" options:"global,cn,us,de" default:"global"`
|
||||
IsSharepoint bool `json:"is_sharepoint"`
|
||||
UseOnlineAPI bool `json:"use_online_api" default:"true"`
|
||||
APIAddress string `json:"api_url_address" default:"https://api.oplist.org/onedrive/renewapi"`
|
||||
ClientID string `json:"client_id"`
|
||||
ClientSecret string `json:"client_secret"`
|
||||
RedirectUri string `json:"redirect_uri" required:"true" default:"https://api.oplist.org/onedrive/callback"`
|
||||
RefreshToken string `json:"refresh_token" required:"true"`
|
||||
SiteId string `json:"site_id"`
|
||||
ChunkSize int64 `json:"chunk_size" type:"number" default:"5"`
|
||||
CustomHost string `json:"custom_host" help:"Custom host for onedrive download link"`
|
||||
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
|
||||
EnableDirectUpload bool `json:"enable_direct_upload" default:"false" help:"Enable direct upload from client to OneDrive"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "Onedrive",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "/",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &Onedrive{}
|
||||
})
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package onedrive_app
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Region string `json:"region" type:"select" required:"true" options:"global,cn,us,de" default:"global"`
|
||||
ClientID string `json:"client_id" required:"true"`
|
||||
ClientSecret string `json:"client_secret" required:"true"`
|
||||
TenantID string `json:"tenant_id"`
|
||||
Email string `json:"email"`
|
||||
ChunkSize int64 `json:"chunk_size" type:"number" default:"5"`
|
||||
CustomHost string `json:"custom_host" help:"Custom host for onedrive download link"`
|
||||
DisableDiskUsage bool `json:"disable_disk_usage" default:"false"`
|
||||
EnableDirectUpload bool `json:"enable_direct_upload" default:"false" help:"Enable direct upload from client to OneDrive"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "OnedriveAPP",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "/",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &OnedriveAPP{}
|
||||
})
|
||||
}
|
||||
@@ -1,803 +0,0 @@
|
||||
package onedrive_sharelink
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/drivers/base"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/net"
|
||||
streamPkg "github.com/OpenListTeam/OpenList/v4/internal/stream"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/cron"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/http_range"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/singleflight"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/avast/retry-go"
|
||||
"github.com/pkg/errors"
|
||||
log "github.com/sirupsen/logrus"
|
||||
)
|
||||
|
||||
const (
|
||||
headerTTL = 25 * time.Minute
|
||||
driveTokenTTL = 20 * time.Minute
|
||||
directLinkTTL = 20 * time.Minute
|
||||
simpleUploadLimit = 250 * 1024 * 1024
|
||||
uploadSessionChunk = 10 * 1024 * 1024
|
||||
)
|
||||
|
||||
type OnedriveSharelink struct {
|
||||
model.Storage
|
||||
cron *cron.Cron
|
||||
Addition
|
||||
|
||||
headerMu sync.RWMutex
|
||||
sg singleflight.Group[http.Header]
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Init(ctx context.Context) error {
|
||||
// Initialize error variable
|
||||
var err error
|
||||
|
||||
// If there is "-my" in the URL, it is NOT a SharePoint link
|
||||
d.IsSharepoint = !strings.Contains(d.ShareLinkURL, "-my")
|
||||
|
||||
// Initialize cron job to run every hour
|
||||
d.cron = cron.NewCron(time.Hour * 1)
|
||||
d.cron.Do(func() {
|
||||
var err error
|
||||
h, err := d.getHeaders(ctx)
|
||||
if err != nil {
|
||||
log.Errorf("%+v", err)
|
||||
return
|
||||
}
|
||||
d.storeHeaders(h)
|
||||
})
|
||||
|
||||
// Get initial headers
|
||||
h, err := d.getHeaders(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.storeHeaders(h)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
files, err := d.getFiles(ctx, dir.GetPath())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
folderSizes, err := d.driveChildrenFolderSizes(ctx, dir.GetPath())
|
||||
if err != nil {
|
||||
log.Warnf("onedrive_sharelink: failed to get folder sizes for %s: %+v", dir.GetPath(), err)
|
||||
}
|
||||
|
||||
// Convert the slice of files to the required model.Obj format
|
||||
return utils.SliceConvert(files, func(src Item) (model.Obj, error) {
|
||||
obj := fileToObj(src)
|
||||
if size, ok := folderSizes[obj.GetName()]; ok {
|
||||
obj.Size = size
|
||||
}
|
||||
obj.Path = stdpath.Join(dir.GetPath(), obj.GetName())
|
||||
return obj, nil
|
||||
})
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
// Get the unique ID of the file
|
||||
uniqueId := file.GetID()
|
||||
// Cut the first char and the last char
|
||||
uniqueId = uniqueId[1 : len(uniqueId)-1]
|
||||
url := d.downloadLinkPrefix + uniqueId
|
||||
|
||||
header, err := d.getValidHeaders(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if args.Redirect {
|
||||
directURL, err := d.resolveDirectDownloadURL(ctx, file, url, header)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
expiration := directLinkTTL
|
||||
return &model.Link{
|
||||
URL: directURL,
|
||||
Expiration: &expiration,
|
||||
}, nil
|
||||
}
|
||||
|
||||
return &model.Link{
|
||||
URL: url,
|
||||
Header: header,
|
||||
RangeReader: rangeReaderFunc(func(ctx context.Context, hr http_range.Range) (io.ReadCloser, error) {
|
||||
return d.rangeReadWithRefresh(ctx, url, hr)
|
||||
}),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) MakeDir(ctx context.Context, parentDir model.Obj, dirName string) error {
|
||||
token, err := d.getValidDriveAccessToken(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
apiURL := injectAccessToken(d.drivePathAPIURL(parentDir.GetPath())+"/children", token)
|
||||
body := map[string]any{
|
||||
"name": dirName,
|
||||
"folder": map[string]any{},
|
||||
"@microsoft.graph.conflictBehavior": "fail",
|
||||
}
|
||||
resp, err := d.doJSON(ctx, http.MethodPost, apiURL, body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
switch resp.StatusCode {
|
||||
case http.StatusOK, http.StatusCreated:
|
||||
return nil
|
||||
case http.StatusConflict:
|
||||
return errs.ObjectAlreadyExists
|
||||
default:
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return fmt.Errorf("failed to create folder, status code: %d, body: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Move(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
// TODO move obj, optional
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Rename(ctx context.Context, srcObj model.Obj, newName string) error {
|
||||
// TODO rename obj, optional
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Copy(ctx context.Context, srcObj, dstDir model.Obj) error {
|
||||
// TODO copy obj, optional
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Remove(ctx context.Context, obj model.Obj) error {
|
||||
// TODO remove obj, optional
|
||||
return errs.NotImplement
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) Put(ctx context.Context, dstDir model.Obj, stream model.FileStreamer, up driver.UpdateProgress) error {
|
||||
info, err := d.createUploadInfo(ctx, stdpath.Join(dstDir.GetPath(), stream.GetName()), stream.GetSize())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if info.ChunkSize == 0 {
|
||||
return d.uploadContent(ctx, info.UploadURL, stream, up)
|
||||
}
|
||||
return d.uploadToSession(ctx, info.UploadURL, stream, up)
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) GetDetails(ctx context.Context) (*model.StorageDetails, error) {
|
||||
if d.DisableDiskUsage {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
size, err := d.driveItemSize(ctx, "/")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &model.StorageDetails{
|
||||
DiskUsage: model.DiskUsage{
|
||||
TotalSpace: size,
|
||||
UsedSpace: size,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) driveItemSize(ctx context.Context, path string) (int64, error) {
|
||||
token, err := d.getValidDriveAccessToken(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
apiURL := injectAccessToken(d.drivePathAPIURL(path), token)
|
||||
resp, err := d.doJSON(ctx, http.MethodGet, apiURL, nil)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return 0, fmt.Errorf("failed to get folder details, status code: %d, body: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
var item struct {
|
||||
Size int64 `json:"size"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&item); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return item.Size, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) driveChildrenFolderSizes(ctx context.Context, path string) (map[string]int64, error) {
|
||||
token, err := d.getValidDriveAccessToken(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rawURL := d.drivePathAPIURL(path) + "/children?$select=name,size,folder"
|
||||
sizes := make(map[string]int64)
|
||||
for rawURL != "" {
|
||||
resp, err := d.doJSON(ctx, http.MethodGet, injectAccessToken(rawURL, token), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var data struct {
|
||||
Value []struct {
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
Folder any `json:"folder"`
|
||||
} `json:"value"`
|
||||
NextLink string `json:"@odata.nextLink"`
|
||||
}
|
||||
func() {
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
err = fmt.Errorf("failed to get children details, status code: %d, body: %s", resp.StatusCode, string(body))
|
||||
return
|
||||
}
|
||||
err = json.NewDecoder(resp.Body).Decode(&data)
|
||||
}()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, item := range data.Value {
|
||||
if item.Folder != nil {
|
||||
sizes[item.Name] = item.Size
|
||||
}
|
||||
}
|
||||
rawURL = data.NextLink
|
||||
}
|
||||
return sizes, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) GetDirectUploadTools() []string {
|
||||
if !d.EnableDirectUpload {
|
||||
return nil
|
||||
}
|
||||
return []string{"HttpDirect"}
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) GetDirectUploadInfo(ctx context.Context, tool string, dstDir model.Obj, fileName string, fileSize int64) (any, error) {
|
||||
if !d.EnableDirectUpload {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
if tool != "HttpDirect" {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
return d.createUploadInfo(ctx, stdpath.Join(dstDir.GetPath(), fileName), fileSize)
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) createUploadInfo(ctx context.Context, path string, fileSize int64) (*model.HttpDirectUploadInfo, error) {
|
||||
token, err := d.getValidDriveAccessToken(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if fileSize >= 0 && fileSize <= simpleUploadLimit {
|
||||
return &model.HttpDirectUploadInfo{
|
||||
UploadURL: injectAccessToken(d.drivePathAPIURL(path)+"/content", token),
|
||||
Method: http.MethodPut,
|
||||
}, nil
|
||||
}
|
||||
apiURL := injectAccessToken(d.drivePathAPIURL(path)+"/createUploadSession", token)
|
||||
body := map[string]any{
|
||||
"item": map[string]any{
|
||||
"@microsoft.graph.conflictBehavior": "rename",
|
||||
},
|
||||
}
|
||||
resp, err := d.doJSON(ctx, http.MethodPost, apiURL, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return nil, fmt.Errorf("failed to create upload session, status code: %d, body: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
var data uploadSessionResp
|
||||
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if data.UploadURL == "" {
|
||||
return nil, fmt.Errorf("failed to get upload URL from response")
|
||||
}
|
||||
return &model.HttpDirectUploadInfo{
|
||||
UploadURL: data.UploadURL,
|
||||
ChunkSize: uploadSessionChunk,
|
||||
Method: http.MethodPut,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) uploadContent(ctx context.Context, uploadURL string, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
if up == nil {
|
||||
up = func(float64) {}
|
||||
}
|
||||
reader := driver.NewLimitedUploadStream(ctx, &driver.ReaderUpdatingProgress{
|
||||
Reader: &driver.SimpleReaderWithSize{
|
||||
Reader: file,
|
||||
Size: file.GetSize(),
|
||||
},
|
||||
UpdateProgress: up,
|
||||
})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadURL, reader)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.ContentLength = file.GetSize()
|
||||
resp, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusCreated {
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return fmt.Errorf("failed to upload content, status code: %d, body: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) uploadToSession(ctx context.Context, uploadURL string, file model.FileStreamer, up driver.UpdateProgress) error {
|
||||
if up == nil {
|
||||
up = func(float64) {}
|
||||
}
|
||||
if file.GetSize() <= 0 {
|
||||
return d.uploadSessionChunk(ctx, uploadURL, file, 0, 0, file.GetSize())
|
||||
}
|
||||
ss, err := streamPkg.NewStreamSectionReader(file, uploadSessionChunk, &up)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var finish int64
|
||||
for finish < file.GetSize() {
|
||||
if utils.IsCanceled(ctx) {
|
||||
return ctx.Err()
|
||||
}
|
||||
left := file.GetSize() - finish
|
||||
byteSize := min(left, int64(uploadSessionChunk))
|
||||
rd, err := ss.GetSectionReader(finish, byteSize)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = retry.Do(
|
||||
func() error {
|
||||
if _, err := rd.Seek(0, io.SeekStart); err != nil {
|
||||
return err
|
||||
}
|
||||
return d.uploadSessionChunk(ctx, uploadURL, rd, finish, byteSize, file.GetSize())
|
||||
},
|
||||
retry.Context(ctx),
|
||||
retry.Attempts(3),
|
||||
retry.DelayType(retry.BackOffDelay),
|
||||
retry.Delay(time.Second),
|
||||
)
|
||||
ss.FreeSectionReader(rd)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
finish += byteSize
|
||||
up(float64(finish) * 100 / float64(file.GetSize()))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) uploadSessionChunk(ctx context.Context, uploadURL string, reader io.Reader, start, size, total int64) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPut, uploadURL, driver.NewLimitedUploadStream(ctx, reader))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.ContentLength = size
|
||||
if total > 0 {
|
||||
req.Header.Set("Content-Range", fmt.Sprintf("bytes %d-%d/%d", start, start+size-1, total))
|
||||
}
|
||||
resp, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
switch {
|
||||
case resp.StatusCode >= 500 && resp.StatusCode <= 504:
|
||||
return fmt.Errorf("server error: %d", resp.StatusCode)
|
||||
case resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusCreated && resp.StatusCode != http.StatusAccepted:
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return errors.New(string(data))
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) drivePathAPIURL(path string) string {
|
||||
drivePath := stdpath.Join(d.driveRootPath, path)
|
||||
drivePath = utils.FixAndCleanPath(drivePath)
|
||||
if drivePath == "/" {
|
||||
return d.DriveURL + "/root"
|
||||
}
|
||||
return fmt.Sprintf("%s/root:%s:", d.DriveURL, utils.EncodePath(drivePath, true))
|
||||
}
|
||||
|
||||
func injectAccessToken(rawURL, token string) string {
|
||||
if token == "" {
|
||||
return rawURL
|
||||
}
|
||||
u, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return rawURL
|
||||
}
|
||||
values := u.Query()
|
||||
if strings.HasPrefix(token, "access_token=") {
|
||||
values.Set("access_token", strings.TrimPrefix(token, "access_token="))
|
||||
} else {
|
||||
values.Set("access_token", token)
|
||||
}
|
||||
u.RawQuery = values.Encode()
|
||||
return u.String()
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) doJSON(ctx context.Context, method, rawURL string, body any) (*http.Response, error) {
|
||||
var reader io.Reader
|
||||
if body != nil {
|
||||
data, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reader = bytes.NewReader(data)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, method, rawURL, reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Accept", "application/json;odata.metadata=minimal")
|
||||
if body != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
return base.HttpClient.Do(req)
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) getValidDriveAccessToken(ctx context.Context) (string, error) {
|
||||
d.headerMu.RLock()
|
||||
token := d.DriveAccessToken
|
||||
expired := time.Since(time.Unix(d.DriveTokenTime, 0)) > driveTokenTTL
|
||||
d.headerMu.RUnlock()
|
||||
if token != "" && !expired {
|
||||
return token, nil
|
||||
}
|
||||
if err := d.refreshDriveContext(ctx); err != nil {
|
||||
d.headerMu.RLock()
|
||||
token = d.DriveAccessToken
|
||||
d.headerMu.RUnlock()
|
||||
if token != "" {
|
||||
log.Warnf("onedrive_sharelink: use cached drive access token after refresh failure: %+v", err)
|
||||
return token, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
d.headerMu.RLock()
|
||||
defer d.headerMu.RUnlock()
|
||||
return d.DriveAccessToken, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) refreshDriveContext(ctx context.Context) error {
|
||||
header, err := d.getValidHeaders(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, d.ShareLinkURL, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header = cloneHeader(header)
|
||||
resp, err := NewNoRedirectCLient().Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
redirectURL := resp.Header.Get("Location")
|
||||
if redirectURL == "" && resp.Request != nil && resp.Request.URL != nil {
|
||||
redirectURL = resp.Request.URL.String()
|
||||
}
|
||||
if redirectURL == "" {
|
||||
return fmt.Errorf("share link did not return redirect URL")
|
||||
}
|
||||
return d.refreshDriveContextFromRedirect(ctx, redirectURL, header)
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) refreshDriveContextFromRedirect(ctx context.Context, redirectURL string, header http.Header) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, redirectURL, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header = cloneHeader(header)
|
||||
resp, err := base.HttpClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return fmt.Errorf("onedrive page request failed, status code: %d", resp.StatusCode)
|
||||
}
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ctxInfo, err := parsePageContext(body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rootPath, err := driveRootPathFromRedirect(redirectURL, ctxInfo.ListURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.headerMu.Lock()
|
||||
d.DriveURL = ctxInfo.DriveInfo.DriveURL
|
||||
d.DriveAccessToken = ctxInfo.DriveInfo.DriveAccessToken
|
||||
d.DriveTokenTime = time.Now().Unix()
|
||||
d.driveRootPath = rootPath
|
||||
d.headerMu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
func driveRootPathFromRedirect(redirectURL, listURL string) (string, error) {
|
||||
u, err := url.Parse(redirectURL)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
id := u.Query().Get("id")
|
||||
if id == "" {
|
||||
return "/", nil
|
||||
}
|
||||
if listURL == "" {
|
||||
return "/", nil
|
||||
}
|
||||
if id == listURL {
|
||||
return "/", nil
|
||||
}
|
||||
prefix := strings.TrimRight(listURL, "/") + "/"
|
||||
if strings.HasPrefix(id, prefix) {
|
||||
return utils.FixAndCleanPath(strings.TrimPrefix(id, strings.TrimRight(listURL, "/"))), nil
|
||||
}
|
||||
return "/", nil
|
||||
}
|
||||
|
||||
var pageContextRE = regexp.MustCompile(`(?s)var _spPageContextInfo=(\{.*?\});_spPageContextInfo`)
|
||||
|
||||
func parsePageContext(body []byte) (*pageContextInfo, error) {
|
||||
match := pageContextRE.FindSubmatch(body)
|
||||
if len(match) < 2 {
|
||||
return nil, fmt.Errorf("failed to find _spPageContextInfo")
|
||||
}
|
||||
var info pageContextInfo
|
||||
if err := json.Unmarshal(match[1], &info); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if info.DriveInfo.DriveURL == "" {
|
||||
return nil, fmt.Errorf("failed to get drive URL from page context")
|
||||
}
|
||||
if info.DriveInfo.DriveAccessToken == "" {
|
||||
return nil, fmt.Errorf("failed to get drive access token from page context")
|
||||
}
|
||||
return &info, nil
|
||||
}
|
||||
|
||||
//func (d *OnedriveSharelink) Other(ctx context.Context, args model.OtherArgs) (interface{}, error) {
|
||||
// return nil, errs.NotSupport
|
||||
//}
|
||||
|
||||
var _ driver.Driver = (*OnedriveSharelink)(nil)
|
||||
|
||||
// rangeReadWithRefresh tries once with current headers, and if the response
|
||||
// looks invalid (error status or html login page), it refreshes headers and retries.
|
||||
func (d *OnedriveSharelink) rangeReadWithRefresh(ctx context.Context, url string, hr http_range.Range) (io.ReadCloser, error) {
|
||||
tryOnce := func(header http.Header) (io.ReadCloser, error) {
|
||||
h := cloneHeader(header)
|
||||
if h == nil {
|
||||
h = http.Header{}
|
||||
}
|
||||
h = http_range.ApplyRangeToHttpHeader(hr, h)
|
||||
resp, err := net.RequestHttp(ctx, http.MethodGet, h, url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ct := strings.ToLower(resp.Header.Get("Content-Type"))
|
||||
if strings.Contains(ct, "text/html") {
|
||||
_ = resp.Body.Close()
|
||||
return nil, fmt.Errorf("unexpected html response")
|
||||
}
|
||||
return resp.Body, nil
|
||||
}
|
||||
|
||||
header, err := d.getValidHeaders(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if body, err := tryOnce(header); err == nil {
|
||||
return body, nil
|
||||
}
|
||||
|
||||
// refresh and retry once
|
||||
header, err = d.refreshHeaders(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return tryOnce(header)
|
||||
}
|
||||
|
||||
type rangeReaderFunc func(ctx context.Context, hr http_range.Range) (io.ReadCloser, error)
|
||||
|
||||
func (f rangeReaderFunc) RangeRead(ctx context.Context, hr http_range.Range) (io.ReadCloser, error) {
|
||||
return f(ctx, hr)
|
||||
}
|
||||
|
||||
func cloneHeader(header http.Header) http.Header {
|
||||
if header == nil {
|
||||
return nil
|
||||
}
|
||||
return header.Clone()
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) resolveDirectDownloadURL(ctx context.Context, file model.Obj, rawURL string, header http.Header) (string, error) {
|
||||
var errs []error
|
||||
if obj, ok := unwrapObject(file); ok {
|
||||
if obj.SPItemURL != "" {
|
||||
directURL, err := d.resolveSPItemDownloadURL(ctx, obj.SPItemURL, header)
|
||||
if err == nil {
|
||||
return directURL, nil
|
||||
}
|
||||
errs = append(errs, err)
|
||||
}
|
||||
if obj.ContentDownloadURL != "" {
|
||||
return obj.ContentDownloadURL, nil
|
||||
}
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
req.Header = cloneHeader(header)
|
||||
if req.Header == nil {
|
||||
req.Header = http.Header{}
|
||||
}
|
||||
|
||||
resp, err := NewNoRedirectCLient().Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
location := resp.Header.Get("Location")
|
||||
if location == "" {
|
||||
errs = append(errs, fmt.Errorf("download.aspx returned no redirect location, status code: %d", resp.StatusCode))
|
||||
return "", fmt.Errorf("onedrive_sharelink: direct download URL unavailable: %v", errs)
|
||||
}
|
||||
u, err := req.URL.Parse(location)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return u.String(), nil
|
||||
}
|
||||
|
||||
type spItemDownloadResp struct {
|
||||
ContentDownloadURL string `json:"@content.downloadUrl"`
|
||||
}
|
||||
|
||||
func unwrapObject(obj model.Obj) (*Object, bool) {
|
||||
for {
|
||||
switch o := obj.(type) {
|
||||
case *Object:
|
||||
return o, true
|
||||
case model.ObjUnwrap:
|
||||
obj = o.Unwrap()
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) resolveSPItemDownloadURL(ctx context.Context, spItemURL string, header http.Header) (string, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, spItemURL, nil)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
req.Header = cloneHeader(header)
|
||||
if req.Header == nil {
|
||||
req.Header = http.Header{}
|
||||
}
|
||||
req.Header.Set("Accept", "application/json;odata.metadata=minimal")
|
||||
|
||||
resp, err := NewNoRedirectCLient().Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return "", fmt.Errorf("sp item metadata request failed, status code: %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
var data spItemDownloadResp
|
||||
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if data.ContentDownloadURL == "" {
|
||||
return "", fmt.Errorf("sp item metadata response missing @content.downloadUrl")
|
||||
}
|
||||
return data.ContentDownloadURL, nil
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) headerSnapshot() http.Header {
|
||||
d.headerMu.RLock()
|
||||
defer d.headerMu.RUnlock()
|
||||
return cloneHeader(d.Headers)
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) storeHeaders(header http.Header) {
|
||||
if header == nil {
|
||||
return
|
||||
}
|
||||
d.headerMu.Lock()
|
||||
d.Headers = header
|
||||
d.HeaderTime = time.Now().Unix()
|
||||
d.headerMu.Unlock()
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) headersExpired() bool {
|
||||
d.headerMu.RLock()
|
||||
defer d.headerMu.RUnlock()
|
||||
return time.Since(time.Unix(d.HeaderTime, 0)) > headerTTL
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) refreshHeaders(ctx context.Context) (http.Header, error) {
|
||||
header, err, _ := d.sg.Do("refresh", func() (http.Header, error) {
|
||||
h, e := d.getHeaders(ctx)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
d.storeHeaders(h)
|
||||
return h, nil
|
||||
})
|
||||
return header, err
|
||||
}
|
||||
|
||||
func (d *OnedriveSharelink) getValidHeaders(ctx context.Context) (http.Header, error) {
|
||||
if h := d.headerSnapshot(); h != nil && !d.headersExpired() {
|
||||
return h, nil
|
||||
}
|
||||
h, err := d.refreshHeaders(ctx)
|
||||
if err != nil {
|
||||
if h2 := d.headerSnapshot(); h2 != nil {
|
||||
log.Warnf("onedrive_sharelink: use cached headers after refresh failure: %+v", err)
|
||||
return h2, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return h, nil
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
package openlist
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Address string `json:"url" required:"true"`
|
||||
MetaPassword string `json:"meta_password"`
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
Token string `json:"token"`
|
||||
PassIPToUpsteam bool `json:"pass_ip_to_upsteam" default:"true"`
|
||||
PassUAToUpsteam bool `json:"pass_ua_to_upsteam" default:"true"`
|
||||
ForwardArchiveReq bool `json:"forward_archive_requests" default:"true"`
|
||||
PassRefreshFlagToUpsteam bool `json:"pass_refresh_flag_to_upsteam" default:"false"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "OpenList",
|
||||
LocalSort: true,
|
||||
DefaultRoot: "/",
|
||||
ProxyRangeOption: true,
|
||||
LinkCacheMode: driver.LinkCacheAuto,
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &OpenList{}
|
||||
})
|
||||
}
|
||||
@@ -1,182 +0,0 @@
|
||||
package openlist_share
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
stdpath "path"
|
||||
"strings"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/errs"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
"github.com/OpenListTeam/OpenList/v4/server/common"
|
||||
"github.com/go-resty/resty/v2"
|
||||
)
|
||||
|
||||
type OpenListShare struct {
|
||||
model.Storage
|
||||
Addition
|
||||
serverArchivePreview bool
|
||||
}
|
||||
|
||||
func (d *OpenListShare) Config() driver.Config {
|
||||
return config
|
||||
}
|
||||
|
||||
func (d *OpenListShare) GetAddition() driver.Additional {
|
||||
return &d.Addition
|
||||
}
|
||||
|
||||
func (d *OpenListShare) Init(ctx context.Context) error {
|
||||
d.Addition.Address = strings.TrimSuffix(d.Addition.Address, "/")
|
||||
var settings common.Resp[map[string]string]
|
||||
_, _, err := d.request("/public/settings", http.MethodGet, func(req *resty.Request) {
|
||||
req.SetResult(&settings)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d.serverArchivePreview = settings.Data["share_archive_preview"] == "true"
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *OpenListShare) Drop(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *OpenListShare) List(ctx context.Context, dir model.Obj, args model.ListArgs) ([]model.Obj, error) {
|
||||
var resp common.Resp[FsListResp]
|
||||
_, _, err := d.request("/fs/list", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ListReq{
|
||||
PageReq: model.PageReq{
|
||||
Page: 1,
|
||||
PerPage: 0,
|
||||
},
|
||||
Path: stdpath.Join(fmt.Sprintf("/@s/%s", d.ShareId), dir.GetPath()),
|
||||
Password: d.Pwd,
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []model.Obj
|
||||
for _, f := range resp.Data.Content {
|
||||
file := model.ObjThumb{
|
||||
Object: model.Object{
|
||||
Name: f.Name,
|
||||
Path: stdpath.Join(dir.GetPath(), f.Name),
|
||||
Modified: f.Modified,
|
||||
Ctime: f.Created,
|
||||
Size: f.Size,
|
||||
IsFolder: f.IsDir,
|
||||
HashInfo: utils.FromString(f.HashInfo),
|
||||
},
|
||||
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
|
||||
}
|
||||
files = append(files, &file)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *OpenListShare) Link(ctx context.Context, file model.Obj, args model.LinkArgs) (*model.Link, error) {
|
||||
path := utils.FixAndCleanPath(stdpath.Join(d.ShareId, file.GetPath()))
|
||||
u := fmt.Sprintf("%s/sd%s?pwd=%s", d.Address, path, d.Pwd)
|
||||
return &model.Link{URL: u}, nil
|
||||
}
|
||||
|
||||
func (d *OpenListShare) GetArchiveMeta(ctx context.Context, obj model.Obj, args model.ArchiveArgs) (model.ArchiveMeta, error) {
|
||||
if !d.serverArchivePreview || !d.ForwardArchiveReq {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
var resp common.Resp[ArchiveMetaResp]
|
||||
_, code, err := d.request("/fs/archive/meta", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Path: stdpath.Join(fmt.Sprintf("/@s/%s", d.ShareId), obj.GetPath()),
|
||||
Password: d.Pwd,
|
||||
Refresh: false,
|
||||
})
|
||||
})
|
||||
if code == 202 {
|
||||
return nil, errs.WrongArchivePassword
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var tree []model.ObjTree
|
||||
if resp.Data.Content != nil {
|
||||
tree = make([]model.ObjTree, 0, len(resp.Data.Content))
|
||||
for _, content := range resp.Data.Content {
|
||||
tree = append(tree, &content)
|
||||
}
|
||||
}
|
||||
return &model.ArchiveMetaInfo{
|
||||
Comment: resp.Data.Comment,
|
||||
Encrypted: resp.Data.Encrypted,
|
||||
Tree: tree,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *OpenListShare) ListArchive(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) ([]model.Obj, error) {
|
||||
if !d.serverArchivePreview || !d.ForwardArchiveReq {
|
||||
return nil, errs.NotImplement
|
||||
}
|
||||
var resp common.Resp[ArchiveListResp]
|
||||
_, code, err := d.request("/fs/archive/list", http.MethodPost, func(req *resty.Request) {
|
||||
req.SetResult(&resp).SetBody(ArchiveListReq{
|
||||
ArchiveMetaReq: ArchiveMetaReq{
|
||||
ArchivePass: args.Password,
|
||||
Path: stdpath.Join(fmt.Sprintf("/@s/%s", d.ShareId), obj.GetPath()),
|
||||
Password: d.Pwd,
|
||||
Refresh: false,
|
||||
},
|
||||
PageReq: model.PageReq{
|
||||
Page: 1,
|
||||
PerPage: 0,
|
||||
},
|
||||
InnerPath: args.InnerPath,
|
||||
})
|
||||
})
|
||||
if code == 202 {
|
||||
return nil, errs.WrongArchivePassword
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []model.Obj
|
||||
for _, f := range resp.Data.Content {
|
||||
file := model.ObjThumb{
|
||||
Object: model.Object{
|
||||
Name: f.Name,
|
||||
Modified: f.Modified,
|
||||
Ctime: f.Created,
|
||||
Size: f.Size,
|
||||
IsFolder: f.IsDir,
|
||||
HashInfo: utils.FromString(f.HashInfo),
|
||||
},
|
||||
Thumbnail: model.Thumbnail{Thumbnail: f.Thumb},
|
||||
}
|
||||
files = append(files, &file)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (d *OpenListShare) Extract(ctx context.Context, obj model.Obj, args model.ArchiveInnerArgs) (*model.Link, error) {
|
||||
if !d.serverArchivePreview || !d.ForwardArchiveReq {
|
||||
return nil, errs.NotSupport
|
||||
}
|
||||
path := utils.FixAndCleanPath(stdpath.Join(d.ShareId, obj.GetPath()))
|
||||
u := fmt.Sprintf("%s/sad%s?pwd=%s&inner=%s&pass=%s",
|
||||
d.Address,
|
||||
path,
|
||||
d.Pwd,
|
||||
utils.EncodePath(args.InnerPath, true),
|
||||
url.QueryEscape(args.Password))
|
||||
return &model.Link{URL: u}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*OpenListShare)(nil)
|
||||
@@ -1,27 +0,0 @@
|
||||
package openlist_share
|
||||
|
||||
import (
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/driver"
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/op"
|
||||
)
|
||||
|
||||
type Addition struct {
|
||||
driver.RootPath
|
||||
Address string `json:"url" required:"true"`
|
||||
ShareId string `json:"sid" required:"true"`
|
||||
Pwd string `json:"pwd"`
|
||||
ForwardArchiveReq bool `json:"forward_archive_requests" default:"true"`
|
||||
}
|
||||
|
||||
var config = driver.Config{
|
||||
Name: "OpenListShare",
|
||||
LocalSort: true,
|
||||
NoUpload: true,
|
||||
DefaultRoot: "/",
|
||||
}
|
||||
|
||||
func init() {
|
||||
op.RegisterDriver(func() driver.Driver {
|
||||
return &OpenListShare{}
|
||||
})
|
||||
}
|
||||
@@ -1,111 +0,0 @@
|
||||
package openlist_share
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/OpenListTeam/OpenList/v4/internal/model"
|
||||
"github.com/OpenListTeam/OpenList/v4/pkg/utils"
|
||||
)
|
||||
|
||||
type ListReq struct {
|
||||
model.PageReq
|
||||
Path string `json:"path" form:"path"`
|
||||
Password string `json:"password" form:"password"`
|
||||
Refresh bool `json:"refresh"`
|
||||
}
|
||||
|
||||
type ObjResp struct {
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
IsDir bool `json:"is_dir"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Created time.Time `json:"created"`
|
||||
Sign string `json:"sign"`
|
||||
Thumb string `json:"thumb"`
|
||||
Type int `json:"type"`
|
||||
HashInfo string `json:"hashinfo"`
|
||||
}
|
||||
|
||||
type FsListResp struct {
|
||||
Content []ObjResp `json:"content"`
|
||||
Total int64 `json:"total"`
|
||||
Readme string `json:"readme"`
|
||||
Write bool `json:"write"`
|
||||
Provider string `json:"provider"`
|
||||
}
|
||||
|
||||
type ArchiveMetaReq struct {
|
||||
ArchivePass string `json:"archive_pass"`
|
||||
Password string `json:"password"`
|
||||
Path string `json:"path"`
|
||||
Refresh bool `json:"refresh"`
|
||||
}
|
||||
|
||||
type TreeResp struct {
|
||||
ObjResp
|
||||
Children []TreeResp `json:"children"`
|
||||
hashCache *utils.HashInfo
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetSize() int64 {
|
||||
return t.Size
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetName() string {
|
||||
return t.Name
|
||||
}
|
||||
|
||||
func (t *TreeResp) ModTime() time.Time {
|
||||
return t.Modified
|
||||
}
|
||||
|
||||
func (t *TreeResp) CreateTime() time.Time {
|
||||
return t.Created
|
||||
}
|
||||
|
||||
func (t *TreeResp) IsDir() bool {
|
||||
return t.ObjResp.IsDir
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetHash() utils.HashInfo {
|
||||
return utils.FromString(t.HashInfo)
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetID() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetPath() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (t *TreeResp) GetChildren() []model.ObjTree {
|
||||
ret := make([]model.ObjTree, 0, len(t.Children))
|
||||
for _, child := range t.Children {
|
||||
ret = append(ret, &child)
|
||||
}
|
||||
return ret
|
||||
}
|
||||
|
||||
func (t *TreeResp) Thumb() string {
|
||||
return t.ObjResp.Thumb
|
||||
}
|
||||
|
||||
type ArchiveMetaResp struct {
|
||||
Comment string `json:"comment"`
|
||||
Encrypted bool `json:"encrypted"`
|
||||
Content []TreeResp `json:"content"`
|
||||
RawURL string `json:"raw_url"`
|
||||
Sign string `json:"sign"`
|
||||
}
|
||||
|
||||
type ArchiveListReq struct {
|
||||
model.PageReq
|
||||
ArchiveMetaReq
|
||||
InnerPath string `json:"inner_path"`
|
||||
}
|
||||
|
||||
type ArchiveListResp struct {
|
||||
Content []ObjResp `json:"content"`
|
||||
Total int64 `json:"total"`
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user