webp fix v2
This commit is contained in:
@@ -122,6 +122,7 @@
|
||||
|
||||
### 踩坑记录(都是我自己写错的)
|
||||
1. **`gifFrameCount` 恒返回 0**:漏跳 7 字节 Logical Screen Descriptor,
|
||||
|
||||
`pos` 停在 LSD 首字节(宽度低位)→ 首轮命中 `default` 返回 0。
|
||||
LSD 的 packed 在 `data[10]`(=pos+4),**必须无条件 pos += 7 再按 flag 加色表**。
|
||||
2. **`buildJPEGWithOrientation` panic**:`entry[10:12]` 只有 2 字节却调 `PutUint32`。
|
||||
@@ -132,3 +133,57 @@
|
||||
5. **WebP 桩不产出字节** → 依赖真实编码器的端到端断言在本机无法成立,
|
||||
已改为 `t.Skip`(桩环境跳过),像素级映射由单测覆盖。
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 部署后线上验证(用户已部署 box-dev-api)
|
||||
|
||||
新增 `dev/devcheck-avatar.py`(头像链路,可复跑)。**A 分档线上确认生效**:
|
||||
上传 1600x1600 JPEG(124,941 B)→ 存储为 **512x512 WebP, 1,496 B**。
|
||||
|
||||
### 🔴 线上验证暴露的真实缺口(待修)
|
||||
**EXIF 归一的位置太晚,比例校验看不到归一后的方向。**
|
||||
- `s3.go:110` `asp.check()` 只用 `image.DecodeConfig` 拿宽高,**不读 EXIF**;
|
||||
- EXIF 归一在 `ConvertStaticImageToWebP` 内部(第 240+ 行),**晚于比例校验**。
|
||||
- 后果:手机竖拍照片(像素 4032x3024 + Orientation=6,实际显示 3024x4032)
|
||||
直传头像会被判「比例不符 1:1(偏差 0.333)」而拒绝——**尽管用户看到的是方形**。
|
||||
实测线上确实如此:传 1536x512+EXIF=6 被拒,头像 URL 未变。
|
||||
- 前端走 cropper(canvas 会归一方向)不受影响,故只影响**绕过前端的直传**。
|
||||
- 修法方向:`asp.check` 先读 `jpegOrientation`,对 5/6/7/8 交换宽高再比比例。
|
||||
别为一次校验就完整解码(大图内存),只做 EXIF 段解析 + 宽高交换即可。
|
||||
|
||||
### 验证脚本踩坑
|
||||
- 提问成功页**不渲染图片**(`templates/question/success.html` 无图片区),
|
||||
图片只在 `question.go:50` 的**详情页**渲染;且提问图 key **不带 `picture/` 前缀**
|
||||
(是 `<年>/<月>/<纳秒>.<ext>`),头像才带 `picture/`。→ 断言正则要分开写。
|
||||
- `/user/profile/update` 是完整资料表单,`UpdateProfile` 有 12+ 个 `required` 字段
|
||||
(`internal/form/user.go:7`)。只发 `_csrf` 会被挡在 Bind 阶段、头像根本不上传。
|
||||
做法:从 profile 页**抓取所有 input/select/textarea 的 name=value 回填**。
|
||||
- 头像上传成功是 **302**,头像 URL 在跳转后的页面里(不是响应体)。
|
||||
|
||||
## 补完 B1 + B3(2026-10-15 夜)
|
||||
|
||||
### B1 补完:EXIF 比例校验缺口
|
||||
- `aspect.check` 签名改为 `(cfg image.Config, orientation int) error`——**不再自己DecodeConfig**,
|
||||
由调用方传入已解析的宽高,顺带消除一次重复的文件头读取。
|
||||
- 内部:`orientationSwapsDimensions(orientation)` 为真(5/6/7/8)则交换 w/h 再比比例。
|
||||
- `imagecheck` 新增 `CheckAspectRatioSize(w, h, wantW, wantH, tol)`;
|
||||
`CheckAspectRatio(r,...)` 改为薄封装(内部走 DecodeConfig + 调Size 版),行为不变。
|
||||
- `exif.go` 抽出 `orientationSwapsDimensions`,`applyJPEGOrientation` 里的 switch 改为调它(去重)。
|
||||
|
||||
### B3:合并重复解码(峰值内存减半)
|
||||
- 根因:`uploadPictureToS3` 里 `image.Decode` 为拿 format 解一遍 → `ConvertStaticImageToWebP` 内再解一遍。
|
||||
- 改法:`image.DecodeConfig` 一次同时拿 **format + 宽高**(不解像素),
|
||||
供「像素上限校验 / 比例校验 / WebP 透传判断」三处共用;完整解码只发生在真正要转码时。
|
||||
- `ConvertStaticImageToWebP` 签名 `(io.Reader, spec)` → `([]byte, spec)`:
|
||||
调用方手里已有原始字节,不必再包一层 `bytes.Reader`(也不必内部 `io.ReadAll` 复制)。
|
||||
内部随之删掉重复的 `CheckPixelLimit`(调用方已在解码前校验过)。
|
||||
- `route/question/page.go` 同步:删掉 `image.Decode`取 format 与 `DecodeConfig`取 cfg 的两次调用,
|
||||
改为一次 `DecodeConfig`;新增本地 `fitsWithinSide(cfg, maxSide)`。
|
||||
- `dev/s3stub/s3_stub.go` 同步签名(去掉 `io` import)。
|
||||
- **实测堆峰值增量**(runtime.MemStats,含保留引用强制不回收):
|
||||
| 输入 | 一次解码 | 两次解码 |
|
||||
|---|---|---|
|
||||
| JPEG 4000×3000 (2.3MB) | 18.1 MB | 36.1 MB |
|
||||
| PNG 3000×3000 (33KB) | 36.1 MB | 72.1 MB |
|
||||
→ 正好 2 倍。1C1G 上并发上传大图时这是决定性的。
|
||||
|
||||
@@ -44,6 +44,13 @@
|
||||
- **EXIF 方向必须在转码前归一**(`exif.go` 手工解析 APP1,无外部依赖)。
|
||||
`image.Decode` 与 Pillow 默认都不读 EXIF;**转成 WebP 后 EXIF 被丢弃、方向永久丢失**。
|
||||
顺序铁律:**先归一方向 → 再限边缩放**(归一会交换宽高)。
|
||||
⚠️ **比例校验必须用同一套方向**(`aspect.check(cfg, orientation)` + `orientationSwapsDimensions`),
|
||||
否则手机竖拍照片(像素横躺 + Orientation=6)直传头像/背景会被误判「比例不符」而拒。
|
||||
比例校验只交换宽高、**不解码像素**(大图内存)。前端 cropper 路径不受影响(canvas 已归一)。
|
||||
- **上传管线只完整解码一次**:`image.DecodeConfig` 一次同时拿 format + 宽高,
|
||||
供像素上限 / 比例 / WebP 透传三处共用;完整解码只在真正转码时发生。
|
||||
`ConvertStaticImageToWebP(data []byte, spec)` 收字节而非 `io.Reader`。
|
||||
实测堆峰值:JPEG 4000×3000 单解 18MB / 双解 36MB(正好 2 倍),1C1G 上很关键。
|
||||
- GIF 帧数上限 `maxGIFFrames = 200`:`CheckPixelLimit` 只看逻辑屏尺寸,
|
||||
几百帧的大 GIF 体积小但能让 `cwebp` 长时间吃 CPU。
|
||||
- **method=6 已评估并否决**:实测耗时只增 1.2~1.9x(不是原以为的 5~10x),
|
||||
|
||||
@@ -0,0 +1,346 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""头像上传链路端到端验证(批次 A 分档 + B1 EXIF 归一)。
|
||||
|
||||
为什么走头像而不是提问图:
|
||||
- 提问图那条路要经过内容审核/提问入库才能拿到 key,链路长且易受其它因素干扰;
|
||||
- 头像强制 1:1 且 POST /user/profile/update 直接返回页面,便于立即取回 CDN URL。
|
||||
|
||||
验证两件事:
|
||||
A 分档生效:上传 1600x1600 的大图,头像档 maxSide=512 -> 存盘应为 512x512
|
||||
(改前是 1920,实测体积差 ~7 倍)
|
||||
B1 EXIF Orientation=6 归一:上传「像素 512x256 + EXIF 顺时针 90」的方形图,
|
||||
归一后仍应是 512x512(宽高互换但比例不变,故改用体积+尺寸双断言,
|
||||
并额外验证竖长条图 256x512 + Orientation=6 -> 应变成 512x256)
|
||||
|
||||
用法:
|
||||
python dev/devcheck-avatar.py [--base https://box-dev-api.337845818.best]
|
||||
依赖:Pillow(仅用于本地生成测试图片)
|
||||
清理:TamaBox ban --email <输出中的测试邮箱> --yes
|
||||
"""
|
||||
import argparse
|
||||
import http.cookiejar
|
||||
import io
|
||||
import json
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
||||
"(KHTML, like Gecko) Chrome/124.0 Safari/537.36")
|
||||
RESULTS = []
|
||||
|
||||
|
||||
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: D102
|
||||
return None
|
||||
|
||||
|
||||
class Resp:
|
||||
def __init__(self, status, headers, body):
|
||||
self.status = status
|
||||
self.h = {k.lower(): v for k, v in headers.items()}
|
||||
self.body = body
|
||||
|
||||
@property
|
||||
def loc(self):
|
||||
return self.h.get("location", "")
|
||||
|
||||
@property
|
||||
def text(self):
|
||||
return self.body.decode("utf-8", "replace")
|
||||
|
||||
|
||||
class Client:
|
||||
def __init__(self, base):
|
||||
self.base = base
|
||||
self.cj = http.cookiejar.CookieJar()
|
||||
self.op = urllib.request.build_opener(
|
||||
urllib.request.HTTPCookieProcessor(self.cj), NoRedirect())
|
||||
|
||||
def _send(self, req, timeout=120):
|
||||
try:
|
||||
with self.op.open(req, timeout=timeout) as r:
|
||||
return Resp(r.status, dict(r.headers), r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
try:
|
||||
body = e.read()
|
||||
except Exception: # noqa: BLE001
|
||||
body = b""
|
||||
return Resp(e.code, dict(e.headers), body)
|
||||
except Exception as e: # noqa: BLE001
|
||||
return Resp(-1, {}, ("TRANSPORT_ERROR: %s" % e).encode())
|
||||
|
||||
def req(self, method, path, data=None, content_type=None, timeout=120):
|
||||
url = path if path.startswith("http") else self.base + path
|
||||
headers = {"User-Agent": UA, "Referer": self.base + path.split("?")[0]}
|
||||
body = None
|
||||
if data is not None:
|
||||
body = data if isinstance(data, bytes) else urllib.parse.urlencode(data).encode()
|
||||
headers["Content-Type"] = content_type or "application/x-www-form-urlencoded"
|
||||
return self._send(urllib.request.Request(url, data=body, headers=headers, method=method),
|
||||
timeout)
|
||||
|
||||
def get(self, p, **kw):
|
||||
r = self.req("GET", p, **kw)
|
||||
return r if r.status != -1 else self.req("GET", p, **kw)
|
||||
|
||||
def post(self, p, data=None, **kw):
|
||||
return self.req("POST", p, data=data, **kw)
|
||||
|
||||
def get_binary(self, url):
|
||||
return self._send(urllib.request.Request(
|
||||
url, headers={"User-Agent": UA}, method="GET"), timeout=60)
|
||||
|
||||
|
||||
def step(desc, ok, extra=""):
|
||||
RESULTS.append((desc, ok, extra))
|
||||
print(" [%s] %s%s" % ("PASS" if ok else "FAIL", desc, (" (%s)" % extra) if extra else ""))
|
||||
return ok
|
||||
|
||||
|
||||
def csrf_of(html):
|
||||
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def captcha(c):
|
||||
j = json.loads(c.get("/api/captcha/generate").text)
|
||||
pid, q = j["data"]["id"], j["data"]["question"]
|
||||
m = re.match(r"\s*(\d+)\s*([+\-×])\s*(\d+)\s*=", q)
|
||||
a, op, b = int(m.group(1)), m.group(2), int(m.group(3))
|
||||
return "%s:%d" % (pid, {"+": a + b, "-": a - b, "×": a * b}[op])
|
||||
|
||||
|
||||
def multipart(fields, files):
|
||||
boundary = "----tbav%s" % uuid.uuid4().hex
|
||||
buf = io.BytesIO()
|
||||
for k, v in fields.items():
|
||||
buf.write(("--%s\r\n" % boundary).encode())
|
||||
buf.write(('Content-Disposition: form-data; name="%s"\r\n\r\n' % k).encode())
|
||||
buf.write(("%s\r\n" % v).encode())
|
||||
for name, fn, ct, data in files:
|
||||
buf.write(("--%s\r\n" % boundary).encode())
|
||||
buf.write(('Content-Disposition: form-data; name="%s"; filename="%s"\r\n' % (name, fn)).encode())
|
||||
buf.write(("Content-Type: %s\r\n\r\n" % ct).encode())
|
||||
buf.write(data)
|
||||
buf.write(b"\r\n")
|
||||
buf.write(("--%s--\r\n" % boundary).encode())
|
||||
return buf.getvalue(), "multipart/form-data; boundary=%s" % boundary
|
||||
|
||||
|
||||
def sniff(d):
|
||||
if len(d) >= 12 and d[:4] == b"RIFF" and d[8:12] == b"WEBP":
|
||||
return "webp", webp_dims(d)
|
||||
if d[:3] == b"\xff\xd8\xff":
|
||||
return "jpeg", jpeg_dims(d)
|
||||
if d[:8] == b"\x89PNG\r\n\x1a\n":
|
||||
return "png", (struct.unpack(">II", d[16:24]) if len(d) >= 24 else None)
|
||||
return "unknown", None
|
||||
|
||||
|
||||
def webp_dims(d):
|
||||
c = d[12:16]
|
||||
if c == b"VP8 " and len(d) >= 30:
|
||||
return (struct.unpack("<H", d[26:28])[0] & 0x3FFF,
|
||||
struct.unpack("<H", d[28:30])[0] & 0x3FFF)
|
||||
if c == b"VP8L" and len(d) >= 25:
|
||||
b = struct.unpack("<I", d[21:25])[0]
|
||||
return (b & 0x3FFF) + 1, ((b >> 14) & 0x3FFF) + 1
|
||||
if c == b"VP8X" and len(d) >= 30:
|
||||
return ((d[24] | d[25] << 8 | d[26] << 16) + 1,
|
||||
(d[27] | d[28] << 8 | d[29] << 16) + 1)
|
||||
return None
|
||||
|
||||
|
||||
def jpeg_dims(d):
|
||||
i = 2
|
||||
while i + 9 < len(d):
|
||||
if d[i] != 0xFF:
|
||||
i += 1
|
||||
continue
|
||||
m = d[i + 1]
|
||||
if m in (0xC0, 0xC1, 0xC2, 0xC3):
|
||||
h, w = struct.unpack(">HH", d[i + 5:i + 9])
|
||||
return (w, h)
|
||||
if m in (0xD8, 0xD9) or 0xD0 <= m <= 0xD7:
|
||||
i += 2
|
||||
continue
|
||||
i += 2 + struct.unpack(">H", d[i + 2:i + 4])[0]
|
||||
return None
|
||||
|
||||
|
||||
def make_avatar(w, h, orientation=None):
|
||||
"""生成一张带明显方位特征的方形头像。"""
|
||||
from PIL import Image, ImageDraw
|
||||
im = Image.new("RGB", (w, h), (30, 120, 200))
|
||||
d = ImageDraw.Draw(im)
|
||||
# 顶部一条粗红条:旋转 90 度后它会跑到侧面,方向一眼可辨
|
||||
d.rectangle([0, 0, w, max(2, h // 12)], fill=(230, 30, 30))
|
||||
for i in range(0, w, max(8, w // 20)):
|
||||
d.line([(i, h // 4), (i, h)], fill=(255, 255, 255), width=2)
|
||||
buf = io.BytesIO()
|
||||
if orientation:
|
||||
exif = im.getexif()
|
||||
exif[0x0112] = orientation
|
||||
im.save(buf, "JPEG", quality=90, exif=exif)
|
||||
else:
|
||||
im.save(buf, "JPEG", quality=90)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def scrape_profile_fields(html):
|
||||
"""从 profile 页抓取所有 input 的 name/value,保证回填必填项。
|
||||
|
||||
UpdateProfile 的必填字段有十来个(昵称/介绍/5 个二维码颜色/点大小/透明度…),
|
||||
漏一个就会因 `valid:"required"` 被 Bind 挡下、头像根本不会走到上传分支。
|
||||
"""
|
||||
fields = {}
|
||||
for m in re.finditer(r'<input\b[^>]*>', html, re.I):
|
||||
tag = m.group(0)
|
||||
nm = re.search(r'name="([^"]+)"', tag, re.I)
|
||||
if not nm:
|
||||
continue
|
||||
name = nm.group(1)
|
||||
if name in ("avatar", "background", "backgroundimage", "_csrf", "submit"):
|
||||
continue
|
||||
typ = re.search(r'type="([^"]+)"', tag, re.I)
|
||||
if typ and typ.group(1).lower() in ("checkbox", "radio"):
|
||||
# 只保留勾选态的,避免把「本想取消」的项又勾上
|
||||
if not re.search(r'\bchecked\b', tag, re.I):
|
||||
continue
|
||||
val = re.search(r'value="([^"]*)"', tag, re.I)
|
||||
fields[name] = val.group(1) if val else ""
|
||||
# select 走第一个非空 option
|
||||
for m in re.finditer(r'<select\b[^>]*name="([^"]+)"[^>]*>(.*?)</select>', html, re.I | re.S):
|
||||
name, body = m.group(1), m.group(2)
|
||||
opts = re.findall(r'<option\b([^>]*)>', body, re.I)
|
||||
chosen = ""
|
||||
for o in opts:
|
||||
vm = re.search(r'value="([^"]*)"', o)
|
||||
v = vm.group(1) if vm else ""
|
||||
if re.search(r'\bselected\b', o, re.I):
|
||||
chosen = v
|
||||
break
|
||||
if not chosen:
|
||||
chosen = v
|
||||
fields[name] = chosen
|
||||
# textarea
|
||||
for m in re.finditer(r'<textarea\b[^>]*name="([^"]+)"[^>]*>(.*?)</textarea>', html, re.I | re.S):
|
||||
fields[m.group(1)] = re.sub(r"<[^>]+>", "", m.group(2)).strip()
|
||||
return fields
|
||||
|
||||
|
||||
def upload_avatar(c, data, fname="avatar.jpg"):
|
||||
page = c.get("/user/profile")
|
||||
if page.status != 200:
|
||||
return None, "profile HTTP %s" % page.status
|
||||
fields = scrape_profile_fields(page.text)
|
||||
fields["_csrf"] = csrf_of(page.text)
|
||||
body, ct = multipart(fields, [("avatar", fname, "image/jpeg", data)])
|
||||
res = c.post("/user/profile/update", data=body, content_type=ct)
|
||||
# 成功时是 302 跳回 /user/profile,头像 URL 在跳转后的页面里
|
||||
if res.status == 302 and res.loc:
|
||||
res = c.get(res.loc if res.loc.startswith("/") else "/user/profile")
|
||||
if res.status != 200:
|
||||
return None, "update 后页面 HTTP %s" % res.status
|
||||
# 从页面里取头像 URL:<img src="https://r2.../picture/..."> 或 og:image
|
||||
urls = re.findall(r'https://r2\.[^"\'\s<>]+/picture/[0-9]+/\d{2}/\d{2}/[0-9a-f]+', res.text)
|
||||
if not urls:
|
||||
m = re.search(r'(https://r2\.[^"\'\s<>]+/picture/[^"\'\s<>]+)', res.text)
|
||||
urls = [m.group(1)] if m else []
|
||||
if urls:
|
||||
return urls[-1], None
|
||||
# 没匹配到:落盘返回页并打印可疑片段,便于定位是「上传被拒」还是「正则没覆盖」
|
||||
dump = "C:/Users/Tamakyi Gekka/AppData/Local/Temp/tbav_profile.html"
|
||||
with open(dump, "wb") as f:
|
||||
f.write(res.body)
|
||||
alerts = re.findall(r'class="uk-alert[^"]*"[^>]*>(.*?)</div>', res.text, re.S)
|
||||
hint = " | ".join(re.sub(r"\s+", " ", re.sub(r"<[^>]+>", " ", a)).strip()[:80]
|
||||
for a in alerts[:3])
|
||||
return None, "未匹配到 picture URL(已存 %s)%s%s" % (dump, (" 提示:" + hint) if hint else "",
|
||||
(" HTTP %s" % res.status))
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--base", default="https://box-dev-api.337845818.best")
|
||||
args = ap.parse_args()
|
||||
base = args.base.rstrip("/")
|
||||
pwd = "TbAv!2026"
|
||||
ts = time.strftime("%m%d%H%M%S")
|
||||
email, domain = "tbav%s@example.com" % ts, "tbav%s" % ts
|
||||
print("== 账号:%s / %s ==" % (email, domain))
|
||||
|
||||
# 方形 1600x1600:验证 A 分档(应存成 512x512)
|
||||
square = make_avatar(1600, 1600)
|
||||
# EXIF Orientation=6(顺时针 90):像素存成横的 1536x512,归一后显示为竖的 512x1536,
|
||||
# 再经头像档限边 512 -> 512x512。
|
||||
# 注意必须用「像素横躺」的样本:头像有 1:1 比例校验,若像素本身就是竖的会被先拒掉,
|
||||
# 那样就测不到 EXIF 分支了。
|
||||
exif_shot = make_avatar(1536, 512, orientation=6)
|
||||
print(" 本地样本 square 1600x1600%7d bytes" % len(square))
|
||||
print(" 本地样本 exifshot 1536x512 (EXIF=6)%d bytes" % len(exif_shot))
|
||||
|
||||
c = Client(base)
|
||||
r = c.get("/register")
|
||||
reg = c.post("/register", {
|
||||
"email": email, "domain": domain, "name": "TBAv%s" % ts,
|
||||
"password": pwd, "repeat_password": pwd,
|
||||
"g-recaptcha-response": captcha(c), "language": "zh-CN",
|
||||
"_csrf": csrf_of(r.text),
|
||||
})
|
||||
if not step("注册", reg.status == 302, "HTTP %s -> %s" % (reg.status, reg.loc)):
|
||||
return 1
|
||||
|
||||
c = Client(base)
|
||||
r = c.get("/login")
|
||||
lg = c.post("/login", {"email": email, "password": pwd,
|
||||
"g-recaptcha-response": captcha(c), "_csrf": csrf_of(r.text)})
|
||||
if not step("登录", lg.status == 302, "HTTP %s -> %s" % (lg.status, lg.loc)):
|
||||
return 1
|
||||
|
||||
# ---- A 分档 ----
|
||||
url, err = upload_avatar(c, square)
|
||||
if not step("上传方形大头像(1600x1600)", url is not None, err or (url or "")):
|
||||
return 1
|
||||
img = c.get_binary(url)
|
||||
st, data = img.status, img.body
|
||||
if not step("拉回头像", st == 200 and bool(data), "HTTP %s" % st):
|
||||
return 1
|
||||
kind, dims = sniff(data)
|
||||
print(" 存储结果:%s %s %d bytes" % (kind, dims, len(data)))
|
||||
step("A 分档生效:大图被缩到头像档 512(改前是 1920)",
|
||||
dims == (512, 512), "实际 %s" % (dims,))
|
||||
|
||||
# ---- B1 EXIF ----
|
||||
# 关键:两次上传的 URL 必须不同,否则说明第二次没生效(回退到了旧头像)。
|
||||
url2, err2 = upload_avatar(c, exif_shot)
|
||||
if not step("上传 EXIF 横躺头像(1536x512, Orientation=6)", bool(url2), err2 or (url2 or "")):
|
||||
return 0
|
||||
if url2 == url:
|
||||
step("B1 EXIF 归一(第二次上传未生效,URL 未变)", False,
|
||||
"两次上传返回同一 URL,无法区分;请检查是否被 1:1 校验拒绝")
|
||||
return 0
|
||||
img2 = c.get_binary(url2)
|
||||
kind2, dims2 = sniff(img2.body)
|
||||
print(" 存储结果:%s %s %d bytes" % (kind2, dims2, len(img2.body)))
|
||||
# 归一后:像素 1536x512 -> 旋转成 512x1536 -> 再限边 512 -> 512x512
|
||||
step("B1 EXIF Orientation=6 归一(1536x512 -> 512x1536 -> 512x512)",
|
||||
dims2 == (512, 512), "实际 %s" % (dims2,))
|
||||
# 体积对照:未归一时 1536x512 的 WebP 明显更大
|
||||
step("B1 归一后体积已按 512 档压过(应与方形样本同量级)",
|
||||
len(img2.body) < 6000, "%d bytes" % len(img2.body))
|
||||
|
||||
print("\n== 结果:%d/%d 通过 ==" % (sum(1 for _, ok, _ in RESULTS if ok), len(RESULTS)))
|
||||
print("清理:TamaBox ban --email %s --yes" % email)
|
||||
return 0 if all(ok for _, ok, _ in RESULTS) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,370 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""图片上传链路端到端验证(批次 A / B1 / B2)。
|
||||
|
||||
针对已部署环境验证三件事:
|
||||
B1 EXIF Orientation=6 的竖拍 JPEG 转换后必须变成竖图(核心验收点)
|
||||
B2 动画 GIF 必须被转成 .webp(cwebp 可用性 + 提问路径与头像路径一致性)
|
||||
A 转换后不比原图小时必须回退原图(用一张极小线框图验证)
|
||||
|
||||
做法:注册账号 -> 匿名提问(multipart 同时带三张图)-> 解析成功页里的 CDN URL
|
||||
-> 逐张拉回字节,解析真实容器格式与尺寸。
|
||||
|
||||
用法:
|
||||
python dev/devcheck-image.py [--base https://box-dev-api.337845818.best]
|
||||
|
||||
依赖:Pillow(仅用于在本地生成测试图片,不参与被测逻辑)。
|
||||
注意:会在目标环境真实创建账号并发送带图提问(测试邮箱前缀 tbimg<ts>@)。
|
||||
清理:TamaBox ban --email <该邮箱> --yes
|
||||
"""
|
||||
import argparse
|
||||
import http.cookiejar
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
||||
"(KHTML, like Gecko) Chrome/124.0 Safari/537.36")
|
||||
RESULTS = []
|
||||
CREATED_EMAIL = ""
|
||||
|
||||
|
||||
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: D102
|
||||
return None
|
||||
|
||||
|
||||
class Resp:
|
||||
def __init__(self, status, headers, body):
|
||||
self.status = status
|
||||
self.h = {k.lower(): v for k, v in headers.items()}
|
||||
self.body = body
|
||||
|
||||
@property
|
||||
def loc(self):
|
||||
return self.h.get("location", "")
|
||||
|
||||
@property
|
||||
def text(self):
|
||||
return self.body.decode("utf-8", "replace")
|
||||
|
||||
|
||||
class Client:
|
||||
def __init__(self, base):
|
||||
self.base = base
|
||||
self.cj = http.cookiejar.CookieJar()
|
||||
self.op = urllib.request.build_opener(
|
||||
urllib.request.HTTPCookieProcessor(self.cj), NoRedirect())
|
||||
|
||||
def _send(self, req, timeout=120):
|
||||
try:
|
||||
with self.op.open(req, timeout=timeout) as r:
|
||||
return Resp(r.status, dict(r.headers), r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
try:
|
||||
body = e.read()
|
||||
except Exception: # noqa: BLE001
|
||||
body = b""
|
||||
return Resp(e.code, dict(e.headers), body)
|
||||
except Exception as e: # noqa: BLE001
|
||||
return Resp(-1, {}, ("TRANSPORT_ERROR: %s" % e).encode())
|
||||
|
||||
def req(self, method, path, data=None, content_type=None, timeout=120):
|
||||
url = path if path.startswith("http") else self.base + path
|
||||
headers = {"User-Agent": UA, "Referer": self.base + path.split("?")[0]}
|
||||
body = None
|
||||
if data is not None:
|
||||
body = data if isinstance(data, bytes) else urllib.parse.urlencode(data).encode()
|
||||
headers["Content-Type"] = content_type or "application/x-www-form-urlencoded"
|
||||
return self._send(urllib.request.Request(url, data=body, headers=headers, method=method),
|
||||
timeout)
|
||||
|
||||
def get(self, p, **kw):
|
||||
r = self.req("GET", p, **kw)
|
||||
return r if r.status != -1 else self.req("GET", p, **kw)
|
||||
|
||||
def post(self, p, data=None, **kw):
|
||||
return self.req("POST", p, data=data, **kw)
|
||||
|
||||
def get_binary(self, url):
|
||||
return self._send(urllib.request.Request(
|
||||
url, headers={"User-Agent": UA}, method="GET"), timeout=60)
|
||||
|
||||
|
||||
def step(desc, ok, extra=""):
|
||||
RESULTS.append((desc, ok, extra))
|
||||
print(" [%s] %s%s" % ("PASS" if ok else "FAIL", desc, (" (%s)" % extra) if extra else ""))
|
||||
return ok
|
||||
|
||||
|
||||
def csrf_of(html):
|
||||
m = re.search(r'name="_csrf"\s+value="([^"]+)"', html)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def captcha(c):
|
||||
j = json.loads(c.get("/api/captcha/generate").text)
|
||||
pid, q = j["data"]["id"], j["data"]["question"]
|
||||
m = re.match(r"\s*(\d+)\s*([+\-×])\s*(\d+)\s*=", q)
|
||||
a, op, b = int(m.group(1)), m.group(2), int(m.group(3))
|
||||
return "%s:%d" % (pid, {"+": a + b, "-": a - b, "×": a * b}[op])
|
||||
|
||||
|
||||
def multipart(fields, files):
|
||||
boundary = "----tbimg%s" % uuid.uuid4().hex
|
||||
buf = io.BytesIO()
|
||||
for k, v in fields.items():
|
||||
buf.write(("--%s\r\n" % boundary).encode())
|
||||
buf.write(('Content-Disposition: form-data; name="%s"\r\n\r\n' % k).encode())
|
||||
buf.write(("%s\r\n" % v).encode())
|
||||
for name, fn, ct, data in files:
|
||||
buf.write(("--%s\r\n" % boundary).encode())
|
||||
buf.write(('Content-Disposition: form-data; name="%s"; filename="%s"\r\n' % (name, fn)).encode())
|
||||
buf.write(("Content-Type: %s\r\n\r\n" % ct).encode())
|
||||
buf.write(data)
|
||||
buf.write(b"\r\n")
|
||||
buf.write(("--%s--\r\n" % boundary).encode())
|
||||
return buf.getvalue(), "multipart/form-data; boundary=%s" % boundary
|
||||
|
||||
|
||||
# ---------- 容器格式解析(纯 stdlib) ----------
|
||||
|
||||
def sniff(data):
|
||||
"""返回 (kind, (w,h))。kind in {webp, jpeg, gif, png, unknown}"""
|
||||
if len(data) >= 12 and data[:4] == b"RIFF" and data[8:12] == b"WEBP":
|
||||
return "webp", webp_dims(data)
|
||||
if data[:3] == b"\xff\xd8\xff":
|
||||
return "jpeg", jpeg_dims(data)
|
||||
if data[:3] == b"GIF":
|
||||
return "gif", gif_dims(data)
|
||||
if data[:8] == b"\x89PNG\r\n\x1a\n":
|
||||
return "png", png_dims(data)
|
||||
return "unknown", None
|
||||
|
||||
|
||||
def webp_dims(d):
|
||||
c = d[12:16]
|
||||
if c == b"VP8 " and len(d) >= 30:
|
||||
return (struct.unpack("<H", d[26:28])[0] & 0x3FFF,
|
||||
struct.unpack("<H", d[28:30])[0] & 0x3FFF)
|
||||
if c == b"VP8L" and len(d) >= 25:
|
||||
b = struct.unpack("<I", d[21:25])[0]
|
||||
return (b & 0x3FFF) + 1, ((b >> 14) & 0x3FFF) + 1
|
||||
if c == b"VP8X" and len(d) >= 30:
|
||||
return ((d[24] | d[25] << 8 | d[26] << 16) + 1,
|
||||
(d[27] | d[28] << 8 | d[29] << 16) + 1)
|
||||
return None
|
||||
|
||||
|
||||
def jpeg_dims(d):
|
||||
i = 2
|
||||
while i + 9 < len(d):
|
||||
if d[i] != 0xFF:
|
||||
i += 1
|
||||
continue
|
||||
m = d[i + 1]
|
||||
if m in (0xC0, 0xC1, 0xC2, 0xC3):
|
||||
h, w = struct.unpack(">HH", d[i + 5:i + 9])
|
||||
return (w, h)
|
||||
if m in (0xD8, 0xD9) or 0xD0 <= m <= 0xD7:
|
||||
i += 2
|
||||
continue
|
||||
i += 2 + struct.unpack(">H", d[i + 2:i + 4])[0]
|
||||
return None
|
||||
|
||||
|
||||
def gif_dims(d):
|
||||
return struct.unpack("<HH", d[6:10]) if len(d) >= 10 else None
|
||||
|
||||
|
||||
def png_dims(d):
|
||||
return struct.unpack(">II", d[16:24]) if len(d) >= 24 else None
|
||||
|
||||
|
||||
# ---------- 测试图片生成(Pillow 仅此一处使用) ----------
|
||||
|
||||
def make_images():
|
||||
from PIL import Image, ImageDraw
|
||||
import io as _io
|
||||
|
||||
# B1:EXIF Orientation=6。像素存 240x120,正确归一后应显示 120x240。
|
||||
base = Image.new("RGB", (240, 120))
|
||||
dr = ImageDraw.Draw(base)
|
||||
for i in range(0, 240, 20):
|
||||
dr.line([(i, 0), (i, 120)], fill=(200, 30, 30), width=3)
|
||||
dr.rectangle([0, 0, 239, 119], outline=(0, 0, 255), width=4)
|
||||
exif = base.getexif()
|
||||
exif[0x0112] = 6
|
||||
buf = _io.BytesIO()
|
||||
base.save(buf, "JPEG", quality=88, exif=exif)
|
||||
portrait = buf.getvalue()
|
||||
|
||||
# B2:6 帧动画 GIF。加大到 640x480,确保转换后仍小于原图(否则会走回退)。
|
||||
frames = []
|
||||
pal_frames = []
|
||||
for f in range(6):
|
||||
im = Image.new("RGB", (640, 480), (240, 240, 240))
|
||||
d = ImageDraw.Draw(im)
|
||||
d.ellipse([40 + f * 60, 120, 200 + f * 60, 280], fill=(220, 40, 40))
|
||||
d.rectangle([300, 60 + f * 20, 600, 200], fill=(30, 90, 220))
|
||||
pal_frames.append(im.convert("P", palette=Image.ADAPTIVE, colors=64))
|
||||
gbuf = _io.BytesIO()
|
||||
pal_frames[0].save(gbuf, "GIF", save_all=True, append_images=pal_frames[1:],
|
||||
duration=120, loop=0)
|
||||
animated = gbuf.getvalue()
|
||||
|
||||
# A:极小线框图(200x200),WebP 有损编码后反而比 PNG 大 -> 应回退原图。
|
||||
wire = Image.new("RGB", (200, 200), (255, 255, 255))
|
||||
d = ImageDraw.Draw(wire)
|
||||
for i in range(0, 200, 20):
|
||||
d.line([(i, 0), (i, 199)], fill=(0, 0, 0))
|
||||
d.line([(0, i), (199, i)], fill=(0, 0, 0))
|
||||
wbuf = _io.BytesIO()
|
||||
wire.save(wbuf, "PNG")
|
||||
wireframe = wbuf.getvalue()
|
||||
|
||||
return {"portrait": portrait, "anim": animated, "wire": wireframe}
|
||||
|
||||
|
||||
def main():
|
||||
global CREATED_EMAIL
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--base", default="https://box-dev-api.337845818.best")
|
||||
args = ap.parse_args()
|
||||
base = args.base.rstrip("/")
|
||||
pwd = "TbImg!2026"
|
||||
ts = time.strftime("%m%d%H%M%S")
|
||||
email, domain = "tbimg%s@example.com" % ts, "tbimg%s" % ts
|
||||
CREATED_EMAIL = email
|
||||
print("== 账号:%s / %s ==" % (email, domain))
|
||||
|
||||
imgs = make_images()
|
||||
for k, v in imgs.items():
|
||||
print(" 本地样本 %-9s %7d bytes" % (k, len(v)))
|
||||
|
||||
c = Client(base)
|
||||
r = c.get("/register")
|
||||
reg = c.post("/register", {
|
||||
"email": email, "domain": domain, "name": "TBImg%s" % ts,
|
||||
"password": pwd, "repeat_password": pwd,
|
||||
"g-recaptcha-response": captcha(c), "language": "zh-CN",
|
||||
"_csrf": csrf_of(r.text),
|
||||
})
|
||||
if not step("注册", reg.status == 302, "HTTP %s -> %s" % (reg.status, reg.loc)):
|
||||
return 1
|
||||
|
||||
c = Client(base)
|
||||
r = c.get("/login")
|
||||
lg = c.post("/login", {"email": email, "password": pwd,
|
||||
"g-recaptcha-response": captcha(c), "_csrf": csrf_of(r.text)})
|
||||
if not step("登录", lg.status == 302, "HTTP %s -> %s" % (lg.status, lg.loc)):
|
||||
return 1
|
||||
|
||||
page = c.get("/_/%s" % domain)
|
||||
if not step("打开提问箱", page.status == 200, "HTTP %s" % page.status):
|
||||
return 1
|
||||
|
||||
# route/question/page.go 只取 f.Images[0](单张上限),
|
||||
# 故每张样本各发一条提问,逐张验证。
|
||||
fetched = []
|
||||
for key, (fname, ctype) in (("portrait", ("portrait.jpg", "image/jpeg")),
|
||||
("anim", ("anim.gif", "image/gif")),
|
||||
("wire", ("wire.png", "image/png"))):
|
||||
box = c.get("/_/%s" % domain)
|
||||
body, ct = multipart({
|
||||
"content": "图片上传链路验证 %s(可忽略)" % key,
|
||||
"g-recaptcha-response": captcha(c),
|
||||
"receive_reply_email": "", "private": "", "ask_password": "",
|
||||
"start_chat_val": "", "_csrf": csrf_of(box.text),
|
||||
}, [("images", fname, ctype, imgs[key])])
|
||||
|
||||
res = c.post("/_/%s" % domain, data=body, content_type=ct)
|
||||
if not step("提交带图提问 [%s]" % key, res.status in (200, 302),
|
||||
"HTTP %s -> %s" % (res.status, res.loc)):
|
||||
continue
|
||||
|
||||
# 成功页不渲染图片(templates/question/success.html 无图片区),
|
||||
# 故从成功页取出 question ID,再访问详情页取图片 key。
|
||||
qid = re.search(r'/_/%s/(\d+)' % re.escape(domain), res.text)
|
||||
if not qid:
|
||||
step(" └ 取提问 ID [%s]" % key, False, "成功页未给出 /_/<domain>/<id>")
|
||||
continue
|
||||
detail = c.get("/_/%s/%s" % (domain, qid.group(1)))
|
||||
# 提问图的 key 不带 picture/ 前缀(是 "<年>/<月>/<纳秒>.<ext>"),
|
||||
# 模板拼成 https://<CDN>/<key>,故按 CDN host + 数字开头路径来匹配。
|
||||
urls = sorted(set(re.findall(
|
||||
r'https://r2\.[^"\'\s<>]+/\d{4}/\d{1,2}/\d+\.\w+', detail.text)))
|
||||
if not urls:
|
||||
step(" └ 详情页回显图片 [%s]" % key, False,
|
||||
"HTTP %s,未找到 picture URL" % detail.status)
|
||||
continue
|
||||
step(" └ 详情页回显图片 [%s]" % key, True, "%d 个" % len(urls))
|
||||
|
||||
for u in urls:
|
||||
st, data = c.get_binary(u)
|
||||
if st != 200 or not data:
|
||||
step(" └ 拉回图片 [%s]" % key, False, "HTTP %s" % st)
|
||||
continue
|
||||
kind, dims = sniff(data)
|
||||
fetched.append((key, kind, dims, len(data)))
|
||||
print(" %-14s %-7s %-12s %7d bytes" % (u.rsplit("/", 1)[-1], kind, dims, len(data)))
|
||||
|
||||
|
||||
print("\n== 断言 ==")
|
||||
|
||||
# B1
|
||||
portrait = [f for f in fetched if f[0] == "portrait" and f[1] in ("webp", "jpeg")
|
||||
and f[2] == (120, 240)]
|
||||
land = [f for f in fetched if f[0] == "portrait" and f[1] in ("webp", "jpeg")
|
||||
and f[2] == (240, 120)]
|
||||
if portrait:
|
||||
step("B1 EXIF Orientation=6 已归一(240x120 -> 120x240)", True,
|
||||
"%s, %d bytes" % (portrait[0][1], portrait[0][3]))
|
||||
elif land:
|
||||
step("B1 EXIF Orientation=6 已归一", False, "仍是 240x120,说明未应用 EXIF 旋转")
|
||||
else:
|
||||
step("B1 EXIF 方向归一", False, "未匹配到 240x120 / 120x240 的图片,实际: %s" %
|
||||
[(f[1], f[2]) for f in fetched])
|
||||
|
||||
# B2:GIF 样本必须变成 webp;关键是按 key 归属判定,而不是看整体有没有 webp
|
||||
gif_res = [f for f in fetched if f[0] == "anim"]
|
||||
if not gif_res:
|
||||
step("B2 动画 GIF 转 WebP", False, "未取到 anim 样本结果")
|
||||
elif gif_res[0][1] == "webp":
|
||||
step("B2 动画 GIF 已转为 WebP", True,
|
||||
"%dx%d, %d bytes(原 GIF %d bytes,-%.0f%%)" % (
|
||||
gif_res[0][2][0], gif_res[0][2][1], gif_res[0][3], len(imgs["anim"]),
|
||||
(1 - gif_res[0][3] / len(imgs["anim"])) * 100))
|
||||
elif gif_res[0][1] == "gif":
|
||||
step("B2 动画 GIF 已转为 WebP", False,
|
||||
"仍是原始 GIF %d bytes -> cwebp 不可用或转换失败" % gif_res[0][3])
|
||||
else:
|
||||
step("B2 动画 GIF 转 WebP", False, "得到 %s(预期 webp)" % gif_res[0][1])
|
||||
|
||||
# A:线框图应回退原图(PNG),而非转成更大的 WebP
|
||||
wire = [f for f in fetched if f[0] == "wire"]
|
||||
if not wire:
|
||||
step("A 线框图兜底回退", False, "未取到 wire 样本结果")
|
||||
elif wire[0][1] == "png":
|
||||
step("A 转换无收益时回退原图(线框图保持 PNG)", True,
|
||||
"%d bytes(原 PNG %d bytes)" % (wire[0][3], len(imgs["wire"])))
|
||||
else:
|
||||
step("A 线框图兜底回退", False,
|
||||
"被转成了 %s(%d bytes)——本应回退原图" % (wire[0][1], wire[0][3]))
|
||||
|
||||
print("\n== 结果:%d/%d 通过 ==" % (sum(1 for _, ok, _ in RESULTS if ok), len(RESULTS)))
|
||||
print("清理:TamaBox ban --email %s --yes" % email)
|
||||
return 0 if all(ok for _, ok, _ in RESULTS) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -7,7 +7,6 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"io"
|
||||
"mime/multipart"
|
||||
)
|
||||
|
||||
@@ -35,7 +34,7 @@ func UploadBackgroundToS3(file multipart.File, fileHeader *multipart.FileHeader)
|
||||
return "", nil
|
||||
}
|
||||
|
||||
func ConvertStaticImageToWebP(r io.Reader, spec EncodeSpec) ([]byte, error) { return nil, nil }
|
||||
func ConvertStaticImageToWebP(data []byte, spec EncodeSpec) ([]byte, error) { return nil, nil }
|
||||
|
||||
func ConvertGifToAnimatedWebP(gifData []byte) ([]byte, error) { return nil, nil }
|
||||
|
||||
|
||||
@@ -63,17 +63,24 @@ const DefaultAspectTolerance = 0.01
|
||||
// 与 CheckPixelLimit 一致:只做 DecodeConfig 级轻量检查,不解码全图。
|
||||
// 注意:该方法会读取(消耗)传入的 reader,调用方需自行重建/复位 reader。
|
||||
func CheckAspectRatio(r io.Reader, wantW, wantH float64, tolerance float64) error {
|
||||
if wantW <= 0 || wantH <= 0 {
|
||||
return nil
|
||||
}
|
||||
cfg, _, err := image.DecodeConfig(r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if cfg.Width <= 0 || cfg.Height <= 0 {
|
||||
return CheckAspectRatioSize(cfg.Width, cfg.Height, wantW, wantH, tolerance)
|
||||
}
|
||||
|
||||
// CheckAspectRatioSize 是 CheckAspectRatio 的「宽高已知」版本。
|
||||
// 调用方若已从文件头拿到尺寸(或需要先按 EXIF Orientation 修正宽高后再比对),
|
||||
// 用本函数可避免重复读取文件头。
|
||||
func CheckAspectRatioSize(w, h int, wantW, wantH float64, tolerance float64) error {
|
||||
if wantW <= 0 || wantH <= 0 {
|
||||
return nil
|
||||
}
|
||||
actual := float64(cfg.Width) / float64(cfg.Height)
|
||||
if w <= 0 || h <= 0 {
|
||||
return nil
|
||||
}
|
||||
actual := float64(w) / float64(h)
|
||||
if math.Abs(actual-wantW/wantH) > tolerance {
|
||||
return ErrBadAspectRatio
|
||||
}
|
||||
|
||||
@@ -106,6 +106,16 @@ func tiffOrientation(tiff []byte) int {
|
||||
return 0
|
||||
}
|
||||
|
||||
// orientationSwapsDimensions 报告该 Orientation 是否会交换宽高。
|
||||
// 2/3/4 只是镜像或 180 度旋转,宽高不变;5/6/7/8 会把横躺的像素转正,宽高互换。
|
||||
func orientationSwapsDimensions(orientation int) bool {
|
||||
switch orientation {
|
||||
case exifOrientLeftTop, exifOrientRightTop, exifOrientRightBottom, exifOrientLeftBottom:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// orientedImage 把源图按 EXIF Orientation 重映射为目标坐标,供 draw.Draw
|
||||
// 一次性拷贝,避免逐像素重复计算源矩形。
|
||||
type orientedImage struct {
|
||||
@@ -158,9 +168,8 @@ func applyJPEGOrientation(img image.Image, orientation int) image.Image {
|
||||
return img
|
||||
}
|
||||
dstW, dstH := sw, sh
|
||||
switch orientation {
|
||||
case exifOrientLeftTop, exifOrientRightTop, exifOrientRightBottom, exifOrientLeftBottom:
|
||||
dstW, dstH = sh, sw // 这四种会交换宽高
|
||||
if orientationSwapsDimensions(orientation) {
|
||||
dstW, dstH = sh, sw
|
||||
}
|
||||
|
||||
dst := image.NewRGBA(image.Rect(0, 0, dstW, dstH))
|
||||
|
||||
@@ -208,7 +208,7 @@ func TestConvertStaticImageToWebPAppliesEXIFOrientation(t *testing.T) {
|
||||
t.Fatalf("raw jpeg dims = %dx%d, want 40x30", cfg.Width, cfg.Height)
|
||||
}
|
||||
|
||||
out, err := ConvertStaticImageToWebP(bytes.NewReader(data), EncodeSpec{MaxSide: 1600, Quality: 80})
|
||||
out, err := ConvertStaticImageToWebP(data, EncodeSpec{MaxSide: 1600, Quality: 80})
|
||||
if err != nil {
|
||||
t.Fatalf("ConvertStaticImageToWebP: %v", err)
|
||||
}
|
||||
@@ -220,3 +220,50 @@ func TestConvertStaticImageToWebPAppliesEXIFOrientation(t *testing.T) {
|
||||
t.Fatalf("webp dims = %dx%d, want 30x40 (orientation 6 applied)", w, h)
|
||||
}
|
||||
}
|
||||
|
||||
// 回归:比例校验必须与转码时的 EXIF 归一用同一套方向。
|
||||
//
|
||||
// 手机竖拍照片存成 40x30 像素但 EXIF 声明 Orientation=6,肉眼看到的是 30x40 的竖图。
|
||||
// 若比例校验只看原始像素宽高(30/40 的横躺比例),一张合规的方形头像会被判为
|
||||
// 「比例不符」而拒——这正是线上验证曾发现的缺口。
|
||||
func TestAspectCheckAppliesEXIFOrientation(t *testing.T) {
|
||||
// 像素 40x30 + Orientation=6 → 归一后 30x40(竖图 3:4,不是 1:1)
|
||||
if err := (&aspect{W: 1, H: 1}).check(image.Config{Width: 40, Height: 30}, exifOrientRightTop); err == nil {
|
||||
t.Fatal("40x30 + ori6 归一后是 3:4 竖图,头像应判为比例不符")
|
||||
}
|
||||
|
||||
// 像素 40x30 + Orientation=1 → 仍是 3:4 竖图,同样应拒
|
||||
if err := (&aspect{W: 1, H: 1}).check(image.Config{Width: 40, Height: 30}, exifOrientTopLeft); err == nil {
|
||||
t.Fatal("40x30 ori1 是 3:4 竖图,头像应判为比例不符")
|
||||
}
|
||||
|
||||
// 像素 40x40 + Orientation=6 → 归一后仍 40x40,头像应放行
|
||||
if err := (&aspect{W: 1, H: 1}).check(image.Config{Width: 40, Height: 40}, exifOrientRightTop); err != nil {
|
||||
t.Fatalf("40x40 + ori6 应放行: %v", err)
|
||||
}
|
||||
|
||||
// 背景 16:9:像素 160x90 + Orientation=6 → 归一后 90x160(9:16)应拒
|
||||
if err := (&aspect{W: 16, H: 9}).check(image.Config{Width: 160, Height: 90}, exifOrientRightTop); err == nil {
|
||||
t.Fatal("160x90 + ori6 归一后是 9:16 竖图,背景应判为比例不符")
|
||||
}
|
||||
|
||||
// nil aspect 表示该用途不校验比例(如通用图)
|
||||
if err := (*aspect)(nil).check(image.Config{Width: 4000, Height: 3}, exifOrientRightTop); err != nil {
|
||||
t.Fatalf("nil aspect 不应校验: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 回归:EXIF 归一会交换宽高的四种方向,与不交换的三种。
|
||||
func TestOrientationSwapsDimensions(t *testing.T) {
|
||||
swaps := map[int]bool{
|
||||
exifOrientTopLeft: false, exifOrientTopRight: false,
|
||||
exifOrientBottomRight: false, exifOrientBottomLeft: false,
|
||||
exifOrientLeftTop: true, exifOrientRightTop: true,
|
||||
exifOrientRightBottom: true, exifOrientLeftBottom: true,
|
||||
}
|
||||
for ori, want := range swaps {
|
||||
if got := orientationSwapsDimensions(ori); got != want {
|
||||
t.Errorf("orientationSwapsDimensions(%d) = %v, want %v", ori, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+35
-34
@@ -43,11 +43,19 @@ type aspect struct {
|
||||
W, H float64
|
||||
}
|
||||
|
||||
func (a *aspect) check(data []byte) error {
|
||||
// check 校验给定尺寸是否满足本用途的宽高比约束。
|
||||
// orientation 是 EXIF Orientation:手机竖拍照片解出的像素是横躺的(5/6/7/8 会
|
||||
// 交换宽高),若直接用原始像素宽高比对,合规的竖图会被误判为比例不符而拒。
|
||||
// 修正只交换宽高、不解码像素,大图内存不受影响。
|
||||
func (a *aspect) check(cfg image.Config, orientation int) error {
|
||||
if a == nil {
|
||||
return nil
|
||||
}
|
||||
return imagecheck.CheckAspectRatio(bytes.NewReader(data), a.W, a.H, imagecheck.DefaultAspectTolerance)
|
||||
w, h := cfg.Width, cfg.Height
|
||||
if orientationSwapsDimensions(orientation) {
|
||||
w, h = h, w
|
||||
}
|
||||
return imagecheck.CheckAspectRatioSize(w, h, a.W, a.H, imagecheck.DefaultAspectTolerance)
|
||||
}
|
||||
|
||||
// EncodeSpec 描述一次上传的 WebP 编码档位。
|
||||
@@ -107,31 +115,31 @@ func uploadPictureToS3(file multipart.File, fileHeader *multipart.FileHeader, as
|
||||
return "", errors.New("unsupported file type: not a valid image")
|
||||
}
|
||||
|
||||
// 2.5 用途宽高比校验(头像 1:1 / 背景 16:9),绕过前端裁剪的直传在此被拦截
|
||||
if err := asp.check(fileData); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
fileReader := bytes.NewReader(fileData)
|
||||
|
||||
// 3. 解码前先校验像素总量(防解压炸弹),再解码图片获取格式;解码失败则直接返回错误。
|
||||
if err := imagecheck.CheckPixelLimit(bytes.NewReader(fileData)); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
_, format, err := image.Decode(fileReader)
|
||||
// 3. 只解析文件头:一次拿到格式与宽高,供像素上限校验、比例校验、
|
||||
// WebP 透传判断共用。image.DecodeConfig 不解像素,因此这里不产生解码开销。
|
||||
// (此前为拿 format 先 image.Decode 一遍、转换时再解一遍,同一张图解码两次,
|
||||
// 4096×4096 上限下峰值内存翻倍,1C1G 机器并发上传有 OOM 风险。)
|
||||
cfg, format, err := image.DecodeConfig(bytes.NewReader(fileData))
|
||||
if err != nil {
|
||||
return "", errors.Wrap(err, "image decode failed, file may be corrupted")
|
||||
}
|
||||
|
||||
// 3.5 像素总量校验(防解压炸弹),必须在任何完整解码之前。
|
||||
if cfg.Width > 0 && cfg.Height > 0 &&
|
||||
int64(cfg.Width)*int64(cfg.Height) > imagecheck.MaxImagePixels {
|
||||
return "", imagecheck.ErrImageTooLarge
|
||||
}
|
||||
|
||||
// 3.6 用途宽高比校验(头像 1:1 / 背景 16:9),绕过前端裁剪的直传在此被拦截。
|
||||
// 比例校验必须与转码时的 EXIF 归一保持同一套方向,否则手机竖拍照片
|
||||
// (像素横躺 + Orientation=6)会被误判为比例不符。
|
||||
if err := asp.check(cfg, jpegOrientation(fileData)); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
var uploadData []byte
|
||||
var contentType string
|
||||
|
||||
// 重置 reader,以便后续可能的 resize 操作
|
||||
if _, err := fileReader.Seek(0, io.SeekStart); err != nil {
|
||||
return "", errors.Wrap(err, "reset reader")
|
||||
}
|
||||
|
||||
// 4. 按格式转换或直接使用原图
|
||||
switch format {
|
||||
case "gif":
|
||||
@@ -146,13 +154,12 @@ func uploadPictureToS3(file multipart.File, fileHeader *multipart.FileHeader, as
|
||||
case "webp":
|
||||
// 已是 WebP:仅当超尺寸上限时才重编(重编会再丢一代画质),
|
||||
// 尺寸合规则原样上传。
|
||||
cfg, _, err := image.DecodeConfig(bytes.NewReader(fileData))
|
||||
if err == nil && fitsWithin(cfg.Width, cfg.Height, spec.MaxSide) {
|
||||
if fitsWithin(cfg.Width, cfg.Height, spec.MaxSide) {
|
||||
uploadData = fileData
|
||||
contentType = "image/webp"
|
||||
break
|
||||
}
|
||||
webpData, err := ConvertStaticImageToWebP(fileReader, spec)
|
||||
webpData, err := ConvertStaticImageToWebP(fileData, spec)
|
||||
if err != nil || len(webpData) >= len(fileData) {
|
||||
uploadData = fileData
|
||||
contentType = mimeTypeFromFormat(format)
|
||||
@@ -162,7 +169,7 @@ func uploadPictureToS3(file multipart.File, fileHeader *multipart.FileHeader, as
|
||||
}
|
||||
default:
|
||||
// 静态图片(JPEG / PNG 等)转静态 WebP
|
||||
webpData, err := ConvertStaticImageToWebP(fileReader, spec)
|
||||
webpData, err := ConvertStaticImageToWebP(fileData, spec)
|
||||
if err != nil || len(webpData) >= len(fileData) {
|
||||
// 转换失败、或转换后并不比原图小(小图标/线框图/已优化 JPEG
|
||||
// 上有损 WebP 可能反而更大)→ 一律沿用原图。
|
||||
@@ -211,16 +218,10 @@ func uploadPictureToS3(file multipart.File, fileHeader *multipart.FileHeader, as
|
||||
// ConvertStaticImageToWebP 将静态图片(JPEG/PNG 等)按给定档位转换为 WebP 格式。
|
||||
// spec.MaxSide 同时约束宽与高(此前只限宽,竖图长边不受约束)。
|
||||
// 调用方负责比较转换前后的体积,决定是否采用转换结果。
|
||||
func ConvertStaticImageToWebP(r io.Reader, spec EncodeSpec) ([]byte, error) {
|
||||
// 先整体读入并校验像素总量(防解压炸弹),再完整解码。
|
||||
data, err := io.ReadAll(r)
|
||||
if err != nil {
|
||||
return nil, errors.Wrap(err, "read image")
|
||||
}
|
||||
if err := imagecheck.CheckPixelLimit(bytes.NewReader(data)); err != nil {
|
||||
return nil, errors.Wrap(err, "check image pixel limit")
|
||||
}
|
||||
|
||||
//
|
||||
// 接收已读入的字节而非 io.Reader:调用方手里已有原始字节,避免再包一层
|
||||
// bytes.Reader 重复持有整份数据。
|
||||
func ConvertStaticImageToWebP(data []byte, spec EncodeSpec) ([]byte, error) {
|
||||
img, _, err := image.Decode(bytes.NewReader(data))
|
||||
if err != nil {
|
||||
return nil, errors.Wrap(err, "decode image")
|
||||
|
||||
+17
-15
@@ -668,6 +668,11 @@ type uploadImageOptions struct {
|
||||
|
||||
var ErrUploadImageSizeTooLarge = errors.New("图片文件大小不能大于 5Mb")
|
||||
|
||||
// fitsWithinSide 判断图片宽高是否都已在 maxSide 的边长限制内。
|
||||
func fitsWithinSide(cfg image.Config, maxSide int) bool {
|
||||
return cfg.Width > 0 && cfg.Height > 0 && cfg.Width <= maxSide && cfg.Height <= maxSide
|
||||
}
|
||||
|
||||
func uploadImage(ctx context.Context, opts uploadImageOptions) error {
|
||||
imgFile := opts.Image
|
||||
fileName := imgFile.Filename
|
||||
@@ -696,17 +701,16 @@ func uploadImage(ctx context.Context, opts uploadImageOptions) error {
|
||||
var contentType string
|
||||
var uploadKey string
|
||||
|
||||
// 检测原始格式(解码前先校验像素总量,防止解压炸弹)
|
||||
if err := imagecheck.CheckPixelLimit(bytes.NewReader(originalData)); err != nil {
|
||||
if errors.Is(err, imagecheck.ErrImageTooLarge) {
|
||||
return imagecheck.ErrImageTooLarge
|
||||
}
|
||||
// 检测原始格式(只解析文件头,不解像素):既拿到 format 也拿到宽高,
|
||||
// 供像素总量校验与 WebP 透传判断共用。此前此处 image.Decode 完整解码一遍
|
||||
// 只为取 format,转换时还要再解一遍,峰值内存翻倍。
|
||||
cfg, format, err := image.DecodeConfig(bytes.NewReader(originalData))
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "decode image config")
|
||||
}
|
||||
|
||||
_, format, err := image.Decode(bytes.NewReader(originalData))
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "decode image")
|
||||
if cfg.Width > 0 && cfg.Height > 0 &&
|
||||
int64(cfg.Width)*int64(cfg.Height) > imagecheck.MaxImagePixels {
|
||||
return imagecheck.ErrImageTooLarge
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
@@ -728,19 +732,17 @@ func uploadImage(ctx context.Context, opts uploadImageOptions) error {
|
||||
uploadKey = fmt.Sprintf("%d/%d/%d%s", now.Year(), now.Month(), now.UnixNano(), ".webp")
|
||||
}
|
||||
default:
|
||||
// JPEG / PNG 等静态图片转换为 WebP。这里只取 DecodeConfig(不波纹解像素),
|
||||
// 避免在像素量已经 CheckPixelLimit 校验过的超大图上再解码一遍。
|
||||
cfg, _, cfgErr := image.DecodeConfig(bytes.NewReader(originalData))
|
||||
// JPEG / PNG 等静态图片转换为 WebP。
|
||||
// 已是 WebP 且尺寸合规 -> 原样上传:再编码会白丢一代有损画质。
|
||||
if format == "webp" && cfgErr == nil &&
|
||||
cfg.Width <= storage.GeneralEncodeSpec.MaxSide && cfg.Height <= storage.GeneralEncodeSpec.MaxSide {
|
||||
if format == "webp" &&
|
||||
fitsWithinSide(cfg, storage.GeneralEncodeSpec.MaxSide) {
|
||||
uploadData = originalData
|
||||
contentType = "image/webp"
|
||||
uploadKey = fmt.Sprintf("%d/%d/%d%s", now.Year(), now.Month(), now.UnixNano(), fileExt)
|
||||
break
|
||||
}
|
||||
|
||||
webpData, err := storage.ConvertStaticImageToWebP(bytes.NewReader(originalData), storage.GeneralEncodeSpec)
|
||||
webpData, err := storage.ConvertStaticImageToWebP(originalData, storage.GeneralEncodeSpec)
|
||||
if err != nil || len(webpData) >= len(originalData) {
|
||||
// 转换失败,或转换后并不比原图小(小图标/线框图/已优化 JPEG 上有损 WebP
|
||||
// 可能反而更大)—— 一律沿用原图。
|
||||
|
||||
Reference in New Issue
Block a user