退信核查按原因分类:只有限流/临时性退信才剔除补发
问题:退信核查此前凡是退信一律从断点清单剔除、下次重跑补发。 但退信分两类——「您的账号外发频率超过邮件系统限制」这类限流退信重发 可成功;「地址不存在」这类硬退信重发也发不出去,只会浪费每日发信额度、 拖垮发信信誉。 改动: - 新增退信分类 rate/hard/unknown:优先用 DSN 状态码 5.x.x/4.x.x 判定, 同一封退信里的多个收件人可分别归类;无 DSN 时回退正文关键词匹配 (硬退信优先于限流,宁可不补发可疑地址) - --prune-state 只剔除「可重试」退信;永久失败不剔除,即不再补发 - 新增无效地址清单 broadcast_invalid.json:硬退信地址记入后每次运行 直接跳过(含 --reset-state),附 --reset-invalid / --no-invalid-list - 新增 --prune-unknown / --invalid-file / --bounce-report 参数 - 新增退信明细报告 broadcast_bounce_report.json(含每个收件人判定依据) - 修正 _part_text 取正文的两个解码坑:utf-8 正文默认 base64 传输编码未 解码、str 形态 payload 走 raw-unicode-escape 致中文变 \uXXXX,两者都会 让中文关键词匹配静默失效 - 补充退信主题关键词(delivery failed、未能送达、无法送达 等) - README 补充退信分类表与无效地址清单说明 - 新增 .gitignore,移除误入库的 __pycache__/broadcast.cpython-313.pyc
This commit is contained in:
+11
@@ -0,0 +1,11 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
|
||||
# 运行时生成的状态/报告文件(含真实用户邮箱,不入库)
|
||||
broadcast_state.json
|
||||
broadcast_report.json
|
||||
broadcast_invalid.json
|
||||
broadcast_bounce_report.json
|
||||
broadcast_bounces_seen.json
|
||||
*.tmp
|
||||
@@ -17,6 +17,9 @@ README — TamaBox 站内信群发工具(mail-broadcast)
|
||||
single.subject.txt 单文件模式主题(一行)
|
||||
broadcast_state.json (运行后生成)断点续发记录,已发邮箱重跑自动跳过
|
||||
broadcast_report.json (运行后生成)每次运行的监控报告
|
||||
broadcast_invalid.json (运行后生成)无效地址清单:硬退信地址,永久跳过
|
||||
broadcast_bounce_report.json (运行后生成)退信核查明细(含分类与判定依据)
|
||||
broadcast_bounces_seen.json (运行后生成)已核查退信的 Message-ID,避免重复报告
|
||||
|
||||
参数文件 params.ini
|
||||
-------------------
|
||||
@@ -135,22 +138,40 @@ conf/app.ini 路径的解析优先级:
|
||||
# 单文件模式:所有人发 templates/single.html(不按语言)
|
||||
python3 broadcast.py --single --send --to you@example.com
|
||||
|
||||
退信核查(验证真实送达)
|
||||
------------------------
|
||||
退信核查(验证真实送达 · 按原因分类处理)
|
||||
----------------------------------------
|
||||
背景:SMTP 返回 250 只代表服务器收下了,不代表送达。阿里云企业邮箱等对
|
||||
「发送频率超限」「收件人不存在」等情况往往是先收下、再异步把退信通知投到
|
||||
发件账号的收件箱——脚本当时的「发送成功」并不真实。
|
||||
|
||||
# 发送完过几分钟,扫描近 3 天的退信通知,列出实际未送达的收件人(仅核查)
|
||||
**退信不是一类,必须分类处理**(v3 起):
|
||||
|
||||
| 类别 | 典型原因 | 脚本动作 |
|
||||
| ---- | -------- | -------- |
|
||||
| 可重试 `rate` | 「您的账号外发频率超过邮件系统限制」、系统繁忙、DSN 4.x.x | 从断点清单剔除 → 下次/本次补发 |
|
||||
| 永久失败 `hard` | 地址不存在、用户不存在、DSN 5.x.x | **不补发**;记入无效地址清单,永久跳过 |
|
||||
| 未分类 `unknown` | 判不出来 | 默认不补发(保守);`--prune-unknown` 可强制补发 |
|
||||
|
||||
以前是「凡是退信一律剔除补发」,结果**地址不存在的死信也会被反复重发**——
|
||||
既浪费每日发信额度,又拖垮发信信誉。现在只有真正可重试的才会补发。
|
||||
|
||||
判定优先级:DSN 状态码(5.x.x / 4.x.x,最权威,且**同一封退信里的多个收件人
|
||||
可分别归类**)→ 正文硬退信关键词 → 限流/临时性关键词 → 正文 SMTP 状态码 → 未分类。
|
||||
硬退信优先于限流:宁可少补发一个可疑地址,也不给死信反复重发。
|
||||
|
||||
# 发送完过几分钟,扫描退信并分类报告(仅核查,不改动任何清单)
|
||||
python3 broadcast.py --check-bounces
|
||||
|
||||
# 仅剔除:把这些用户从断点续发清单剔除后结束,本次不发送
|
||||
#(下次任意一次重跑会自动补上这批人)
|
||||
# 仅剔除:把「可重试」的退信从断点续发清单剔除后结束,本次不发送
|
||||
#(下次任意一次重跑会自动补上这批人;地址不存在的不会补发)
|
||||
python3 broadcast.py --check-bounces --prune-state
|
||||
|
||||
# 剔除并补发:剔除后继续正常发送流程,本次就把退信用户补上
|
||||
# 剔除并补发:剔除后继续正常发送流程,本次就把可重试的用户补上
|
||||
python3 broadcast.py --check-bounces --prune-state --send
|
||||
|
||||
# 未分类的也按可重试一并剔除补发(确认不是死信后再用)
|
||||
python3 broadcast.py --check-bounces --prune-state --prune-unknown
|
||||
|
||||
# 可选:--imap-host imap.xxx.com(默认由 SMTP 域名推导 smtp.→imap.)
|
||||
# --imap-port 993;扫描起点 --since auto(默认,见下)
|
||||
扫描起点(避免把群发无关的退信算进来):
|
||||
@@ -158,11 +179,24 @@ conf/app.ini 路径的解析优先级:
|
||||
时间开始——即只核查本次群发发出的那些邮件的退信
|
||||
- --since 2026-09-07 可显式指定起点日期
|
||||
- 断点清单为空时回退为 --since-days N(默认 3 天)
|
||||
|
||||
无效地址清单 broadcast_invalid.json
|
||||
-----------------------------------
|
||||
硬退信(地址不存在等)的地址会写进这里,之后**每次运行都直接跳过**,
|
||||
即使加了 --reset-state 也不会再发——重发也发不出去,只会浪费额度。
|
||||
条目里保留了退信原因、主题与时间,便于核对到底是哪些地址失效了。
|
||||
|
||||
python3 broadcast.py --reset-invalid # 清空清单(确认地址已修正后用)
|
||||
python3 broadcast.py --no-invalid-list ... # 本次忽略清单(既不跳过也不写入)
|
||||
python3 broadcast.py --invalid-file /path/to/other.json # 换一个清单文件
|
||||
|
||||
说明:
|
||||
- 用的是 conf/app.ini [mail] 的账号与密码(需邮箱已开启 IMAP,密码为邮箱登录密码)
|
||||
- 交互模式的运行方式「4) 退信核查」里同样三选一:仅核查 / 仅剔除 / 剔除并补发
|
||||
- 交互模式的运行方式「4) 退信核查」里同样三选一:仅核查 / 仅剔除 / 剔除并补发,
|
||||
选择「仅剔除 / 剔除并补发」时会再问一次是否连未分类的一起补发
|
||||
- 已核查过的退信(按 Message-ID)记录在 broadcast_bounces_seen.json,
|
||||
重复核查不会重复报告/重复剔除
|
||||
- 每次核查另写一份明细报告 broadcast_bounce_report.json(含每个收件人的类别与判定依据)
|
||||
|
||||
安全机制
|
||||
--------
|
||||
|
||||
Binary file not shown.
+397
-68
@@ -30,6 +30,8 @@ TamaBox 站内信群发工具(独立项目,零依赖,不依赖、不修改
|
||||
- 断点续发:状态文件记录已发邮箱,重跑自动跳过;--reset-state 清空重发
|
||||
- 监控报告:每次运行写 JSON 报告(--report / --no-report)
|
||||
- SMTP 拒收捕获:send_message 返回的 refused 计入失败
|
||||
- 退信分类核查:只有「限流/临时性」退信才剔除补发;「地址不存在」这类
|
||||
永久失败不补发,并记入无效地址清单(broadcast_invalid.json)永久跳过
|
||||
|
||||
用法示例:
|
||||
# 1) 演练:列出收件人 + 统计(不发信)
|
||||
@@ -74,13 +76,15 @@ from email.mime.text import MIMEText
|
||||
from email.utils import formataddr, make_msgid
|
||||
from urllib.parse import quote_plus
|
||||
|
||||
SCRIPT_VERSION = "2026-09-07.bouncecheck.v2"
|
||||
SCRIPT_VERSION = "2026-09-07.bounceclass.v3"
|
||||
|
||||
DEFAULT_STATE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_state.json")
|
||||
DEFAULT_REPORT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_report.json")
|
||||
DEFAULT_TEMPLATES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "templates")
|
||||
DEFAULT_PARAMS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "params.ini")
|
||||
DEFAULT_BOUNCE_SEEN = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_bounces_seen.json")
|
||||
DEFAULT_INVALID = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_invalid.json")
|
||||
DEFAULT_BOUNCE_REPORT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_bounce_report.json")
|
||||
|
||||
# 语言展示标签(未知语言直接显示语言码)
|
||||
LANG_LABEL = {
|
||||
@@ -671,13 +675,22 @@ def reset_state(state_path):
|
||||
# ---------------------------------------------------------------------------
|
||||
# 退信核查(IMAP):SMTP 250 ≠ 真正送达。企业邮箱(如阿里云)对限流、不存在的
|
||||
# 收件人常「先收下、后异步退信」,退信通知会投到发件账号的收件箱。这里用标准库
|
||||
# imaplib 登录 IMAP 扫描退信,解析出真正投递失败的收件人;配合 --prune-state
|
||||
# 把这些用户从断点续发清单剔除,下次重跑自动补发。
|
||||
# imaplib 登录 IMAP 扫描退信,解析出真正投递失败的收件人。
|
||||
#
|
||||
# 关键:退信不是一类,必须分类处理——
|
||||
# rate(限流/临时性,如「您的账号外发频率超过邮件系统限制」)
|
||||
# → 稍后重发大概率成功:从断点续发清单剔除,下次重跑自动补发
|
||||
# hard(永久失败,如「收件人地址不存在」/ DSN 5.x.x)
|
||||
# → 重发也发不出去,只会浪费额度、拖垮发信信誉:
|
||||
# 不从断点清单剔除(即不补发),并记入无效地址清单永久跳过
|
||||
# unknown(判不出来)→ 默认不剔除(保守),可用 --prune-unknown 强制剔除
|
||||
# ---------------------------------------------------------------------------
|
||||
BOUNCE_SUBJECT_KEYWORDS = (
|
||||
"退信", "投递失败", "无法投递", "邮件被退回", "投递状态", "发送失败",
|
||||
"undeliver", "delivery status", "returned mail", "failure notice",
|
||||
"mail delivery subsystem", "mail could not be delivered", "delivery failure",
|
||||
"退信", "投递失败", "无法投递", "未能送达", "无法送达", "邮件被退回",
|
||||
"被退回", "退回通知", "投递状态", "发送失败", "发送不成功",
|
||||
"undeliver", "delivery status", "delivery failed", "delivery failure",
|
||||
"returned mail", "failure notice", "not delivered",
|
||||
"mail delivery subsystem", "mail could not be delivered",
|
||||
)
|
||||
EMAIL_RE = re.compile(r"[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}")
|
||||
# 中文退信常见「标签:地址」写法 + RFC DSN 的 Final-Recipient 等
|
||||
@@ -691,6 +704,50 @@ LABELED_PATTERNS = (
|
||||
)
|
||||
SYSTEM_LOCALPARTS = ("mailer-daemon", "postmaster", "mail-daemon", "noreply", "no-reply")
|
||||
|
||||
# --- 退信分类关键词 ---------------------------------------------------------
|
||||
# 判定优先级:DSN 状态码(5.x.x/4.x.x,最权威) > 硬退信关键词 > 限流关键词 > 未知
|
||||
CAT_RATE = "rate" # 限流/临时性失败 → 可重试,剔除后补发
|
||||
CAT_HARD = "hard" # 地址不存在等永久失败 → 不补发,记入无效地址清单
|
||||
CAT_UNKNOWN = "unknown" # 判不出来 → 默认不剔除(保守),--prune-unknown 可强制
|
||||
|
||||
CAT_LABEL = {
|
||||
CAT_RATE: "可重试 · 限流/临时性",
|
||||
CAT_HARD: "永久失败 · 地址无效",
|
||||
CAT_UNKNOWN: "未分类(保守:不补发)",
|
||||
}
|
||||
|
||||
# 限流 / 临时性失败(可稍后重试)
|
||||
RATE_LIMIT_KEYWORDS = (
|
||||
"您的账号外发频率超过邮件系统限制", # 阿里云企业邮箱限流退信原文
|
||||
"外发频率超过", "发送频率超过", "超过邮件系统限制", "发送频率受限",
|
||||
"发送流量超限", "超出发送限额", "发送数量超限", "超出邮件系统限制",
|
||||
"系统繁忙", "稍后重试", "稍后重新发送", "请稍后再试", "请稍后重新发送",
|
||||
"rate limit", "too many", "throttl", "try again later", "try later",
|
||||
"too many recipients", "temporarily", "temporary failure", "temporary local",
|
||||
"greylist", "grey list", "deferred", "deferral", "connection timed out",
|
||||
"try again", "please retry", "later time",
|
||||
)
|
||||
|
||||
# 永久失败:收件人/邮箱根本不存在
|
||||
HARD_BOUNCE_KEYWORDS = (
|
||||
"地址不存在", "用户不存在", "无此用户", "账号不存在", "帐户不存在",
|
||||
"收件人不存在", "不存在该用户", "邮箱不存在", "没有这个邮箱",
|
||||
"没有此邮箱", "查无此用户", "无效地址", "无效的收件人", "无效的收件地址",
|
||||
"收件人地址错误", "该邮箱不存在", "未知的用户",
|
||||
"no such user", "no such recipient", "no such mailbox", "no such address",
|
||||
"user unknown", "unknown user", "unknown recipient", "unknown address",
|
||||
"recipient address rejected", "recipient not found", "recipient rejected",
|
||||
"mailbox unavailable", "mailbox not found", "mailbox does not exist",
|
||||
"address does not exist", "address not found", "address rejected",
|
||||
"invalid recipient", "invalid address", "invalid mailbox",
|
||||
"bad destination", "bad address syntax", "user not found",
|
||||
"does not exist", "550 5.1.1", "no mailbox here", "mailbox disabled",
|
||||
)
|
||||
|
||||
# SMTP 状态码识别:增强状态码 5.1.1 / 三位码 550
|
||||
_SMTP_ENHANCED_RE = re.compile(r"\b([245])\.(\d{1,3})\.(\d{1,3})\b")
|
||||
_SMTP_CODE_RE = re.compile(r"(?<![\d.])[245]\d{2}(?![\d.])")
|
||||
|
||||
|
||||
def _decode_hdr(value):
|
||||
"""解码 MIME 编码的邮件头(Subject 等)为可读文本。"""
|
||||
@@ -705,6 +762,30 @@ def _decode_hdr(value):
|
||||
return str(value)
|
||||
|
||||
|
||||
def _part_payload_text(part):
|
||||
"""取单个 part 的文本。
|
||||
|
||||
两个坑都要避开:
|
||||
- 直接 get_payload(decode=True) 遇到 str 形态的非 ASCII payload 会走
|
||||
raw-unicode-escape,中文变成 \\uXXXX 字面串,关键词匹配全部失效;
|
||||
- 反过来只用 get_payload() 又可能拿到 base64/QP 的原文(utf-8 正文默认
|
||||
base64 传输编码),同样匹配不到。
|
||||
所以:非 ASCII 的 str 直接用(已解码),其余交给 decode=True 解 base64/QP。
|
||||
"""
|
||||
try:
|
||||
raw = part.get_payload()
|
||||
if isinstance(raw, str) and not raw.isascii():
|
||||
return raw
|
||||
raw = part.get_payload(decode=True)
|
||||
if isinstance(raw, bytes):
|
||||
return raw.decode(part.get_content_charset() or "utf-8", "replace")
|
||||
if isinstance(raw, str):
|
||||
return raw
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
return ""
|
||||
|
||||
|
||||
def _part_text(msg, limit=30000):
|
||||
"""抽取邮件正文文本(plain / html / DSN 原始块),拼接后截断。"""
|
||||
chunks = []
|
||||
@@ -712,12 +793,7 @@ def _part_text(msg, limit=30000):
|
||||
for part in msg.walk():
|
||||
ct = part.get_content_type()
|
||||
if ct in ("text/plain", "text/html"):
|
||||
try:
|
||||
payload = part.get_payload(decode=True)
|
||||
if payload:
|
||||
chunks.append(payload.decode(part.get_content_charset() or "utf-8", "replace"))
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
chunks.append(_part_payload_text(part))
|
||||
elif ct == "message/delivery-status":
|
||||
blocks = part.get_payload()
|
||||
if isinstance(blocks, list):
|
||||
@@ -730,17 +806,20 @@ def _part_text(msg, limit=30000):
|
||||
if sum(len(c) for c in chunks) > limit:
|
||||
break
|
||||
else:
|
||||
try:
|
||||
payload = msg.get_payload(decode=True)
|
||||
if payload:
|
||||
chunks.append(payload.decode(msg.get_content_charset() or "utf-8", "replace"))
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
return "\n".join(chunks)[:limit]
|
||||
chunks.append(_part_payload_text(msg))
|
||||
return "\n".join(c for c in chunks if c)[:limit]
|
||||
|
||||
|
||||
def _dsn_failed_recipients(msg):
|
||||
"""标准 DSN(message/delivery-status)中 Action: failed 的收件人。"""
|
||||
# DSN 里表示「已送达」的 Action,遇到就跳过(其余 failed/delayed/expired 都算失败)
|
||||
DSN_OK_ACTIONS = ("delivered", "relayed", "expanded")
|
||||
|
||||
|
||||
def _dsn_recipient_details(msg):
|
||||
"""标准 DSN(message/delivery-status)中每个收件人的失败详情。
|
||||
|
||||
返回 [{email, action, status, diagnostic}],逐个带自己的 Status /
|
||||
Diagnostic-Code,因此一封退信里的多个收件人可以分别归类。
|
||||
"""
|
||||
out = []
|
||||
for part in msg.walk():
|
||||
if part.get_content_type() != "message/delivery-status":
|
||||
@@ -751,15 +830,69 @@ def _dsn_failed_recipients(msg):
|
||||
for blk in blocks:
|
||||
try:
|
||||
final = str(blk.get("Final-Recipient") or blk.get("Original-Recipient") or "")
|
||||
action = str(blk.get("Action") or "failed").lower()
|
||||
action = str(blk.get("Action") or "failed").strip().lower()
|
||||
status = str(blk.get("Status") or "").strip()
|
||||
diagnostic = str(blk.get("Diagnostic-Code") or "").strip()
|
||||
except Exception: # noqa: BLE001
|
||||
continue
|
||||
if action in DSN_OK_ACTIONS:
|
||||
continue
|
||||
m = EMAIL_RE.search(final)
|
||||
if m and "failed" in action:
|
||||
out.append(m.group(0).strip(".").lower())
|
||||
if not m:
|
||||
continue
|
||||
out.append({
|
||||
"email": m.group(0).strip(".").lower(),
|
||||
"action": action,
|
||||
"status": status,
|
||||
"diagnostic": diagnostic,
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def _smtp_code_class(text):
|
||||
"""从 DSN Status / Diagnostic-Code 取 SMTP 状态码首位:5=永久、4=临时、2=成功。"""
|
||||
if not text:
|
||||
return None
|
||||
m = _SMTP_ENHANCED_RE.search(text) # 增强状态码,如 5.1.1 / 4.4.7
|
||||
if m:
|
||||
return m.group(1)
|
||||
m = _SMTP_CODE_RE.search(text) # 三位码,如 550 / 451
|
||||
if m:
|
||||
return m.group(0)[0]
|
||||
return None
|
||||
|
||||
|
||||
def classify_bounce(status="", diagnostic="", text=""):
|
||||
"""判定退信类别,返回 (类别, 判定依据)。
|
||||
|
||||
顺序:DSN 状态码(最权威)→ 正文/诊断信息里的硬退信关键词 → 限流关键词
|
||||
→ 正文里的 SMTP 状态码 → unknown。
|
||||
硬退信优先于限流:宁可不补发一个可疑地址,也不要反复给死信浪费额度。
|
||||
"""
|
||||
for src in (status, diagnostic):
|
||||
c = _smtp_code_class(src or "")
|
||||
if c == "5":
|
||||
return CAT_HARD, f"DSN 状态码 {(src or '').strip()[:40]}"
|
||||
if c == "4":
|
||||
return CAT_RATE, f"DSN 状态码 {(src or '').strip()[:40]}"
|
||||
|
||||
body = "\n".join(x for x in (text, diagnostic) if x)
|
||||
low = body.lower()
|
||||
for kw in HARD_BOUNCE_KEYWORDS:
|
||||
if kw.lower() in low:
|
||||
return CAT_HARD, f"关键词「{kw}」"
|
||||
for kw in RATE_LIMIT_KEYWORDS:
|
||||
if kw.lower() in low:
|
||||
return CAT_RATE, f"关键词「{kw}」"
|
||||
|
||||
c = _smtp_code_class(text or "")
|
||||
if c == "5":
|
||||
return CAT_HARD, "正文 SMTP 5xx"
|
||||
if c == "4":
|
||||
return CAT_RATE, "正文 SMTP 4xx"
|
||||
return CAT_UNKNOWN, ""
|
||||
|
||||
|
||||
def _own_ok(addr, own_addr):
|
||||
a = addr.strip(".").lower()
|
||||
if not a or a == own_addr.lower():
|
||||
@@ -781,28 +914,40 @@ def is_bounce_message(msg):
|
||||
|
||||
|
||||
def extract_bounced_recipients(msg, own_addr):
|
||||
"""从退信中解析投递失败的收件人列表(已去重、剔除发件人与系统地址)。"""
|
||||
"""从退信中解析投递失败的收件人,并逐个分类。
|
||||
|
||||
返回 [{email, category, reason, detail}](已去重、剔除发件人与系统地址)。
|
||||
"""
|
||||
if not is_bounce_message(msg):
|
||||
return []
|
||||
recipients = []
|
||||
# 1) 标准 DSN 最可靠
|
||||
for r in _dsn_failed_recipients(msg):
|
||||
if _own_ok(r, own_addr) and r not in recipients:
|
||||
recipients.append(r)
|
||||
# 2) 中文退信的「标签:地址」写法
|
||||
if not recipients:
|
||||
|
||||
found = []
|
||||
|
||||
def add(em, category, reason, detail=""):
|
||||
em = (em or "").strip().strip(".").lower()
|
||||
if not _own_ok(em, own_addr):
|
||||
return
|
||||
if any(x["email"] == em for x in found):
|
||||
return
|
||||
found.append({"email": em, "category": category,
|
||||
"reason": reason, "detail": (detail or "")[:160]})
|
||||
|
||||
# 1) 标准 DSN 最可靠:每个收件人带自己的 Status/Diagnostic-Code,可逐个分类
|
||||
for d in _dsn_recipient_details(msg):
|
||||
cat, reason = classify_bounce(d["status"], d["diagnostic"], "")
|
||||
add(d["email"], cat, reason, d["status"] or d["diagnostic"])
|
||||
|
||||
# 2) 中文退信的「标签:地址」写法 / 3) 正文兜底:整封共用一个分类结果
|
||||
if not found:
|
||||
text = _part_text(msg)
|
||||
cat, reason = classify_bounce("", "", text)
|
||||
for pat in LABELED_PATTERNS:
|
||||
for m in re.finditer(pat, text, re.IGNORECASE):
|
||||
e = m.group("e")
|
||||
if _own_ok(e, own_addr) and e.lower() not in recipients:
|
||||
recipients.append(e.lower())
|
||||
# 3) 兜底:正文里出现的其他地址(剔除发件人自身与 daemon/postmaster)
|
||||
if not recipients:
|
||||
for e in EMAIL_RE.findall(_part_text(msg)):
|
||||
if _own_ok(e, own_addr) and e.lower() not in recipients:
|
||||
recipients.append(e.lower())
|
||||
return recipients
|
||||
add(m.group("e"), cat, reason, "")
|
||||
if not found:
|
||||
for e in EMAIL_RE.findall(text):
|
||||
add(e, cat, reason, "")
|
||||
return found
|
||||
|
||||
|
||||
def prune_state(state_path, emails):
|
||||
@@ -834,6 +979,88 @@ def prune_state(state_path, emails):
|
||||
return removed
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 无效地址清单:硬退信(收件人不存在 / DSN 5.x.x)的地址记在这里。
|
||||
# 这些地址重发也发不出去,所以:
|
||||
# - 不从断点续发清单剔除(即不会「补发」)
|
||||
# - 记入本清单,后续每次运行(即使 --reset-state)都会直接跳过
|
||||
# 只有确实要恢复发送时,才手动删条目或 --reset-invalid 清空。
|
||||
# ---------------------------------------------------------------------------
|
||||
def load_invalid(path):
|
||||
"""读取无效地址清单,返回 {email: {"reason","detail","subject","date","ts"}}。"""
|
||||
if not os.path.isfile(path):
|
||||
return {}
|
||||
try:
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
raw = json.load(f)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"[警告] 读取无效地址清单失败,按空清单处理:{e}")
|
||||
return {}
|
||||
out = {}
|
||||
if isinstance(raw, dict):
|
||||
items = raw.items()
|
||||
elif isinstance(raw, list):
|
||||
items = [(it.get("email"), it) for it in raw if isinstance(it, dict)]
|
||||
else:
|
||||
return out
|
||||
for em, info in items:
|
||||
em = str(em or "").strip().lower()
|
||||
if not em:
|
||||
continue
|
||||
info = info if isinstance(info, dict) else {"reason": str(info)}
|
||||
out[em] = {
|
||||
"reason": str(info.get("reason") or ""),
|
||||
"detail": str(info.get("detail") or ""),
|
||||
"subject": str(info.get("subject") or ""),
|
||||
"date": str(info.get("date") or ""),
|
||||
"ts": str(info.get("ts") or ""),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def record_invalid(path, entries):
|
||||
"""合并写入无效地址清单。entries: {email: {reason, detail, subject, date}}。
|
||||
返回新增条数(已有条目只更新原因)。"""
|
||||
if not entries:
|
||||
return 0
|
||||
data = load_invalid(path)
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
added = 0
|
||||
for em, info in entries.items():
|
||||
em = (em or "").strip().lower()
|
||||
if not em:
|
||||
continue
|
||||
if em not in data:
|
||||
added += 1
|
||||
data[em] = {
|
||||
"reason": str(info.get("reason") or ""),
|
||||
"detail": str(info.get("detail") or ""),
|
||||
"subject": str(info.get("subject") or ""),
|
||||
"date": str(info.get("date") or ""),
|
||||
"ts": now,
|
||||
}
|
||||
try:
|
||||
tmp = path + ".tmp"
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump({k: data[k] for k in sorted(data)}, f, ensure_ascii=False, indent=2)
|
||||
os.replace(tmp, path)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"[警告] 写入无效地址清单失败:{e}")
|
||||
return 0
|
||||
return added
|
||||
|
||||
|
||||
def reset_invalid(path):
|
||||
"""清空无效地址清单(慎用:这些地址多半仍是不可达的)。"""
|
||||
try:
|
||||
if os.path.isfile(path):
|
||||
os.remove(path)
|
||||
return True
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"[警告] 清除无效地址清单失败:{e}")
|
||||
return False
|
||||
|
||||
|
||||
def resolve_scan_start(args):
|
||||
"""退信扫描起点:--since 显式日期 > auto 时取断点清单最早一条发送记录的
|
||||
时间(精确到该时刻,不加余量)> 无记录时回退 --since-days。
|
||||
@@ -857,8 +1084,10 @@ def resolve_scan_start(args):
|
||||
|
||||
def check_bounces(cfg, args):
|
||||
"""登录发件邮箱 IMAP,扫描近 N 天的退信并解析失败收件人。
|
||||
返回 {email: (subject, date_str)};已处理过的邮件(Message-ID)记录在
|
||||
broadcast_bounces_seen.json,重复核查不会重复报告。"""
|
||||
|
||||
返回 {email: {"subject","date","category","reason","detail"}};已处理过的
|
||||
邮件(Message-ID)记录在 broadcast_bounces_seen.json,重复核查不会重复报告。
|
||||
category 见 CAT_RATE / CAT_HARD / CAT_UNKNOWN。"""
|
||||
smtp_host = cfg["mail_smtp"] or ""
|
||||
imap_host = (args.imap_host or "").strip()
|
||||
if not imap_host:
|
||||
@@ -893,7 +1122,7 @@ def check_bounces(cfg, args):
|
||||
except Exception: # noqa: BLE001
|
||||
seen = set()
|
||||
|
||||
bounced = {} # email -> (subject, date)
|
||||
bounced = {} # email -> {"subject","date","category","reason","detail"}
|
||||
scanned, bounce_cnt, newly_seen = 0, 0, []
|
||||
for i, mid in enumerate(ids, 1):
|
||||
uid = mid.decode() if isinstance(mid, bytes) else str(mid)
|
||||
@@ -924,33 +1153,93 @@ def check_bounces(cfg, args):
|
||||
bounce_cnt += 1
|
||||
newly_seen.append(msg_id)
|
||||
for r in rcpts:
|
||||
bounced.setdefault(r, (subject, date))
|
||||
print(f" [退信 {i}/{len(ids)}] {subject[:40]} → {', '.join(rcpts) if rcpts else '(未解析出收件人)'}")
|
||||
bounced.setdefault(r["email"], {
|
||||
"subject": subject, "date": date,
|
||||
"category": r["category"], "reason": r["reason"], "detail": r["detail"],
|
||||
})
|
||||
if rcpts:
|
||||
desc = " ".join(
|
||||
f"{r['email']}[{CAT_LABEL.get(r['category'], r['category'])}]" for r in rcpts
|
||||
)
|
||||
else:
|
||||
desc = "(未解析出收件人)"
|
||||
print(f" [退信 {i}/{len(ids)}] {subject[:40]} → {desc}")
|
||||
|
||||
try:
|
||||
conn.logout()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
# ---- 按类别分组处理:只有「可重试」的才剔除补发 ----
|
||||
groups = {CAT_RATE: [], CAT_HARD: [], CAT_UNKNOWN: []}
|
||||
for em, info in bounced.items():
|
||||
groups.setdefault(info["category"], []).append(em)
|
||||
prune_cats = [CAT_RATE] + ([CAT_UNKNOWN] if getattr(args, "prune_unknown", False) else [])
|
||||
|
||||
print("\n========================================================")
|
||||
print(f"退信核查完成:扫描 {scanned} 封未处理邮件,其中退信 {bounce_cnt} 封,"
|
||||
f"涉及 {len(bounced)} 个收件人。")
|
||||
if bounced:
|
||||
for r, (subj, dt) in sorted(bounced.items()):
|
||||
print(f" - {r} ({dt} · {subj[:36]})")
|
||||
print("\n这些用户实际未收到邮件。")
|
||||
for cat in (CAT_RATE, CAT_HARD, CAT_UNKNOWN):
|
||||
ems = sorted(groups.get(cat, []))
|
||||
if not ems:
|
||||
continue
|
||||
print(f"\n [{CAT_LABEL[cat]}] {len(ems)} 个:")
|
||||
for em in ems:
|
||||
info = bounced[em]
|
||||
why = info["reason"] or "未识别到明确原因"
|
||||
meta = " · ".join(x for x in (info["date"], info["subject"][:32], why) if x)
|
||||
print(f" - {em} ({meta})")
|
||||
|
||||
if not bounced:
|
||||
print("未发现新的退信,此前发送均正常送达(至少未被退回)。")
|
||||
else:
|
||||
# 硬退信:不补发 + 记入无效地址清单永久跳过
|
||||
hard = sorted(groups.get(CAT_HARD, []))
|
||||
if hard and not getattr(args, "no_invalid_list", False):
|
||||
n = record_invalid(
|
||||
args.invalid_file,
|
||||
{em: bounced[em] for em in hard},
|
||||
)
|
||||
print(f"\n[无效地址] {len(hard)} 个永久失败地址已记入 {args.invalid_file}"
|
||||
f"(新增 {n} 条);这些地址后续运行(含 --reset-state)会直接跳过,不再浪费额度。")
|
||||
print(" 确认地址已修正后,删掉该文件或 --reset-invalid 可恢复发送。")
|
||||
elif hard:
|
||||
print(f"\n[无效地址] {len(hard)} 个永久失败地址未写入清单(--no-invalid-list),"
|
||||
f"但它们不会被补发。")
|
||||
|
||||
# 可重试:从断点清单剔除,下次/本次补发
|
||||
retryable = [em for cat in prune_cats for em in groups.get(cat, [])]
|
||||
if args.prune_state:
|
||||
n = prune_state(args.state, bounced.keys())
|
||||
print(f"[剔除] 已从断点续发清单剔除 {n} 条记录:{args.state}")
|
||||
n = prune_state(args.state, retryable)
|
||||
print(f"[剔除] 已从断点续发清单剔除 {n} 条「可重试」记录:{args.state}")
|
||||
if getattr(args, "send", False):
|
||||
print(" 本次发送将继续进行,这些用户会被包含在内(剔除并补发)。")
|
||||
else:
|
||||
print(" 本次不发送;下次重跑(同参数)会自动给这些用户补发。")
|
||||
else:
|
||||
print("可选后续:--prune-state 仅剔除不发送;--prune-state --send 剔除后立即补发;")
|
||||
print(" 或交互模式运行方式 4 里选择对应核查方式。")
|
||||
else:
|
||||
print("未发现新的退信,此前发送均正常送达(至少未被退回)。")
|
||||
elif retryable:
|
||||
print(f"\n[提示] 有 {len(retryable)} 个「可重试」退信等待补发。可选:"
|
||||
f"--prune-state 仅剔除;--prune-state --send 剔除后立即补发。")
|
||||
unknown = groups.get(CAT_UNKNOWN, [])
|
||||
if unknown and not getattr(args, "prune_unknown", False):
|
||||
print(f"[提示] 有 {len(unknown)} 个退信未能分类,默认不补发(保守)。"
|
||||
f"确认是限流后可加 --prune-unknown 一并剔除补发。")
|
||||
|
||||
# 退信核查明细报告(便于事后核对,不受 --no-report 影响)
|
||||
try:
|
||||
with open(args.bounce_report, "w", encoding="utf-8") as f:
|
||||
json.dump({
|
||||
"generated_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"script_version": SCRIPT_VERSION,
|
||||
"scan_from": start_dt.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"scan_from_desc": start_desc,
|
||||
"scanned": scanned,
|
||||
"bounce_messages": bounce_cnt,
|
||||
"counts": {c: len(groups.get(c, [])) for c in (CAT_RATE, CAT_HARD, CAT_UNKNOWN)},
|
||||
"recipients": {em: bounced[em] for em in sorted(bounced)},
|
||||
}, f, ensure_ascii=False, indent=2)
|
||||
print(f"\n[退信报告] 已写入:{args.bounce_report}")
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"[警告] 写入退信报告失败(不影响本次结果):{e}")
|
||||
|
||||
if newly_seen:
|
||||
try:
|
||||
@@ -1106,10 +1395,11 @@ def run_interactive(cfg, templates, extra_vars, args, config_path):
|
||||
|
||||
if mode == "4":
|
||||
args.check_bounces = True
|
||||
print("\n退信核查方式:")
|
||||
print(" 1) 仅核查 —— 只报告哪些用户没收到,不改动断点清单(默认)")
|
||||
print(" 2) 仅剔除 —— 把退信用户从断点清单剔除后结束,本次不发送")
|
||||
print(" 3) 剔除并补发 —— 剔除后继续正常发送流程(发送前仍有统计与确认)")
|
||||
print("\n退信核查方式(会先按原因分类:限流/临时性 → 可重试;地址不存在 → 永久失败):")
|
||||
print(" 1) 仅核查 —— 只报告哪些用户没收到、各自什么原因,不改动任何清单(默认)")
|
||||
print(" 2) 仅剔除 —— 只把「限流等可重试」的退信从断点清单剔除后结束,本次不发送")
|
||||
print(" (「地址不存在」的不会剔除,另记入无效地址清单永久跳过)")
|
||||
print(" 3) 剔除并补发 —— 同上剔除后继续正常发送流程(发送前仍有统计与确认)")
|
||||
while True:
|
||||
try:
|
||||
sub = input("核查方式 [1/2/3](直接回车 = 1): ").strip()
|
||||
@@ -1118,11 +1408,16 @@ def run_interactive(cfg, templates, extra_vars, args, config_path):
|
||||
if sub in ("", "1", "2", "3"):
|
||||
break
|
||||
print(" 请输入 1、2 或 3")
|
||||
if sub == "2":
|
||||
args.prune_state = True
|
||||
elif sub == "3":
|
||||
if sub in ("2", "3"):
|
||||
args.prune_state = True
|
||||
if sub == "3":
|
||||
args.send = True
|
||||
if args.prune_state:
|
||||
try:
|
||||
unk = input("未分类的退信是否也按「可重试」一并剔除补发?(y/N): ").strip().lower()
|
||||
except EOFError:
|
||||
unk = ""
|
||||
args.prune_unknown = unk in ("y", "yes")
|
||||
return site_url, box_prefix, extra_vars
|
||||
|
||||
# ---- 模板模式:多语言 / 单文件通用 ----
|
||||
@@ -1238,8 +1533,21 @@ def main():
|
||||
help="退信核查:登录发件邮箱 IMAP 扫描退信通知,找出实际未送达的收件人"
|
||||
"(SMTP 250 不代表送达;限流等退信是异步投到发件箱的)")
|
||||
parser.add_argument("--prune-state", action="store_true",
|
||||
help="配合 --check-bounces:把退信用户从断点续发清单剔除后结束(仅剔除,"
|
||||
"本次不发送);再加 --send 则剔除后立即补发")
|
||||
help="配合 --check-bounces:把「可重试」退信(限流/临时性)从断点续发清单"
|
||||
"剔除后结束(仅剔除,本次不发送);再加 --send 则剔除后立即补发。"
|
||||
"「地址不存在」等永久失败不会被剔除(不补发)")
|
||||
parser.add_argument("--prune-unknown", action="store_true",
|
||||
help="配合 --check-bounces:无法分类的退信也按「可重试」处理,一并剔除补发"
|
||||
"(默认不剔除,避免给死信地址反复重发)")
|
||||
parser.add_argument("--invalid-file", default=DEFAULT_INVALID, metavar="FILE",
|
||||
help=f"无效地址清单(硬退信地址)路径;默认脚本同目录 broadcast_invalid.json。"
|
||||
f"清单里的地址每次运行都会跳过,即使 --reset-state")
|
||||
parser.add_argument("--reset-invalid", action="store_true",
|
||||
help="清空无效地址清单(慎用:这些地址多半仍不可达)")
|
||||
parser.add_argument("--no-invalid-list", action="store_true",
|
||||
help="本次不使用无效地址清单:既不跳过已知无效地址,也不写入新的硬退信")
|
||||
parser.add_argument("--bounce-report", default=DEFAULT_BOUNCE_REPORT, metavar="FILE",
|
||||
help="退信核查明细报告路径(JSON);默认脚本同目录 broadcast_bounce_report.json")
|
||||
parser.add_argument("--imap-host", default="",
|
||||
help="IMAP 服务器地址(默认由 SMTP 域名推导:smtp.xxx → imap.xxx)")
|
||||
parser.add_argument("--imap-port", type=int, default=993,
|
||||
@@ -1267,6 +1575,13 @@ def main():
|
||||
args.interactive = not (args.send or args.to or args.yes or args.dry_run
|
||||
or args.check_bounces)
|
||||
|
||||
# ---- 无效地址清单维护(--reset-invalid):手动恢复发送硬退信地址时用 ----
|
||||
if args.reset_invalid:
|
||||
if reset_invalid(args.invalid_file):
|
||||
print(f"[无效地址清单] 已清空:{args.invalid_file}")
|
||||
else:
|
||||
print(f"[无效地址清单] 文件不存在,无需清空:{args.invalid_file}")
|
||||
|
||||
# 载入模板
|
||||
templates, lang_list = load_templates(args.templates_dir)
|
||||
single_tpl = load_single_template(args.templates_dir)
|
||||
@@ -1401,6 +1716,19 @@ def main():
|
||||
recipients = fetch_recipients(cfg, args, lang_expr=lang_expr, tbl=users_table)
|
||||
counts = fetch_counts(cfg, args, tbl=users_table)
|
||||
|
||||
# 跳过历史硬退信地址(收件人不存在 / DSN 5.x.x)——重发也发不出去,
|
||||
# 且不受 --reset-state 影响;只有 --no-invalid-list 或清空清单才会恢复。
|
||||
invalid_skipped = 0
|
||||
if not args.no_invalid_list:
|
||||
invalid_map = load_invalid(args.invalid_file)
|
||||
if invalid_map:
|
||||
before = len(recipients)
|
||||
recipients = [r for r in recipients
|
||||
if (r["email"] or "").strip().lower() not in invalid_map]
|
||||
invalid_skipped = before - len(recipients)
|
||||
print(f" [无效地址清单] 已载入 {len(invalid_map)} 条"
|
||||
f"({args.invalid_file}),本次跳过 {invalid_skipped} 个地址")
|
||||
|
||||
print("数据库用户统计:")
|
||||
print(f" 总用户数(含已注销) : {counts['total']}")
|
||||
print(f" 已注销用户数 : {counts['deactivated']}")
|
||||
@@ -1496,6 +1824,7 @@ def main():
|
||||
"failed": 0,
|
||||
"smtp_rejected": 0,
|
||||
"skipped_already_sent": skipped,
|
||||
"skipped_invalid": invalid_skipped,
|
||||
"failed_list": [],
|
||||
"missing_placeholders": [],
|
||||
"test_mode": bool(args.to),
|
||||
@@ -1646,9 +1975,9 @@ def main():
|
||||
print(f"[注意] 模板中有未提供值的占位符(已替换为空串):{', '.join(sorted(all_missing))}")
|
||||
if not args.to and args.send:
|
||||
print(f"断点续发状态已写入:{args.state}(重跑脚本会自动跳过已发用户)")
|
||||
print("[提示] SMTP 250 不代表一定送达(限流常被异步退回发件箱)。"
|
||||
print("[提示] SMTP 250 不代表一定送达(限流/无效地址常被异步退回发件箱)。"
|
||||
"发送完过几分钟可运行「python3 broadcast.py --check-bounces --prune-state」"
|
||||
"核查退信并让失败用户下次自动补发。")
|
||||
"核查退信:限流类会自动剔除补发,地址不存在类记入无效清单不再重发。")
|
||||
if failed:
|
||||
print("失败清单:")
|
||||
for e, err in failed:
|
||||
|
||||
Reference in New Issue
Block a user