Compare commits

...
2 Commits
Author SHA1 Message Date
tamakyi 909e853500 gitignore: 忽略 .workbuddy 项目数据目录 2026-09-07 18:30:04 +08:00
tamakyi bc971deb37 退信核查按原因分类:只有限流/临时性退信才剔除补发
问题:退信核查此前凡是退信一律从断点清单剔除、下次重跑补发。
但退信分两类——「您的账号外发频率超过邮件系统限制」这类限流退信重发
可成功;「地址不存在」这类硬退信重发也发不出去,只会浪费每日发信额度、
拖垮发信信誉。

改动:
- 新增退信分类 rate/hard/unknown:优先用 DSN 状态码 5.x.x/4.x.x 判定,
  同一封退信里的多个收件人可分别归类;无 DSN 时回退正文关键词匹配
  (硬退信优先于限流,宁可不补发可疑地址)
- --prune-state 只剔除「可重试」退信;永久失败不剔除,即不再补发
- 新增无效地址清单 broadcast_invalid.json:硬退信地址记入后每次运行
  直接跳过(含 --reset-state),附 --reset-invalid / --no-invalid-list
- 新增 --prune-unknown / --invalid-file / --bounce-report 参数
- 新增退信明细报告 broadcast_bounce_report.json(含每个收件人判定依据)
- 修正 _part_text 取正文的两个解码坑:utf-8 正文默认 base64 传输编码未
  解码、str 形态 payload 走 raw-unicode-escape 致中文变 \uXXXX,两者都会
  让中文关键词匹配静默失效
- 补充退信主题关键词(delivery failed、未能送达、无法送达 等)
- README 补充退信分类表与无效地址清单说明
- 新增 .gitignore,移除误入库的 __pycache__/broadcast.cpython-313.pyc
2026-09-07 18:26:09 +08:00
4 changed files with 452 additions and 75 deletions
+14
View File
@@ -0,0 +1,14 @@
# Python
__pycache__/
*.py[cod]
# WorkBuddy 项目数据(会话记忆等)
.workbuddy/
# 运行时生成的状态/报告文件(含真实用户邮箱,不入库)
broadcast_state.json
broadcast_report.json
broadcast_invalid.json
broadcast_bounce_report.json
broadcast_bounces_seen.json
*.tmp
+41 -7
View File
@@ -17,6 +17,9 @@ README — TamaBox 站内信群发工具(mail-broadcast
single.subject.txt 单文件模式主题(一行) single.subject.txt 单文件模式主题(一行)
broadcast_state.json (运行后生成)断点续发记录,已发邮箱重跑自动跳过 broadcast_state.json (运行后生成)断点续发记录,已发邮箱重跑自动跳过
broadcast_report.json (运行后生成)每次运行的监控报告 broadcast_report.json (运行后生成)每次运行的监控报告
broadcast_invalid.json (运行后生成)无效地址清单:硬退信地址,永久跳过
broadcast_bounce_report.json (运行后生成)退信核查明细(含分类与判定依据)
broadcast_bounces_seen.json (运行后生成)已核查退信的 Message-ID,避免重复报告
参数文件 params.ini 参数文件 params.ini
------------------- -------------------
@@ -135,22 +138,40 @@ conf/app.ini 路径的解析优先级:
# 单文件模式:所有人发 templates/single.html(不按语言) # 单文件模式:所有人发 templates/single.html(不按语言)
python3 broadcast.py --single --send --to you@example.com python3 broadcast.py --single --send --to you@example.com
退信核查(验证真实送达) 退信核查(验证真实送达 · 按原因分类处理
------------------------ ----------------------------------------
背景:SMTP 返回 250 只代表服务器收下了,不代表送达。阿里云企业邮箱等对 背景:SMTP 返回 250 只代表服务器收下了,不代表送达。阿里云企业邮箱等对
「发送频率超限」「收件人不存在」等情况往往是先收下、再异步把退信通知投到 「发送频率超限」「收件人不存在」等情况往往是先收下、再异步把退信通知投到
发件账号的收件箱——脚本当时的「发送成功」并不真实。 发件账号的收件箱——脚本当时的「发送成功」并不真实。
# 发送完过几分钟,扫描近 3 天的退信通知,列出实际未送达的收件人(仅核查) **退信不是一类,必须分类处理**v3 起):
| 类别 | 典型原因 | 脚本动作 |
| ---- | -------- | -------- |
| 可重试 `rate` | 「您的账号外发频率超过邮件系统限制」、系统繁忙、DSN 4.x.x | 从断点清单剔除 → 下次/本次补发 |
| 永久失败 `hard` | 地址不存在、用户不存在、DSN 5.x.x | **不补发**;记入无效地址清单,永久跳过 |
| 未分类 `unknown` | 判不出来 | 默认不补发(保守);`--prune-unknown` 可强制补发 |
以前是「凡是退信一律剔除补发」,结果**地址不存在的死信也会被反复重发**——
既浪费每日发信额度,又拖垮发信信誉。现在只有真正可重试的才会补发。
判定优先级:DSN 状态码(5.x.x / 4.x.x,最权威,且**同一封退信里的多个收件人
可分别归类**)→ 正文硬退信关键词 → 限流/临时性关键词 → 正文 SMTP 状态码 → 未分类。
硬退信优先于限流:宁可少补发一个可疑地址,也不给死信反复重发。
# 发送完过几分钟,扫描退信并分类报告(仅核查,不改动任何清单)
python3 broadcast.py --check-bounces python3 broadcast.py --check-bounces
# 仅剔除:把这些用户从断点续发清单剔除后结束,本次不发送 # 仅剔除:把「可重试」的退信从断点续发清单剔除后结束,本次不发送
#(下次任意一次重跑会自动补上这批人) #(下次任意一次重跑会自动补上这批人;地址不存在的不会补发
python3 broadcast.py --check-bounces --prune-state python3 broadcast.py --check-bounces --prune-state
# 剔除并补发:剔除后继续正常发送流程,本次就把退信用户补上 # 剔除并补发:剔除后继续正常发送流程,本次就把可重试的用户补上
python3 broadcast.py --check-bounces --prune-state --send python3 broadcast.py --check-bounces --prune-state --send
# 未分类的也按可重试一并剔除补发(确认不是死信后再用)
python3 broadcast.py --check-bounces --prune-state --prune-unknown
# 可选:--imap-host imap.xxx.com(默认由 SMTP 域名推导 smtp.→imap. # 可选:--imap-host imap.xxx.com(默认由 SMTP 域名推导 smtp.→imap.
# --imap-port 993;扫描起点 --since auto(默认,见下) # --imap-port 993;扫描起点 --since auto(默认,见下)
扫描起点(避免把群发无关的退信算进来): 扫描起点(避免把群发无关的退信算进来):
@@ -158,11 +179,24 @@ conf/app.ini 路径的解析优先级:
时间开始——即只核查本次群发发出的那些邮件的退信 时间开始——即只核查本次群发发出的那些邮件的退信
- --since 2026-09-07 可显式指定起点日期 - --since 2026-09-07 可显式指定起点日期
- 断点清单为空时回退为 --since-days N(默认 3 天) - 断点清单为空时回退为 --since-days N(默认 3 天)
无效地址清单 broadcast_invalid.json
-----------------------------------
硬退信(地址不存在等)的地址会写进这里,之后**每次运行都直接跳过**,
即使加了 --reset-state 也不会再发——重发也发不出去,只会浪费额度。
条目里保留了退信原因、主题与时间,便于核对到底是哪些地址失效了。
python3 broadcast.py --reset-invalid # 清空清单(确认地址已修正后用)
python3 broadcast.py --no-invalid-list ... # 本次忽略清单(既不跳过也不写入)
python3 broadcast.py --invalid-file /path/to/other.json # 换一个清单文件
说明: 说明:
- 用的是 conf/app.ini [mail] 的账号与密码(需邮箱已开启 IMAP,密码为邮箱登录密码) - 用的是 conf/app.ini [mail] 的账号与密码(需邮箱已开启 IMAP,密码为邮箱登录密码)
- 交互模式的运行方式「4) 退信核查」里同样三选一:仅核查 / 仅剔除 / 剔除并补发 - 交互模式的运行方式「4) 退信核查」里同样三选一:仅核查 / 仅剔除 / 剔除并补发
选择「仅剔除 / 剔除并补发」时会再问一次是否连未分类的一起补发
- 已核查过的退信(按 Message-ID)记录在 broadcast_bounces_seen.json - 已核查过的退信(按 Message-ID)记录在 broadcast_bounces_seen.json
重复核查不会重复报告/重复剔除 重复核查不会重复报告/重复剔除
- 每次核查另写一份明细报告 broadcast_bounce_report.json(含每个收件人的类别与判定依据)
安全机制 安全机制
-------- --------
Binary file not shown.
+397 -68
View File
@@ -30,6 +30,8 @@ TamaBox 站内信群发工具(独立项目,零依赖,不依赖、不修改
- 断点续发:状态文件记录已发邮箱,重跑自动跳过;--reset-state 清空重发 - 断点续发:状态文件记录已发邮箱,重跑自动跳过;--reset-state 清空重发
- 监控报告:每次运行写 JSON 报告(--report / --no-report - 监控报告:每次运行写 JSON 报告(--report / --no-report
- SMTP 拒收捕获:send_message 返回的 refused 计入失败 - SMTP 拒收捕获:send_message 返回的 refused 计入失败
- 退信分类核查:只有「限流/临时性」退信才剔除补发;「地址不存在」这类
永久失败不补发,并记入无效地址清单(broadcast_invalid.json)永久跳过
用法示例: 用法示例:
# 1) 演练:列出收件人 + 统计(不发信) # 1) 演练:列出收件人 + 统计(不发信)
@@ -74,13 +76,15 @@ from email.mime.text import MIMEText
from email.utils import formataddr, make_msgid from email.utils import formataddr, make_msgid
from urllib.parse import quote_plus from urllib.parse import quote_plus
SCRIPT_VERSION = "2026-09-07.bouncecheck.v2" SCRIPT_VERSION = "2026-09-07.bounceclass.v3"
DEFAULT_STATE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_state.json") DEFAULT_STATE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_state.json")
DEFAULT_REPORT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_report.json") DEFAULT_REPORT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_report.json")
DEFAULT_TEMPLATES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "templates") DEFAULT_TEMPLATES_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "templates")
DEFAULT_PARAMS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "params.ini") DEFAULT_PARAMS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "params.ini")
DEFAULT_BOUNCE_SEEN = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_bounces_seen.json") DEFAULT_BOUNCE_SEEN = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_bounces_seen.json")
DEFAULT_INVALID = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_invalid.json")
DEFAULT_BOUNCE_REPORT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "broadcast_bounce_report.json")
# 语言展示标签(未知语言直接显示语言码) # 语言展示标签(未知语言直接显示语言码)
LANG_LABEL = { LANG_LABEL = {
@@ -671,13 +675,22 @@ def reset_state(state_path):
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 退信核查(IMAP):SMTP 250 ≠ 真正送达。企业邮箱(如阿里云)对限流、不存在的 # 退信核查(IMAP):SMTP 250 ≠ 真正送达。企业邮箱(如阿里云)对限流、不存在的
# 收件人常「先收下、后异步退信」,退信通知会投到发件账号的收件箱。这里用标准库 # 收件人常「先收下、后异步退信」,退信通知会投到发件账号的收件箱。这里用标准库
# imaplib 登录 IMAP 扫描退信,解析出真正投递失败的收件人;配合 --prune-state # imaplib 登录 IMAP 扫描退信,解析出真正投递失败的收件人
# 把这些用户从断点续发清单剔除,下次重跑自动补发。 #
# 关键:退信不是一类,必须分类处理——
# rate(限流/临时性,如「您的账号外发频率超过邮件系统限制」)
# → 稍后重发大概率成功:从断点续发清单剔除,下次重跑自动补发
# hard(永久失败,如「收件人地址不存在」/ DSN 5.x.x)
# → 重发也发不出去,只会浪费额度、拖垮发信信誉:
# 不从断点清单剔除(即不补发),并记入无效地址清单永久跳过
# unknown(判不出来)→ 默认不剔除(保守),可用 --prune-unknown 强制剔除
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
BOUNCE_SUBJECT_KEYWORDS = ( BOUNCE_SUBJECT_KEYWORDS = (
"退信", "投递失败", "无法投递", "邮件被退回", "投递状态", "发送失败", "退信", "投递失败", "无法投递", "未能送达", "无法送达", "邮件被退回",
"undeliver", "delivery status", "returned mail", "failure notice", "被退回", "退回通知", "投递状态", "发送失败", "发送不成功",
"mail delivery subsystem", "mail could not be delivered", "delivery failure", "undeliver", "delivery status", "delivery failed", "delivery failure",
"returned mail", "failure notice", "not delivered",
"mail delivery subsystem", "mail could not be delivered",
) )
EMAIL_RE = re.compile(r"[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}") EMAIL_RE = re.compile(r"[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}")
# 中文退信常见「标签:地址」写法 + RFC DSN 的 Final-Recipient 等 # 中文退信常见「标签:地址」写法 + RFC DSN 的 Final-Recipient 等
@@ -691,6 +704,50 @@ LABELED_PATTERNS = (
) )
SYSTEM_LOCALPARTS = ("mailer-daemon", "postmaster", "mail-daemon", "noreply", "no-reply") SYSTEM_LOCALPARTS = ("mailer-daemon", "postmaster", "mail-daemon", "noreply", "no-reply")
# --- 退信分类关键词 ---------------------------------------------------------
# 判定优先级:DSN 状态码(5.x.x/4.x.x,最权威) > 硬退信关键词 > 限流关键词 > 未知
CAT_RATE = "rate" # 限流/临时性失败 → 可重试,剔除后补发
CAT_HARD = "hard" # 地址不存在等永久失败 → 不补发,记入无效地址清单
CAT_UNKNOWN = "unknown" # 判不出来 → 默认不剔除(保守),--prune-unknown 可强制
CAT_LABEL = {
CAT_RATE: "可重试 · 限流/临时性",
CAT_HARD: "永久失败 · 地址无效",
CAT_UNKNOWN: "未分类(保守:不补发)",
}
# 限流 / 临时性失败(可稍后重试)
RATE_LIMIT_KEYWORDS = (
"您的账号外发频率超过邮件系统限制", # 阿里云企业邮箱限流退信原文
"外发频率超过", "发送频率超过", "超过邮件系统限制", "发送频率受限",
"发送流量超限", "超出发送限额", "发送数量超限", "超出邮件系统限制",
"系统繁忙", "稍后重试", "稍后重新发送", "请稍后再试", "请稍后重新发送",
"rate limit", "too many", "throttl", "try again later", "try later",
"too many recipients", "temporarily", "temporary failure", "temporary local",
"greylist", "grey list", "deferred", "deferral", "connection timed out",
"try again", "please retry", "later time",
)
# 永久失败:收件人/邮箱根本不存在
HARD_BOUNCE_KEYWORDS = (
"地址不存在", "用户不存在", "无此用户", "账号不存在", "帐户不存在",
"收件人不存在", "不存在该用户", "邮箱不存在", "没有这个邮箱",
"没有此邮箱", "查无此用户", "无效地址", "无效的收件人", "无效的收件地址",
"收件人地址错误", "该邮箱不存在", "未知的用户",
"no such user", "no such recipient", "no such mailbox", "no such address",
"user unknown", "unknown user", "unknown recipient", "unknown address",
"recipient address rejected", "recipient not found", "recipient rejected",
"mailbox unavailable", "mailbox not found", "mailbox does not exist",
"address does not exist", "address not found", "address rejected",
"invalid recipient", "invalid address", "invalid mailbox",
"bad destination", "bad address syntax", "user not found",
"does not exist", "550 5.1.1", "no mailbox here", "mailbox disabled",
)
# SMTP 状态码识别:增强状态码 5.1.1 / 三位码 550
_SMTP_ENHANCED_RE = re.compile(r"\b([245])\.(\d{1,3})\.(\d{1,3})\b")
_SMTP_CODE_RE = re.compile(r"(?<![\d.])[245]\d{2}(?![\d.])")
def _decode_hdr(value): def _decode_hdr(value):
"""解码 MIME 编码的邮件头(Subject 等)为可读文本。""" """解码 MIME 编码的邮件头(Subject 等)为可读文本。"""
@@ -705,6 +762,30 @@ def _decode_hdr(value):
return str(value) return str(value)
def _part_payload_text(part):
"""取单个 part 的文本。
两个坑都要避开:
- 直接 get_payload(decode=True) 遇到 str 形态的非 ASCII payload 会走
raw-unicode-escape,中文变成 \\uXXXX 字面串,关键词匹配全部失效;
- 反过来只用 get_payload() 又可能拿到 base64/QP 的原文(utf-8 正文默认
base64 传输编码),同样匹配不到。
所以:非 ASCII 的 str 直接用(已解码),其余交给 decode=True 解 base64/QP。
"""
try:
raw = part.get_payload()
if isinstance(raw, str) and not raw.isascii():
return raw
raw = part.get_payload(decode=True)
if isinstance(raw, bytes):
return raw.decode(part.get_content_charset() or "utf-8", "replace")
if isinstance(raw, str):
return raw
except Exception: # noqa: BLE001
pass
return ""
def _part_text(msg, limit=30000): def _part_text(msg, limit=30000):
"""抽取邮件正文文本(plain / html / DSN 原始块),拼接后截断。""" """抽取邮件正文文本(plain / html / DSN 原始块),拼接后截断。"""
chunks = [] chunks = []
@@ -712,12 +793,7 @@ def _part_text(msg, limit=30000):
for part in msg.walk(): for part in msg.walk():
ct = part.get_content_type() ct = part.get_content_type()
if ct in ("text/plain", "text/html"): if ct in ("text/plain", "text/html"):
try: chunks.append(_part_payload_text(part))
payload = part.get_payload(decode=True)
if payload:
chunks.append(payload.decode(part.get_content_charset() or "utf-8", "replace"))
except Exception: # noqa: BLE001
pass
elif ct == "message/delivery-status": elif ct == "message/delivery-status":
blocks = part.get_payload() blocks = part.get_payload()
if isinstance(blocks, list): if isinstance(blocks, list):
@@ -730,17 +806,20 @@ def _part_text(msg, limit=30000):
if sum(len(c) for c in chunks) > limit: if sum(len(c) for c in chunks) > limit:
break break
else: else:
try: chunks.append(_part_payload_text(msg))
payload = msg.get_payload(decode=True) return "\n".join(c for c in chunks if c)[:limit]
if payload:
chunks.append(payload.decode(msg.get_content_charset() or "utf-8", "replace"))
except Exception: # noqa: BLE001
pass
return "\n".join(chunks)[:limit]
def _dsn_failed_recipients(msg): # DSN 里表示「已送达」的 Action,遇到就跳过(其余 failed/delayed/expired 都算失败)
"""标准 DSNmessage/delivery-status)中 Action: failed 的收件人。""" DSN_OK_ACTIONS = ("delivered", "relayed", "expanded")
def _dsn_recipient_details(msg):
"""标准 DSNmessage/delivery-status)中每个收件人的失败详情。
返回 [{email, action, status, diagnostic}],逐个带自己的 Status /
Diagnostic-Code,因此一封退信里的多个收件人可以分别归类。
"""
out = [] out = []
for part in msg.walk(): for part in msg.walk():
if part.get_content_type() != "message/delivery-status": if part.get_content_type() != "message/delivery-status":
@@ -751,15 +830,69 @@ def _dsn_failed_recipients(msg):
for blk in blocks: for blk in blocks:
try: try:
final = str(blk.get("Final-Recipient") or blk.get("Original-Recipient") or "") final = str(blk.get("Final-Recipient") or blk.get("Original-Recipient") or "")
action = str(blk.get("Action") or "failed").lower() action = str(blk.get("Action") or "failed").strip().lower()
status = str(blk.get("Status") or "").strip()
diagnostic = str(blk.get("Diagnostic-Code") or "").strip()
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
continue continue
if action in DSN_OK_ACTIONS:
continue
m = EMAIL_RE.search(final) m = EMAIL_RE.search(final)
if m and "failed" in action: if not m:
out.append(m.group(0).strip(".").lower()) continue
out.append({
"email": m.group(0).strip(".").lower(),
"action": action,
"status": status,
"diagnostic": diagnostic,
})
return out return out
def _smtp_code_class(text):
"""从 DSN Status / Diagnostic-Code 取 SMTP 状态码首位:5=永久、4=临时、2=成功。"""
if not text:
return None
m = _SMTP_ENHANCED_RE.search(text) # 增强状态码,如 5.1.1 / 4.4.7
if m:
return m.group(1)
m = _SMTP_CODE_RE.search(text) # 三位码,如 550 / 451
if m:
return m.group(0)[0]
return None
def classify_bounce(status="", diagnostic="", text=""):
"""判定退信类别,返回 (类别, 判定依据)。
顺序:DSN 状态码(最权威)→ 正文/诊断信息里的硬退信关键词 → 限流关键词
→ 正文里的 SMTP 状态码 → unknown。
硬退信优先于限流:宁可不补发一个可疑地址,也不要反复给死信浪费额度。
"""
for src in (status, diagnostic):
c = _smtp_code_class(src or "")
if c == "5":
return CAT_HARD, f"DSN 状态码 {(src or '').strip()[:40]}"
if c == "4":
return CAT_RATE, f"DSN 状态码 {(src or '').strip()[:40]}"
body = "\n".join(x for x in (text, diagnostic) if x)
low = body.lower()
for kw in HARD_BOUNCE_KEYWORDS:
if kw.lower() in low:
return CAT_HARD, f"关键词「{kw}"
for kw in RATE_LIMIT_KEYWORDS:
if kw.lower() in low:
return CAT_RATE, f"关键词「{kw}"
c = _smtp_code_class(text or "")
if c == "5":
return CAT_HARD, "正文 SMTP 5xx"
if c == "4":
return CAT_RATE, "正文 SMTP 4xx"
return CAT_UNKNOWN, ""
def _own_ok(addr, own_addr): def _own_ok(addr, own_addr):
a = addr.strip(".").lower() a = addr.strip(".").lower()
if not a or a == own_addr.lower(): if not a or a == own_addr.lower():
@@ -781,28 +914,40 @@ def is_bounce_message(msg):
def extract_bounced_recipients(msg, own_addr): def extract_bounced_recipients(msg, own_addr):
"""从退信中解析投递失败的收件人列表(已去重、剔除发件人与系统地址)。""" """从退信中解析投递失败的收件人,并逐个分类。
返回 [{email, category, reason, detail}](已去重、剔除发件人与系统地址)。
"""
if not is_bounce_message(msg): if not is_bounce_message(msg):
return [] return []
recipients = []
# 1) 标准 DSN 最可靠 found = []
for r in _dsn_failed_recipients(msg):
if _own_ok(r, own_addr) and r not in recipients: def add(em, category, reason, detail=""):
recipients.append(r) em = (em or "").strip().strip(".").lower()
# 2) 中文退信的「标签:地址」写法 if not _own_ok(em, own_addr):
if not recipients: return
if any(x["email"] == em for x in found):
return
found.append({"email": em, "category": category,
"reason": reason, "detail": (detail or "")[:160]})
# 1) 标准 DSN 最可靠:每个收件人带自己的 Status/Diagnostic-Code,可逐个分类
for d in _dsn_recipient_details(msg):
cat, reason = classify_bounce(d["status"], d["diagnostic"], "")
add(d["email"], cat, reason, d["status"] or d["diagnostic"])
# 2) 中文退信的「标签:地址」写法 / 3) 正文兜底:整封共用一个分类结果
if not found:
text = _part_text(msg) text = _part_text(msg)
cat, reason = classify_bounce("", "", text)
for pat in LABELED_PATTERNS: for pat in LABELED_PATTERNS:
for m in re.finditer(pat, text, re.IGNORECASE): for m in re.finditer(pat, text, re.IGNORECASE):
e = m.group("e") add(m.group("e"), cat, reason, "")
if _own_ok(e, own_addr) and e.lower() not in recipients: if not found:
recipients.append(e.lower()) for e in EMAIL_RE.findall(text):
# 3) 兜底:正文里出现的其他地址(剔除发件人自身与 daemon/postmaster add(e, cat, reason, "")
if not recipients: return found
for e in EMAIL_RE.findall(_part_text(msg)):
if _own_ok(e, own_addr) and e.lower() not in recipients:
recipients.append(e.lower())
return recipients
def prune_state(state_path, emails): def prune_state(state_path, emails):
@@ -834,6 +979,88 @@ def prune_state(state_path, emails):
return removed return removed
# ---------------------------------------------------------------------------
# 无效地址清单:硬退信(收件人不存在 / DSN 5.x.x)的地址记在这里。
# 这些地址重发也发不出去,所以:
# - 不从断点续发清单剔除(即不会「补发」)
# - 记入本清单,后续每次运行(即使 --reset-state)都会直接跳过
# 只有确实要恢复发送时,才手动删条目或 --reset-invalid 清空。
# ---------------------------------------------------------------------------
def load_invalid(path):
"""读取无效地址清单,返回 {email: {"reason","detail","subject","date","ts"}}。"""
if not os.path.isfile(path):
return {}
try:
with open(path, "r", encoding="utf-8") as f:
raw = json.load(f)
except Exception as e: # noqa: BLE001
print(f"[警告] 读取无效地址清单失败,按空清单处理:{e}")
return {}
out = {}
if isinstance(raw, dict):
items = raw.items()
elif isinstance(raw, list):
items = [(it.get("email"), it) for it in raw if isinstance(it, dict)]
else:
return out
for em, info in items:
em = str(em or "").strip().lower()
if not em:
continue
info = info if isinstance(info, dict) else {"reason": str(info)}
out[em] = {
"reason": str(info.get("reason") or ""),
"detail": str(info.get("detail") or ""),
"subject": str(info.get("subject") or ""),
"date": str(info.get("date") or ""),
"ts": str(info.get("ts") or ""),
}
return out
def record_invalid(path, entries):
"""合并写入无效地址清单。entries: {email: {reason, detail, subject, date}}。
返回新增条数(已有条目只更新原因)。"""
if not entries:
return 0
data = load_invalid(path)
now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
added = 0
for em, info in entries.items():
em = (em or "").strip().lower()
if not em:
continue
if em not in data:
added += 1
data[em] = {
"reason": str(info.get("reason") or ""),
"detail": str(info.get("detail") or ""),
"subject": str(info.get("subject") or ""),
"date": str(info.get("date") or ""),
"ts": now,
}
try:
tmp = path + ".tmp"
with open(tmp, "w", encoding="utf-8") as f:
json.dump({k: data[k] for k in sorted(data)}, f, ensure_ascii=False, indent=2)
os.replace(tmp, path)
except Exception as e: # noqa: BLE001
print(f"[警告] 写入无效地址清单失败:{e}")
return 0
return added
def reset_invalid(path):
"""清空无效地址清单(慎用:这些地址多半仍是不可达的)。"""
try:
if os.path.isfile(path):
os.remove(path)
return True
except Exception as e: # noqa: BLE001
print(f"[警告] 清除无效地址清单失败:{e}")
return False
def resolve_scan_start(args): def resolve_scan_start(args):
"""退信扫描起点:--since 显式日期 > auto 时取断点清单最早一条发送记录的 """退信扫描起点:--since 显式日期 > auto 时取断点清单最早一条发送记录的
时间(精确到该时刻,不加余量)> 无记录时回退 --since-days。 时间(精确到该时刻,不加余量)> 无记录时回退 --since-days。
@@ -857,8 +1084,10 @@ def resolve_scan_start(args):
def check_bounces(cfg, args): def check_bounces(cfg, args):
"""登录发件邮箱 IMAP,扫描近 N 天的退信并解析失败收件人。 """登录发件邮箱 IMAP,扫描近 N 天的退信并解析失败收件人。
返回 {email: (subject, date_str)};已处理过的邮件(Message-ID)记录在
broadcast_bounces_seen.json,重复核查不会重复报告。""" 返回 {email: {"subject","date","category","reason","detail"}};已处理过的
邮件(Message-ID)记录在 broadcast_bounces_seen.json,重复核查不会重复报告。
category 见 CAT_RATE / CAT_HARD / CAT_UNKNOWN。"""
smtp_host = cfg["mail_smtp"] or "" smtp_host = cfg["mail_smtp"] or ""
imap_host = (args.imap_host or "").strip() imap_host = (args.imap_host or "").strip()
if not imap_host: if not imap_host:
@@ -893,7 +1122,7 @@ def check_bounces(cfg, args):
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
seen = set() seen = set()
bounced = {} # email -> (subject, date) bounced = {} # email -> {"subject","date","category","reason","detail"}
scanned, bounce_cnt, newly_seen = 0, 0, [] scanned, bounce_cnt, newly_seen = 0, 0, []
for i, mid in enumerate(ids, 1): for i, mid in enumerate(ids, 1):
uid = mid.decode() if isinstance(mid, bytes) else str(mid) uid = mid.decode() if isinstance(mid, bytes) else str(mid)
@@ -924,33 +1153,93 @@ def check_bounces(cfg, args):
bounce_cnt += 1 bounce_cnt += 1
newly_seen.append(msg_id) newly_seen.append(msg_id)
for r in rcpts: for r in rcpts:
bounced.setdefault(r, (subject, date)) bounced.setdefault(r["email"], {
print(f" [退信 {i}/{len(ids)}] {subject[:40]}{', '.join(rcpts) if rcpts else '(未解析出收件人)'}") "subject": subject, "date": date,
"category": r["category"], "reason": r["reason"], "detail": r["detail"],
})
if rcpts:
desc = " ".join(
f"{r['email']}[{CAT_LABEL.get(r['category'], r['category'])}]" for r in rcpts
)
else:
desc = "(未解析出收件人)"
print(f" [退信 {i}/{len(ids)}] {subject[:40]}{desc}")
try: try:
conn.logout() conn.logout()
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
pass pass
# ---- 按类别分组处理:只有「可重试」的才剔除补发 ----
groups = {CAT_RATE: [], CAT_HARD: [], CAT_UNKNOWN: []}
for em, info in bounced.items():
groups.setdefault(info["category"], []).append(em)
prune_cats = [CAT_RATE] + ([CAT_UNKNOWN] if getattr(args, "prune_unknown", False) else [])
print("\n========================================================") print("\n========================================================")
print(f"退信核查完成:扫描 {scanned} 封未处理邮件,其中退信 {bounce_cnt} 封," print(f"退信核查完成:扫描 {scanned} 封未处理邮件,其中退信 {bounce_cnt} 封,"
f"涉及 {len(bounced)} 个收件人。") f"涉及 {len(bounced)} 个收件人。")
if bounced: for cat in (CAT_RATE, CAT_HARD, CAT_UNKNOWN):
for r, (subj, dt) in sorted(bounced.items()): ems = sorted(groups.get(cat, []))
print(f" - {r} {dt} · {subj[:36]}") if not ems:
print("\n这些用户实际未收到邮件。") continue
print(f"\n [{CAT_LABEL[cat]}] {len(ems)} 个:")
for em in ems:
info = bounced[em]
why = info["reason"] or "未识别到明确原因"
meta = " · ".join(x for x in (info["date"], info["subject"][:32], why) if x)
print(f" - {em} {meta}")
if not bounced:
print("未发现新的退信,此前发送均正常送达(至少未被退回)。")
else:
# 硬退信:不补发 + 记入无效地址清单永久跳过
hard = sorted(groups.get(CAT_HARD, []))
if hard and not getattr(args, "no_invalid_list", False):
n = record_invalid(
args.invalid_file,
{em: bounced[em] for em in hard},
)
print(f"\n[无效地址] {len(hard)} 个永久失败地址已记入 {args.invalid_file}"
f"(新增 {n} 条);这些地址后续运行(含 --reset-state)会直接跳过,不再浪费额度。")
print(" 确认地址已修正后,删掉该文件或 --reset-invalid 可恢复发送。")
elif hard:
print(f"\n[无效地址] {len(hard)} 个永久失败地址未写入清单(--no-invalid-list),"
f"但它们不会被补发。")
# 可重试:从断点清单剔除,下次/本次补发
retryable = [em for cat in prune_cats for em in groups.get(cat, [])]
if args.prune_state: if args.prune_state:
n = prune_state(args.state, bounced.keys()) n = prune_state(args.state, retryable)
print(f"[剔除] 已从断点续发清单剔除 {n} 条记录:{args.state}") print(f"[剔除] 已从断点续发清单剔除 {n}「可重试」记录:{args.state}")
if getattr(args, "send", False): if getattr(args, "send", False):
print(" 本次发送将继续进行,这些用户会被包含在内(剔除并补发)。") print(" 本次发送将继续进行,这些用户会被包含在内(剔除并补发)。")
else: else:
print(" 本次不发送;下次重跑(同参数)会自动给这些用户补发。") print(" 本次不发送;下次重跑(同参数)会自动给这些用户补发。")
else: elif retryable:
print("可选后续:--prune-state 仅剔除不发送;--prune-state --send 剔除后立即补发;") print(f"\n[提示] 有 {len(retryable)} 个「可重试」退信等待补发。可选:"
print(" 或交互模式运行方式 4 里选择对应核查方式") f"--prune-state 仅剔除;--prune-state --send 剔除后立即补发")
else: unknown = groups.get(CAT_UNKNOWN, [])
print("未发现新的退信,此前发送均正常送达(至少未被退回)。") if unknown and not getattr(args, "prune_unknown", False):
print(f"[提示] 有 {len(unknown)} 个退信未能分类,默认不补发(保守)。"
f"确认是限流后可加 --prune-unknown 一并剔除补发。")
# 退信核查明细报告(便于事后核对,不受 --no-report 影响)
try:
with open(args.bounce_report, "w", encoding="utf-8") as f:
json.dump({
"generated_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
"script_version": SCRIPT_VERSION,
"scan_from": start_dt.strftime("%Y-%m-%d %H:%M:%S"),
"scan_from_desc": start_desc,
"scanned": scanned,
"bounce_messages": bounce_cnt,
"counts": {c: len(groups.get(c, [])) for c in (CAT_RATE, CAT_HARD, CAT_UNKNOWN)},
"recipients": {em: bounced[em] for em in sorted(bounced)},
}, f, ensure_ascii=False, indent=2)
print(f"\n[退信报告] 已写入:{args.bounce_report}")
except Exception as e: # noqa: BLE001
print(f"[警告] 写入退信报告失败(不影响本次结果):{e}")
if newly_seen: if newly_seen:
try: try:
@@ -1106,10 +1395,11 @@ def run_interactive(cfg, templates, extra_vars, args, config_path):
if mode == "4": if mode == "4":
args.check_bounces = True args.check_bounces = True
print("\n退信核查方式:") print("\n退信核查方式(会先按原因分类:限流/临时性 → 可重试;地址不存在 → 永久失败)")
print(" 1) 仅核查 —— 只报告哪些用户没收到,不改动断点清单(默认)") print(" 1) 仅核查 —— 只报告哪些用户没收到、各自什么原因,不改动任何清单(默认)")
print(" 2) 仅剔除 —— 把退信用户从断点清单剔除后结束,本次不发送") print(" 2) 仅剔除 —— 只把「限流等可重试」的退信从断点清单剔除后结束,本次不发送")
print(" 3) 剔除并补发 —— 剔除后继续正常发送流程(发送前仍有统计与确认") print(" (「地址不存在」的不会剔除,另记入无效地址清单永久跳过")
print(" 3) 剔除并补发 —— 同上剔除后继续正常发送流程(发送前仍有统计与确认)")
while True: while True:
try: try:
sub = input("核查方式 [1/2/3](直接回车 = 1: ").strip() sub = input("核查方式 [1/2/3](直接回车 = 1: ").strip()
@@ -1118,11 +1408,16 @@ def run_interactive(cfg, templates, extra_vars, args, config_path):
if sub in ("", "1", "2", "3"): if sub in ("", "1", "2", "3"):
break break
print(" 请输入 1、2 或 3") print(" 请输入 1、2 或 3")
if sub == "2": if sub in ("2", "3"):
args.prune_state = True
elif sub == "3":
args.prune_state = True args.prune_state = True
if sub == "3":
args.send = True args.send = True
if args.prune_state:
try:
unk = input("未分类的退信是否也按「可重试」一并剔除补发?(y/N): ").strip().lower()
except EOFError:
unk = ""
args.prune_unknown = unk in ("y", "yes")
return site_url, box_prefix, extra_vars return site_url, box_prefix, extra_vars
# ---- 模板模式:多语言 / 单文件通用 ---- # ---- 模板模式:多语言 / 单文件通用 ----
@@ -1238,8 +1533,21 @@ def main():
help="退信核查:登录发件邮箱 IMAP 扫描退信通知,找出实际未送达的收件人" help="退信核查:登录发件邮箱 IMAP 扫描退信通知,找出实际未送达的收件人"
"(SMTP 250 不代表送达;限流等退信是异步投到发件箱的)") "(SMTP 250 不代表送达;限流等退信是异步投到发件箱的)")
parser.add_argument("--prune-state", action="store_true", parser.add_argument("--prune-state", action="store_true",
help="配合 --check-bounces:把退信用户从断点续发清单剔除后结束(仅剔除," help="配合 --check-bounces:把「可重试」退信(限流/临时性)从断点续发清单"
"本次不发送);再加 --send 则剔除后立即补发") "剔除后结束(仅剔除,本次不发送);再加 --send 则剔除后立即补发"
"「地址不存在」等永久失败不会被剔除(不补发)")
parser.add_argument("--prune-unknown", action="store_true",
help="配合 --check-bounces:无法分类的退信也按「可重试」处理,一并剔除补发"
"(默认不剔除,避免给死信地址反复重发)")
parser.add_argument("--invalid-file", default=DEFAULT_INVALID, metavar="FILE",
help=f"无效地址清单(硬退信地址)路径;默认脚本同目录 broadcast_invalid.json。"
f"清单里的地址每次运行都会跳过,即使 --reset-state")
parser.add_argument("--reset-invalid", action="store_true",
help="清空无效地址清单(慎用:这些地址多半仍不可达)")
parser.add_argument("--no-invalid-list", action="store_true",
help="本次不使用无效地址清单:既不跳过已知无效地址,也不写入新的硬退信")
parser.add_argument("--bounce-report", default=DEFAULT_BOUNCE_REPORT, metavar="FILE",
help="退信核查明细报告路径(JSON);默认脚本同目录 broadcast_bounce_report.json")
parser.add_argument("--imap-host", default="", parser.add_argument("--imap-host", default="",
help="IMAP 服务器地址(默认由 SMTP 域名推导:smtp.xxx → imap.xxx") help="IMAP 服务器地址(默认由 SMTP 域名推导:smtp.xxx → imap.xxx")
parser.add_argument("--imap-port", type=int, default=993, parser.add_argument("--imap-port", type=int, default=993,
@@ -1267,6 +1575,13 @@ def main():
args.interactive = not (args.send or args.to or args.yes or args.dry_run args.interactive = not (args.send or args.to or args.yes or args.dry_run
or args.check_bounces) or args.check_bounces)
# ---- 无效地址清单维护(--reset-invalid):手动恢复发送硬退信地址时用 ----
if args.reset_invalid:
if reset_invalid(args.invalid_file):
print(f"[无效地址清单] 已清空:{args.invalid_file}")
else:
print(f"[无效地址清单] 文件不存在,无需清空:{args.invalid_file}")
# 载入模板 # 载入模板
templates, lang_list = load_templates(args.templates_dir) templates, lang_list = load_templates(args.templates_dir)
single_tpl = load_single_template(args.templates_dir) single_tpl = load_single_template(args.templates_dir)
@@ -1401,6 +1716,19 @@ def main():
recipients = fetch_recipients(cfg, args, lang_expr=lang_expr, tbl=users_table) recipients = fetch_recipients(cfg, args, lang_expr=lang_expr, tbl=users_table)
counts = fetch_counts(cfg, args, tbl=users_table) counts = fetch_counts(cfg, args, tbl=users_table)
# 跳过历史硬退信地址(收件人不存在 / DSN 5.x.x)——重发也发不出去,
# 且不受 --reset-state 影响;只有 --no-invalid-list 或清空清单才会恢复。
invalid_skipped = 0
if not args.no_invalid_list:
invalid_map = load_invalid(args.invalid_file)
if invalid_map:
before = len(recipients)
recipients = [r for r in recipients
if (r["email"] or "").strip().lower() not in invalid_map]
invalid_skipped = before - len(recipients)
print(f" [无效地址清单] 已载入 {len(invalid_map)}"
f"{args.invalid_file}),本次跳过 {invalid_skipped} 个地址")
print("数据库用户统计:") print("数据库用户统计:")
print(f" 总用户数(含已注销) : {counts['total']}") print(f" 总用户数(含已注销) : {counts['total']}")
print(f" 已注销用户数 : {counts['deactivated']}") print(f" 已注销用户数 : {counts['deactivated']}")
@@ -1496,6 +1824,7 @@ def main():
"failed": 0, "failed": 0,
"smtp_rejected": 0, "smtp_rejected": 0,
"skipped_already_sent": skipped, "skipped_already_sent": skipped,
"skipped_invalid": invalid_skipped,
"failed_list": [], "failed_list": [],
"missing_placeholders": [], "missing_placeholders": [],
"test_mode": bool(args.to), "test_mode": bool(args.to),
@@ -1646,9 +1975,9 @@ def main():
print(f"[注意] 模板中有未提供值的占位符(已替换为空串):{', '.join(sorted(all_missing))}") print(f"[注意] 模板中有未提供值的占位符(已替换为空串):{', '.join(sorted(all_missing))}")
if not args.to and args.send: if not args.to and args.send:
print(f"断点续发状态已写入:{args.state}(重跑脚本会自动跳过已发用户)") print(f"断点续发状态已写入:{args.state}(重跑脚本会自动跳过已发用户)")
print("[提示] SMTP 250 不代表一定送达(限流常被异步退回发件箱)。" print("[提示] SMTP 250 不代表一定送达(限流/无效地址常被异步退回发件箱)。"
"发送完过几分钟可运行「python3 broadcast.py --check-bounces --prune-state」" "发送完过几分钟可运行「python3 broadcast.py --check-bounces --prune-state」"
"核查退信并让失败用户下次自动补发。") "核查退信:限流类会自动剔除补发,地址不存在类记入无效清单不再重发。")
if failed: if failed:
print("失败清单:") print("失败清单:")
for e, err in failed: for e, err in failed: