CANN/cannbot-skills白盒测试模板

代码模板【免费下载链接】cannbot-skillsCANNBot 是面向 CANN 开发的用于提升开发效率的系列智能体本仓库为其提供可复用的 Skills 模块。项目地址: https://gitcode.com/cann/cannbot-skills常量DTYPE_MAP { float16: torch.float16, bfloat16: torch.bfloat16, float32: torch.float32, float64: torch.float64, int8: torch.int8, int16: torch.int16, int32: torch.int32, int64: torch.int64, bool: torch.bool, } TOLERANCE { torch.float32: (1e-4, 1e-4), torch.float16: (1e-3, 1e-3), torch.bfloat16: (1e-3, 1e-3), torch.float64: (1e-6, 1e-6), # 非 float 类型不走 TOLERANCE用 torch.equal 精确比较 }Tensor 构造Tensor 构造分两种场景由 S5_mapping_spec.md 中各 tensor 的 param_type 标注决定param_typeJSON spec 结构构造方式REQUIREDdict含 shape/dtype/_data_range单次make_data调用DYNAMIClist[dict]每个子 tensor 含 shape/dtype/_data_range逐子 tensormake_data调用返回list[Tensor]两种场景共用make_data函数和统一的isinstance(spec, list)运行时分支无需子 agent 按 param_type 生成两套代码。输入 tensorshape 和 dtype 直接从case[tensors][inputs]读取无需推导。REQUIRED 输入dict和 DYNAMIC 输入list[dict]通过isinstance(spec, list)统一处理。根据每个 tensor 的_data_range字段选择构造方式def make_data(shape, dtype, data_range, value_domainNone): 根据 data_range 构造不同值域的 tensor。value_domain 约束生成范围。 if value_domain and value_domain[type] range: eff_lo value_domain.get(min) eff_hi value_domain.get(max) eff_lo eff_lo if eff_lo is not None else -10.0 eff_hi eff_hi if eff_hi is not None else 10.0 if data_range normal: return torch.rand(shape, dtypedtype) * (eff_hi - eff_lo) eff_lo elif data_range negative: neg_hi min(0, eff_hi) if eff_lo neg_hi: return torch.rand(shape, dtypedtype) * (neg_hi - eff_lo) eff_lo return torch.rand(shape, dtypedtype) * (eff_hi - eff_lo) eff_lo elif data_range near_zero: nz_lo max(eff_lo, -0.01) nz_hi min(eff_hi, 0.01) if nz_lo nz_hi: return torch.rand(shape, dtypedtype) * (nz_hi - nz_lo) nz_lo return torch.rand(shape, dtypedtype) * (eff_hi - eff_lo) eff_lo elif data_range tiny_pos: tp_lo max(eff_lo, 1e-7) tp_hi min(eff_hi, 1e-5) if tp_lo tp_hi: return torch.rand(shape, dtypedtype) * (tp_hi - tp_lo) tp_lo return torch.rand(shape, dtypedtype) * (eff_hi - eff_lo) eff_lo if data_range zero: return torch.zeros(shape, dtypedtype) elif data_range extreme: dtype_max {torch.float16: 65504.0, torch.bfloat16: 3.3895e38, torch.float32: 3.4e38} return torch.full(shape, dtype_max.get(dtype, 3.4e38), dtypedtype) elif data_range negative: return -torch.rand(shape, dtypedtype) * 10 elif data_range tiny_pos: return torch.ones(shape, dtypedtype) * 1e-6 elif data_range all_ones: return torch.ones(shape, dtypedtype) elif data_range near_zero: return (torch.rand(shape, dtypedtype) - 0.5) * 0.02 elif data_range with_inf: t torch.randn(shape, dtypedtype) t.view(-1)[0] float(inf) return t elif data_range with_nan: t torch.randn(shape, dtypedtype) t.view(-1)[0] float(nan) return t else: if value_domain: t value_domain[type] if t positive: return torch.rand(shape, dtypedtype) * 10 0.01 elif t non_negative: return torch.rand(shape, dtypedtype) * 10 elif t non_zero: r torch.randn(shape, dtypedtype) return torch.where(r.abs() 0.1, torch.ones_like(r), r) return torch.randn(shape, dtypedtype) tensors p[tensors] params p[params] inputs {} for name, spec in tensors[inputs].items(): if spec is None: inputs[name] None continue if isinstance(spec, list): inputs[name] [ make_data(sub[shape], DTYPE_MAP[sub[dtype]], sub.get(_data_range, normal), sub.get(_value_domain)).npu() for sub in spec ] else: dr spec.get(_data_range, normal) vd spec.get(_value_domain) inputs[name] make_data(spec[shape], DTYPE_MAP[spec[dtype]], dr, vd).npu()可选输入S5 映射中 optional tensor 为 None 时跳过构造传 None 给算子。输出 tensor预分配场景部分算子 API 要求调用方预分配输出 tensor。此时从case[tensors][outputs]构造。REQUIRED 输出dict和 DYNAMIC 输出list[dict]同样通过isinstance(spec, list)统一处理outputs_prealloc {} for name, spec in tensors[outputs].items(): if spec is None: continue if isinstance(spec, list): # DYNAMICTensorList逐子 tensor 预分配 outputs_prealloc[name] [ torch.empty(sub[shape], dtypeDTYPE_MAP[sub[dtype]]).npu() for sub in spec ] else: # REQUIRED单 tensor outputs_prealloc[name] torch.empty(spec[shape], dtypeDTYPE_MAP[spec[dtype]]).npu()如果算子 API 不要求预分配API 内部创建输出则跳过此步。标量属性从case[params]读取。根据算子接口需要提取对应的标量参数params p[params] attr_1 params.get(attr_name_1, default_1) attr_2 params.get(attr_name_2, default_2)断言断言逻辑按 output 的 param_type 派发。子 agent 根据 S5_mapping_spec.md 中各 output 的标注选择对应模板。REQUIRED 模式单 tensor 输出shape 检查assert output.shape expected_shapedtype 检查assert output.dtype expected_dtype数值对比逐输出对比精度校验失败时必须转为 XFAIL见 02-constraints.md 铁律for i, (npu_out, ref_out) in enumerate(zip(npu_outputs, ref_outputs)): if npu_out.dtype not in TOLERANCE: # 非 float 类型精确比较 assert torch.equal(npu_out.cpu(), ref_out), fOutput[{i}] value mismatch continue rtol, atol TOLERANCE[npu_out.dtype] try: torch.testing.assert_close(npu_out.cpu().float(), ref_out.cpu().float(), rtolrtol, atolatol, equal_nanTrue) except AssertionError as e: pytest.xfail(fOutput[{i}] precision mismatch: {e})DYNAMIC 模式TensorList 输出shape 检查逐子 tensor 检查assert sub_out.shape expected_sub_shapedtype 检查逐子 tensor 检查assert sub_out.dtype expected_sub_dtype数值对比嵌套迭代逐输出名 → 逐子 tensor精度校验失败时必须转为 XFAIL# npu_out 和 ref_out 均为 list[Tensor]逐子 tensor 对比 for i, (sub_npu, sub_ref) in enumerate(zip(npu_out_list, ref_out_list)): assert sub_npu.shape sub_ref.shape, fSub-tensor[{i}] shape mismatch assert sub_npu.dtype sub_ref.dtype, fSub-tensor[{i}] dtype mismatch if sub_npu.dtype not in TOLERANCE: assert torch.equal(sub_npu.cpu(), sub_ref), fSub-tensor[{i}] value mismatch continue rtol, atol TOLERANCE[sub_npu.dtype] try: torch.testing.assert_close(sub_npu.cpu().float(), sub_ref.cpu().float(), rtolrtol, atolatol, equal_nanTrue) except AssertionError as e: pytest.xfail(fSub-tensor[{i}] precision mismatch: {e})如果某些参数组合没有 reference 实现只做 shape/dtype 检查parametrize ids直接使用 S5 映射中的case[id]pytest.mark.parametrize(p, PARAMS, idslambda c: c[id]) def test_{op_name}(p): ...✅/❌ 示例# ✅ 正确conftest.py 注册 --cases-file测试文件通过 getoption 读取 # conftest.py: import pytest def pytest_addoption(parser): parser.addoption(--cases-file, defaultS5_mapped_cases_low.json) # S6_test_{op_name}.py: import json, os _CASES_DIR os.path.dirname(os.path.abspath(__file__)) def pytest_generate_tests(metafunc): if p in metafunc.fixturenames: cases_file metafunc.config.getoption(--cases-file, S5_mapped_cases_low.json) with open(os.path.join(_CASES_DIR, cases_file)) as _f: cases json.load(_f)[cases] metafunc.parametrize(p, cases, idslambda c: c[id]) # ❌ 错误pytest_addoption 写在测试模块中命令行 --cases-file 无法注册 # S6_test_{op_name}.py: def pytest_addoption(parser): # 不会被 pytest 识别为 conftest hook parser.addoption(--cases-file, defaultS5_mapped_cases_low.json) # ❌ 错误手写参数 PARAMS [{dtype: float16, D: 32}]【免费下载链接】cannbot-skillsCANNBot 是面向 CANN 开发的用于提升开发效率的系列智能体本仓库为其提供可复用的 Skills 模块。项目地址: https://gitcode.com/cann/cannbot-skills创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考