💾TVM论文笔记

😀总体框架

• DL Framework 导出计算图 给到 tvm

• tvm 进行graph level的优化
• op fuse
• constant folding
• static memory planning
• data layout transformation

图优化

op fuse

data layout transformation

Operator codegen

tensor expression

• output shape + how each element is computed

schedule primitives

Cooperation Data Fetching

An alternative to the shared-nothing approach is to fetch data cooperatively

Tensorization

• 逻辑上分离tensor-intrinsic和hardware-intrinsic

• macro kernel: 用hardware-intrinsic实现的, 最小规模的任务, 包含:
• 语义, 依然用tensor expression表达
• lowering规则, 看例子是用一个函数定义, 函数

• tensoize: 将一个大的computation decompose到对应的macro kernel

Latency Hidding

• overlap memory op and compute

• CPU/GPU : 硬件通过在多个wrap中调度切换隐式实现.

• TPU: 采用 decoupled access-execute (DAE)结构, mem op 和 compute的overlap需要有软件显式完成.

AutoMating Optimization

定义搜索空间 － schedule space specification

• 通过 开发者通过 schedule template specification API来声明可调参数, 这需要kernel开发者的先验知识

• generic master template 自动生成可调参数

ML Based Cost Model

• Blackbox auto-tuning: 不看硬件, 暴力大量尝试cfg, 看实际结果来选

• Predefined cost model: 依据硬件, 预定义一个cost model

• ML based cost model: 用黑盒硬件, 预训练一个ml based cost model.

Design ML Model

• 需要在 quality / speed 之间做平衡

• gradient tree boosting model

Schedule Exploer

多机协作搜索

小结

• op fuse

• data layout transformation kernel自动生成: cost model + Schedule template + Scheduler Expoler , 三者分别用到的技术:

• ML based cost model

• tensor expression 自动/手动 定义大量可调参数 + hardware aware primitives

• 并行模拟退火