chop.passes.graph.analysis.add_metadata#
add_common_metadata_analysis_pass#
- chop.passes.graph.analysis.add_metadata.add_common_metadata.add_common_metadata_analysis_pass(graph, pass_args={'add_value': True, 'dummy_in': None, 'force_device_meta': False})[source]#
add common metadata
- Parameters:
graph (MaseGraph) – a MaseGraph
pass_args (_type_, optional, "add_value" controls whether tensor values would be added to the meta data, defaults to True) – this pass does not need any arguments, defaults to None
pass_args can take
dummy_in: a dictionary of dummy inputs to the graph
add_value: a boolean to control whether tensor values would be added to the meta data in the “value” field
force_device_meta: a boolean to force everything to be on device=”meta”
{ "dummy_in": dummy_in, # this would be a dictionary of dummy inputs (actual tensors) "add_value": True, # if True, real values of tensors would be added to the metadata "value" field "force_device_meta": False # if True, everything would be forced to be on device="meta" for a symbolic run }
- Returns:
return a tuple of a MaseGraph and an empty dict (no additional info to return)
- Return type:
tuple(MaseGraph, Dict)
The common metadata of a Mase node in a Mase graph describes the constraints of the node for any static analysis or possible transformation. The metadata has a tree structure, e.g.
- common
mase_op -> str : the mase op of the node, e.g. placeholder, linear, relu
mase_type -> str : the mase type of the node, e.g. module, builtin_func, module_related_func
- args -> {}
$name : name of the arg
(if the arg is a tensor)
type -> type of the arg, e.g. fixed point or float
precision -> format of the type, e.g. (10, 5)
shape -> shape of the arg
(if the arg is not a tensor)
value of the arg
- results -> {}
$name : name of the result
(if the result is a tensor)
type -> type of the result, e.g. fixed point or float
precision -> format of the type, e.g. (10, 5)
shape -> shape of the result
(if the result is not a tensor)
value of the result
Examples:
A linear layer in a mase graph:
%fc1 : [num_users=1] = call_module[target=fc1](args = (%flatten,), kwargs = {})
A linear layer after this pass:
{ "common": { "mase_type": "module_related_func", "mase_op": "linear", "args": { "data_in_0": { "shape": [1, 784], "torch_dtype": torch.float32, "type": "float", "precision": [32], }, "weight": {"type": "float", "precision": [32], "shape": [784, 784]}, "bias": {"type": "float", "precision": [32], "shape": [784]}, }, "results": { "data_out_0": { "type": "float", "precision": [32], "shape": [1, 784], "torch_dtype": torch.float32, } }, }, "software": {}, "hardware": {}, }
A relu layer in a mase graph:
%relu : [num_users=1] = call_function[target=torch.nn.functional.relu](args = (%fc1,), kwargs = {inplace: False})
A relu layer after this pass:
{ "common": { "mase_type": "module_related_func", "mase_op": "relu", "results": { "data_out_0": { "type": "float", "precision": [32], "shape": [1, 784], "torch_dtype": torch.float32, } }, "args": { "data_in_0": { "shape": [1, 784], "torch_dtype": torch.float32, "type": "float", "precision": [32], }, "inplace": False, }, }, "software": {}, "hardware": {}, }
A flatten op in a mase graph:
%flatten : [num_users=1] = call_function[target=torch.flatten](args = (%x,), kwargs = {start_dim: 1, end_dim: -1})
A flatten op after this pass:
{ "common": { "mase_type": "implicit_func", "mase_op": "flatten", "results": { "data_out_0": { "type": "float", "precision": [32], "shape": [1, 784], "torch_dtype": torch.float32, } }, "args": { "data_in_0": { "shape": [1, 28, 28], "torch_dtype": torch.float32, "type": "float", "precision": [32], }, "start_dim": 1, "end_dim": -1, }, }, "software": {}, "hardware": {}, }
add_software_metadata_analysis_pass#
- chop.passes.graph.analysis.add_metadata.add_software_metadata.add_software_metadata_analysis_pass(graph, pass_args=None)[source]#
add software metadata
- Parameters:
graph (MaseGraph) – a MaseGraph
pass_args (_type_, optional) – this pass does not need any arguments, defaults to None
pass_args is normally None for this pass
- Returns:
return a tuple of a MaseGraph and an empty dict (no additional info to return)
- Return type:
tuple(MaseGraph, Dict)
add_hardware_metadata_analysis_pass#
- chop.passes.graph.analysis.add_metadata.add_hardware_metadata.add_hardware_metadata_analysis_pass(graph, pass_args=None)[source]#
add hardware metadata
- Parameters:
graph (MaseGraph) – a MaseGraph
pass_args (_type_, optional) – this pass does not need any arguments, defaults to None
- Returns:
return a tuple of a MaseGraph and an empty dict (no additional info to return)
- Return type:
tuple(MaseGraph, Dict)
The hardware metadata of a Mase node in a Mase graph describes the constraints of the node for any static analysis or possible transformation. The metadata has a tree structure, e.g.
- hardware
is_implicit -> bool : whether the node is mapped on hardware or software annotation only
verilog_param -> {} : parameters need for customise the hardware module
toolchain -> str : tool chain for code generation, must be INTERNAL, EXTERNAL or HLS
module -> str : the name of the used hardware module
device_id -> int : the ID of the device where the node is mapped, default = -1
- interface -> {}
- namename of the parameters
storage : the hardware interface implemented, must be BRAM
transpose : whether the data needs to be transposed before emitting
dependence_files -> [] : the dependent files for the generated module
The verilog parameters follow the following naming rules:
Hardware signal naming rules
Data with tensor types are explicit as hardware signals, such as weight and bias, and Data with scalar/tuple types are implicit as parameters (TODO).
Each op is a node with a set of inputs, outputs and parameters
The input is named by: data_in_0 (data_in_0_ready, data_in_valid), data_in_1,
The output is named by: data_out_0 (data_out_0_ready, data_out_valid), data_out_1, ..
The parameters are named by PyTorch names: weight (weight_ready, weight_valid), bias (bias_ready, bias_valid)
Hardware parameters naming rules Parameters with tensor types are explicit as hardware signals, such as weight and bias, and parameters with scalar/tuple types are implicit as hardware parameters.
- Taking data_in_0 for example:
DATA_IN_0_PRECISION_0
DATA_IN_0_PRECISION_1
…
(depending on how many precision parameters we have.
The order matches the same order as the mase precision metadata)
DATA_IN_0_TENSOR_SIZE_DIM_0
DATA_IN_0_TENSOR_SIZE_DIM_1
DATA_IN_0_TENSOR_SIZE_DIM_2
DATA_IN_0_PARALLELISM_DIM_0
DATA_IN_0_PARALLELISM_DIM_1
DATA_IN_0_PARALLELISM_DIM_2
(This means that the number of iterations = tensor_size / spatial_size)
Implicit parameters are directly translated into verilog parameters, e.g. STRIDE DIM
Examples:
A linear layer in a mase graph with the following common metadata:
%fc1 : [num_users=1] = call_module[target=fc1](args = (%flatten,), kwargs = {})
{ "common": { "mase_type": "module_related_func", "mase_op": "linear", "args": { "data_in_0": { "shape": [1, 784], "torch_dtype": torch.float32, "type": "float", "precision": [32], }, "weight": {"type": "float", "precision": [32], "shape": [784, 784]}, "bias": {"type": "float", "precision": [32], "shape": [784]}, }, "results": { "data_out_0": { "type": "float", "precision": [32], "shape": [1, 784], "torch_dtype": torch.float32, } }, }, "software": {}, "hardware": {}, }
The hardware metadata of the linear layer after this pass:
{ "common": {...}, "software": {}, "hardware": { "is_implicit": False, "interface": { "weight": {"storage": "BRAM", "transpose": False}, "bias": {"storage": "BRAM", "transpose": False}, }, "toolchain": "INTERNAL", "module": "fixed_linear", "device_id": -1, "dependence_files": [ "cast/fixed_cast.sv", "fixed_arithmetic/fixed_dot_product.sv", "fixed_arithmetic/fixed_vector_mult.sv", "fixed_arithmetic/register_slice.sv", "fixed_arithmetic/fixed_accumulator.sv", "fixed_arithmetic/fixed_adder_tree.sv", "fixed_arithmetic/fixed_adder_tree_layer.sv", "fixed_arithmetic/fixed_mult.sv", "common/join2.sv", "linear/fixed_linear.sv", ], "verilog_param": { "DATA_IN_0_PRECISION_0": 8, "DATA_IN_0_PRECISION_1": 3, "DATA_IN_0_TENSOR_SIZE_DIM_0": 1, "DATA_IN_0_PARALLELISM_DIM_0": 1, "DATA_IN_0_TENSOR_SIZE_DIM_1": 784, "DATA_IN_0_PARALLELISM_DIM_1": 784, "DATA_IN_0_TENSOR_SIZE_DIM_2": 1, "DATA_IN_0_PARALLELISM_DIM_2": 1, "WEIGHT_PRECISION_0": 8, "WEIGHT_PRECISION_1": 3, "WEIGHT_TENSOR_SIZE_DIM_0": 784, "WEIGHT_PARALLELISM_DIM_0": 784, "WEIGHT_TENSOR_SIZE_DIM_1": 784, "WEIGHT_PARALLELISM_DIM_1": 784, "WEIGHT_TENSOR_SIZE_DIM_2": 1, "WEIGHT_PARALLELISM_DIM_2": 1, "BIAS_PRECISION_0": 8, "BIAS_PRECISION_1": 3, "BIAS_TENSOR_SIZE_DIM_0": 784, "BIAS_PARALLELISM_DIM_0": 784, "BIAS_TENSOR_SIZE_DIM_1": 1, "BIAS_PARALLELISM_DIM_1": 1, "BIAS_TENSOR_SIZE_DIM_2": 1, "BIAS_PARALLELISM_DIM_2": 1, "DATA_OUT_0_PRECISION_0": 8, "DATA_OUT_0_PRECISION_1": 3, "DATA_OUT_0_TENSOR_SIZE_1_DIM_0": 1, "DATA_OUT_0_PARALLELISM_1_DIM_0": 1, "DATA_OUT_0_TENSOR_SIZE_1_DIM_1": 784, "DATA_OUT_0_PARALLELISM_1_DIM_1": 784, "DATA_OUT_0_TENSOR_SIZE_1_DIM_2": 1, "DATA_OUT_0_PARALLELISM_1_DIM_2": 1, }, }, }
A relu layer in a mase graph with the following common metadata:
%relu : [num_users=1] = call_function[target=torch.nn.functional.relu](args = (%fc1,), kwargs = {inplace: False})
{ "common": { "mase_type": "module_related_func", "mase_op": "relu", "results": { "data_out_0": { "type": "float", "precision": [32], "shape": [1, 784], "torch_dtype": torch.float32, } }, "args": { "data_in_0": { "shape": [1, 784], "torch_dtype": torch.float32, "type": "float", "precision": [32], }, "inplace": False, }, }, "software": {}, "hardware": {}, }
The hardware metadata of the relu layer after this pass:
{ "common": {...}, "software": {}, "hardware": { "is_implicit": False, "interface": {"inplace": {}}, "toolchain": "INTERNAL", "module": "fixed_relu", "device_id": -1, "dependence_files": ["activations/fixed_relu.sv"], "verilog_param": { "DATA_IN_0_PRECISION_0": 8, "DATA_IN_0_PRECISION_1": 3, "DATA_IN_0_TENSOR_SIZE_DIM_0": 1, "DATA_IN_0_PARALLELISM_DIM_0": 1, "DATA_IN_0_TENSOR_SIZE_DIM_1": 784, "DATA_IN_0_PARALLELISM_DIM_1": 784, "DATA_IN_0_TENSOR_SIZE_DIM_2": 1, "DATA_IN_0_PARALLELISM_DIM_2": 1, "INPLACE": False, "DATA_OUT_0_PRECISION_0": 8, "DATA_OUT_0_PRECISION_1": 3, "DATA_OUT_0_TENSOR_SIZE_1_DIM_0": 1, "DATA_OUT_0_PARALLELISM_1_DIM_0": 1, "DATA_OUT_0_TENSOR_SIZE_1_DIM_1": 784, "DATA_OUT_0_PARALLELISM_1_DIM_1": 784, "DATA_OUT_0_TENSOR_SIZE_1_DIM_2": 1, "DATA_OUT_0_PARALLELISM_1_DIM_2": 1, }, }, }