Tutorials

The following tutorials walk through the main flows of MASE, from model training to deployment.

Hint

For a quick introduction, see the Quickstart page. The tutorials on this page dive deeper into various features and use cases.

See below a summary of the main tutorials:

Overview of the main Mase tutorials.

Core Tutorials

The following tutorials show how to use Mase for standard model compression and optimization flows, such as Finetuning, Quantization and Pruning.

• Tutorial 1: Introduction to the Mase IR, MaseGraph and Torch FX passes
• Tutorial 2: Finetuning Bert for Sequence Classification using a LoRA adapter
• Tutorial 3: Running Quantization-Aware Training (QAT) on Bert
• Tutorial 4: Unstructured Pruning on Bert

Architecture Search

The following show how to effectively run Neural Architecture Search (NAS) and mixed-precision search to find optimal architecture configurations for a given task or dataset.

• Tutorial 6: Mixed Precision Quantization Search with Mase and Optuna

Inference Deployment

The following tutorials show how to deploy an optimized model to a range of platforms, including FPGA and GPU clusters.

• Tutorial 7: Deploying a Model for Inference on Distributed Clusters
• Tutorial 8: Autogenerating an FPGA accelerator for a Transformer Model
• Tutorial 9: Running Kernel Fusion for Inference Acceleration on GPUs

Advanced Topics

• Advanced: TensorRT Quantization Tutorial
  • Section 1. INT8 Quantization
    • Section 1.1 Fake Quantization
    • Section 1.2 Calibration
    • Section 1.3 Quantized Aware Training (QAT)
    • Section 1.4 TensorRT Quantization
    • Section 1.5 Performance Analysis
  • Section 2. FP16 Quantization
  • Section 3. Type-wise Mixed Precision on Larger Model
  • Section 4. Layer-wise Mixed Precision
• Advanced: ONNX Runtime Tutorial
  • Section 1. ONNX Runtime Optimizations
  • Section 2. Quantization
• Advanced: Using Mase CLI
  • Run the train action with the CLI
    • Command line interface
    • Training implementation
    • Models
    • Output
    • Training Logs
    • Test command
    • Quantise transform
    • Quantization-aware Training (QAT)
    • Quantise transform by Type
  • Mixed-precision search on manual model
    • Search for Mixed-Precision Quantization Scheme
      • Search config
      • Launch the Precision Search
      • Search Logs
  • Mixed-precision search on MASE Graph
    • Commands
    • Search Config
    • Run the search

Developer Guide

If you’d like to contribute to Mase, you may find the following resources useful.

• Developer: Guide on how to add a new model into Machop
  • Overall Structure
    • Model
    • Command Line Interface
  • What To Do
  • Get model function
• Developer: How to write documentations in MASE
  • The sphinx flow
  • Automatic documentation generation through docstrings
    • Documentation format
    • Add the pass to the sphinx directory
• Developer: How to extend search
  • How does MASE search work?
  • How to extend search_space?