Hi,
Ash said to post on here so the nice people at Sony can help us troubleshoot getting custom models working on the AI Camera Module.
I'd like to produce a step by step tutorial on building custom models, for say object detection. (see https://youtu.be/Qo4T9EiyCwc for work so far), I can create the onnx or pytorch model for general use, and that bit works fine, however when I get to the step where I want to use the Sony Model Compression toolkit (MCT) this is where things get messy.
The process I'm going through to produce the model is:
1. Collect Images
2. Annotate Images (using labelimg)
3. Split Data - into train and val for training and testing the model
4. Train - run 50 or so epochs of training
5. Evaluate - looking at the graphs that are produced
6. Deploy - converting the model to onnx format using PyTorch
7. Quantize - using either PyTorch or the ONNX library
8. Compress using the MCT
I've created a jupyter notebook that processes the images I've collected and annotated (in Yolo format) - you can grab a copy of the files and notebook here: https://github.com/kevinmcaleer/object_detection - the Test_2.ipynb is the work in progress file.
If I run the imxconv-pt to convert the model I get the following errors:
```
2024-10-22 11:44:25,304 INFO : CODE: [START] Starting SDSPconv
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Caught exception while converting'quantized_model': ERROR: CODE: [VE] Verification Error: Quantization verification failed:
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Conv2D: /model.24/dfl/conv/Conv is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Reshape: /model.24/dfl/Reshape_1 is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Resize: /model.15/Resize is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Resize: /model.11/Resize is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Pad: /model.0/conv/Conv_pad_to1d_1 is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Pad: /model.0/conv/Conv_pad_to1d_0 is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Swish: /model.0/act/Mul_smr is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Conv2D: /model.0/conv/Conv is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Concat: /model.0/conv/Conv_concat is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer StrideSlice: /model.0/conv/Conv_pf(0;0) is not quantized
2024-10-22 11:45:20,872 ERROR: CODE: [GE] Layer StrideSlice: /model.0/conv/Conv_pf(0;0)_dim_1 is not quantized
...
```
However the model was quantized, so I'm not sure why its saying this. If someone from the Sony team can expand upon the quantization step that would really help - I promise to write this up as a mini-course on https://www.kevsrobots.com/learn/ Please provide a working example using Pytorch!
Many thanks,
Kevin McAleer
Ash said to post on here so the nice people at Sony can help us troubleshoot getting custom models working on the AI Camera Module.
I'd like to produce a step by step tutorial on building custom models, for say object detection. (see https://youtu.be/Qo4T9EiyCwc for work so far), I can create the onnx or pytorch model for general use, and that bit works fine, however when I get to the step where I want to use the Sony Model Compression toolkit (MCT) this is where things get messy.
The process I'm going through to produce the model is:
1. Collect Images
2. Annotate Images (using labelimg)
3. Split Data - into train and val for training and testing the model
4. Train - run 50 or so epochs of training
5. Evaluate - looking at the graphs that are produced
6. Deploy - converting the model to onnx format using PyTorch
7. Quantize - using either PyTorch or the ONNX library
8. Compress using the MCT
I've created a jupyter notebook that processes the images I've collected and annotated (in Yolo format) - you can grab a copy of the files and notebook here: https://github.com/kevinmcaleer/object_detection - the Test_2.ipynb is the work in progress file.
If I run the imxconv-pt to convert the model I get the following errors:
```
2024-10-22 11:44:25,304 INFO : CODE: [START] Starting SDSPconv
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Caught exception while converting'quantized_model': ERROR: CODE: [VE] Verification Error: Quantization verification failed:
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Conv2D: /model.24/dfl/conv/Conv is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Reshape: /model.24/dfl/Reshape_1 is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Resize: /model.15/Resize is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Resize: /model.11/Resize is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Pad: /model.0/conv/Conv_pad_to1d_1 is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Pad: /model.0/conv/Conv_pad_to1d_0 is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Swish: /model.0/act/Mul_smr is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Conv2D: /model.0/conv/Conv is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer Concat: /model.0/conv/Conv_concat is not quantized
2024-10-22 11:45:20,871 ERROR: CODE: [GE] Layer StrideSlice: /model.0/conv/Conv_pf(0;0) is not quantized
2024-10-22 11:45:20,872 ERROR: CODE: [GE] Layer StrideSlice: /model.0/conv/Conv_pf(0;0)_dim_1 is not quantized
...
```
However the model was quantized, so I'm not sure why its saying this. If someone from the Sony team can expand upon the quantization step that would really help - I promise to write this up as a mini-course on https://www.kevsrobots.com/learn/ Please provide a working example using Pytorch!
Many thanks,
Kevin McAleer
Statistics: Posted by kevsmac — Tue Oct 22, 2024 10:54 am