PyTorch or ONNX to RKNN: a deployment checklist

“It converted, but the board disagrees” is the most common failure in this pipeline. The cause is rarely the converter and usually one of three decisions made around it: input layout, quantization mode, calibration data.

1. Confirm the model can be converted

pt2rknn covers YOLOv8 and five other network families across RK3562 to RK3588. When an operator is unsupported, the fix is usually in the model, not the tool: replace the custom operator, or split inference into two stages.

2. Choose quantization deliberately

• FP16: negligible accuracy loss, half the size. Good for validating the pipeline.
• INT8: the real speed and memory win, but calibration required.

Always keep an FP16 run as the baseline. When INT8 misbehaves, you need to know whether quantization or deployment code is responsible.

3. Prepare calibration data

INT8 accuracy depends on whether the calibration set represents real inputs:

• 100–300 images is enough, chosen for coverage of lighting and scenes;
• use images that will actually occur at inference, not the extreme cases from training;
• preprocessing must match inference exactly — normalization and channel order.

4. Locate the damage per layer

When accuracy drops, look at per-layer error instead of the aggregate. A sudden jump points at a quantization-sensitive layer; keeping that layer in FP16 while the rest runs INT8 is the usual fix.

5. Still disagreeing on device

Check in this order:

1. Preprocessing parity — BGR vs RGB, normalization, letterbox.
2. Output decoding — matching the export.
3. Input shape — matching the model definition.
4. Thresholds — identical to the PC comparison.

Roughly nine out of ten end up in items 1 and 2.

6. Why a web interface

This pipeline is iterative: change quantization, change calibration, change target chip. A browser UI removes the script-editing step and keeps conversion history, so the next similar model starts from known-good parameters.

---

Try it at pt2rknn.