Tips and Troubleshooting Steps for YOLOX Object Detection on Raspberry Pi

R2026b

This section provides tips for improving results and solutions to common issues when using the Train and Deploy a YOLOX Object Detector on Raspberry Pi Using Transfer Learning example.

Tips for Data Collection

Use these guidelines to collect and label high-quality training data that maximizes detector accuracy with minimal effort.

1. Video Capture

Record 30-60 seconds of video per object class. A single 30-second video at 30 fps provides approximately 900 frames for labeling.

For best results:

  • Record from multiple angles: front, side, top, and 45 degree views.

  • Vary lighting conditions: indoor fluorescent, natural window light, and outdoor.

  • include background variation: plain surfaces, cluttered desks, and environments similar to where the detector would operate.

  • Position the object at different distances from the camera: near, medium, and far.

  • Use a stable surface or tripod to reduce motion blur.

  • Record at 720p (1280x720) minimum resolution. The preferred resolution is1080p.

2. Labeling Tips

Follow these practices to produce accurate and consistent bounding box annotations in the Video Labeler (Computer Vision Toolbox) app.

  • Label 10-20 frames manually before using the point tracker automation in the app.

  • Draw bounding boxes that tightly fit the object without excessive padding.

  • Re-initialize tracking every 100-200 frames or when the bounding box drifts.

  • Review tracked labels by scrubbing through the video every 50-100 frames.

  • Maintain consistent labeling standards — keep the same person labeling if possible.

3. Minimum Data Requirements

Use these thresholds to determine whether your dataset is large enough for effective transfer learning.

  • A minimum of 300 images per class is required for training to converge.

  • For transfer learning with YOLOX, 500 to 900 images per class is recommended to achieve reliable detection accuracy.

  • Beyond 1000 images, returns diminish for small object sets. Focus on increasing dataset diversity rather than adding more images.

Tips for Training

Use these guidelines to select the right model variant, tune hyperparameters, and interpret training progress for your custom detector.

1. Choose a Model Variant

Select a YOLOX variant based on your deployment target.

  • Select nano-coco variant (~3 MB) for the best balance of speed and accuracy for Raspberry Pi® and is the recommended starting point.

  • Select tiny-coco (~8 MB) when higher detection accuracy is more important than inference speed.

  • Select small-coco (~15 MB) when maximizing detection accuracy is the highest priority and slower inference is acceptable.

  • Select medium-coco (~40 MB) only for desktop or powerful edge devices — it is not recommended for Raspberry Pi due to high memory usage and slow inference.

2. Understand Hyperparameters

Adjust these key training parameters to balance accuracy, speed, and memory usage for your system.

  • inputSize — [416, 416, 3] is the recommended balance between accuracy and speed. Use [320, 320, 3] for faster training and inference, or [640, 630, 3] for your system.

  • learningRate — 1e-3 is a good starting point for transfer learning. Lower to 1e-4 if training loss oscillates or become NaN.

  • miniBatchSize — 8 works for most systems. Reduce to 4 or 2 if you encounter out-of-memory errors.

  • maxEpochs — 60 epochs is typical for transfer learning. Increase to 80-100 if validation loss is still decreasing at the end of training.

3. Interpreting Training Progress

Use the training progress plot to diagnose whether training is proceeding normally or required intervention.

Monitor the training progress plot:

  • Healthy training — Both training loss and validation loss decrease and stabilize.

  • Overfitting — Training loss decreases but validation loss increases. Reduce macEpochs or add more training data.

  • Underfitting — Neither loss decreases substantially. Increase maxEpochs or try a larger model variant.

  • Unstable training — Loss oscillates or becomes NaN. Reduce learningRate and try again.

Tips for Deployment

Use these recommendations to configure your Raspberry Pi hardware, camera, and model for reliable real-time object detection.

1. Hardware Setup

Make sure your Raspberry Pi meets these hardware recommendations for stable inference performance.

  • Use Raspberry Pi 5 (4 GB or 8 GB RAM) for best inference performance (~100-150 ms per frame with nano-coco).

  • Raspberry Pi

    4 Model B (4 GB or 8 GB RAM) is supported but provides slower inference (~200-300 ms per frame).

  • Attach a heatsink or active fan to prevent thermal throttling during continuous inference.

  • Use a 5V/5A power supply for Raspberry Pi 5 or 5V/3A for Raspberry Pi 4B.

2. Camera Configuration

Verify these camera settings before running inference to avoid capture errors or resolution mismatches.

  • Verify the camera index on your Raspberry Pi.

  • Match the camera resolution to your detector input size for best results or use a standard resolution (640x480) and let the preprocessing subsystem resize.

  • Test the camera independently before deploying. Run libcamera-hello on the Pi terminal to confirm it works.

3. Model Configuration

Confirm these model settings before building and deploying to Raspberry Pi.

  • Set the trained MAT-file path in the Deep Learning Object Detector (Computer Vision Toolbox) block before building.

  • Set the system target file to ert.tlc (Embedded Coder®) for standalone code generation.

  • Set Hardware board to Raspberry Pi (64bit) in the model configuration dialog box.

Troubleshooting Data Collection Issues

If you encounter problems during video recording, labeling, or frame extraction, use the following solutions to resolve them.

1. Not Enough Training Data

Issue — Fewer than 300 images per class after frame extraction.

Troubleshooting steps:

  • Capture more video footage (2-3 minutes minimum per class).

  • Use a lower SamplingFactor value when calling objectDetectorTrainingData to extract more frames.

  • Record the object in additional scenarios and environments.

  • Data augmentation (applied automatically during training) helps compensate for smaller datasets.

2. Point Tracker Loses Object

Issue — Bounding box drifts away from the object or stops tracking.

Troubleshooting steps:

  • Record video with slower, more stable camera movement.

  • Make sure of consistent lighting. Avoid sudden shadows or backlighting changes.

  • Manually re-initialize tracking every 100-200 frames.

  • Label more frames manually before using automation to give the tracker a stronger initialization.

3. Video Labeler App is Slow

Issue — App freezes or has slow playback.

Troubleshooting steps:

  • Use 720p video instead of 4K to reduce memory usage.

  • Pre-extract frames and import them as an image sequence.

  • Close other applications to free system memory.

  • Work with shorter video segments (1-2 minutes each).

4. Inconsistent Bounding Box Sizes

Issue — Boxes vary in tightness around the object across frames.

Troubleshooting steps:

  • Establish labeling guidelines before starting (for example, include the full object boundary with minimal padding).

  • Review and adjust boxes every 50-100 frames.

  • Use the zoom feature in the Video Labeler app for precise boundary placement.

Troubleshooting Training Issues

If training fails, produces unexpected results, or takes too long, use the following solutions to diagnose and fix the issue.

1. Out of Memory Error

Issue — Training crashes with a memory error.

Troubleshooting steps:

  • Reduce miniBatchSize to 4, 2, or 1.

  • Reduce inputSize to [320, 320, 3].

  • Close other applications to free system memory.

  • Set executionEnvironment to 'cpu' if GPU memory is insufficient.

2. Loss not decreasing or becoming NaN

Issue — Training loss stays constant or jumps to NaN.

Troubleshooting steps:

  • Reduce learningRate to 1e-4 or 5e-5.

  • Verify that bounding boxes in your labeled data are valid (positive width and height, within image bounds).

  • Check for corrupted image files in the training dataset.

  • Try a different model variant (for example, tiny-coco instead of nano-coco).

3. Training is slow

Issue — Training takes several hours on CPU.

Troubleshooting steps:

  • Use a GPU if available by setting executionEnvironment to 'gpu'.

  • Reduce inputSize to [320, 320, 3].

  • Reduce maxEpochs to 30 for a faster initial test.

  • Training on CPU is expected to be slower — allow 1-3 hours depending on dataset size and hardware.

4. Poor validation performance

Issue — Low confidence scores or missed detections on validation images.

Troubleshooting steps:

  • Collect more training data (aim for 500+ images per class).

  • Improve data diversity with more angles, lighting conditions, and backgrounds.

  • Increase maxEpochs to 80 or 100.

  • Try a larger model variant (tiny-coco or small-coco).

  • Lower the detection threshold to 0.2 or 0.25 during evaluation.

  • Review labeling quality — incorrect or inconsistent labels degrade performance.

5. Overfitting

Issue — Training loss is low but validation loss is high or increasing.

Troubleshooting steps:

  • Reduce maxEpochs to stop training earlier (before validation loss diverges).

  • Collect more diverse training data.

  • Increase augmentation strength in the configuration (higher brightness, contrast, saturation values).

  • Verify that the validation set is representative of the data the detector will encounter.

6. Detector training fails to start

Issue — Error occurs before training begins.

Troubleshooting steps:

  • Verify that Computer Vision Toolbox™ and Deep Learning Toolbox™ are installed.

  • Ensure the gTruth.mat file contains a valid groundTruth object.

  • Confirm that classNames is not empty.

  • Check that all images referenced in the training data exist on disk.

Troubleshooting Deployment Issues

If the deployed detector does not start, performs poorly, or causes system instability on Raspberry Pi, use the following solutions to identify and resolve the issue.

1. Cannot connect to Raspberry Pi

Issue — Connection timeout or SSH error when calling raspi.

Troubleshooting steps:

  • Verify that the Raspberry Pi is powered on and fully booted.

  • Ping the Raspberry Pi IP address from your computer.

  • Verify SSH is enabled on the Pi by running sudo raspi-config and checking Interface Options > SSH.

  • Confirm the correct IP address on the Raspberry Pi by running hostname -I.

  • Check that both devices (Raspberry Pi and host computer) are on the same network subnet.

  • Try a direct Ethernet connection instead of Wi-Fi®.

  • Rerun raspisetup in MATLAB® to reconfigure the connection.

2. Unable to detect camera

Issue — Camera block shows an error or returns no image.

Troubleshooting steps:

  • Check the camera ribbon cable connection at both ends (camera board and Pi board).

  • Verify the camera interface is enabled by running sudo raspi-config in the Pi terminal and checking Interface Options > Camera.

  • Test the camera directly on the Pi by running libcamera-hello or libcamera-still -o test.jpg.

  • Make sure the power supply provides sufficient current (cameras draw additional power).

  • Verify camera compatibility — Camera Module V2 and V3 are supported.

3. Camera object already in use

Issue — Error stating the camera or webcam is already in use.

Troubleshooting steps:

  • Clear the persistent webcam object by running clear deploy_yolox_realtime in Command Window.

  • If using a webcam object directly, clear it by running clear cam in Command Window.

  • Restart the MATLAB session if the above does not release the camera.

4. Very slow inference on Raspberry Pi

Issue — More than 500 ms per frame or very low frame rate.

Troubleshooting steps:

  • Use nano-coco model variant instead of larger variants.

  • Reduce input size to [320, 320, 3].

  • Lower the camera resolution to 640x480.

  • Check for CPU thermal throttling by running vcgencmd get_throttled on the Pi terminal.

  • Add a heatsink or active fan if the Pi is overheating.

  • Close background processes on the Pi.

  • Upgrade to Raspberry Pi 5 for better performance.

5. Detector not loading in model

Issue — Error loading the network or parameter mismatch when opening the model.

Troubleshooting steps:

  • Verify the detector variable exists in the workspace by running whos detector in the Command Window.

  • Check that the file path to the deployment MAT-file is correct.

  • Make sure the detector is a yoloxObjectDetector object.

  • Try loading the deployment package directly by running load('yolox_deployment_package.mat') in the Command Window.

  • Regenerate the deployment package from the trained model.

6. Poor detection performance on Raspberry Pi

Issue — Detector works well in MATLAB but produces poor results on the Pi.

Troubleshooting steps:

  • Check that lighting conditions during deployment are similar to training conditions.

  • Verify camera focus and exposure settings.

  • Make sure the camera resolution matches or is compatible with the detector input size.

  • Test with images similar to the training data before testing with live camera input.

  • Lower the detection threshold if objects are being missed.

  • Verify that the correct detector version is deployed (check the MAT-file timestamp).

7. Raspberry Pi freezes or crashes

Issue — System becomes unresponsive or reboots during inference.

Troubleshooting steps:

  • Check the power supply and use 5V/5A for Pi 5 or 5V/3A for Pi 4B.

  • Monitor temperature on the Raspberry Pi terminal by running vcgencmd measure_temp and keep below 80 degrees C.

  • Add a cooling solution (heatsink or active fan).

  • Reduce CPU load by using a smaller model variant or lower resolution.

  • Check SD card health — corrupted cards cause intermittent crashes.

  • Monitor memory usage and increase swap file size if RAM is exhausted.

See Also