# Probabilistic Object Detection

# 1. Overview

**Contributions of this work:**- Introduce a new robotic vision task: probabilistic object detection

- Define a new performance measure for probabilistic object detection: probabilistic detection quality (PDQ)

- Evaluate PDQ

- Evaluate state-of-the-art detectors using PDQ

**Questions**:- Is probabilistic object detection necessary?

- Can/should this be applied in SLAM?

# 2. Motivation

- Current CV paradigm for object detection: deterministic bounding box and semantic class with confidence score/label distribution

- Evaluation metrics based on this idea of object detection, influence training

- Conventional object detectors can over confidently assign incorrect label

- Need spatial and label uncertainty estimates

# 3. Background & Related Work

## 3.1. Evaluation Metrics

- Detections classified with threshold on Intersection over Union (IoU)

- Average precision (AP): sort detections by confidence, compute area under precision-recall curve

- Mean average precision (mAP) - vary IoU threshold from 0.5:0.05:0.95, take mean of APs

## 3.2. Conventional Object Detection

- Typical detector output: bounding boxes and class label scores

- CNNs rapidly improving in accuracy and speed

- Spatial and semantic uncertainty not typically provided

## 3.3. Uncertainty Estimation in Object Detection

- Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban
Remote Sensing Images Using Deep Convolutional Neural Networks (Kampffmeyer, 2016)
- Monte Carlo dropout approximates Bayesian inference

- Pixel-wise classification uncertainty for semantic labels

- Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding (Kendall, 2017)
- Bayesian model outputs pixel-wise semantic segmentation with model uncertainty per class

- What Uncertainties Do We Need in Bayesian Deep
Learning for Computer Vision? (Kendall and Gal, 2017)
- Examine
*aleatoric*(observation) and*epistemic*(model) uncertainty

- Examine

- Evaluating Merging Strategies for Sampling-based Uncertainty
Techniques in Object Detection (Miller, 2019)
- Estimate spatial and classification uncertainties for object detection

- Use uncertainty to accept/reject detections in near open-set conditions

# 4. Probabilistic Object Detection

- Object detection consists of:
- Probability distribution over known labels

- Bounding box: Gaussian distributions for corner positions

# 5. Probability-based Detection Quality (PDQ)

- Frame $f$, evaluate detections $\mathcal{D}_j^f$ with ground truth objects $\mathcal{G}_i^f$

- Foreground Loss $L_{FG}$

- Background Loss $L_{BG}$

- Spatial Quality $Q_{S}$

- Label Quality $Q_{L}$

- Pairwise PDQ $\mathrm{pPDQ}$

**PDQ Score**$\mathrm{pPDQ}$

- Example PDQ scores

# 6. PDQ Evaluation

- Perform experiments with simulated object detectors
- For all ground truth objects, simulate detector by adding
*true variance*to detections

- Independent of true variance, simulated detector gives observations with some random
*reported variance*

- For all ground truth objects, simulate detector by adding

- PDQ score maximized when reported variance best matches true variance

- PDQ directly affected by label probability, vs mAP which only affected by dominant class

# 7. Evaluation of Object Detectors

- Evaluate state-of-the-art detectors
- Convert standard detections to probabilistic detections, assuming $P(\mathbf{x}\in\mathcal{S}_j^f)=1-\epsilon$ for pixels inside bounding box, $P(\mathbf{x}\notin\mathcal{S}_j^f)=\epsilon$ for pixels outside

- Probabilistic detectors
- MC-Dropout SSD: use MC Dropout with SSD-400 detector, BSAS clustering to estimate Gaussians

- probFRCNN: find detections suppressed by nonmax supression with IoU > 0.75, cluster and estimate Gaussians for corners

- Observations
- Probabilistic detectors perform best in PDQ

- Top performing standard detectors have poor spatial quality eg YOLOv3

- mAP measure does not penalise high false positives eg FRCNN X+FPN(0.05)

# 8. Conclusions and Future Work

- Performance metrics strongly influence
*how*object detectors are designed and trained to perform

- New performance measure for probabilistic object detection will steer work towards better spatial uncertainty estimation

- Extend to probabilistic instance segmentation - replace bounding box spatial probability density with segmentation mask density

# 9. Discussion

- Probabilistic object detection better suited to robotic deployment in real world

- How should probabilistic object detection be used in SLAM?
- PDQ metric, probabilistic approach to object detection will steer object detection field towards characterising uncertainty

- Add uncertain object labels to factor graph

- Future SLAM maps should incorporate instance segmentations - these need accurate uncertainty estimates