Skip to content

Density-Based Counting

When you have to count many overlapping, densely packed objects (crowds of people, cells under a microscope, chickens in a coop, trees from satellite imagery), object detection is often the wrong tool. Density-map regression is the right one.

What This Is

Instead of predicting bounding boxes, you predict a density map D with the same spatial shape as the input. Every pixel of D is a real number, and sum(D) = count. Each annotated point becomes a small Gaussian bump at that location; the model learns to regress the sum of those bumps.

Training target:
  for each annotated point (x_i, y_i):  add Gaussian_σ(x - x_i, y - y_i) to D
  integral of D over the image ≈ number of annotated points

Loss: ||D_pred - D_target||²     (MSE on the density map)
Inference count:  n_hat = sum(D_pred)

This reframes "how many are there?" as a dense regression problem. The spatial structure of the density map also gives you where they are, for free.

When You Use It

  • objects are dense, overlapping, or partially occluded (crowds, dense cell clusters, herds)
  • bounding boxes are hard to define or annotate (fuzzy boundaries, highly variable scale)
  • you want a count and a spatial distribution (heatmap of where objects are)
  • the count can range from 0 to thousands per image — detection-then-count scales poorly at the high end

Do Not Use It When

  • objects are sparse and well-separated — detection works fine and gives you boxes for free
  • the task cares about individual identity (re-ID, tracking) — density maps do not preserve identity
  • you need to report bounding boxes to a downstream consumer — density maps do not contain boxes
  • the dataset is tiny (< 50 training images) — density regression needs spatial diversity to learn

Building The Target Map

For annotated points (x_1, y_1), ..., (x_N, y_N):

def build_density_target(points, H, W, sigma=4.0):
    D = np.zeros((H, W), dtype=np.float32)
    for (x, y) in points:
        # place a Gaussian centered at (x, y)
        yy, xx = np.ogrid[:H, :W]
        D += np.exp(-((yy - y)**2 + (xx - x)**2) / (2 * sigma**2))
    D /= (2 * np.pi * sigma**2)  # normalize so sum matches count
    return D

The kernel width σ matters:

  • fixed σ: simple; works when objects have similar size across the image
  • adaptive σ: set σ proportional to the distance to the k-nearest neighbor point — tighter bumps in crowded regions, wider bumps in sparse ones (the MCNN / CSRNet approach)

Architecture: Decoder On A Frozen Backbone

The classical CSRNet-style setup:

frozen encoder (e.g. VGG conv block 1-10, or a SSL-pretrained backbone)
    → features at 1/8 resolution
learned decoder:
    Conv → ReLU → Conv → ReLU → Conv → ReLU
    final 1×1 conv → 1-channel density map
    upsample to original resolution

Only the decoder trains. Typical parameter count: 1-10M for the decoder, versus 20M-100M frozen in the backbone.

This is also the pattern behind segmentation-with-a-frozen-backbone, depth estimation, and any other dense-prediction task. The encoder "sees", the small learned decoder "translates" features into the output space you need.

Loss And Metrics

Training loss. MSE on the density map is standard. Two common refinements:

  • per-pixel weighting — weight foreground (nonzero density) pixels more than background, since most pixels are zero
  • count-consistency term — add |sum(D_pred) - sum(D_target)| to the loss to regularize the total count explicitly

Evaluation metrics. The task is usually scored on count, not density:

  • MAE = mean absolute error on counts (lower is better)
  • MSE = mean squared error on counts (penalizes big misses more)
  • GAME (grid average mean error) = MAE computed on subdivisions of the image; penalizes getting the right count in the wrong place

What To Inspect

  • predicted density map overlaid on the input image — does it light up where the objects are?
  • predicted count vs true count scatter plot — is it unbiased, or systematically over/underestimating?
  • count error binned by true count — the model might be good at small counts and bad at large ones (or vice versa)
  • σ sweep — try σ ∈ {2, 4, 8, 16} and see how count MAE changes; there is usually a clear optimum
  • background false-positive density — does the model put density on empty regions? That is pure overcount.

Failure Pattern

  • σ-miscalibration. Ground-truth σ was 4, but objects in new images are twice the size. The model regresses too-tight bumps; integrals undercount. Always use the same σ (or adaptive σ) at train and eval.
  • resolution mismatch. You train at 256×256 but evaluate on 1024×1024 without rescaling density maps. A density map's integral scales with the area — you must either rescale by (input_area / train_area) or predict at fixed resolution.
  • learning a smooth blob. The decoder learns a smooth low-frequency prior that roughly predicts average density everywhere. Count is approximately right, but the spatial distribution is garbage. Fix: inspect the density map, not just the count.
  • class imbalance. Most pixels are zero; the model learns to predict zero everywhere. Fix: upsample crowded regions or weight the loss by density mass.
  • point annotation error. Annotators dropped points; your targets undercount the real density. You cannot train past the annotation ceiling.

Quick Checks

  • does sum(D_target) equal the point count per image? (sanity-check the target builder)
  • is σ the same at train and eval?
  • is the model's predicted count distribution centered on the true count distribution?
  • does the density map look spatially correct, not just aggregate-correct?
  • is the decoder the only thing learning? (the encoder should be frozen with requires_grad=False)

Practice

Run academy/labs/density-map-counting/src/counting_workflow.py:

  • generates a synthetic dataset of scenes with 5-80 small objects placed randomly (with overlap allowed)
  • builds density targets with σ=4
  • trains three heads on top of a frozen small CNN backbone: naive regression of the count scalar, detection-and-count (connected components on a binary threshold), density-map regression
  • measures MAE per method and per count bin
  • visualizes predicted vs true density maps and count-vs-count scatter

You should leave able to explain when density-map regression beats detection-and-count, why σ matters, and what to look at when your counts are right but your density map is wrong.

Longer Connection

Density regression is one instance of a broader pattern: represent the target as a spatial distribution, not a discrete list. Instances:

  • density maps — counting
  • heatmap regression for keypoints (human pose estimation, landmark detection) — put a Gaussian at each keypoint, regress the heatmap
  • segmentation masks — every pixel gets a class probability
  • saliency maps — every pixel gets an importance score (see Saliency and Explainability)

All of these share the "dense output over the spatial grid" structure, which is also why they share the encoder-decoder architecture with a frozen backbone. Once you know how to train one, you can train any of them with small changes to the output head and the loss.