U-Net · Semantic segmentation

U-Net: outlining the shapes that matter.

We use U-Net to find crosswalk paint in aerial imagery, covering every pixel and every block. Here's how it works, why its "U" shape is unusually well-suited to the job, and where it gets fooled.

Plain-language definition

What is U-Net?

U-Net is a semantic segmentation model. Where YOLO draws boxes, U-Net classifies every pixel. Show it an aerial tile and it returns a mask of the same size, with each pixel labeled "crosswalk" or "not crosswalk."

That distinction matters for planners. A bounding box gives you a rough location. A pixel mask gives you a shape you can measure (area, length, width, orientation) and convert into a polygon you can drop straight into ArcGIS or QGIS.

If YOLO is a quick spotter, U-Net is a careful colorist. It doesn't say "there's a crosswalk over there." It says "these exact 4,832 pixels are the crosswalk."

Why "U-Net"?

The architecture is shaped like the letter U.

The left side compresses the image into "what's in it." The right side expands it back out into "where it is." Skip connections (the dashed arrows) carry fine detail across the U so the output stays sharp.

Input tile Encode Encode Encode Bottleneck "What is in it?" Decode Decode Decode Output mask main flow skip connections (preserve detail)

The encoder loses spatial detail in exchange for understanding. The decoder gets it back. Skip connections let the model reconstruct sharp boundaries, exactly what crosswalk edges need.

From tile to polygon

What U-Net does to one aerial tile.

aerial tile U-Net pixel mask vectorize GeoJSON

Aerial tile → pixel mask → polygons. The polygons are what actually ships in the GeoJSON layer.

In our pipeline

How we use U-Net for crosswalk segmentation.

  1. Tile. The 15 cm/pixel PASDA imagery is cut into overlapping tiles small enough to fit through the model.
  2. Predict. Each tile passes through U-Net; the output is a same-sized mask with per-pixel crosswalk probability.
  3. Stitch. Tile masks are merged into a single citywide raster, with overlap regions averaged to smooth seams.
  4. Vectorize. The raster is converted to polygons. Tiny specks are removed; long, thin painted shapes are kept.
  5. Export. The result is a GeoJSON crosswalk layer that joins to intersections, transit stops, or your priority corridors.

Results

How well it found the crosswalks.

F1 score

0.94Combined precision and recall

A high F1 in segmentation means the model is finding most real crosswalk pixels and rarely mistaking other paint for them.

Precision · Recall

0.93Precision: true positive rate
0.95Recall: coverage of real paint

Recall slightly outpaces precision: the model errs toward finding everything painted, then we filter.

What it produced

3,610Crosswalk polygons
8Philadelphia neighborhoods

Each polygon is a planning-grade feature: usable for joining, measuring, and prioritizing.

Honest limits

Where U-Net stumbles.

  • Look-alike paint. Parking-lot striping, loading-zone hatching, and bus-bay markings can all read as "crosswalk-ish" depending on resolution and angle. Most are filterable; some aren't.
  • Faded markings. Very faded paint produces low-confidence pixels that may fall below the threshold and disappear. That's a gap in data quality, not a model failure, but it shows up in the layer the same way.
  • Snow, leaves, shadows. Anything that occludes or distorts the visual signal of paint will hurt segmentation quality. The imagery's seasonal vintage matters.
  • New construction. If a crosswalk was painted after the imagery vintage, the model can't see it. The output is only as fresh as the tiles.

Practical takeaway: the segmentation layer is excellent for "where are the painted crossings as of this imagery." It is not a substitute for an inventory tied to a permit system. Use it for screening; verify the rest with field walks.

Keep going

Round out the picture.

Companion model · Detection

YOLO 101

U-Net handles paint; YOLO handles signs and lights. Same project, different question: boxes instead of pixels.

Read YOLO 101 →

Try it

See U-Net polygons in the explorer

Pick a neighborhood and watch crosswalk polygons appear. Click through to verify them against Google Street View.

Open the app →