GeoAI Tutorial: Extract Building Footprints from NAIP Imagery with U-Net and SAM

A new end-to-end tutorial shows how to build a GeoAI pipeline that pulls building footprints out of NAIP (National Agriculture Imagery Program) aerial photos. The workflow covers environment setup, tile generation, training a U-Net with a ResNet-34 encoder, sliding-window inference, polygon regularization, and a comparison between zero-shot segmentation (Grounding DINO plus SAM) and a pretrained Mask R-CNN model. The tutorial runs on Google Colab and connects to real-world data from Microsoft Planetary Computer and Overture Maps.

What happened

Researchers published a full walkthrough for extracting building footprints from NAIP aerial imagery, a publicly available high-resolution raster dataset used widely in US geospatial work. The tutorial uses the geoai-py Python library alongside PyTorch, segmentation-models-pytorch, and buildingregulariser.

Parameter	Value
Image chip size	512 x 512 pixels
Stride (sliding window)	256 pixels
Model architecture	U-Net with ResNet-34 encoder
Encoder pre-training	ImageNet weights
Input channels	3 (RGB)
Output classes	2 (building / background)
Batch size	8
Training epochs	12
Learning rate	0.001
Validation split	20%

How the pipeline works

The workflow runs in six broad stages. Training data comes from a Hugging Face dataset hosted at giswqs/geospatial, which includes a georeferenced RGB GeoTIFF and a GeoJSON file of building footprints.

1. Download NAIP raster imagery and vector building labels, then inspect coordinate reference systems and band statistics.
2. Slice the full raster into 512-pixel chips with a 256-pixel overlap. Chips are saved as paired image and mask tiles.
3. Train the U-Net model for 12 epochs and monitor validation loss to catch overfitting early.
4. Run sliding-window inference over an unseen test scene, stitching predictions back into a full-coverage output.
5. Convert raw binary masks into clean building polygons using geometric regularization, then score results with IoU and F1 metrics.
6. Optionally run Grounding DINO and SAM (Segment Anything Model) in zero-shot mode, and compare those outputs against a pretrained Mask R-CNN model, which handles instance segmentation (labeling each building as a separate object rather than a single class map).

Why does zero-shot segmentation matter here?

Training a supervised model like U-Net requires labeled data. Grounding DINO combined with SAM skips that step entirely: you give the model a text prompt such as “building” and it attempts to segment matching regions in the image without any task-specific training. The tutorial uses this as a benchmark, not a recommendation, measuring how well zero-shot methods hold up against a model actually trained on building footprints.

Mask R-CNN adds a third comparison point. Unlike U-Net, which produces a single class map, Mask R-CNN assigns a separate mask to each individual building instance. This matters when downstream tasks need a count of buildings or per-building area measurements.

Real-world extension

The tutorial closes by swapping the sample dataset for live data. NAIP imagery comes from Microsoft Planetary Computer, a cloud geospatial platform, and building labels come from Overture Maps, an open dataset backed by major mapping organizations. This step is marked optional in the config ("run_real_aoi": False by default) but shows the pipeline is not limited to toy examples.

Why it matters

Building footprint extraction has practical uses across urban planning, insurance risk assessment, disaster response, and property analytics. The barrier used to be expensive commercial software or proprietary datasets. This tutorial shows a path using only open-source tools and free-tier cloud compute.

For developers building location-aware products, the techniques here are directly transferable. If you are working on a e-commerce site that needs geographic coverage analysis, or any product tied to physical locations, having a reliable automated footprint layer can replace costly manual digitization.

The comparison between supervised training and zero-shot methods is also useful context as teams decide how much labeled data investment is actually worth it. According to the tutorial structure, the zero-shot path requires no training data but the quality gap versus a fine-tuned model is what the IoU and F1 scores are there to quantify.

Our take

This is a dense, well-structured tutorial that respects the reader’s time by putting all configuration in a single dictionary at the top. The architecture choices (U-Net with ResNet-34, ImageNet initialization) are conservative in the best sense: they work reliably without requiring exotic hardware or massive datasets.

The zero-shot versus fine-tuned comparison is the most practically useful part. Most teams exploring geospatial AI will face exactly this question: do we label data and train, or just throw SAM at it? Running both paths and scoring them with the same metrics is the right way to answer it.

One honest caveat: the tutorial runs on Colab and warns explicitly that training without a GPU “will still run but be much slower.” For production workloads, 12 epochs on 512-pixel tiles at batch size 8 is modest, and you will want to benchmark on your own imagery before committing to this stack. The real-world extension using Planetary Computer and Overture Maps is where the actual deployment questions start, and that section is marked optional and lightly covered.

If your team is evaluating AI-driven spatial analysis for a client project, this tutorial is a solid starting point. For teams that need help connecting these kinds of models to a live product, our AI integration service covers exactly that handoff from prototype to production. You can also see how we approach data-driven projects in our client case studies.

For more context on how AI agents and models are being pushed into real-world pipelines right now, see our coverage of AI agent developments at OpenAI, Anthropic, and Microsoft.

What to do about it

1. Clone or open the tutorial in Google Colab and set the hardware accelerator to a T4 GPU before running anything.
2. Run the pipeline end to end on the sample NAIP data first to confirm your environment is stable.
3. Enable "run_zero_shot": True and "run_pretrained": True in the config dictionary to generate all three comparison outputs.
4. Compare IoU and F1 scores across U-Net, Grounding DINO plus SAM, and Mask R-CNN before deciding which approach fits your data volume and labeling budget.
5. Switch the data sources to Microsoft Planetary Computer and Overture Maps to test against your actual area of interest.

Score all three segmentation approaches before picking one. The numbers will tell you whether the labeling effort is worth it for your specific imagery.

Source: Marktechpost