Skip to main content

Computer Scientists Build "Pedestrian Remover"

Imagine encountering leashed dogs without dog walkers, or shoes filled just with ankles - when scoping out potential apartments using Google Street View. These are the sorts of visual hiccups that an experimental computer vision system occasionally generates when it automatically removes individual pedestrians from images that populate Google Street View.

The as-yet unnamed system removes pedestrians from urban scenes pulled from Google Street View - a service that provides panoramic views of cities, towns and rural areas across the world. Street views are constructed by stitching together overlapping images taken from a moving vehicle.

Computer science graduate student Arturo Flores developed this proof-of-concept system. Flores and computer science professor Serge Belongie presented the work in June at the IEEE International Workshop on Mobile Vision.

The project explores one way that computer vision could be used to preserve privacy in public environments in our digital age. Google Street View currently blurs faces and license plates from its images. Nevertheless, clothes, body shape, and height combined with geographical location can be enough to make some pedestrians with blurred faces personally identifiable. Removing pedestrians in this way helps address privacy concerns.

The new system removes pedestrians and replaces the holes in the images with an approximation of the actual background behind each pedestrian. These corresponding background pixels are pulled from the image taken right before or right after the image in question.

The pedestrian removal is relatively "ghost free" - meaning that the artifacts caused by the pixel swapping are usually not distracting. But the pedestrian remover does occasionally produce strange results - like dogs on leashes with no owners, and shoes with feet but nothing else.

One next step, according to Flores, is to remove groups of pedestrians from single images.

"I'm always trying to get the students to think about applying computer vision to real-world data," said Belongie.

When the automatic pedestrian remover replaced the woman (left) with an image of the building behind her (right), the umbrella remained.


Print Article