Software Erases Pedestrian from Google Street View Images

Google's (News - Alert) Street View, a Google Maps feature, offers live, ground-level photos to give users panoramic views of cities, towns and rural areas across the world. These views are created by stitching together images taken from a moving vehicle, so people or pedestrians walking around, as well as motor vehicles with plate numbers, can appear in the final image.

Although, Google Street View currently blurs faces and license plates from its images, clothes, body shape, and height combined with geographical location can be enough to make some pedestrians personally identifiable even if the face is blurred out, according to scientists at the University of California in San Diego.

That is causing an uproar in some countries, while others are working on how to block the people and car information from those images.

In fact, scientists at the UC San Diego have found ways to take care of that problem. Computer science graduate student Arturo Flores and computer science professor Serge Belongie of UC San Diego have developed proof-of-concept software that removes pedestrians from Google Street View images. This work was presented in June at the IEEE (News - Alert) International Workshop on Mobile Vision.

In this system, the software removes pedestrians and replaces the holes in the images with an approximation of the actual background behind each pedestrian. These corresponding background pixels are pulled from the image taken right before or right after the image in question

The next step, according to Flores, is to remove groups of pedestrians from single images.

While it is relatively “ghost free”, the proof-of-concept system is not perfect. For instance, the pedestrian remover does occasionally produce strange results, like dogs on leashes with no owners, and shoes with feet but nothing else, according to the UC San Diego paper.

In addition, the system struggles to generate background pixels when the pedestrian happens to be walking in the same direction as the vehicle at just the right speed. In these cases, the pedestrian may cover up the same spot in multiple frames, foiling the computer scientists’ pixel-swapping approach to removing pedestrians, the paper explains.

Furthermore, according to the scientists, the pedestrian remover only works in urban settings, where the pixels blocked by people are often on a dominant planar surface, which makes them simpler to replace.

Consequently, the software works well for a person walking by a mural of horses grazing in a pasture because the mural in the background is flat. However, if the same person was on a country road walking by actual horses grazing in a pasture, the system would not be effective because this background is not predominately flat.