Exploring urban safety perception with vision-language models and semantically grounded image counterfactuals

Resumo

Over several decades, urban perception has emerged as an important research domain spanning urban planning, mental well-being, and perception mapping. Foundational theories such as Broken Windows have highlighted the relationship between the built environment and human behavior. More recently, researchers have leveraged street-view imagery together with auxiliary datasets, including crime statistics, health indicators, and demographic information, to model how people perceive urban spaces. Despite their success, existing approaches largely rely on similar machine learning pipelines and provide limited insight into the visual factors that influence perceived safety. Although these models can estimate perceptual attributes, their opaque decision-making processes hinder transparency and reduce their usefulness for informing urban design. Counterfactual image generation offers a promising direction for interpretability, but current methods often produce unrealistic or poorly controlled edits, limiting their practical value. This thesis makes four contributions. First, it reviews the urban perception literature to identify methodological limitations and research opportunities that motivate the subsequent work. Second, it investigates which visual and semantic elements of street-view images are most consistently associated with perceived safety through UrbanFormer, a model that jointly predicts perception scores and explains its own decisions. Third, building on this analysis, it introduces UrbanPD4k, a dataset of manually annotated physical disorder elements, and UrbanCFX, a pipeline that translates counterfactual scene differences into human-readable recommendations, making model outputs more accessible to non-specialists. Fourth, it introduces UrbanCounterViz, a visual analytics system for interpretable and steerable exploration of urban perception that integrates generative modeling and perception analysis. Rather than relying on unconstrained image generation, the system grounds edits in real observed transformations and presents them as photorealistic, inspectable visualizations through interactive, human-in-the-loop exploration. Evaluations based on usage scenarios, expert interviews, and an exploratory user study provide evidence that UrbanCounterViz can produce edits perceived as grounded, interpretable, and credible, supporting its use as an early-stage reasoning tool for data-driven urban design and planning.

Ficha do documento

Tipo

Tese

Ano

2026

Instituição

Fundação Getulio Vargas

Fonte

Repositório da FGV

Idioma

Português

Acesso

Acesso aberto

Identificador

oai:repositorio.fgv.br:10438/41074

Temas

Planejamento UrbanoTecnologiaDadosGovernança

Palavras-chave

Systematic reviewPRISMAUrban perceptionVisual analyticsGuided image editingCounterfactual imagesStreet-view imagerySemantic segmentationRevisão sistemáticaPercepção urbanaAnálise visualEdição guiada de imagensImagens contrafactuaisImagens de ruasSegmentação semânticaPercepção humana de segurançaTecnologiaPlanejamento urbanoInteligência artificialAnálise de imagemSemântica - Processamento de dadosAprendizado do computador