| 摘要: |
Visual features are unsatisfactory to effectively describe the visual semantics. However, single layer based semantic modeling may be not able to cope with complicated semantic contents. In this paper, we pro- pose Hierarchical Deep Semantic Representation (H-DSR), a hierarchical framework which combines se- mantic context modeling with visual features. First, the input image is sampled with spatially fixed grids. Deep features are then extracted for each sample in particular location. Second, using pre-learned classi- fiers, a detection response map is constructed for each patch. Semantic representation is then extracted from the map, which have a sense of latent semantic context. We combine the semantic and visual repre- sentations for joint representation. Third, a hierarchical deep semantic representation is built with recur- rent reconstructions using three layers. The concatenated visual and semantic representations are used as the inputs of subsequent layers for semantic representation extraction. Finally, we verify the effectiveness of H-DSR for visual categorization on two publicly available datasets: Oxford Flowers 17 and UIUC-Sports. Improved performances are obtained over many baseline methods. |
