Objects as Attributes for Scene Classification

Yongsub Lim Applied Algorithm Lab., KAIST 2020-04-29 1

Introduction • Low-level features have used for visual tasks, but not enough as those become higher level • This paper proposes to use objects as attributes of scenes for scene classification – Objects become attributes of scenes 2020-04-29 2

Introduction • Attribute based methods for object recognition have shown promising results • For instance, polar bear can be described as white, fluffy object with paws • Such visual attributes summarize the low-level features into object parts , and other properties 2020-04-29 3

Introduction Not easy to distinguish these scenes based on just texture statistics!

2020-04-29 4

Object bank representation • Object filters are to characterize local image properties related to the presence/absence of objects 2020-04-29 5

Comparing to other popular methods Low-level based methods produce very similar results for images having very different meaning OB easily distinguish due to the semantic information 2020-04-29 6

Object bank representation • OB achieve reasonable recognition results on a very small number of scene training examples 2020-04-29 7

Object bank representation • OB is also good for a small number of object detectors 2020-04-29 8

What are ‘objects’ for object filters?

• More objects, better performance – – Semantic hierarchy becomes more prominent It is not a good way • One observation is that not all objects are of equal importance in natural images – Just need detectors for a few most important objects – There is a result that 3000~4000 concepts are enough to annotate video data 2020-04-29 9

Objects are not equally important • In this paper, 200 most frequent objects obtained from popular image datasets and image search engines are used 2020-04-29 10

Hierarchy of Selected Objects 2020-04-29 11

Experiments: basic-level • This paper uses a much simpler classifier than SPM 2020-04-29 12

Experiments: basic-level • OB is not a replacement of low-level image features, it offers important complementary information of the images 2020-04-29 13

Experiments: super-ordinate level • It is tested on UIUC-Sports dataset – Activities and events become classes • OB is even better than state-of-the-art 73.4% which uses all given object outlines and identities 2020-04-29 14

Experiments: super-ordinate level • We can see that a more semantic-level image representation overcomes confusion caused by low level features – eg. sailing and rowing 2020-04-29 15

Summary • Consider objects as attributes of scenes, use object bank representation for images • Need a modest number of objects which occurs much more frequently than the majority • OB is not only good itself not capture, it can , but also because it provides information which low-level features did boost performance significantly by combining features 2020-04-29 16