pochih/CBIR

Intro

This repository contains a CBIR (content-based image retrieval) system

Extract query image's feature, and retrieve similar ones from image database

Image src

Part1: Feature Extraction

In this system, I implement several popular image features:

• color-based
  • RGB histogram
• texture-based
  • gabor filter
• shape-based
  • daisy
  • edge histogram
  • HOG (histogram of gradient)
• deep methods
  • VGG net
  • Residual net

all features are modulized

Feature Fusion

Some features are not robust enough, and turn to feature fusion

• fusion.py

Dimension Reduction

The curse of dimensionality told that vectors in high dimension will sometimes lose distance property

• Random Projection

Part2: Evaluation

CBIR system retrieves images based on feature similarity

Robustness of system is evaluated by MMAP (mean MAP), the evaluation formulas is refer to here

• image AP : average of precision at each hit
  • depth=K means the system will return top-K images
  • a correct image in top-K is called a hit
  • AP = (hit1.precision + hit2.precision + ... + hitH.precision) / H
• class1 MAP = (class1.img1.AP + class1.img2.AP + ... + class1.imgM.AP) / M
• MMAP = (class1.MAP + class2.MAP + ... + classN.MAP) / N

Implementation of evaluation can found at evaluate.py

My database contains 25 classes, each class with 20 images, 500 images in total, depth=K will return top-K images from database

Method	color	daisy	edge	gabor	HOG	vgg19	resnet152
Mean MAP (depth=10)	0.614	0.468	0.301	0.346	0.450	0.914	0.944

Part3: Image Retrieval (return top 5 of each method)

Let me show some results of the system

query1 - women dress

query

color

daisy

edge

gabor

HOG

VGG19

Resnet152

query2 - orange

query

color

daisy

edge

gabor

HOG

VGG19

Resnet152

query3 - NBA jersey

query

color

daisy

edge

gabor

HOG

VGG19

Resnet152

query4 - snack

query

color

daisy

edge

gabor

HOG

VGG19

Resnet152

Part4: Usage of Repository

If you are interesting with the results, and want to try your own images,

Please refer to USAGE.md

The details are written inside.

Author

Po-Chih Huang / @pochih