OpenCV实现图像搜索引擎(Image Search Engine)

雨点打透心脏的1/2处 2022-09-15 00:08 371阅读 0赞
一.原理
  1. 1. 图像搜索原理
  2. 图像搜索算法基本可以分为如下步骤:
  3. 提取图像特征。如采用SIFT、指纹算法函数、哈希函数、bundling features算法等。当然如知乎中所言,也可以针对特定的图像集群采用特定的模式设计算法,从而提高匹配的精度。如已知所有图像的中间部分在颜色空间或构图上有显著的区别,就可以加强对中间部分的分析,从而更加高效地提取图像特征。
  4. 图像特征的存储。一般将图像特征量化为数据存放于索引表中,并存储在外部存储介质中,搜索图片时仅搜索索引表中的图像特征,按匹配程度从高到低查找类似图像。对于图像尺寸分辩率不同的情况可以采用降低采样或归一化方法。
  5. 相似度匹配。如存储的是特征向量,则比较特征向量之间的加权后的平方距离。如存储的是散列码,则比较Hamming距离。初筛后,还可以进一步筛选最佳图像集。
  6. 2. 图片搜索引擎算法及框架设计
  7. 基本步骤
  8. 采用颜色空间特征提取器和构图空间特征提取器提取图像特征。
  9. 图像索引表构建驱动程序生成待搜索图像库的图像特征索引表。
  10. 图像搜索引擎驱动程序执行搜索命令,生成原图图像特征并传入图片搜索匹配器。
  11. 图片搜索匹配内核执行搜索匹配任务。返回前limit个最佳匹配图像。

color_descriptor.py

  1. # -*- coding: utf-8 -*-
  2. # @Time : 2021/10/9 9:44
  3. # @Author :
  4. import cv2
  5. import numpy
  6. """ 颜色空间特征提取器ColorDescriptor 类成员bins。记录HSV色彩空间生成的色相、饱和度及明度分布直方图的最佳bins分配。bins分配过多则可能导致程序效率低下,匹配难度和匹配要求过分苛严;bins分配过少则会导致匹配精度不足,不能表证图像特征。 成员函数describe(self, image)。将图像从BGR色彩空间转为HSV色彩空间(此处应注意OpenCV读入图像的色彩空间为BGR而非RGB)。生成左上、右上、左下、右下、中心部分的掩模。中心部分掩模的形状为椭圆形。这样能够有效区分中心部分和边缘部分,从而在getHistogram()方法中对不同部位的色彩特征做加权处理。 """
  7. class ColorDescriptor:
  8. __slot__ = ["bins"]
  9. def __init__(self, bins):
  10. self.bins = bins
  11. def getHistogram(self, image, mask, isCenter):
  12. """ :param image: :param mask: :param isCenter: :return: """
  13. # get histogram
  14. imageHistogram = cv2.calcHist([image], [0, 1, 2], mask, self.bins, [0, 180, 0, 256, 0, 256])
  15. # normalize
  16. imageHistogram = cv2.normalize(imageHistogram, imageHistogram).flatten()
  17. if isCenter:
  18. weight = 5.0
  19. for index in range(len(imageHistogram)):
  20. imageHistogram[index] *= weight
  21. return imageHistogram
  22. def describe(self, image):
  23. image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
  24. features = []
  25. # get dimension and center
  26. height, width = image.shape[0], image.shape[1]
  27. centerX, centerY = int(width * 0.5), int(height * 0.5)
  28. # initialize mask dimension
  29. segments = [(0, centerX, 0, centerY), (0, centerX, centerY, height), (centerX, width, 0, centerY),
  30. (centerX, width, centerY, height)]
  31. # initialize center part
  32. axesX, axesY = int(width * 0.75) / 2, int(height * 0.75) / 2
  33. ellipseMask = numpy.zeros([height, width], dtype="uint8")
  34. # img, center, axes, angle, startAngle, endAngle, color, thickness=None, lineType=None, shift=None
  35. cv2.ellipse(ellipseMask, (centerX, centerY), (int(axesX), int(axesY)), 0, 0, 360, 255, -1)
  36. # initialize corner part
  37. for startX, endX, startY, endY in segments:
  38. cornerMask = numpy.zeros([height, width], dtype="uint8")
  39. cv2.rectangle(cornerMask, (startX, startY), (endX, endY), 255, -1)
  40. cornerMask = cv2.subtract(cornerMask, ellipseMask)
  41. # get histogram of corner part
  42. imageHistogram = self.getHistogram(image, cornerMask, False)
  43. features.append(imageHistogram)
  44. # get histogram of center part
  45. imageHistogram = self.getHistogram(image, ellipseMask, True)
  46. features.append(imageHistogram)
  47. # return
  48. return features

index.py

  1. # -*- coding: utf-8 -*-
  2. # @Time : 2021/10/9 9:45
  3. # @Author :
  4. import color_descriptor
  5. import structure_descriptor
  6. import glob
  7. import argparse
  8. import cv2
  9. """ 图像索引表构建驱动index.py。 引入color_descriptor和structure_descriptor。用于解析图片库图像,获得色彩空间特征向量和构图空间特征向量。 用argparse设置命令行参数。参数包括图片库路径、色彩空间特征索引表路径、构图空间特征索引表路径。 用glob获得图片库路径。 生成索引表文本并写入csv文件。 可采用如下命令行形式启动驱动程序: python index.py --dataset dataset --colorindex color_index.csv --structure structure_index.csv """
  10. searchArgParser = argparse.ArgumentParser()
  11. searchArgParser.add_argument("-d", "--dataset", required=True,
  12. help="Path to the directory that contains the images to be indexed")
  13. searchArgParser.add_argument("-c", "--colorindex", required=True,
  14. help="Path to where the computed color index will be stored")
  15. searchArgParser.add_argument("-s", "--structureindex", required=True,
  16. help="Path to where the computed structure index will be stored")
  17. arguments = vars(searchArgParser.parse_args())
  18. idealBins = (8, 12, 3)
  19. colorDesriptor = color_descriptor.ColorDescriptor(idealBins)
  20. output = open(arguments["colorindex"], "w")
  21. for imagePath in glob.glob(arguments["dataset"] + "/*.jpg"):
  22. imageName = imagePath[imagePath.rfind("/") + 1:]
  23. image = cv2.imread(imagePath)
  24. features = colorDesriptor.describe(image)
  25. # write features to file
  26. features = [str(feature).replace("\n", "") for feature in features]
  27. output.write("%s,%s\n" % (imageName, ",".join(features)))
  28. # close index file
  29. output.close()
  30. idealDimension = (16, 16)
  31. structureDescriptor = structure_descriptor.StructureDescriptor(idealDimension)
  32. output = open(arguments["structureindex"], "w")
  33. for imagePath in glob.glob("dataset" + "/*.jpg"):
  34. imageName = imagePath[imagePath.rfind("/") + 1:]
  35. image = cv2.imread(imagePath)
  36. structures = structureDescriptor.describe(image)
  37. # write structures to file
  38. structures = [str(structure).replace("\n", "") for structure in structures]
  39. output.write("%s,%s\n" % (imageName, ",".join(structures)))
  40. # close index file
  41. output.close()

searchEngine.py

  1. # -*- coding: utf-8 -*-
  2. # @Time : 2021/10/9 9:45
  3. # @Author :
  4. import color_descriptor
  5. import structure_descriptor
  6. import searcher
  7. import argparse
  8. import cv2
  9. """ 图像搜索引擎驱动searchEngine.py。 引入color_descriptor和structure_descriptor。用于解析待匹配(搜索)的图像,获得色彩空间特征向量和构图空间特征向量。 用argparse设置命令行参数。参数包括图片库路径、色彩空间特征索引表路径、构图空间特征索引表路径、待搜索图片路径。 生成索引表文本并写入csv文件。 可采用如下命令行形式启动驱动程序: python searchEngine.py -c color_index.csv -s structure_index.csv -r dataset -q query/pyramid.jpg dataset为图片库路径。color_index.csv为色彩空间特征索引表路径。structure_index.csv为构图空间特征索引表路径,query/pyramid.jpg为待搜索图片路径。 """
  10. searchArgParser = argparse.ArgumentParser()
  11. searchArgParser.add_argument("-c", "--colorindex", required = True, help = "Path to where the computed color index will be stored")
  12. searchArgParser.add_argument("-s", "--structureindex", required = True, help = "Path to where the computed structure index will be stored")
  13. searchArgParser.add_argument("-q", "--query", required = True, help = "Path to the query image")
  14. searchArgParser.add_argument("-r", "--resultpath", required = True, help = "Path to the result path")
  15. searchArguments = vars(searchArgParser.parse_args())
  16. idealBins = (8, 12, 3)
  17. idealDimension = (16, 16)
  18. colorDescriptor = color_descriptor.ColorDescriptor(idealBins)
  19. structureDescriptor = structure_descriptor.StructureDescriptor(idealDimension)
  20. queryImage = cv2.imread(searchArguments["query"])
  21. colorIndexPath = searchArguments["colorindex"]
  22. structureIndexPath = searchArguments["structureindex"]
  23. resultPath = searchArguments["resultpath"]
  24. queryFeatures = colorDescriptor.describe(queryImage)
  25. queryStructures = structureDescriptor.describe(queryImage)
  26. imageSearcher = searcher.Searcher(colorIndexPath, structureIndexPath)
  27. searchResults = imageSearcher.search(queryFeatures, queryStructures)
  28. print(searchResults)
  29. for imageName, score in searchResults:
  30. queryResult = cv2.imread( imageName)
  31. cv2.imshow("Result Score: " + str(int(score)) + " (lower is better)", queryResult)
  32. cv2.imshow("Query", queryImage)
  33. cv2.waitKey(0)

searcher.py

  1. # -*- coding: utf-8 -*-
  2. # @Time : 2021/10/9 9:45
  3. # @Author :
  4. import numpy
  5. import csv
  6. import re
  7. """ 图片搜索匹配内核Searcher。 类成员colorIndexPath和structureIndexPath。记录色彩空间特征索引表路径和结构特征索引表路径。 成员函数solveColorDistance(self, features, queryFeatures, eps = 1e-5)。求features和queryFeatures特征向量的二范数。eps是为了避免除零错误。 成员函数solveStructureDistance(self, structures, queryStructures, eps = 1e-5)。同样是求特征向量的二范数。eps是为了避免除零错误。需作统一化处理,color和structure特征向量距离相对比例适中,不可过分偏颇。 成员函数searchByColor(self, queryFeatures)。使用csv模块的reader方法读入索引表数据。采用re的split方法解析数据格式。用字典searchResults存储query图像与库中图像的距离,键为图库内图像名imageName,值为距离distance。 成员函数transformRawQuery(self, rawQueryStructures)。将未处理的query图像矩阵转为用于匹配的特征向量形式。 成员函数searchByStructure(self, rawQueryStructures)。类似4。 成员函数search(self, queryFeatures, rawQueryStructures, limit = 3)。将searchByColor方法和searchByStructure的结果汇总,获得总匹配分值,分值越低代表综合距离越小,匹配程度越高。返回前limit个最佳匹配图像。 """
  8. class Searcher:
  9. __slot__ = ["colorIndexPath", "structureIndexPath"]
  10. def __init__(self, colorIndexPath, structureIndexPath):
  11. self.colorIndexPath, self.structureIndexPath = colorIndexPath, structureIndexPath
  12. def solveColorDistance(self, features, queryFeatures, eps=1e-5):
  13. distance = 0.5 * numpy.sum([((a - b) ** 2) / (a + b + eps) for a, b in zip(features, queryFeatures)])
  14. return distance
  15. def solveStructureDistance(self, structures, queryStructures, eps=1e-5):
  16. distance = 0
  17. normalizeRatio = 5e3
  18. for index in range(len(queryStructures)):
  19. for subIndex in range(len(queryStructures[index])):
  20. a = structures[index][subIndex]
  21. b = queryStructures[index][subIndex]
  22. distance += (a - b) ** 2 / (a + b + eps)
  23. return distance / normalizeRatio
  24. def searchByColor(self, queryFeatures):
  25. searchResults = { }
  26. with open(self.colorIndexPath) as indexFile:
  27. reader = csv.reader(indexFile)
  28. for line in reader:
  29. features = []
  30. for feature in line[1:]:
  31. feature = feature.replace("[", "").replace("]", "")
  32. findStartPosition = 0
  33. feature = re.split("\s+", feature)
  34. rmlist = []
  35. for index, strValue in enumerate(feature):
  36. if strValue == "":
  37. rmlist.append(index)
  38. for _ in range(len(rmlist)):
  39. currentIndex = rmlist[-1]
  40. rmlist.pop()
  41. del feature[currentIndex]
  42. feature = [float(eachValue) for eachValue in feature]
  43. features.append(feature)
  44. distance = self.solveColorDistance(features, queryFeatures)
  45. searchResults[line[0]] = distance
  46. indexFile.close()
  47. # print "feature", sorted(searchResults.iteritems(), key = lambda item: item[1], reverse = False)
  48. return searchResults
  49. def transformRawQuery(self, rawQueryStructures):
  50. queryStructures = []
  51. for substructure in rawQueryStructures:
  52. structure = []
  53. for line in substructure:
  54. for tripleColor in line:
  55. structure.append(float(tripleColor))
  56. queryStructures.append(structure)
  57. return queryStructures
  58. def searchByStructure(self, rawQueryStructures):
  59. searchResults = { }
  60. queryStructures = self.transformRawQuery(rawQueryStructures)
  61. with open(self.structureIndexPath) as indexFile:
  62. reader = csv.reader(indexFile)
  63. for line in reader:
  64. structures = []
  65. for structure in line[1:]:
  66. structure = structure.replace("[", "").replace("]", "")
  67. structure = re.split("\s+", structure)
  68. if structure[0] == "":
  69. structure = structure[1:]
  70. structure = [float(eachValue) for eachValue in structure]
  71. structures.append(structure)
  72. distance = self.solveStructureDistance(structures, queryStructures)
  73. searchResults[line[0]] = distance
  74. indexFile.close()
  75. # print "structure", sorted(searchResults.iteritems(), key = lambda item: item[1], reverse = False)
  76. return searchResults
  77. def search(self, queryFeatures, rawQueryStructures, limit=3):
  78. featureResults = self.searchByColor(queryFeatures)
  79. structureResults = self.searchByStructure(rawQueryStructures)
  80. results = { }
  81. for key, value in featureResults.items():
  82. results[key] = value + structureResults[key]
  83. results = sorted(results.items(), key=lambda item: item[1], reverse=False)
  84. return results[: limit]

structure_descriptor.py

  1. # -*- coding: utf-8 -*-
  2. # @Time : 2021/10/9 9:45
  3. # @Author :
  4. import cv2
  5. """ 构图空间特征提取器StructureDescriptor。 类成员dimension。将所有图片归一化(降低采样)为dimension所规定的尺寸。由此才能够用于统一的匹配和构图空间特征的生成。 成员函数describe(self, image)。将图像从BGR色彩空间转为HSV色彩空间(此处应注意OpenCV读入图像的色彩空间为BGR而非RGB)。返回HSV色彩空间的矩阵,等待在搜索引擎核心中的下一步处理。 """
  6. class StructureDescriptor:
  7. __slot__ = ["dimension"]
  8. def __init__(self, dimension):
  9. self.dimension = dimension
  10. def describe(self, image):
  11. image = cv2.resize(image, self.dimension, interpolation=cv2.INTER_CUBIC)
  12. # image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
  13. return image
二.执行
  1. # 构建索引
  2. python index.py --dataset dataset --colorindex color_index.csv --structure structure_index.csv
  3. # 搜图
  4. python searchEngine.py -c color_index.csv -s structure_index.csv -r dataset -q query/pyramid.jpg

在这里插入图片描述
在这里插入图片描述

参考:https://blog.csdn.net/coderhuhy/article/details/46575667

发表评论

表情:
评论列表 (有 0 条评论,371人围观)

还没有评论,来说两句吧...

相关阅读