The important task of library book inventory, or shelf-reading, requires humans to remove each book from a library shelf, open the front cover, scan a barcode, and reshelve the book. It is a labor-intensive and often error-prone process. Technologies such as 2D barcode scanning or radio frequency identification (RFID) tags have recently been proposed to improve this process. They both incur significant upfront costs and require a large investment of time to fit books with special tags before the system can be productive. A vision-based automation system is proposed to improve this process without those prohibitively high upfront costs. This low-cost shelf-reading system uses a hand-held imaging device such as a smartphone to capture book spine images and a server that processes feature descriptors in these images for book identification. Existing color feature descriptors for feature matching typically use grayscale feature detectors, which omit important color edges. Also, photometric-invariant color feature descriptors require unnecessary computations to provide color descriptor information. This paper presents the development of a simple color enhancement feature descriptor called Color Difference-of-Gaussians SIFT (CDSIFT). CDSIFT is well suited for library inventory process automation, and this paper introduces such a system for this unique application. 1. Introduction Taking inventory is a daunting task in any industry, especially when the number of items reaches into the multimillions, as is the case with most major libraries. It turns into a very challenging and costly task because each item has to be accounted for without the benefit of automation. A comparison done by the On-line Computer Library Center shows that libraries in the United States alone circulate more books every day than the shipping giant FedEx delivers packages. Approximately 5.4 million books are checked out daily from libraries across the US. Furthermore, libraries worldwide hold an estimated 16 billion volumes, and this number continues to grow. Even allocating just one second per book, a full inventory would require over 507 man-years. When equipment such as a barcode scanner is used, each book must be taken off the shelf, its cover opened, the barcode scanned, and then reshelved. Even with such improved technology, the amount of time and labor required is still substantial. Another promising alternative is to use radiofrequency identification (RFID) chips. This approach, however, requires replacing existing call numbers, special labels, or barcodes, constituting a
References
[1]
D. J. Lee, Y. Chang, J. K. Archibald, and C. Pitzak, “Matching book-spine images for library shelf-reading process automation,” in Proceedings of the 4th IEEE Conference on Automation Science and Engineering (CASE '08), pp. 738–743, August 2008.
[2]
F. Pernkopf, “Detection of surface defects on raw steel blocks using Bayesian network classifiers,” Pattern Analysis and Applications, vol. 7, no. 3, pp. 333–342, 2004.
[3]
T. M？enp？？, J. Viertola, and M. Pietik？inen, “Optimising colour and texture features for real-time visual inspection,” Pattern Analysis and Applications, vol. 6, no. 3, pp. 169–175, 2003.
[4]
S. M. Bhandarkar, X. Luo, R. F. Daniels, and E. W. Tollner, “Automated planning and optimization of lumber production using machine vision and computed tomography,” IEEE Transactions on Automation Science and Engineering, vol. 5, no. 4, pp. 677–695, 2008.
[5]
P. J. Elbischger, H. Bischof, P. Regitnig, and G. A. Holzapfel, “Automatic analysis of collagen fiber orientation in the outermost layer of human arteries,” Pattern Analysis and Applications, vol. 7, no. 3, pp. 269–284, 2004.
[6]
Y. Cheng and M. A. Jafari, “Vision-based online process control in manufacturing applications,” IEEE Transactions on Automation Science and Engineering, vol. 5, no. 1, pp. 140–153, 2008.
[7]
P. Soda, G. Iannello, and M. Vento, “A multiple expert system for classifying fluorescent intensity in antinuclear autoantibodies analysis,” Pattern Analysis and Applications, vol. 12, no. 3, pp. 215–226, 2009.
[8]
G. A. Khuwaja and A. N. Abu-Rezq, “Bi-modal breast cancer classification system,” Pattern Analysis and Applications, vol. 7, no. 3, pp. 235–242, 2004.
[9]
M. J. Swain and D. H. Ballard, “Color indexing,” International Journal of Computer Vision, vol. 7, no. 1, pp. 11–32, 1991.
[10]
A. Mojsilovi？ and E. Soljanin, “Color quantization and processing by fibonacci lattices,” IEEE Transactions on Image Processing, vol. 10, no. 11, pp. 1712–1725, 2001.
[11]
G. J. Burghouts and J. M. Geusebroek, “Performance evaluation of local colour invariants,” Computer Vision and Image Understanding, vol. 113, no. 1, pp. 48–62, 2009.
[12]
K. E. A. van de Sande, T. Gevers, and C. G. M. Snoek, “Evaluating color descriptors for object and scene recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1582–1596, 2010.
[13]
S. G. Fowers, D. J. Lee, and D. K. Wilde, “Color DoG: a three-channel color feature detector for embedded systems,” in Proceedings of the 27th Intelligent Robots and Computer Vision: Algorithms and Techniques, vol. 7539, Jan 2010.
[14]
S. G. Fowers, D.-J. Lee, and G. Xiong, “Improved library shelf reading using color feature matching of book-spine images,” in Proceedings of the International Conference on Control, Automation, Robotics and Vision, December 2010.
[15]
D. Slater and G. Healey, “The illumination-invariant recognition of 3D objects using local color invariants,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 206–210, 1996.
[16]
G. D. Finlayson, “Color in perspective,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 10, pp. 1034–1038, 1996.
[17]
T. Gevers and A. W. M. Smeulders, “Color based object recognition,” Lecture Notes in Computer Science, vol. 1310, pp. 319–326, 1997.
[18]
I. Biederman, “Recognition-by-Components: a theory of human image understanding,” Psychological Review, vol. 94, no. 2, pp. 115–147, 1987.
[19]
C. Harris and M. Stephens, “A combined corner and edge detector,” in Proceedings of the Alvey Vision Conference, vol. 15, 1988.
[20]
P. Beaudet, “Rotationally invariant image operators,” in Proceedings of the International Conference of Pattern Recognition, pp. 579–583, 1978.
[21]
K. Paler, J. F？glein, J. Illingworth, and J. Kittler, “Local ordered grey levels as an aid to corner detection,” Pattern Recognition, vol. 17, no. 5, pp. 535–543, 1984.
[22]
O. A. Zuniga and R. Haralick, “Corner detection using the facet model,” in Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 30–37, 1983.
[23]
K. Mikolajczyk and C. Schmid, “Scale & affine invariant interest point detectors,” International Journal of Computer Vision, vol. 60, no. 1, pp. 63–86, 2004.
[24]
K. Mikolajczyk and C. Schmid, “A performance evaluation of local descriptors,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615–1630, 2005.
[25]
F. Godtliebsen, J. S. Marron, and P. Chaudhuri, “Statistical significance of features in digital images,” Image and Vision Computing, vol. 22, no. 13, pp. 1093–1104, 2004.
[26]
R. Deriche and G. Giraudon, “A computational approach for corner and vertex detection,” International Journal of Computer Vision, vol. 10, no. 2, pp. 101–124, 1993.
[27]
L. Kitchen and A. Rosenfeld, “Gray-level corner detection,” Pattern Recognition Letters, vol. 1, no. 2, pp. 95–102, 1993.
[28]
K. Rangarajan, M. Shah, and D. van Brackle, “Optimal corner detector,” Computer Vision Graph Image Processing, vol. 48, no. 2, pp. 230–245, 1989.
[29]
H. Wang and M. Brady, “A practical solution to corner detection,” in Proceedings of the IEEE International Conference on Image Processing, vol. 1, pp. 919–923, November1994.
[30]
P. L. Rosin, “Augmenting corner descriptors,” Graphical Models and Image Processing, vol. 58, no. 3, pp. 286–294, 1996.
[31]
J. van de Weijer and C. Schmid, “Coloring local feature extraction,” in Proceedings of the Computer Vision European Conference on Computer Vision (ECCV '06), pp. 334–348, 2006.
[32]
A. Bosch, A. Zisserman, and X. Mu？oz, “Scene classification using a hybrid generative/discriminative approach,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 712–727, 2008.
[33]
R. Datta, D. Joshi, J. Li, and J. Z. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Computing Surveys, vol. 40, no. 2, pp. 1–60, 2008.
[34]
R. O. Stehling, M. A. Nascimento, and A. X. Falc？o, “A compact and efficient image retrieval approach based on border/interior pixel classification,” in Proceedings of the 11th International Conference on Information and Knowledge Management (CIKM '02), pp. 102–109, November 2002.
[35]
I. Laptev, “Improving object detection with boosted histograms,” Image and Vision Computing, vol. 27, no. 5, pp. 535–544, 2009.
[36]
J. Almeida, A. Rocha, R. Torres, and S. Goldenstein, “Making colors worth more than a thousand words,” in Proceedings of the 23rd Annual ACM Symposium on Applied Computing (SAC '08), pp. 1180–1186, November 2008.
[37]
J. M. Geusebroek, R. van den Boomgaard, A. W. M. Smeulders, and H. Geerts, “Color invariance,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 12, pp. 1338–1350, 2001.
[38]
A. Abdel-Hakim and A. Farag, “CSIFT: a SIFT descriptor with color invariant characteristics,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), vol. 2, pp. 1978–1983, October 2006.
[39]
J. van de Weijer, T. Gevers, and J. Geusebroek, “Color edge detection by photometric quasi-invariants,” in Proceedings of the 9th IEEE International Conference On Computer Vision, pp. 1520–1525, October 2003.
[40]
J. van de Weijer, T. Gevers, and J.-M. Geusebroek, “Edge and corner detection by photometric quasi-invariants,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 625–630, 2005.
[41]
H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, “SURF: Speeded up robust features,” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, 2008.
[42]
D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
[43]
Y. T. Tsai, Q. Wang, and S. You, “CDIKP: a highly-compact local feature descriptor,” in Proceedings of the 19th International Conference on Pattern Recognition (ICPR '08), p. 1, December 2008.
