WO2014190374A1

WO2014190374A1 - Method, apparatus and system for displaying data

Info

Publication number: WO2014190374A1
Application number: PCT/AU2014/000536
Authority: WO
Inventors: Matthew Lloyd Byrne; Michael Alexander ROBINS
Original assignee: Curicon Pty Ltd
Priority date: 2013-05-31
Filing date: 2014-05-21
Publication date: 2014-12-04
Also published as: AU2014273829A1

Abstract

A method of displaying data for use in determining attributes of an item is disclosed. One or more images of the item are received. At least a selected one of the received images is analysed to determine one or more attributes of the selected image. The determined attributes are compared to data associated with one or more further images. The data associated with at least one of the further images is retrieved depending on the comparison. The retrieved data is displayed for use in determining the attributes of the item.

Description

METHOD, APPARATUS AND SYSTEM FOR DISPLAYING DATA

Technical Field

[0001] The present invention relates generally to image processing and, in particular, to a method and apparatus for displaying data for use in determining attributes of an item, and to a computer program product including a computer readable medium having recorded thereon a computer program for displaying data for use in determining attributes of an item.

Background

[0002] Electronic commerce ("e-commerce") websites, such as EBay™, have become increasingly popular. Some e-commerce websites have huge market revenues and provide both buyers and sellers with the convenience of shopping online. When viewing an item listed on an e-commerce website, individuals typically fall into one of two groups. The first group have an intimate knowledge of the listed item. The first group may be able to identify the item and determine attributes of the item based on viewing logos and brand colouring displayed in an image of the item. However, the second group may be totally unfamiliar with the item.

[0003] In some markets, such as the collectables market, it is difficult if not impossible for most individuals to determine certain attributes of an item from viewing an image of the item alone. Identification in such markets has conventionally relied upon experience and knowledge of the individual viewing the item and/or contextual clues which may aid in determining the attributes of the item.

[0004] In order to aid buyers in distinguishing items, as well as providing an image of an item being sold, a seller is typically required to provide data representing attributes of the item for use in providing a description of the item. The data representing the attributes of the item may be referred to as "attribute data". The description may include data representing item attributes such as the name of the item, a category, a model number, condition of the item, brand, style and features of the item. The description may include data representing other item attributes such as a recommended retail price, an average price for the item, origin of the item, year of release of the item, name of an artist and an indication of the rarity of the item. For a collectable item, such as a comic book, the description may include data representing other item attributes such as a volume number, title and issue number. [0005] The seller is typically required to enter the data representing the item attributes into an electronic "listing form" comprising one or more data fields. For example, the listing form may include a data field for each of the item attributes discussed above, such as a title field, a category field, a model number field, a brand field etc. However, it is often time consuming for the seller to determine the attributes and enter the data representing the attributes (or "attribute data") into such a form, particularly when a number of different items are required to be listed. In this instance, the seller is required to determine and enter the attribute data for each listed item often resulting in high redundancy.

[0006] Further, in order to determine the attributes of an item, the seller often requires an intimate knowledge of the item. For some markets, such as the antiques and collectables market, it may be problematic to list an item as it may not be immediately clear to the seller what the item is, or what the attributes of the item are, such as the name, age, origin or value of the item.

[0007] A potential buyer using an e-commerce website currently faces difficulty in searching for an item that the buyer may like to purchase or obtain a value for the item from an e- commerce website as the buyer may not be able to identify the item, describe any of the attributes of the item or know how to define the item for a text based search.

Summary

[0008] It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.

[0009] Disclosed are arrangements which seek to address the above problems by providing a method for displaying data for use in determining attributes of an item, where the item may be identified based on one or more images of the item. An image of the item is used to retrieve data from an image database. The image database contains data representing images classified based on visual features and other attributes.

[0010] According to one aspect of the present disclosure, there is provided a method of displaying data for use in determining attributes of an item, said method comprising:

receiving one or more images of the item;

analysing at least a selected one of the received images to determine one or more attributes of the selected image; comparing the determined attributes to data associated with one or more further images; retrieving the data associated with at least one of the further images depending on the comparison; and

displaying the retrieved data for use in determining the attributes of the item.

[0011] According to another aspect of the present disclosure, there is provided a system for displaying data for use in determining attributes of an item, said system comprising:

a memory for storing data and a computer program;

a processor coupled to the memory for executing the computer program, the computer program comprising instructions for:

receiving one or more images of the item;

analysing at least a selected one of the received images to determine one or more attributes of the selected image;

comparing the determined attributes to data associated with one or more further images;

retrieving the data associated with at least one of the further images depending on the comparison; and

[0012] According to another aspect of the present disclosure, there is provided an apparatus for displaying data for use in determining attributes of an item, said method comprising:

means for receiving one or more images of the item;

means for analysing at least a selected one of the received images to determine one or more attributes of the selected image;

means for comparing the determined attributes to data associated with one or more further images;

means for retrieving the data associated with at least one of the further images depending on the comparison; and

means for displaying the retrieved data for use in determining the attributes of the item.

[0013] According to another aspect of the present disclosure, there is provided a computer readable medium comprising a computer program stored thereon for displaying data for use in determining attributes of an item, said program comprising:

code for receiving one or more images of the item; code for analysing at least a selected one of the received images to determine one or more attributes of the selected image;

code for comparing the determined attributes to data associated with one or more further images;

code for retrieving the data associated with at least one of the further images depending on the comparison; and

code for displaying the retrieved data for use in determining the attributes of the item.

[0014] According to another aspect of the present disclosure there is provided a computer program product including a computer readable medium having recorded thereon a computer program for implementing the method described above.

[0015] According to another aspect of the present disclosure, there is provided a method of adding data for an item to an e-commerce website, said method comprising:

receiving one or more images of the item;

comparing the determined attributes to data associated with one or more further images stored in an image database;

retrieving the data associated with at least one of the further images stored in the image database depending on the comparison; and

adding the retrieved data to an e-commerce website for the item.

[0016]

According to another aspect of the present disclosure, there is provided a system for adding data for an item to an e-commerce website, said system comprising:

a memory for storing data and a computer program;

receiving one or more images of the item;

comparing the determined attributes to data associated with one or more further images stored in an image database; retrieving the data associated with at least one of the further images stored in the image database depending on the comparison; and

adding the retrieved data to an e-commerce website for the item.

[0017]

According to another aspect of the present disclosure, there is provided a computer readable medium comprising a computer program stored thereon for adding data for an item to an e-commerce website, said method comprising:

code for receiving one or more images of the item;

code for analysing at least a selected one of the received images to determine one or more attributes of the selected image;

code for comparing the determined attributes to data associated with one or more further images stored in an image database;

code for retrieving the data associated with at least one of the further images stored in the image database depending on the comparison; and

code for adding the retrieved data to an e-commerce website for the item.

[0018]

According to another aspect of the present disclosure, there is provided an apparatus for adding data for an item to an e-commerce website, said apparatus comprising:

means for receiving one or more images of the item;

means for comparing the determined attributes to data associated with one or more further images stored in an image database;

means for retrieving the data associated with at least one of the further images stored in the image database depending on the comparison; and

means for adding the retrieved data to an e-commerce website for the item.

[0019] Other aspects of the invention are also disclosed. Brief Description of the Drawings

[0020] At least one embodiment of the present invention will now be described with reference to the drawings and appendices, in which:

[0021] Fig. 1 is a flow diagram of a method of listing an item according to the present disclosure;

[0022] Figs. 2A and 2B form a schematic block diagram of a general purpose computer system upon which arrangements described can be practiced;

[0023] Figs. 3 is a schematic block diagram of a computer module upon which a server computer can be practiced;

[0024] Fig. 4 is a flow diagram showing a method of selecting an image;

[0025] Fig. 5 is a flow diagram showing a method of performing geometric operations on an image;

[0026] Fig. 6 is a flow diagram showing a method of isolating an object in an image;

[0027] Fig 7. Is a flow diagram showing a method of determining feature descriptors for an image;

[0028] Fig. 8 is a flow diagram showing a method of comparing images; and

[0029] Fig. 9 shows a user interface displaying a representation of a downloaded image.

Detailed Description including Best Mode

[0030] Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears. [0031] A method 100 of retrieving data for use in determining attributes of an item is described below with reference to Figs. 1 to 8. The retrieved data represents the attributes of the item. The attributes represented by the retrieved data may include, for example, a name of the item, a category, a model number, condition of the item, brand, style and/or features of the item. The attributes represented by the retrieved data may also include a recommended retail price, an average price for the item, origin of the item, year of release of the item, name of an artist and an indication of the rarity of the item. For a collectable item, such as a comic book, the attributes represented by the retrieved data may include a volume number, title and issue number.

[0032] The data retrieved in accordance with the method 100 may be used to identify the item, without an intimate knowledge of the item being required. The retrieved data may be added to a listing of the item on an e-commerce website, for example, where the item is being listed for sale on the e-commerce website. The retrieved data may also be used for cataloguing the item on the e-commerce website. For example, a seller may wish to catalogue their item or items on the e-commerce website for use in possible future sale or sales. The cataloguing may also be used for keeping track of a seller's items and/or to determine the value of the items. Such cataloguing has conventionally been problematic for sellers due to the complexity of identifying each item, determining the attributes of the item and providing required attribute data for each item. The data retrieved for an item may also be used for displaying website locations and/or physical locations (e.g., an address of a traditional retail store) from which the item may be purchased.

[0033] The item may be identified and the attribute data retrieved based on one or more images of the item. Visual features and attributes are determined based on an analysis of a selected one of the images of the item. The determined visual features and attributes are used to compare the image against data stored in a database containing representations of images of items. The database containing the image representations may be referred to the "image database". As described below, the determined visual features and attributes of the selected image may be represented by data in the form of feature descriptors.

[0034] Each of the images in the image database is classified based on one or more visual features and attributes of the associated database image, and an item represented in the associated image. The visual features and attributes of each of the database images may be represented by data (e.g., metadata and/or feature descriptors) which may be stored in the image database to represent each database image. As described below, each database image is stored in the image database as data (e.g., metadata, feature descriptors) representing the database image. In another arrangement, as well as the data representing the associated database image, actual image data (e.g., a joint photo-graphics expert group (JPEG) image file, an exchangeable image file (EXIF), a portable network graphics file (PNG) or a graphics interchange format (GIF) file), for displaying the associated database image may also be stored in the image database. In still another arrangement, an identifier representing the item represented by each database image may be stored in the image database.

[0035] The data stored in the image database for each database image may represent the nature and properties of an item represented by each database image. For example, the data stored in the image database for each database image may include, but is not limited to, data representing attributes of the item including a brand of the item, a title for the item, an average price of the item, conditional pricing of the item (e.g., a price set when certain physical conditions are met), release date of the item, number of copies or number of the items that are available for sale, popularity of the item, trending nature of the item, historical data associated with the item, category information for the item and a model number for the item.

[0036] As described below, data is retrieved from the image database based on a comparison of an image of an item and the database images. One or more data fields of an electronic listing form may be populated using the retrieved data in order to provide a description of the item for a listing of the item on an e-commerce website or the like. The data fields of such an electronic listing form may be populated for use in cataloguing the item (e.g., the retrieved data may be used to fill a catalogue entry).

[0037] Figs. 2A and 2B depict a general-purpose computer system 200, upon which the various arrangements described can be practiced.

[0038] As seen in Fig. 2A, the computer system 200 includes: a computer module 201 ; input devices such as a keyboard 202, a mouse pointer device 203, a scanner 226, a camera 227, and a microphone 280; and output devices including a printer 215, a display device 214 and loudspeakers 217. The camera 227 may be a webcam, for example.

[0039] An external Modulator-Demodulator (Modem) transceiver device 216 may be used by the computer module 201 for communicating to and from a communications network 220 via a connection 221. The communications network 220 may be a wide-area network (WAN), such as the Internet, a cellular telecommunications network, or a private WAN. Where the connection 221 is a telephone line, the modem 216 may be a traditional "dial-up" modem. Alternatively, where the connection 221 is a high capacity (e.g., cable) connection, the modem 216 may be a broadband modem. For example, the connection 221 may be an asymmetric digital subscriber line (ADSL), a very-high-bit-rate digital subscriber line (VDSL) or a symmetric digital subscriber line (SDSL) connection. A wireless modem may also be used for wireless connection to the communications network 220.

[0040] As seen in Fig. 2A, a server computer module 300 is connected to the network 220. The computer module 201 may communicate with the server computer module 300 via the communications network 220. The server computer module 300 will be further described below with reference to Fig. 3.

[0041] The computer module 201 typically includes at least one processor unit 205. In one arrangement, the computer module 201 may comprise a graphical processing unit (GPU). For example, the processor unit 205 may comprise one or more GPUs. The computer module 201 typically includes a memory unit 206. For example, the memory unit 206 may have semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The computer module 201 also includes an number of input/output (I/O) interfaces including: an audio-video interface 207 that couples to the video display 214, loudspeakers 217 and microphone 280; an I/O interface 213 that couples to the keyboard 202, mouse 203, scanner 226, camera 227 and optionally a joystick or other human interface device (not illustrated); and an interface 208 for the external modem 216 and printer 215. In some implementations, the modem 216 may be incorporated within the computer module 201 , for example within the interface 208. The computer module 201 also has a local network interface 211 , which permits coupling of the computer system 200 via a connection 223 to a local-area communications network 222, known as a Local Area Network (LAN). As illustrated in Fig. 2A, the local communications network 222 may also couple to the wide network 220 via a connection 224, which would typically include a so-called "firewall" device or device of similar functionality. The local network interface 211 may comprise an Ethernet circuit card, a Bluetooth^® wireless arrangement or an IEEE 802.11 wireless arrangement; however, numerous other types of interfaces may be practiced for the interface 211.

[0042] The I/O interfaces 208 and 213 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 209 are provided and typically include a hard disk drive (HDD) 210. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. In one arrangement, the storage devices 209 may include a solid state drive. An optical disk drive 212 is typically provided to act as a non-volatile source of data. Portable memory devices, such optical disks (e.g., CD-ROM, DVD, Blu-ray Disc™), USB-RAM, portable, external hard drives, and floppy disks, for example, may be used as appropriate sources of data to the system 200.

[0043] The components 205 to 213 of the computer module 201 typically communicate via an interconnected bus 204 and in a manner that results in a conventional mode of operation of the computer system 200 known to those in the relevant art. For example, the processor 205 is coupled to the system bus 204 using a connection 218. Likewise, the memory 206 and optical disk drive 212 are coupled to the system bus 204 by connections 219. Examples of computers on which the described arrangements can be practised include IBM-PC's and compatibles, Oracle™ Sparcstations, Apple Mac™, ARM™ or like computer systems. The described arrangements may also be practised on virtual computers or the like.

[0044] One or more steps of methods described below may be implemented using the computer system 200 wherein the processes of Figs. 1 and 4 to 7, to be described, may be implemented as one or more software application programs 233 executable within the computer system 200. In particular, one or more of the steps of the described methods are affected by instructions 231 (see Fig. 2B) in the software 233 that are carried out within the computer system 200. The software instructions 231 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the described methods and a second part and the corresponding code modules manage a user interface between the first part and the user.

[0045] The software may be stored in a computer readable medium, including the storage devices described below, for example. The software 233 is typically stored in the HDD 210 or the memory 206. However, in one arrangement, the software 233 may be stored in a solid state drive. The software is typically loaded into the computer system 200 from the computer readable medium, and then executed by the computer system 200. Thus, for example, the software 233 may be stored on an optically readable disk storage medium (e.g., CD-ROM) 225 that is read by the optical disk drive 212. A computer readable medium having such software or computer program recorded on the computer readable medium is a computer program product. The use of the computer program product in the computer system 200 preferably effects an advantageous apparatus for implementing the described methods.

[0046] In some instances, the application programs 233 may be supplied to the user encoded on one or more CD-ROMs 225 and read via the corresponding drive 212, or alternatively may be read by the user from the networks 220 or 222. Still further, the software can also be loaded into the computer system 200 from other computer readable media. Computer readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and/or data to the computer system 200 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray™ Disc, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 201. Examples of transitory or non-tangible computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 201 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like.

[0047] The second part of the application programs 233 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented upon the display 214. Through manipulation of typically the keyboard 202 and the mouse 203, a user of the computer system 200 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 217 and user voice commands input via the microphone 280.

[0048] Fig. 2B is a detailed schematic block diagram of the processor 205 and a "memory" 234. The memory 234 represents a logical aggregation of all the memory modules (including the HDD 209 and semiconductor memory 206) that can be accessed by the computer module 201 in Fig. 2A.

[0049] When the computer module 201 is initially powered up, a power-on self-test (POST) program 250 executes. The POST program 250 is typically stored in a ROM 249 of the semiconductor memory 206 of Fig. 2A. A hardware device such as the ROM 249 storing software is sometimes referred to as firmware. The POST program 250 examines hardware within the computer module 201 to ensure proper functioning and typically checks the processor 205, the memory 234 (209, 206), and a basic input-output systems software (BIOS) module 251 , also typically stored in the ROM 249, for correct operation. Once the POST program 250 has run successfully, the BIOS 251 activates the hard disk drive 210 of Fig. 2A. Activation of the hard disk drive 210 causes a bootstrap loader program 252 that is resident on the hard disk drive 210 to execute via the processor 205. This loads an operating system 253 into the RAM memory 206, upon which the operating system 253 commences operation. The operating system 253 is a system level application, executable by the processor 205, to fulfil various high level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface.

[0050] The operating system 253 manages the memory 234 (209, 206) to ensure that each process or application running on the computer module 201 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different types of memory available in the system 200 of Fig. 2A must be used properly so that each process can run effectively. Accordingly, the aggregated memory 234 is not intended to illustrate how particular segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible by the computer system 200 and how such is used. In one arrangement, the memory may include the memory of a video card device.

[0051] As shown in Fig. 2B, the processor 205 includes a number of functional modules including a control unit 239, an arithmetic logic unit (ALU) 240, and a local or internal memory 248, sometimes called a cache memory. The cache memory 248 typically include a number of storage registers 244 - 246 in a register section. One or more internal buses 241 functionally interconnect these functional modules. The processor 205 typically also has one or more interfaces 242 for communicating with external devices via the system bus 204, using a connection 218. The memory 234 is coupled to the bus 204 using a connection 219.

[0052] The application program 233 includes a sequence of instructions 231 that may include conditional branch and loop instructions. The program 233 may also include data 232 which is used in execution of the program 233. The instructions 231 and the data 232 are stored in memory locations 228, 229, 230 and 235, 236, 237, respectively. Depending upon the relative size of the instructions 231 and the memory locations 228-230, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 230. Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 228 and 229.

[0053] In general, the processor 205 is given a set of instructions which are executed therein. In one arrangement, instructions may be executed in parallel where some instructions are executed on a different processor (e.g., by executing on non-CPU cores where the computer module 201 comprises one or more GPUs). As described above, the processor 205 may comprise one or more GPUs. The processor 205 waits for a subsequent input, to which the processor 205 reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 202, 203, data received from an external source across one of the networks 220, 202, data retrieved from one of the storage devices 206, 209 or data retrieved from a storage medium 225 inserted into the corresponding reader 212, all depicted in Fig. 2A. The execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 234.

[0054] The disclosed arrangements use input variables 254, which are stored in the memory 234 in corresponding memory locations 255, 256, 257. The disclosed arrangements produce output variables 261 , which are stored in the memory 234 in corresponding memory locations 262, 263, 264. Intermediate variables 258 may be stored in memory locations 259, 260, 266 and 267.

[0055] Referring to the processor 205 of Fig. 2B, the registers 244, 245, 246, the arithmetic logic unit (ALU) 240, and the control unit 239 work together to perform sequences of micro- operations needed to perform "fetch, decode, and execute" cycles for every instruction in the instruction set making up the program 233. Each fetch, decode, and execute cycle comprises:

[0056] a fetch operation, which fetches or reads an instruction 231 from a memory location 228, 229, 230;

[0057] a decode operation in which the control unit 239 determines which instruction has been fetched; and

[0058] an execute operation in which the control unit 239 and/or the ALU 240 execute the instruction. [0059] Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 239 stores or writes a value to a memory location 232.

[0060] One or more steps or sub-processes in the processes of Figs. 1 and 4 to 8 is associated with one or more segments of the program 233 and is performed by the register section 244, 245, 247, the ALU 240, and the control unit 239 in the processor 205 working together to perform the fetch, decode, and execute cycles for every instruction in the instruction set for the noted segments of the program 233.

[0061] The described methods may alternatively be implemented in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of the described methods. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories.

[0062] The server computer module 300 typically includes at least one processor unit 305. Again, in one arrangement, the computer module 300 may comprise a GPU. In one arrangement, the processor 305 may comprise one or more GPUs. The computer module 300 typically includes a memory unit 306. For example, the memory unit 306 may have semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The computer module 300 also includes a number of input/output (I/O) interfaces including: an audio-video interface 307. The computer module 300 also includes an interface 308 for an external modem transceiver device 316 for communicating to and from the communications network 220 via a connection 321.

[0063] As seen in Fig. 3, storage devices 309 are provided and typically include a hard disk drive (HDD) 310. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. Again, in one arrangement, the storage devices 309 may include a solid state drive. An optical disk drive 312 is typically provided to act as a non-volatile source of data. Portable memory devices, such optical disks (e.g., CD-ROM, DVD, Blu-ray Disc™), USB-RAM, portable, external hard drives, and floppy disks, for example, may be used as appropriate sources of data to the system 300.

[0064] The components 305 to 313 of the computer module 300 typically communicate via an interconnected bus 304 and in a manner that results in a conventional mode of operation of the computer module 300 known to those in the relevant art. Again, examples of computers on which the server computer module 300 can be practised include IBM-PC's and compatibles, Sun Sparcstations, Apple Mac™, ARM™ or a like computer systems. As described above, the described arrangements may also be practised on virtual computers or the like.

[0065] One or more steps of the described methods may be implemented using the computer module 300 wherein one or more of the processes to be described, may be implemented as one or more software application programs 333 executable within the computer module, as described above for the software application programs 233.

[0066] In one arrangement, one or more steps of the described methods may be implemented as one or more software application programs executable within a smart-phone, tablet device or similar portable computing device connected to the network 220. Accordingly, the processing performed to execute one or more steps of the described methods may be distributed across the computer module 201 , the server computer module 300 and/or a portable computing device (e.g,. a smart-phone, tablet device or the like) connected to the network 220. For example, steps 105 and 107 of the method 100 described below may be executed within a smart-phone, tablet device or similar portable computing device.

[0067] The method 100 will be described below by way of example where data is retrieved and displayed for determining attributes of a Spider-man comic. The retrieved data and determined attributes may be used to identify the Spider-man comic and may be added to a listing of the Spider-man comic on an e-commerce website. As another example, the retrieved data may be added as a catalogue entry on the e-commerce website in order to catalogue the Spider-man comic in a collection of such comics. However, it will be understood that the method 100 may be used to retrieve and display data, and to identify any item, such as for example, antique and pop culture collectables (e.g., coins, stamps, magazines, comics, books, furniture), manufactured products, or any other consumer item that a user may wish to identify, list and/or catalogue on an e-commerce website. The method 100 may also be used with other types of items such as packaged food. For example, a user may wish to identify foreign food items which are labelled in a foreign language. In this instance, the method 100 may be used for border control such as by a customs agent or the like wishing to control the types of foods being brought into a region or country.

[0068] As described below, he method 100 is implemented as one or more software code modules of the software application program 333 resident in the hard disk drive 310 and being controlled in its execution by the processor 305. However, in one arrangement, each step of the method 100 may be implemented as a separate software code module. In one arrangement, one or more steps of the method 100, such as steps 105 and 107 described below, may be implemented on the computer module 201. In another arrangement, one or more steps of the method 100, such as steps 105 and 107 of the method 100 described below may be executed within a smart-phone, tablet device or similar portable computing device.

[0069] The method 100 begins at step 101 , where the software program 333, under execution of the processor 305, is used for receiving one or more images of the item. For example, the user may capture a series of video images of the item using the computer module 201 and the camera 227. Alternatively, only a single image of the item may be captured by the user. The processor 205 uploads the images to the server computer module 300 and the images may be stored within the storage devices 309 of the service computer module 300.

[0070] As described above, in one arrangement, the camera 227 may be in the form of a webcam and the image(s) may be captured using the webcam. Alternatively, the video images of the item may be captured using a smart-phone, tablet device or similar device and then be uploaded to the computer module 201 or server computer module 300 via the network 220. The video images captured at step 101 may be stored in the storage device 209 before being uploaded to the server computer module 300. For example, the video images may be captured over a period of time prior to execution of the method 100 with the captured images being stored in the storage device 209 prior to being uploaded to the computer module 300. In one arrangement, the resolution of a video image or still image captured by the camera 227 or other device may be equal to or exceed 640x480 pixels in size with a quality of two or greater megapixels. If the captured images are of sufficient quality or size, the images may be down- sampled or compressed (e.g., using JPEG compression) prior to the images being uploaded to the server computer module 300 in order to minimise processing time. Such down-sampling or compression may also be performed to minimise transmission time of the images from the computer module 201 to the server computer module 300.

[0071] At the next step 103, the software program 333, under execution of the processor 305, selects one of the images received at step 101 for further analysis. As described below, in step 103 to 109, the method 100 is used for analysing one of the images received at step 103 to determine one or more attributes of the selected image. A method 400 of selecting an image, as executed at step 103, will be described below with reference to Fig. 4. [0072] Then at the next step 105, geometric operations are performed on the selected image (i.e., the image selected at step 103) by the software program 333. The geometric operations may be used for analysing at least the image selected at step 103 to determine one or more attributes of the selected image, as described in detail below. A method 500 of performing geometric operations on an image, as executed at step 105, will be described in detail below with reference to Fig. 5. The processed image may be stored within the storage module 309 of the computer module 300. As described above, in one arrangement, the method 500 may be implemented as one or more software code modules of the software application program 233 resident in the hard disk drive 210 and being controlled in its execution by the processor 205.

[0073] At the next step 107, background isolation may be performed on the selected image to isolate a graphic object, representing the item in the selected image, from the background of the selected image. A bilateral blur filter may be applied to the selected image. The filter may be used to blur the image. The selected image may also be converted to grayscale and an adaptive threshold applied to the image. Edge detection may then be performed on the image to detect edges within the image in order to isolate the object. A method 600 of isolating an object representing an item in an image, as executed at step 107, will be described in detail below with reference to Fig. 6.

[0074] Then at the next step 109, the software program 333, under execution of the processor 305, determines feature descriptors for the selected image. In one arrangement, the feature descriptors may be based on one or more hue, saturation, and value (HSV) histograms for the image selected at step 103. The feature descriptors may also be based on a reduced HSV histogram determined for the selected image. Reduced HSV histograms may represent a region(s) of space in the selected image, rather than the full image. As an example, in one arrangement, an HSV histogram of a region in a top right corner of the selected image is determined. An HSV histogram of a region in a top left corner of the selected image may also be determined. The histogram of the top right corner and the histogram of the top left corner of the selected image may be stored as separate feature descriptors. The feature descriptors may also be based on geometric based features.

[0075] The feature descriptors determined at step 109 may be stored in the storage module 309 of the server computer module 300 for use in comparing the selected image to data (e.g., metadata, feature descriptors) representing one or more further images stored in a database of images (i.e., the "image database") configured within the hard disk drive 310. A method 700 of determining feature descriptors for an image, as executed at step 109, will be described in detail below with reference to Fig. 7.

[0076] At the next step 1 11 , the software application program 333, under execution of the processor 305, compares the selected image to data representing one or more images stored within a database (i.e., the "image database") configured within the hard disk drive 310. In one arrangement, the image database may be stored within RAM configured within the memory unit 306. The method 100 is configured to perform the comparison as efficiently as possible by comparing the selected image to as few of the database images as possible. As described below, images may be added to a candidate list based on similarity. In this instance, the selected image is compared to the images in the candidate list instead of comparing the image to all of the images in the image database.

[0077] In one arrangement, at step 11 1 , the software application program 333 analyses the selected image (i.e., the image selected at step 103) to determine if the image belongs to one of a plurality of predetermined image clusters. Each image in such a predetermined image cluster shares visually similar attributes. For example, in the case of images of a series of comics (e.g., a series of Spider-man comics), each of the images may be placed in the same cluster based on shared colours in the images. Further, each of the images may be placed in the same cluster based on similarity in lettering of the series title. As another example, each image that has similar general feature descriptors may be placed in a particular cluster. If the software application 333 determines that the image selected at step 103 shares similar feature descriptors to the feature descriptors of a particular cluster, then at step 1 11 the software application 333 compares the selected image only to the images of the particular cluster.

[0078] A method 800 of comparing images, as executed at step 11 1 , will be described in detail below with reference to Fig. 8. The method 800 may be used for comparing attributes determined for the selected image to data (e.g., metadata) stored in the image database representing one or more images.

[0079] In one arrangement, the images of the item (e.g., the Spider-man comic) may be received by the software application program 333 at step 101 already in the form of data (e.g., metadata and/or feature descriptors) representing the images rather than actual image data (e.g., a joint photo-graphics expert group (JPEG) image file). In such an arrangement, the method 100 may proceed directly to step 111 of the method 100 as described above. [0080] Then at step 113, if the software application program 333 identifies an image in a candidate list (i.e., comprising one or more database images) matching the image selected at step 103, based on the comparison, then the method 100 proceeds to step 115. Otherwise, the method 100 proceeds to step 119. As described below, images may be added to a candidate list based on similarity. In another arrangement, at step 113, the software application program 333 may determine if the selected image matched an image in one or more particular image clusters.

[0081] At step 115, the software application program 333, under execution of the processor 305, displays the matching image on the display 214. For example, Fig. 9 shows a user interface 900 showing a representation of an image 903 which in the present example is a Spider-man comic. The user interface 900 also shows a representation of the image 901 of the Spider-man comic as captured by the user and as selected at step 103. In the example of Fig. 9, the image 901 is determined at step 113 to match the image 903. In one arrangement, the image 903 is displayed at step 115 based on data downloaded from the image database to the computer module 201 at step 115. For example, where an identifier representing the image 903 of the Spider-man comic is stored in the image database, the identifier may be downloaded from the image database to the computer module 201. The computer module 201 may then retrieve the image 903, from an external source such as another server connected to the network 220, based on the downloaded identifier, before displaying the retrieved image 903. In another arrangement where actual image data (e.g., an image file) for displaying the image 903 is stored in the image database, the image data may be downloaded from the image database at step 115 to the computer module 201 for displaying the image 903 on the display 214.

[0082] Then at step 117, the software application program 333, under execution of the processor 305, is used for retrieving the data associated with the matched image in the image database. Accordingly, the data is retrieved at step 117 depending on the comparison performed at step 111. In the example of Fig. 9, the data retrieved at step 117 may include the title of the comic (i.e., the Amazing Spider-man, Spider-man Vs. The Chameleon). The data may also include an average price (i.e., $1500) of the comic, the release date of the comic (i.e., March 1963) and the number of copies of the comic that are available for sale (i.e., 3 available).

[0083] The data retrieved from the database at step 117 may also include a link or links directing the user to a uniform resource locator (URL) for a website or websites where the item may be purchased. Such links enable the user to buy the item efficiently. For the example of Fig. 9, the data retrieved at step 117 may also include a link or links to website locations from which to purchase the Amazing Spider-man comic represented by the matched image. In other arrangement, the data retrieved from the database at step 117 may describe a physical location (e.g., a street address) where the item may be purchased.

[0084] In one arrangement, the data retrieved from the database at step 117 may include, but is not limited to, data representing other attributes of the item such as popularity of the item, trending nature of the item, historical data associated with the item, category information for the item, a model number for the item and a brand of the item, as well as any other of the attributes described above, or any other suitable attributes.

[0085] The software application program 333 may also be configured for displaying the retrieved data at step 117 for use in determining the attributes of the item. The item may be identified based on the determined attributes, for example, where a user does not have an intimate knowledge of the item. For example, the data retrieved at step 117 may be downloaded to the computer module 201 and displayed on the display 214. As seen in Fig. 9, the description fields 905, 906, 907 and 908 are displayed on the user interface 900. Accordingly, the retrieved data displayed on the user interface 900 may be used for populating data fields of an electronic form on the e-commerce website for use in providing a description of the comic when listing the item on the e-commerce website. For example, the data in the description fields 905, 906, 907 and 908 may be used in populating the electronic form for listing the comic on the e-commerce website. In one arrangement, the retrieved data may be "cherry- picked" by the user (e.g., the comic issue may not match the comic to be listed but series data may match the comic to be listed). In the example of the comic, the cherry-picked data may be used for adding a description to a listing of the comic on an e-commerce website.

[0086] In one arrangement, the description fields of the electronic form may be automatically filled based on the attribute data retrieved at step 117. For example, the user interface 900 including the images 901 and 903 and the description, as seen in Fig. 9, may be used in an e- commerce website such as EBay™. The user interface 900 of Fig. 9 also includes icons 909 and 910. The icon 909, titled "one click sell", may be selected by a user to initiate a sale of the item represented in the images 901 and 903 (i.e., the Spider-man comic). The icon 910, which is titled "Buy" in the example of Fig. 9, may be selected in order to redirect the user to another uniform resource locator (URL) where the item may be purchased. Other attribute data corresponding to metrics such as popularity of the item, trending nature of the item and historical data, as well as any other of the attributes described above, may also be displayed on the user interface 900. [0087] In one arrangement, the data (e.g., metadata, feature descriptors) stored in the image database may link to a table in the image database. The table may comprise data for aggregated items available in traditional retail stores or online stores. The user interface 900 or a similar means may be used to provide purchase options, including pricing and shipping information, to a user. The purchase options may be stored in the image database or may be aggregated from external sources such as the traditional retail stores or online stores. As an example, the purchase options may include a price estimate for the item.

[0088] In one arrangement, the retrieved data used to populate the description fields (e.g., 905, 906, 907 and 908) of the user interface 900 may persist to a further user interface screen, upon the one click sell icon 909 being selected, for example. The further user interface screen may be a "selling" screen which lists the data from the description fields in further data fields. Accordingly, the data from the description fields is used to populate the data fields of the selling screen. Other data may also be used to populate one or more description fields of the selling screen.

[0089] In one arrangement, the retrieved data used to populate the description fields (e.g., 905, 906, 907 and 908) of the user interface 900 may persist to a further user interface screen, upon an "add to catalogue" icon 911 being selected, for example. The further user interface screen may be a "catalogue" screen (used by a user to keep track of their items) which lists the data from the description fields in further data fields. Accordingly, the data from the description fields is used to populate the data fields of the catalogue screen. Other data may also be used to populate one or more description fields of the catalogue screen. The data in the data fields of the catalogue screen and the image 901 may be stored in a database as a catalogue of comics.

[0090] At step 1 19, the software application program 333, under execution of the processor 305, performs a more exhaustive comparison by comparing the image selected at step 103 to the data for all of the database images stored within the image database configured within the hard disk drive 310. Then at step 121 , if the software application program 333 identifies an image in the image database matching the image selected at step 103, based on the comparison performed at step 119, then the method 100 proceeds to step 1 15. Otherwise, the method 100 concludes.

[0091] In the instance that no matching image is determined at step 119, in one arrangement, a screen is downloaded by the software application to the computer module 201 for display on the display 214 notifying the user that no matching image has been determined in the image database. In one arrangement, if no match is found at step 119, images associated with visually or categorically similar items may be displayed. In one arrangement, the software application program 333 may prompt the user to enter attribute data for the item for storage in the image database configured within the hard disk drive 310. The image selected at step 103 may also be stored in the image database. Accordingly, the image database may be periodically populated with attribute data for further images when no matching image is determined at step 1 19.

[0092] The method 400 of selecting an image for further processing, as executed at step 103, will now be described with reference to Fig. 4. The method 100 is implemented as one or more software code modules of the software application program 333 resident in the hard disk drive 310 and being controlled in its execution by the processor 305. In one arrangement, each step of the method 400 may be implemented as a separate software code module. In one arrangement, the method 400 may alternatively be implemented as one or more software code modules of the software application program 233 resident in the hard disk drive 210 and being controlled in its execution by the processor 205. In another arrangement, one more steps of the method 400 may be executed within a smart-phone, tablet device or similar portable computing device connected to the network 220.

[0093] The method 400 begins at step 401 , where the images captured by the user are accessed by the software application program 333 from the storage devices 309 and one of the images is selected for further processing. The selected image is typically a colour image.

[0094] Then at step 403, the selected image is converted to gray-scale. However, in the instance that the selected image is originally a grayscale image, then step 403 may be skipped.

[0095] Following step 403, the method 400 proceeds to step 405, where the software program 333 performs edge detection on the selected image to locate edges of objects within the image. In one arrangement, Canny edge detection may be performed on the selected image.

[0096] At the next step 407, the software program 333, under execution of the processor 305, identifies contours in the image based on any detected edges within the image. Then at step 409, if the number of contours is less than a threshold number of contours, then the method 400 proceeds to step 413. The threshold number of contours is adaptive and is predetermined for a first image received at step 101 of the method 100. Subsequently, the threshold number may be determined based on the average number of contours in the images for a predetermined number of contiguous images (e.g., thirty images). A decrease in the number of contours in a current image compared to a previously received image (i.e., as represented by the threshold) suggests blurring and/or loss of focus. Accordingly, at step 413, the current image is discarded. Otherwise, the method 400 proceeds to step 411 where the current image is selected for further analysis. At step 411, the shape of the current image, as defined by contours in the current image, is used to mask the original colour image as selected at step 401. The selected colour image shows the item and is used in the following steps 105 to 111 throughout the method 100 while the grayscale image determined at step 403 may be discarded. The selected image may be stored within the storage devices 309. Alternatively, the selected image may be stored in the memory 209 or storage modules 209 of the computer module 201 and/or a smart-phone or the like before being transmitted to the computer module 300.

[0097] The method 500 of performing geometric operations on an image, as executed at step 105, will be described in detail below with reference to Fig. 5. The method 500 may be implemented as one or more software code modules of the software application program 333 resident in the hard disk drive 310 and being controlled in its execution by the processor 305. In one arrangement, each step of the method 500 may be implemented as a separate software code module. In one arrangement, the method 500 may be implemented as one or more software code modules of the software application program 233 resident in the hard disk drive 210 and being controlled in its execution by the processor 205. In another arrangement, one or more steps of the method 500 may be executed within a smart-phone, tablet device or similar portable computing device connected to the network 220.

[0098] The method 500 begins at step 501 , where the selected image (i.e., the image selected at step 103) is accessed by the software application program 333 from the storage devices 309 of the server computer module 300. Then at the next step 503, the software application program 333 determines the colour space (e.g., RGB, RGBA) of the selected image. The colour space may be stored within the memory 306 of the server computer module 300. At the next step 505, the image is scaled according to an original resolution of the selected image accessed at step 501. The selected image may also be rotated at step 505. The orientation of the selected image may be determined based on visual features within the selected image. The method 500 concludes at the next step 507, where the scaled image is down-sampled (or compressed) if the selected image is of sufficient quality or size. For example, JPEG compression may be performed on the scaled image. The down-sampling provides an advantage during the transmission of image. The image processed in accordance with the method 500 may be stored within the storage devices 309. [0099] The method 600 of isolating an object in an image, as executed at step 107, will be described in detail below with reference to Fig. 6. The method 600 may be implemented as one or more software code modules of the software application program 333 resident in the hard disk drive 310 and being controlled in its execution by the processor 305. In one arrangement, each step of the method 600 may be implemented as a separate software code module. As described above, in one arrangement, the method 600 may be implemented as one or more software code modules of the software application program 233 resident in the hard disk drive 210 and being controlled in its execution by the processor 205. In another arrangement, one or more steps of the method 600 may be executed within a smart-phone, tablet device or similar portable computing device connected to the network 220.

[00100] The method 600 begins at step 601 , where the image processed in accordance with the method 500 is accessed from the storage devices 309 of the server computer module 300. Then at the next step 603, the software program 333 under execution of the processor 305 applies a bilateral blur filter to the accessed image. The filter blurs the image generally while preserving the integrity of edges in the image. In one arrangement, peripheral regions within the accessed image may be subject to additional blur filters such as a Gaussian blur filter.

[00101] At the next step 605, the blurred image is converted to a grayscale image. In one arrangement, the image is converted to an eight (8) bit grayscale image. The grayscale image may be stored in the storage devices 309 of the server computer module 300. Then at the next step 607, the software program 333, under execution of the processor 305, applies an adaptive threshold to the grayscale image. The adaptive threshold may be applied using a transformation according to Equation (1) as follows:

(1) where T(x,y) represents the threshold for each pixel in the image. The mean threshold is a mean threshold of a block of pixels surrounding the pixel (x,y) minus the weighted mean. Each pixel in the surrounding block is weighted differently. In one arrangement, inversion may also be performed at step 607. [00102] The method 600 continues to the next step 609, where the software program 333 performs Canny edge detection on the threshold image output from step 607 to detect edges within the threshold image. Then at step 611 , based on any detected edges, contours are estimated for each portion of an object corresponding to the detected edges.

[00103] At the next step 613, the contours are clustered based on proximity to neighbouring contours. A sum of the contours is determined to produce an n-sided polygon. The n-sided polygon 'covers' the object and provides separation between a foreground area and background area in the threshold image.

[00104] The method 600 concludes at the next step 615, where the software program 333 applies the polygon determined in step 613 as a mask to the original input image (pre-bilateral filter) to produce a colour image with the object in colour and the background in white. Accordingly, the object is isolated in the colour image.

[00105] The method 700 of determining feature descriptors for an image, as executed at step 109, will be described in detail below with reference to Fig. 7. The method 700 is implemented as one or more software code modules of the software application program 333 resident in the hard disk drive 310 and being controlled in its execution by the processor 305. In one arrangement, each step of the method 700 may be implemented as a separate software code module. In one arrangement, one more steps of the method 700 may be executed within a smart-phone, tablet device or similar portable computing device connected to the network 220.

[00106] The method 700 begins at step 701 , where the image selected at step 103 is accessed by the software application program 333 from the storage devices 309 of the server computer module 300. Then at the next step 703, the software application 333, under execution of the processor 305, determines attribute data for the selected image. The determined attribute data may include one or more HSV histograms for the image. In one arrangement, three histograms are determined for the selected image, in the form of one histogram for each channel of the image.

[00107] The attribute data determined at step 703 may also include a reduced HSV histogram. The reduced HSV histogram may similarly be in the form of three histograms (i.e., one histogram for each channel of the image). In one arrangement, one or more histograms may be combined into a "fingerprint" single object based on a quantized image or quantized region. The selected image may be quantized using a number of means, k, based on randomly placed centres on the data representing the image. Accuracy of such a quantization is determined based on a variable, epsilon. The attribute data determined at step 703 may also include data representing geometric based features, such as corners within the image, areas which rapidly change brightness (e.g., lines) and textural defects. The geometric features may be stored within the hard disk drive 310 as keypoints within the image. Each keypoint may contain attributes including coordinates, size, angle, response (strength) and octave associated with the keypoint. The attributes of the geometric features may also include a class identifier representing an image cluster to which the image belongs.

[00108] The method 700 continues at the next step 705, where feature descriptors are determined based on the attribute data determined at step 703. The feature descriptors may be determined at step 705 using any suitable method. In one arrangement, the feature descriptors are determined at step 705 using the ORB algorithm as described in the article by Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011 , November), entitled "ORB: an efficient alternative to SIFT or SURF", In Computer Vision (ICCV), 2011 IEEE International Conference on (pp. 2564-2571). IEEE, hereby incorporated by reference in its entirety as if fully set forth herein. As an example, the feature descriptors may be determined based on keypoints determined at step 703.

[00109] Matching features in a selected image to features in a template image may be difficult. The feature matching may be difficult since for the feature in the selected image to match a feature in the template image, the features must be in exactly the same position and orientation in both the images. Accordingly, the feature descriptors are determined at step 705 to allow for a greater degree of flexibility when comparing images. The determined feature descriptors may be scale invariant, so that a feature in the selected image will match another feature in an image stored in the image database even if the feature in the database image is larger or smaller than the feature in the selected image. The determined feature descriptors may also be rotation invariant, so that a feature in the selected image will match another feature in the database image even if the feature in the database image has been rotated from a stored rotation of the feature in the selected image. The determined feature descriptors are used for performing feature matching as the features in the selected image are unlikely to perfectly match the features of the database images.

[00110] The method 700 concludes at the next step 707, where the feature descriptors are stored in the database configured within the hard disk drive 310, for use in matching the image. [00111] The method 800 of comparing images, as executed at step 111 , will be described in detail below with reference to Fig. 8. The method 800 is implemented as one or more software code modules of the software application program 333 resident in the hard disk drive 310 and being controlled in its execution by the processor 305. In one arrangement, each step of the method 700 may be implemented as a separate software code module. In one arrangement, one more steps of the method 700 may be executed within a smart-phone, tablet device or similar portable computing device connected to the network 220.

[00112] The method 800 begins at step 801, where the image selected at step 103 is accessed by the software application program 333 from the storage devices 309 of the server computer module 300. The accessed selected image may be stored in the RAM memory 306. The software application program 333 also accesses the attributes of the selected image, such as the HSV histograms and feature descriptors, determined in accordance with the method 700. Again, the accessed attributes of the selected image may be stored in the RAM memory 306. The following steps of the method 800 are used for comparing the attributes determined for the selected image (e.g., as determined in the methods 400, 500 and 600) to data (e.g., metadata, feature descriptors) representing one or more further images stored within the image database configured within the hard disk drive 310 or memory 306.

[00113] As seen in Fig. 8, steps 802 and 804, are performed in parallel. Similarly, the other steps of the method 800 may be performed in parallel. In another arrangement, steps 802 and 804, and the other steps of the method 800 may be performed sequentially.

[00114] At step 802, the software program 333, under execution of the processor 305, performs a histogram comparison to compare the three (3) different channels (i.e., HSV) of the selected image as determined at step 703 to data representing the channels of one or more images in the image database. Data for the database images used at step 802 may be stored in the RAM memory 306 prior to execution of the step 802. Then at step 803, any database image having histograms with a cumulative "similarity score" that exceeds a predetermined threshold is added to a "candidate list" stored in the RAM memory 306. The similarity score indicates how similar the selected image is to one of the database images. In one arrangement, the similarity score indicates how similar colours in a quantized selected image are to colours in a quantized database image. As an example, if a quantized selected image and a quantized database image have a high colour similarity, the similarity score may be 0.8. As another example, if colours in the quantized selected image and colours in the quantized database image completely match, the similarity score may be .0. Further, if colours in the quantized selected image and colours in the quantized database image do not match, the similarity score may be negative.

[00115] At step 804, the software program 333, under execution of the processor 305, determines a generalised feature descriptor (e.g., a GIST descriptor) of the selected image. The generalised feature descriptor may be determined at step 804 using any suitable method. In one arrangement, the generalised feature descriptor is determined at step 804 using the method described in the article by Oliva Aude, and Antonio Torralba., entitled "Modeling the shape of the scene: A holistic representation of the spatial envelope." International journal of computer vision 42, no. 3 (2001): 145-175, hereby incorporated by reference in its entirety as if fully set forth herein. In particular, the generalised feature descriptor may be determined based on a computational model of the recognition of real world scenes that bypasses segmentation and processing of individual objects or regions. The generalised feature descriptor may be determined based on a very low dimensional representation of a scene in the selected image, that is referred to as a Spatial Envelope, using a set of perceptual dimensions (e.g., naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene.

[00116] The generalised feature descriptor determined at step 804 is compared to feature descriptors of one or more images in the image database. Data for the database images used at step 804 may be stored in the RAM memory 306 prior to execution of the step 804.

[00117] Then at step 805, any database image found to be similar to the selected image at step 802 is added to a "candidate list" stored in the RAM memory 306.

[00118] In one arrangement, the software program 333, under execution of the processor 305, may be used in the method 400 for performing optical character recognition (OCR) on the selected image to detect any text on the object corresponding to the item as determined in the method 400. For example, text may be detected on the cover of the item. Any suitable OCR method may be used. In one arrangement, the OCR library "Tesseract" is used to scan the object in the selected image to detect text. Then, any detected text that matches a word in an ordinary dictionary in addition to any custom defined words (e.g., Spider-man) is used to perform a full fuzzy string search on the data representing the images in the image database. Any image in the image database having associated data (e.g., metadata, feature descriptors) that matches the detected text is added to the candidate list. In one arrangement, only images in the image database having associated data which matches the detected text according to a predetermined threshold is added to the candidate list.

[00119] In one arrangement, a comic window may also be detected on the selected image. A comic window is typically a white region, measuring approximately 4x2cm, positioned on a comic. Typically, the comic window is positioned on the upper half of a comic. However, the comic window may alternatively be positioned anywhere on a comic such as on the upper left hand side of a comic. Optical character recognition (OCR) may be performed on the comic window to determine issue details such as a price, issue number and month of release. In such an arrangement, any image in the image database having associated data (e.g., metadata, feature descriptors) that matches the detected issue details is added to the candidate list.

[00120] In one arrangement, the software program 333 may be used in the method 400 for detecting a barcode region in the selected image. For example, a barcode region may be detected by scanning the selected image and applying adaptive thresholding followed by edge detection on the selected image. Any edges detected in the selected image having similar dimensions to a barcode are isolated if the edges form a rectangular region. The detected rectangular region is determined to be a barcode if the rectangular region has a high spatial frequency in the vertical or horizontal domain and is positioned on the edges of an object representing an item. In another arrangement, the barcode region may be detected by applying a Sobel operator to the gray scale image (determined as described above) using a median blur filter.

[00121] Any barcodes detected are typically Universal Product Code (e.g., UPC_A) barcodes. Alternatively, the barcodes may be European Article Number (e.g., EAN13/ISBN) barcodes or matrix barcodes (e.g., Quick Response code, Data Matrix code).

[00122] If the detected barcode region is of sufficiently high resolution, then the barcode region may be read using a suitable barcode reading method to determine a code represented by the barcode. Alternatively, the detected barcode region may be upsampled before being read. The code represented by the barcode may be compared to the barcodes associated with one or more images in the image database using fuzzy string matching. Any image in the image database having an associated barcode that matches the barcode determined as described above is added to the candidate list. [00123] In one arrangement, if an exact matching barcode is not determined as described above, fuzzy string matching may be used to determine a series and/or a publisher for the item. Any determined series or publisher details for the item may compared to data associated with the images stored in the image database to generate the candidate list.

[00124] In one arrangement, the software application program 333, under execution of the processor 305, may be used in the method 400 to perform any suitable face detection method on the selected image. For example, a Haar cascade classifier may be used on the selected image. Then the software program 333 may use eigenfaces stored in a character database, configured within the RAM memory 306, to determine and identify any faces in the selected image. Then, details of any faces determined using the face detection method is compared to data (e.g., metadata, feature descriptors) associated with one or more images in the image database. Any image in the image database having an associated face that matches any faces determined as described above is added to the candidate list.

[00125] Again, the optical character recognition, barcode detection and face detection described above may be performed in parallel or sequentially with the other steps 802, 803, 804 and 805 of the method 800 described above.

[00126] The method 800 continues at the next step 806 following either of steps 803 and 805, where the selected image is compared to each of the images in the candidate list to determine any image in the candidate list that matches the selected image. The feature descriptors of the selected image are compared to the feature descriptors of each of the images in the candidate list using any suitable method. In one arrangement, the feature descriptors are compared using a k-nearest neighbour algorithm. As described above, any matching image in the image database is displayed on the display 214 at step 115. Further, the database image is displayed based with data (e.g., metadata, feature descriptors) associated with the matched image, where the data is retrieved from the image database at step 7.

[00127] As described above, the method 100 allows a seller or buyer to determine attributes of an item in order to identify the item, without the seller or buyer being required to have an intimate knowledge of the item. The method 100 allows the seller or buyer to determine attributes of the item, such as the name, age, origin or value of the item, in order to list the item on an e-commerce website for sale. The method 100 also allows the seller or buyer to list the item on an e-commerce website in a catalogue of items belonging to the seller or buyer. As described above, the methods 100 also allows the buyer to locate where the item may be purchased.

Industrial Applicability

[00128] The arrangements described are applicable to the computer and data processing industries and particularly for the image processing.

[00129] The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

[00130] In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings.

Claims

CLAIMS:

1. A method of displaying data for use in determining attributes of an item, said method comprising:

receiving one or more images of the item;

comparing the determined attributes to data associated with one or more further images; retrieving the data associated with at least one of the further images depending on the comparison; and

2. The method according to claim 1 , further comprising populating data fields of an electronic form using the retrieved data.

3. The method according to claim 1 , further comprising adding the retrieved data to a listing of the item on an e-commerce website.

4. The method according to claim 1 , further comprising displaying the at least one further image.

5. The method according to claim 4, wherein the at least one further image is a similar image.

6. The method according to claim 1 , further comprising isolating an object representing the item in the selected image.

7. The method according to claim 1 , wherein the further images are stored in an image database.

8. The method according to claim 7, wherein the data associated with the further images is stored in the image database.

9. The method according to claim 1 , further comprising determining one or more histograms for selected image.

10. The method according to claim 1 , further comprising determining one of more features for the selected image.

11. The method according to claim 1 , further comprising storing the selected image and the determined attributes in a database as a catalogue of items.

12. The method according to claim 1, wherein the selected ima

ge is received from a personal computer.

13. The method according to claim 1 , wherein the selected image is captured using a webcam.

14. The method according to claim 1 , wherein the selected image is received from a smart- phone.

15. The method according to claim 14, wherein the selected image is captured using a smart-phone.

16. The method according to claim 1 , wherein the selected image is received from a tablet device.

17. The method according to claim 17, wherein the selected image is captured using a tablet device. 8. The method according to claim , further comprising performing background isolation on the selected image prior to the images being received.

19. A system for displaying data for use in determining attributes of an item, said system comprising:

a memory for storing data and a computer program;

receiving one or more images of the item;

analysing at least a selected one of the received images to determine one or more attributes of the selected image; comparing the determined attributes to data associated with one or more further images;

20. An apparatus for displaying data for use in determining attributes of an item, said method comprising:

means for receiving one or more images of the item;

21. A computer readable medium comprising a computer program stored thereon for displaying data for use in determining attributes of an item, said program comprising:

code for receiving one or more images of the item;

22. A method of adding data for an item to an e-commerce website, said method comprising:

receiving one or more images of the item;

adding the retrieved data to an e-commerce website for the item.

23. An apparatus for adding data for an item to an e-commerce website, said apparatus comprising:

means for receiving one or more images of the item;

means for adding the retrieved data to an e-commerce website for the item.

24. A system for adding data for an item to an e-commerce website, said system comprising:

a memory for storing data and a computer program;

receiving one or more images of the item;

adding the retrieved data to an e-commerce website for the item.

25. A computer readable medium comprising a computer program stored thereon for adding data for an item to an e-commerce website, said method comprising:

code for receiving one or more images of the item;

code for analysing at least a selected one of the received images to determine one or more attributes of the selected image; code for comparing the determined attributes to data associated with one or more further images stored in an image database;

code for adding the retrieved data to an e-commerce website for the item.