GoodRelations and Yahoo SearchMonkey

News

2010-02-25:Great news: Yahoo has just turned on the improved rendering of GoodRelations-augmented product pages! Today is the day GoodRelations + RDFa will become a mainstream SEO technique.

File:Yahoo2010.png

2010-02-20:If you use the Magento shop software, there is now a free extension to use GoodRelations for SEO automatically.

2009-11-25:Small bugfix: The value for the gr:has-EAN_UCC-13 property was too short (11 instead of 13 digits). The new value is correct.

2009-10-30: Yahoo Searchmonkey rendering temporarily down: Yahoo crashed the improved rendering temporarily in the process of updating the Yahoo UI - they are working on that but it may take until mid of November for the improved pages to re-appear.

2009-10-10:GoodRelations & Yahoo confirmed in the wild! Our demo page shows up in Yahoo now, see http://tr.im/yahooproduct2

2009-09-22: We removed the XML shortcuts for empty span/div elements from the examples, since that could cause problems with some HTML agents (browsers, applications).
2009-07-24:Yahoo just confirmed to me that GoodRelations and RDFa could be used for optimizing eBay offers in new www.yahoo.com (even if the page has no proper XHTML header)

2009-07-23:The Yahoo! Validator is currently down for maintenance, see this note from Yahoo!. I suspect they are currently fixing the false warning that media:image objects cannot be URIs.

Introduction

Great news for any business in the World, and any Web of Linked Data developer:As of now, Yahoo will display price and offering details and other meta-data of any e-commerce Web page if the site owner uses the free GoodRelations vocabulary.

Previously, such data was only used within special applications developed in the Yahoo ecosystem. Now, every site owner can enhance the search results of his or her offers in Yahoo!.

On this page we will show how you can augment a Web page describing your business and your products with additional data so that Yahoo! will show those details in the results.

At the same time, this data can be used by many novel services in the Web of Linked Data, i.e., you do not only improve your search results in Yahoo! but also become visible for customers in novel search and recommendation engines.

We will use the following specifications:

  1. The RDFa syntax for embedding RDF meta-data in XHTML Web pages.
  2. The GoodRelations Web vocabulary for e-commerce.

Yahoo! currently supports eight types of augmented search results in the standard output, of which the following two are most relevant for a business:

  1. Local (phone numbers, addresses, opening hours, position, etc. of your shops etc.)
  2. Product(prices, images, and product info)

We will not cover the news, video, event, documents, discussion, and games patterns in this recipe.

Important: This document in here goes beyond the original Yahoo! specifications at

because the recipes shown in here make your business visible for

  1. Yahoo!
  2. Yahoo! SearchMonkey and Yahoo! BOSS services, and
  3. all other novel Web of Linked Data services unrelated to Yahoo.

400px

Getting Started

Note: If you want that Yahoo! displays your data in the standard search interface, you are currently limited to one type of additional information per page. That means, you must e.g. have separate HTML pages for

  • your company and
  • one for each individual product or offer.

In the following example, we will use the two Web pages:

Let's assume they initially look as follows:

http://www.heppnetz.de/searchmonkey/company-raw.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<p>Martin Hepp's Yahoo SearchMonkey Mock-up Page - initial page</p>

This page is part of a demo of how a small business can feed its contact etc. descriptions into Yahoo! SearchMonkey and the Web of Linked Data.

Provided by the E-Business & Web Science Research Group (Prof. Hepp) at Universitaet der Bundeswehr Muenchen,

     ``www.unibw.de/ebusiness/``.
   

     ``Company Logo

     ``Hepp Space Ventures Inc.``
     
Hepp Space Ventures Inc. is a non-existent corporation made up for the purpose of this example.

     
Address of our main office:
     ``1234 Hepp Road``,
     ``85577``
     ``Neubiberg``,
     ``Bavaria``,
     ``Germany``.
      Phone: ``+49-89-6004-0``

     Web: ``www.heppnetz.de``

     
      Customer Reviews:
     Rated ``4.5`` on a scale of
     ``1`` to
     ``5``
     by ``14`` users.

     
Opening Hours:
     ``Mon-Fri 10-6, Sat 10-4.30``
     
Parking:

     ``Valet parking & parking lot``

     
Attire:
     ``Business casual``

     
Main type of food:
     ``Black coffee if the waiting gets too long``
   

http://www.heppnetz.de/searchmonkey/product-raw.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<p>Martin Hepp's Yahoo SearchMonkey Mock-up Page - initial page</p>

This page is a demo of how a small business can feed its product and offer descriptions into Yahoo! SearchMonkey and the Web of Linked Data.

Provided by the E-Business & Web Science Research Group (Prof. Hepp) at Universitaet der Bundeswehr Muenchen,

     ``www.unibw.de/ebusiness/``.
   

     Price:
     ``34.99``
     ``USD``
     Product Name:
     ``Personal SCSI 16-bit SCSI Controller``
     
Description:
     ``This low-cost, high-performance SCSI controller allows you to connect up to
     seven professional mass-storage devices to your computer.``

      EAN/UPC code:
     ``0001036378012``
      Article number:
     ``10363780``

     ``Product Image

     Manufacturer:
     ``Hepp Computertechnik``

     ``Product Specifications``
      ``FAQ``
      ``Product Manual``

     
Product Reviews:
     
Average:
     ``4.5``, lowest:
     ``0``, highest:
     ``5`` (total number of reviews:
     ``45``)
    

Now, the first step is that you change the DOCTYPE in the header to "XHTML+RDFa":


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">

Also, make sure that the "head" element includes the proper content type and encoding for XHTML:

<p>Martin Hepp's Yahoo SearchMonkey Mock-up Page</p>

Then, insert the following namespace prefix definitions into the "body" element in both files:

So the complete header should look like

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">

<p>Martin Hepp's Yahoo SearchMonkey Mock-up Page</p>

Describing Your Business

Then, add the additional "div" and "span" elements plus all attributes as shown below for encoding your contact details and opening hours to the file at http://www.heppnetz.de/searchmonkey/company.html:

http://www.heppnetz.de/searchmonkey/company.html - With meta-data:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">

<p>Martin Hepp's Yahoo SearchMonkey Mock-up Page</p>

This page is a demo of how a small business can feed its contact etc. descriptions into Yahoo! SearchMonkey and the Web of Linked Data.

Provided by the E-Business & Web Science Research Group (Prof. Hepp) at Universitaet der Bundeswehr Muenchen,

     ``www.unibw.de/ebusiness/``.
   

     ``

     ``
       ``Company Logo
     ``

     
      
     ``Hepp Space Ventures Inc.``
     
Hepp Space Ventures Inc. is a non-existent corporation made up for the purpose of this example.


Address of our main office:

         ``1234 Hepp Road``,
         ``85577``
         ``Neubiberg``,
         ``Bavaria``,
         ``Germany``.
       

     Phone: ``+49-89-6004-0``

       ``
       ``
     

     Web: ``www.heppnetz.de``

     
      Customer Reviews:
     

         Rated ``4.5`` on a scale of
         ``1`` to
         ``5``
         by ``14`` users.
       

     
Opening Hours:
     ``Mon-Fri 10-6, Sat 10-4.30 [Yahoo commerce]``
     
Parking:

     ``Valet parking & parking lot``
     
Attire:

     ``Business casual``

     ``

     
Main type of food:
     ``Black coffee if the waiting gets too long``

           ``
           ``
         

             ``
             ``
             

         
          

             ``
             ``
             


Describing Each Product

Next, add the additional "div" and "span" elements plus all attributesas shown below for encoding your product details including pricing to the file at http://www.heppnetz.de/searchmonkey/product.html:

http://www.heppnetz.de/searchmonkey/product.html - With meta-data:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">

<p>Martin Hepp's Yahoo SearchMonkey Mock-up Page</p>

This page is a demo of how a small business can feed its product and offer descriptions into Yahoo! SearchMonkey and the Web of Linked Data.

Provided by the E-Business & Web Science Research Group (Prof. Hepp) at Universitaet der Bundeswehr Muenchen,

     ``www.unibw.de/ebusiness/``.
   

     ``

     ``
     ``

       ``
       Price:
         ``34.99``
         ``USD``
         ``
       ``

     

         ``
         ``
         

             ``
             Product Name:
             ``Personal SCSI 16-bit SCSI Controller``
             
Description:
             ``This low-cost, high-performance SCSI controller allows you to connect up to
             seven professional mass-storage devices to your computer.``

              EAN/UPC code:
             ``0001036378012``
              Article number:
             ``10363780``

             ``
               ``Product Image
             ``

               ``
                 Manufacturer:
                 ``Hepp Computertechnik``
                 ``
               ``

             


             ``Product Specifications``
              ``FAQ``
              ``Product Manual``

             
Product Reviews:
             

               ``
                 
Average:
                 ``4.5``, lowest:
                 ``0``, highest:
                 ``5`` (total number of reviews:
                 ``45``)
                ``

             

Publishing the Data

Next, upload the new files to their original location:

Check with a standard browser that the layout of the page is fine.

If you want, you can use stylesheets and other standard design techniques for optimizing the rendering. The only important thing is that the file remains a valid XHTML document. You can use the W3C Markup Validation service at http://validator.w3.org/ to check that.

Now, validate the pages using the Yahoo! Validation Service at the bottom of

You can also invoke it directly using the URI of your page as the respective parameter:

It should show that your mark-up is okay, but not yet included in the Yahoo! index:

700px|alt Yahoo Validation Results

Telling Yahoo and the World

Now, the last thing that remains to be done is telling Yahoo! and the world to consider your new data.

As for Yahoo!, use http://siteexplorer.search.yahoo.com/submit to submit your page.

This requires free registration with Yahoo!.

700px|alt Yahoo Site Explorer

Then, enter the URIs of all pages that you changed. If there are links between the pages, Yahoo! will find all of them, so you won't have to submit hundreds of URIs.

If the page you enhanced following this recipe is not a highly ranked page itself, you should link to it from a prominent page, e.g. your main page, because it will depend on the popularity of your page how quickly the Yahoo! crawler will stop by and index your new data. It can be anything between a few days and several months.

As a last step, you should inform Semantic Web indexing services. Currently, sindice.com is the only service that we know that considers RDFa data, so this should be your first choice.

Go to http://sindice.com/main/submit.

700px

and enter the URIs of all of your enhanced pages there. Then press "submit".

Another popular service, Ping The Semantic Web does not yet support RDFa directly, but you can still make your data known with at trick:

700px|alt Feeding RDFa into PingTheSemanticWeb

Congratulations!

You are now all set - your additional data will sooner or later appear in

  • the Yahoo! search results,
  • numerous novel services based on Yahoo! SearchMonkey and
  • Web of Linked Data applications.

If you want to check whether your data is already included in the Yahoo! SearchMonkey index, you can use the tool at

http://goodrelations-search.appspot.com/

It is basically a variant of the Yahoo! search service that displays all meta-data that is found in the Yahoo! index.

Note: At the time of writing, there are some problems in Yahoo's RDF export, so even though my page has been crawled by Yahoo!, the SeachMonkey index does not return fully correct RDF. Yahoo is aware of the bug.

500px

If you enter

  • wwwurl:

as the search parameter, the tool will list  all meta-data that is currently in the Yahoo! SearchMonkey index.

For example,

  • wwwurl:

http://www.heppnetz.de/searchmonkey/company.html will show the meta-data for

http://www.heppnetz.de/searchmonkey/company.html

You can use the following links to check those pages directly

Information for Web of Linked Data Developers (RDF, OWL, SPARQL, ...)

If you are familiar with the W3C Semantic Web technologies RDF and RDFS or use the Tabulator browser plug-in, you can look at the meta-data in your pages using the RDFa distiller (and validator) written by Ivan Herman (http://www.w3.org/People/Ivan/), available at

http://www.w3.org/2007/08/pyRdfa/

Select - "RDF/XML" or "Turtle" as the output format, - "yes" for warnings, and - "strict" for the parsing type: - Click on "Go!"

The result using the Turtle syntax should look as follows:

a) http://www.heppnetz.de/searchmonkey/company.html

@prefix commerce: <http://search.yahoo.com/searchmonkey/commerce/> .
@prefix dist: <http://www.w3.org/2007/08/pyRdfa/distiller#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix gr: <http://purl.org/goodrelations/v1#> .
@prefix media: <http://search.yahoo.com/searchmonkey/media/> .
@prefix product: <http://search.yahoo.com/searchmonkey/product/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix review: <http://purl.org/stuff/rev#> .
@prefix use: <http://search.yahoo.com/searchmonkey-datatype/use/> .
@prefix vcard: <http://www.w3.org/2006/vcard/ns#> .
@prefix xhv: <http://www.w3.org/1999/xhtml/vocab#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .


<http://www.heppnetz.de/searchmonkey/company.html#business> a gr:BusinessEntity, commerce:Business, vcard:VCard ;
   rdfs:label "Hepp Space Ventures Inc."@en ;
   gr:hasPOS <http://www.heppnetz.de/searchmonkey/company.html#myshop> ;
   gr:legalName "Hepp Space Ventures Inc."@en ;
   gr:offers <http://www.heppnetz.de/searchmonkey/product.html#myoffer> ;
   review:hasReview
       [ a review:Review ;
         review:maxRating "5"^^xsd:integer ;
         review:minRating "1"^^xsd:integer ;
         review:rating "4.5"^^xsd:float ;
         review:totalRatings "14"^^xsd:integer
       ] ;
   commerce:attire "Business casual"@en ;
   commerce:businessCategory "Electronics and Computer"@en ;
   commerce:cuisine "Black coffee if the waiting gets too long"@en ;
   commerce:hoursOfOperation "Mon-Fri 10-6, Sat 10-4.30 [Yahoo commerce]"@en ;
   commerce:parkingOptions "Valet parking & parking lot"@en ;
   media:image <http://www.heppnetz.de/searchmonkey/logo.png> ;
   foaf:page <http://www.heppnetz.de/>, <http://www.heppnetz.de/searchmonkey/company.html>;
   rdfs:seeAlso <http://www.heppnetz.de/searchmonkey/logo.png> ;
   vcard:adr <http://www.heppnetz.de/searchmonkey/company.html#address> ;
   vcard:fn "Hepp Space Ventures Inc."@en ;
   vcard:geo
       [ vcard:latitude "48.0802626"^^xsd:float ;
         vcard:longitude "11.6407428"^^xsd:float
       ] ;
   vcard:tel "+49-89-6004-0"^^xsd:string ;
   vcard:url <http://www.heppnetz.de/> ;
   foaf:depiction <http://www.heppnetz.de/searchmonkey/logo.png> .

<http://www.heppnetz.de/searchmonkey/company.html#myshop> a gr:LocationOfSalesOrServiceProvisioning ;
   gr:hasOpeningHoursSpecification <http://www.heppnetz.de/searchmonkey/company.html#saturdays>, <http://www.heppnetz.de/searchmonkey/company.html#workdays> ;
   vcard:adr <http://www.heppnetz.de/searchmonkey/company.html#address> ;
   vcard:geo
       [ vcard:latitude "48.0802626"^^xsd:float ;
         vcard:longitude "11.6407428"^^xsd:float
       ] .

<http://www.heppnetz.de/searchmonkey/company.html#saturdays> a gr:OpeningHoursSpecification ;
   gr:closes "16:30:00"^^xsd:time ;
   gr:hasOpeningHoursDayOfWeek gr:Saturday ;
   gr:opens "10:00:00"^^xsd:time .

<http://www.heppnetz.de/searchmonkey/company.html#workdays> a gr:OpeningHoursSpecification ;
   gr:closes "18:00:00"^^xsd:time ;
   gr:hasOpeningHoursDayOfWeek
       gr:Friday,
       gr:Monday,
       gr:Thursday,
       gr:Tuesday,
       gr:Wednesday ;
   gr:opens "10:00:00"^^xsd:time .

<http://www.heppnetz.de/searchmonkey/company.html#address> a vcard:Address ;
   vcard:country-name "Germany"@en ;
   vcard:locality "Neubiberg"@en ;
   vcard:postal-code "85577"^^xsd:string ;
   vcard:region "Bavaria"@en ;
   vcard:street-address "1234 Hepp Road"@en .

b) http://www.heppnetz.de/searchmonkey/product.html

@prefix commerce: <http://search.yahoo.com/searchmonkey/commerce/> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix gr: <http://purl.org/goodrelations/v1#> .
@prefix media: <http://search.yahoo.com/searchmonkey/media/> .
@prefix product: <http://search.yahoo.com/searchmonkey/product/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix review: <http://purl.org/stuff/rev#> .
@prefix use: <http://search.yahoo.com/searchmonkey-datatype/use/> .
@prefix vcard: <http://www.w3.org/2006/vcard/ns#> .
@prefix xhv: <http://www.w3.org/1999/xhtml/vocab#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .


<http://www.heppnetz.de/searchmonkey/company.html#business> gr:offers <http://www.heppnetz.de/searchmonkey/product.html#myoffer> .

<http://www.heppnetz.de/searchmonkey/product.html#TypeAndQuantityNode> a gr:TypeAndQuantityNode ;
   gr:amountOfThisGood "1.0"^^xsd:float ;
   gr:hasUnitOfMeasurement "C62"^^xsd:string ;
   gr:typeOfGood <http://www.heppnetz.de/searchmonkey/product.html#product> .

<http://www.heppnetz.de/searchmonkey/product.html#UnitPriceSpecification> a gr:UnitPriceSpecification ;
   gr:hasCurrency "USD"^^xsd:string ;
   gr:hasCurrencyValue "34.99"^^xsd:float ;
   gr:hasUnitOfMeasurement "C62"^^xsd:string .

<http://www.heppnetz.de/searchmonkey/product.html#myoffer> a gr:Offering ;
   gr:availableAtOrFrom <http://www.heppnetz.de/searchmonkey/company.html#myshop> ;
   gr:hasBusinessFunction gr:Sell ;
   gr:hasPriceSpecification <http://www.heppnetz.de/searchmonkey/product.html#UnitPriceSpecification> ;
   gr:includesObject <http://www.heppnetz.de/searchmonkey/product.html#TypeAndQuantityNode> ;
   gr:validFrom "2009-07-20T00:00:00Z"^^xsd:dateTime ;
   gr:validThrough "2010-07-20T00:00:00Z"^^xsd:dateTime ;
   foaf:page <http://www.heppnetz.de/searchmonkey/product.html> .

<http://www.heppnetz.de/searchmonkey/product.html#product> a gr:ProductOrServicesSomeInstancesPlaceholder, product:Product ;
   rdfs:label "Personal SCSI 16-bit SCSI Controller"@en ;
   gr:hasEAN_UCC-13 "0001036378012"^^xsd:string ;
   gr:hasManufacturer <http://www.heppnetz.de/searchmonkey/product.html#heppcomputer> ;
   review:hasReview
       [ a review:Review ;
         review:maxRating "5"^^xsd:integer ;
         review:minRating "0"^^xsd:integer ;
         review:rating "4.5"^^xsd:float ;
         review:totalRatings "45"^^xsd:integer
       ] ;
   media:image <http://www.heppnetz.de/searchmonkey/pscsi.jpg> ;
   product:faq <http://www.heppnetz.de/searchmonkey/heppcomputer.html#faq> ;
   product:identifier "10363780"^^use:sku ;
   product:manual <http://www.heppnetz.de/searchmonkey/heppcomputer.html#manual> ;
   product:manufacturer <http://www.heppnetz.de/searchmonkey/product.html#heppcomputer> ;
   product:specification <http://www.heppnetz.de/searchmonkey/heppcomputer.html#spec> ;
   rdfs:comment "This low-cost, high-performance SCSI controller allows you to connect up to seven professional mass-storage devices to your computer."@en ;
   foaf:page <http://www.heppnetz.de/searchmonkey/product.html>;
   rdfs:seeAlso <http://www.heppnetz.de/searchmonkey/pscsi.jpg> ;
   foaf:depiction <http://www.heppnetz.de/searchmonkey/pscsi.jpg> .

<http://www.heppnetz.de/searchmonkey/product.html#heppcomputer> a gr:BusinessEntity, commerce:Business, vcard:VCard ;
   foaf:page <http://www.heppnetz.de/searchmonkey/heppcomputer.html> ;
   vcard:organization-name "Hepp Computertechnik Inc."@en ;
   vcard:url <http://www.heppnetz.de/searchmonkey/heppcomputer.html> .


Things That Took Us A While To Learn

-)

1. Yahoo accepts images only when the URI is specified this way:

   ``
       ``product image
   ``

The semantically equivalent pattern does not work. It triggers an error in the Yahoo Validator.

2. The ordering of elements matters for Yahoo. For example, the gr:includesObject property MUST ENCLOSE the gr:ProductOrServicesSomeInstancesPlaceholder. Otherwise Yahoo complains about "multiple objects" in one page and will not produce the enhanced rendering.

Additional Resources