{"rowid": 104, "title": "Sitewide Search On A Shoe String", "contents": "One of the questions I got a lot when I was building web sites for smaller businesses was if I could create a search engine for their site. Visitors should be able to search only this site and find things without the maintainer having to put \u201crelated articles\u201d  or \u201cfeatured content\u201d links on every page by hand. \n\nBack when this was all fields this wasn\u2019t easy as you either had to write your own scraping tool, use ht://dig or a paid service from providers like Yahoo, Altavista or later on Google. In the former case you had to swallow the bitter pill of computing and indexing all your content and storing it in a database for quick access and in the latter it hurt your wallet.\n\nTimes have moved on and nowadays you can have the same functionality for free using Yahoo\u2019s \u201cBuild your own search service\u201d \u2013 BOSS. The cool thing about BOSS is that it allows for a massive amount of hits a day and you can mash up the returned data in any format you want. Another good feature of it is that it comes with JSON-P as an output format which makes it possible to use it without any server-side component!\n\nStarting with a working HTML form\n\nIn order to add a search to your site, you start with a simple HTML form which you can use without JavaScript. Most search engines will allow you to filter results by domain. In this case we will search \u201cbbc.co.uk\u201d. If you use Yahoo as your standard search, this could be: \n\n<form id=\"customsearch\" action=\"http://search.yahoo.com/search\">\n\t<div>\n\t\t<label for=\"p\">Search this site:</label>\n\t\t<input type=\"text\" name=\"p\" id=\"term\">\n\t\t<input type=\"hidden\" name=\"vs\" id=\"site\" value=\"bbc.co.uk\">\n\t\t<input type=\"submit\" value=\"go\">\n\t</div>\n</form>\n\nThe Google equivalent is:\n\n<form id=\"customsearch\" action=\"http://www.google.co.uk/search\">\n\t<div>\n\t\t<label for=\"p\">Search this site:</label>\n\t\t<input type=\"text\" name=\"as_q\" id=\"term\">\n\t\t<input type=\"hidden\" name=\"as_sitesearch\" id=\"site\" value=\"bbc.co.uk\">\n\t\t<input type=\"submit\" value=\"go\">\n\t</div>\n</form>\n\nIn any case make sure to use the ID term for the search term and site for the site, as this is what we are going to use for the script. To make things easier, also have an ID called customsearch on the form.\n\nTo use BOSS, you should get your own developer API for BOSS and replace the one in the demo code. There is click tracking on the search results to see how successful your app is, so you should make it your own.\n\nAdding the BOSS magic\n\nBOSS is a REST API, meaning you can use it in any HTTP request or in a browser by simply adding the right parameters to a URL. Say for example you want to search \u201cbbc.co.uk\u201d for \u201cchristmas\u201d all you need to do is open the following URL:\n\nhttp://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&format=xml&appid=YOUR-APPLICATION-ID\n\nTry it out and click it to see the results in XML. We don\u2019t want XML though, which is why we get rid of the format=xml parameter which gives us the same information in JSON:\n\nhttp://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&appid=YOUR-APPLICATION-ID\n\nJSON makes most sense when you can send the output to a function and immediately use it. For this to happen all you need is to add a callback parameter and the JSON will be wrapped in a function call. Say for example we want to call SITESEARCH.found() when the data was retrieved we can do it this way:\n\nhttp://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&callback=SITESEARCH.found&appid=YOUR-APPLICATION-ID\n\nYou can use this immediately in a script node if you want to. The following code would display the total amount of search results for the term christmas on bbc.co.uk as an alert:\n\n<script type=\"text/javascript\">\n\tvar SITESEARCH = {};\n\tSITESEARCH.found = function(o){\n\t\talert(o.ysearchresponse.totalhits);\n\t}\n</script>\n<script type=\"text/javascript\" src=\"http://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&callback=SITESEARCH.found&appid=Kzv_lcHV34HIybw0GjVkQNnw4AEXeyJ9Rb1gCZSGxSRNrcif_HdMT9qTE1y9LdI-\">\n</script>\n\nHowever, for our example, we need to be a bit more clever with this.\n\nEnhancing the search form\n\n\n\n\nHere\u2019s the script that enhances a search form to show results below it.\n\nSITESEARCH = function(){\n\tvar config = {\n\t\tIDs:{\n\t\t\tsearchForm:'customsearch',\n\t\t\tterm:'term',\n\t\t\tsite:'site'\n\t\t},\n\t\tloading:'Loading results...',\n\t\tnoresults:'No results found.',\n\t\tappID:'YOUR-APP-ID',\n\t\tresults:20\n\t};\n\tvar form;\n\tvar out;\n\tfunction init(){\n\t\tif(config.appID === 'YOUR-APP-ID'){\n\t\t\talert('Please get a real application ID!');\n\t\t} else {\n\t\t\tform = document.getElementById(config.IDs.searchForm);\n\t\t\tif(form){\n\t\t\t\tform.onsubmit = function(){\n\t\t\t\t\tvar site = document.getElementById(config.IDs.site).value;\n\t\t\t\t\tvar term = document.getElementById(config.IDs.term).value;\n\t\t\t\t\tif(typeof site === 'string' && typeof term === 'string'){\n\t\t\t\t\t\tif(typeof out !== 'undefined'){\n\t\t\t\t\t\t\tout.parentNode.removeChild(out);\n\t\t\t\t\t\t}\n\t\t\t\t\t\tout = document.createElement('p');\n\t\t\t\t\t\tout.appendChild(document.createTextNode(config.loading));\n\t\t\t\t\t\tform.appendChild(out);\n\t\t\t\t\t\tvar APIurl = 'http://boss.yahooapis.com/ysearch/web/v1/' + \n\t\t\t\t\t\t\t\t\t\t\t\t\tterm + '?callback=SITESEARCH.found&sites=' + \n\t\t\t\t\t\t\t\t\t\t\t\t\tsite + '&count=' + config.results + \n\t\t\t\t\t\t\t\t\t\t\t\t\t'&appid=' + config.appID;\n\t\t\t\t\t\tvar s = document.createElement('script');\n\t\t\t\t\t\ts.setAttribute('src',APIurl);\n\t\t\t\t\t\ts.setAttribute('type','text/javascript');\n\t\t\t\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t\t\t\t\treturn false;\n\t\t\t\t\t}\n\t\t\t\t};\n\t\t\t}\n\t\t}\n\t};\n\tfunction found(o){\n\t\tvar list = document.createElement('ul');\n\t\tvar results = o.ysearchresponse.resultset_web;\n\t\tif(results){\n\t\t\tvar item,link,description;\n\t\t\tfor(var i=0,j=results.length;i<j;i++){\n\t\t\t\titem = document.createElement('li');\n\t\t\t\tlink = document.createElement('a');\n\t\t\t\tlink.setAttribute('href',results[i].clickurl);\n\t\t\t\tlink.innerHTML = results[i].title;\n\t\t\t\titem.appendChild(link);\n\t\t\t\tdescription = document.createElement('p');\n\t\t\t\tdescription.innerHTML = results[i]['abstract'];\n\t\t\t\titem.appendChild(description);\n\t\t\t\tlist.appendChild(item);\n\t\t\t}\n\t\t} else {\n\t\t\tlist = document.createElement('p');\n\t\t\tlist.appendChild(document.createTextNode(config.noresults));\n\t\t}\n\t\tform.replaceChild(list,out);\n\t\tout = list;\n\t};\n\treturn{\n\t\tconfig:config,\n\t\tinit:init,\n\t\tfound:found\n\t};\n}();\n\nOooohhhh scary code! Let\u2019s go through this one bit at a time:\n\nWe start by creating a module called SITESEARCH and give it an configuration object: \n\nSITESEARCH = function(){\n\tvar config = {\n\t\tIDs:{\n\t\t\tsearchForm:'customsearch',\n\t\t\tterm:'term',\n\t\t\tsite:'site'\n\t\t},\n\t\tloading:'Loading results...',\n\t\tappID:'YOUR-APP-ID',\n\t\tresults:20\n\t}\n\nConfiguration objects are a great idea to make your code easy to change and also to override. In this case you can define different IDs than the one agreed upon earlier, define a message to show when the results are loading, when there aren\u2019t any results, the application ID and the number of results that should be displayed.\n\nNote: you need to replace \u201cYOUR-APP-ID\u201d with the real ID you retrieved from BOSS, otherwise the script will complain! \n\nvar form;\nvar out;\nfunction init(){\n\tif(config.appID === 'YOUR-APP-ID'){\n\t\talert('Please get a real application ID!');\n\t} else {\n\nWe define form and out as variables to make sure that all the methods in the module have access to them. We then check if there was a real application ID defined. If there wasn\u2019t, the script complains and that\u2019s that.      \n\nform = document.getElementById(config.IDs.searchForm);\nif(form){\n\tform.onsubmit = function(){\n\t\tvar site = document.getElementById(config.IDs.site).value;\n\t\tvar term = document.getElementById(config.IDs.term).value;\n\t\tif(typeof site === 'string' && typeof term === 'string'){      \n\nIf the application ID was a winner, we check if the form with the provided ID exists and apply an onsubmit event handler. The first thing we get is the values of the site we want to search in and the term that was entered and check that those are strings.\n\nif(typeof out !== 'undefined'){\n\tout.parentNode.removeChild(out);\n}\nout = document.createElement('p');\nout.appendChild(document.createTextNode(config.loading));\nform.appendChild(out);  \n\nIf both are strings we check of out is undefined. We will create a loading message and subsequently the list of search results later on and store them in this variable. So if out is defined, it\u2019ll be an old version of a search (as users will re-submit the form over and over again)  and we need to remove that old version.\n\nWe then create a paragraph with the loading message and append it to the form.\n\nvar APIurl = 'http://boss.yahooapis.com/ysearch/web/v1/' + \n\t\t\t\t\t\t\t\t\t\t\t\tterm + '?callback=SITESEARCH.found&sites=' + \n\t\t\t\t\t\t\t\t\t\t\t\tsite + '&count=' + config.results + \n\t\t\t\t\t\t\t\t\t\t\t\t'&appid=' + config.appID;\n\t\t\t\t\tvar s = document.createElement('script');\n\t\t\t\t\ts.setAttribute('src',APIurl);\n\t\t\t\t\ts.setAttribute('type','text/javascript');\n\t\t\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t\t\t\treturn false;\n\t\t\t\t}\n\t\t\t};\n\t\t}\n\t}\n};\n\nNow it is time to call the BOSS API by assembling a correct REST URL, create a script node and apply it to the head of the document. We return false to ensure the form does not get submitted as we want to stay on the page.\n\nNotice that we are using SITESEARCH.found as the callback method, which means that we need to define this one to deal with the data returned by the API.\n\nfunction found(o){\n\tvar list = document.createElement('ul');\n\tvar results = o.ysearchresponse.resultset_web;\n\tif(results){\n\t\tvar item,link,description;\n\nWe create a new list and then get the resultset_web array from the data returned from the API. If there aren\u2019t any results returned, this array will not exist which is why we need to check for it. Once we done that we can define three variables to repeatedly store the item title we want to display, the link to point to and the description of the link.\n\nfor(var i=0,j=results.length;i<j;i++){\n\titem = document.createElement('li');\n\tlink = document.createElement('a');\n\tlink.setAttribute('href',results[i].clickurl);\n\tlink.innerHTML = results[i].title;\n\titem.appendChild(link);\n\tdescription = document.createElement('p');\n\tdescription.innerHTML = results[i]['abstract'];\n\titem.appendChild(description);\n\tlist.appendChild(item);\n}\n\nWe then loop over the results array and assemble a list of results with the titles in links and paragraphs with the abstract of the site. Notice the bracket notation for abstract as abstract is a reserved word in JavaScript2 :).\n\n} else {\n\t\tlist = document.createElement('p');\n\t\tlist.appendChild(document.createTextNode(config.noresults));\n\t}\n\tform.replaceChild(list,out);\n\tout = list;\n};      \n\nIf there aren\u2019t any results, we define a paragraph with the no results message as list. In any case we replace the old out (the loading message) with the list and re-define out as the list.  \n\nreturn{\n\t\tconfig:config,\n\t\tinit:init,\n\t\tfound:found\n\t};\n}();\n\nAll that is left to do is return the properties and methods we want to make public. In this case found needs to be public as it is accessed by the API return. We return init to make it accessible and config to allow implementers to override any of the properties.\n\nUsing the script\n\n\nIn order to use this script, all you need to do is to add it after the form in the document, override the API key with your own and call init():\n\n<form id=\"customsearch\" action=\"http://search.yahoo.com/search\">\n\t<div>\n\t\t<label for=\"p\">Search this site:</label>\n\t\t<input type=\"text\" name=\"p\" id=\"term\">\n\t\t<input type=\"hidden\" name=\"vs\" id=\"site\" value=\"bbc.co.uk\">\n\t\t<input type=\"submit\" value=\"go\">\n\t</div>\n</form>\n<script type=\"text/javascript\" src=\"boss-site-search.js\"></script>\n<script type=\"text/javascript\">\n\tSITESEARCH.config.appID = 'copy-the-id-you-know-to-get-where';\n\tSITESEARCH.init();\n</script>\n\nWhere to go from here\n\nThis is just a very simple example of what you can do with BOSS. You can define languages and regions, retrieve and display images and news and mix the results with other data sources before displaying them. One very cool feature is that by adding a view=keyterms parameter to the URL you can get the keywords of each of the results to drill deeper into the search. An example for this written in PHP is available on the YDN blog. For JavaScript solutions there is a handy wrapper called yboss available to help you go nuts.", "year": "2008", "author": "Christian Heilmann", "author_slug": "chrisheilmann", "published": "2008-12-04T00:00:00+00:00", "url": "https://24ways.org/2008/sitewide-search-on-a-shoestring/", "topic": "code"}