{"rowid": 104, "title": "Sitewide Search On A Shoe String", "contents": "One of the questions I got a lot when I was building web sites for smaller businesses was if I could create a search engine for their site. Visitors should be able to search only this site and find things without the maintainer having to put \u201crelated articles\u201d  or \u201cfeatured content\u201d links on every page by hand. \n\nBack when this was all fields this wasn\u2019t easy as you either had to write your own scraping tool, use ht://dig or a paid service from providers like Yahoo, Altavista or later on Google. In the former case you had to swallow the bitter pill of computing and indexing all your content and storing it in a database for quick access and in the latter it hurt your wallet.\n\nTimes have moved on and nowadays you can have the same functionality for free using Yahoo\u2019s \u201cBuild your own search service\u201d \u2013 BOSS. The cool thing about BOSS is that it allows for a massive amount of hits a day and you can mash up the returned data in any format you want. Another good feature of it is that it comes with JSON-P as an output format which makes it possible to use it without any server-side component!\n\nStarting with a working HTML form\n\nIn order to add a search to your site, you start with a simple HTML form which you can use without JavaScript. Most search engines will allow you to filter results by domain. In this case we will search \u201cbbc.co.uk\u201d. If you use Yahoo as your standard search, this could be: \n\n<form id=\"customsearch\" action=\"http://search.yahoo.com/search\">\n\t<div>\n\t\t<label for=\"p\">Search this site:</label>\n\t\t<input type=\"text\" name=\"p\" id=\"term\">\n\t\t<input type=\"hidden\" name=\"vs\" id=\"site\" value=\"bbc.co.uk\">\n\t\t<input type=\"submit\" value=\"go\">\n\t</div>\n</form>\n\nThe Google equivalent is:\n\n<form id=\"customsearch\" action=\"http://www.google.co.uk/search\">\n\t<div>\n\t\t<label for=\"p\">Search this site:</label>\n\t\t<input type=\"text\" name=\"as_q\" id=\"term\">\n\t\t<input type=\"hidden\" name=\"as_sitesearch\" id=\"site\" value=\"bbc.co.uk\">\n\t\t<input type=\"submit\" value=\"go\">\n\t</div>\n</form>\n\nIn any case make sure to use the ID term for the search term and site for the site, as this is what we are going to use for the script. To make things easier, also have an ID called customsearch on the form.\n\nTo use BOSS, you should get your own developer API for BOSS and replace the one in the demo code. There is click tracking on the search results to see how successful your app is, so you should make it your own.\n\nAdding the BOSS magic\n\nBOSS is a REST API, meaning you can use it in any HTTP request or in a browser by simply adding the right parameters to a URL. Say for example you want to search \u201cbbc.co.uk\u201d for \u201cchristmas\u201d all you need to do is open the following URL:\n\nhttp://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&format=xml&appid=YOUR-APPLICATION-ID\n\nTry it out and click it to see the results in XML. We don\u2019t want XML though, which is why we get rid of the format=xml parameter which gives us the same information in JSON:\n\nhttp://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&appid=YOUR-APPLICATION-ID\n\nJSON makes most sense when you can send the output to a function and immediately use it. For this to happen all you need is to add a callback parameter and the JSON will be wrapped in a function call. Say for example we want to call SITESEARCH.found() when the data was retrieved we can do it this way:\n\nhttp://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&callback=SITESEARCH.found&appid=YOUR-APPLICATION-ID\n\nYou can use this immediately in a script node if you want to. The following code would display the total amount of search results for the term christmas on bbc.co.uk as an alert:\n\n<script type=\"text/javascript\">\n\tvar SITESEARCH = {};\n\tSITESEARCH.found = function(o){\n\t\talert(o.ysearchresponse.totalhits);\n\t}\n</script>\n<script type=\"text/javascript\" src=\"http://boss.yahooapis.com/ysearch/web/v1/christmas?sites=bbc.co.uk&callback=SITESEARCH.found&appid=Kzv_lcHV34HIybw0GjVkQNnw4AEXeyJ9Rb1gCZSGxSRNrcif_HdMT9qTE1y9LdI-\">\n</script>\n\nHowever, for our example, we need to be a bit more clever with this.\n\nEnhancing the search form\n\n\n\n\nHere\u2019s the script that enhances a search form to show results below it.\n\nSITESEARCH = function(){\n\tvar config = {\n\t\tIDs:{\n\t\t\tsearchForm:'customsearch',\n\t\t\tterm:'term',\n\t\t\tsite:'site'\n\t\t},\n\t\tloading:'Loading results...',\n\t\tnoresults:'No results found.',\n\t\tappID:'YOUR-APP-ID',\n\t\tresults:20\n\t};\n\tvar form;\n\tvar out;\n\tfunction init(){\n\t\tif(config.appID === 'YOUR-APP-ID'){\n\t\t\talert('Please get a real application ID!');\n\t\t} else {\n\t\t\tform = document.getElementById(config.IDs.searchForm);\n\t\t\tif(form){\n\t\t\t\tform.onsubmit = function(){\n\t\t\t\t\tvar site = document.getElementById(config.IDs.site).value;\n\t\t\t\t\tvar term = document.getElementById(config.IDs.term).value;\n\t\t\t\t\tif(typeof site === 'string' && typeof term === 'string'){\n\t\t\t\t\t\tif(typeof out !== 'undefined'){\n\t\t\t\t\t\t\tout.parentNode.removeChild(out);\n\t\t\t\t\t\t}\n\t\t\t\t\t\tout = document.createElement('p');\n\t\t\t\t\t\tout.appendChild(document.createTextNode(config.loading));\n\t\t\t\t\t\tform.appendChild(out);\n\t\t\t\t\t\tvar APIurl = 'http://boss.yahooapis.com/ysearch/web/v1/' + \n\t\t\t\t\t\t\t\t\t\t\t\t\tterm + '?callback=SITESEARCH.found&sites=' + \n\t\t\t\t\t\t\t\t\t\t\t\t\tsite + '&count=' + config.results + \n\t\t\t\t\t\t\t\t\t\t\t\t\t'&appid=' + config.appID;\n\t\t\t\t\t\tvar s = document.createElement('script');\n\t\t\t\t\t\ts.setAttribute('src',APIurl);\n\t\t\t\t\t\ts.setAttribute('type','text/javascript');\n\t\t\t\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t\t\t\t\treturn false;\n\t\t\t\t\t}\n\t\t\t\t};\n\t\t\t}\n\t\t}\n\t};\n\tfunction found(o){\n\t\tvar list = document.createElement('ul');\n\t\tvar results = o.ysearchresponse.resultset_web;\n\t\tif(results){\n\t\t\tvar item,link,description;\n\t\t\tfor(var i=0,j=results.length;i<j;i++){\n\t\t\t\titem = document.createElement('li');\n\t\t\t\tlink = document.createElement('a');\n\t\t\t\tlink.setAttribute('href',results[i].clickurl);\n\t\t\t\tlink.innerHTML = results[i].title;\n\t\t\t\titem.appendChild(link);\n\t\t\t\tdescription = document.createElement('p');\n\t\t\t\tdescription.innerHTML = results[i]['abstract'];\n\t\t\t\titem.appendChild(description);\n\t\t\t\tlist.appendChild(item);\n\t\t\t}\n\t\t} else {\n\t\t\tlist = document.createElement('p');\n\t\t\tlist.appendChild(document.createTextNode(config.noresults));\n\t\t}\n\t\tform.replaceChild(list,out);\n\t\tout = list;\n\t};\n\treturn{\n\t\tconfig:config,\n\t\tinit:init,\n\t\tfound:found\n\t};\n}();\n\nOooohhhh scary code! Let\u2019s go through this one bit at a time:\n\nWe start by creating a module called SITESEARCH and give it an configuration object: \n\nSITESEARCH = function(){\n\tvar config = {\n\t\tIDs:{\n\t\t\tsearchForm:'customsearch',\n\t\t\tterm:'term',\n\t\t\tsite:'site'\n\t\t},\n\t\tloading:'Loading results...',\n\t\tappID:'YOUR-APP-ID',\n\t\tresults:20\n\t}\n\nConfiguration objects are a great idea to make your code easy to change and also to override. In this case you can define different IDs than the one agreed upon earlier, define a message to show when the results are loading, when there aren\u2019t any results, the application ID and the number of results that should be displayed.\n\nNote: you need to replace \u201cYOUR-APP-ID\u201d with the real ID you retrieved from BOSS, otherwise the script will complain! \n\nvar form;\nvar out;\nfunction init(){\n\tif(config.appID === 'YOUR-APP-ID'){\n\t\talert('Please get a real application ID!');\n\t} else {\n\nWe define form and out as variables to make sure that all the methods in the module have access to them. We then check if there was a real application ID defined. If there wasn\u2019t, the script complains and that\u2019s that.      \n\nform = document.getElementById(config.IDs.searchForm);\nif(form){\n\tform.onsubmit = function(){\n\t\tvar site = document.getElementById(config.IDs.site).value;\n\t\tvar term = document.getElementById(config.IDs.term).value;\n\t\tif(typeof site === 'string' && typeof term === 'string'){      \n\nIf the application ID was a winner, we check if the form with the provided ID exists and apply an onsubmit event handler. The first thing we get is the values of the site we want to search in and the term that was entered and check that those are strings.\n\nif(typeof out !== 'undefined'){\n\tout.parentNode.removeChild(out);\n}\nout = document.createElement('p');\nout.appendChild(document.createTextNode(config.loading));\nform.appendChild(out);  \n\nIf both are strings we check of out is undefined. We will create a loading message and subsequently the list of search results later on and store them in this variable. So if out is defined, it\u2019ll be an old version of a search (as users will re-submit the form over and over again)  and we need to remove that old version.\n\nWe then create a paragraph with the loading message and append it to the form.\n\nvar APIurl = 'http://boss.yahooapis.com/ysearch/web/v1/' + \n\t\t\t\t\t\t\t\t\t\t\t\tterm + '?callback=SITESEARCH.found&sites=' + \n\t\t\t\t\t\t\t\t\t\t\t\tsite + '&count=' + config.results + \n\t\t\t\t\t\t\t\t\t\t\t\t'&appid=' + config.appID;\n\t\t\t\t\tvar s = document.createElement('script');\n\t\t\t\t\ts.setAttribute('src',APIurl);\n\t\t\t\t\ts.setAttribute('type','text/javascript');\n\t\t\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t\t\t\treturn false;\n\t\t\t\t}\n\t\t\t};\n\t\t}\n\t}\n};\n\nNow it is time to call the BOSS API by assembling a correct REST URL, create a script node and apply it to the head of the document. We return false to ensure the form does not get submitted as we want to stay on the page.\n\nNotice that we are using SITESEARCH.found as the callback method, which means that we need to define this one to deal with the data returned by the API.\n\nfunction found(o){\n\tvar list = document.createElement('ul');\n\tvar results = o.ysearchresponse.resultset_web;\n\tif(results){\n\t\tvar item,link,description;\n\nWe create a new list and then get the resultset_web array from the data returned from the API. If there aren\u2019t any results returned, this array will not exist which is why we need to check for it. Once we done that we can define three variables to repeatedly store the item title we want to display, the link to point to and the description of the link.\n\nfor(var i=0,j=results.length;i<j;i++){\n\titem = document.createElement('li');\n\tlink = document.createElement('a');\n\tlink.setAttribute('href',results[i].clickurl);\n\tlink.innerHTML = results[i].title;\n\titem.appendChild(link);\n\tdescription = document.createElement('p');\n\tdescription.innerHTML = results[i]['abstract'];\n\titem.appendChild(description);\n\tlist.appendChild(item);\n}\n\nWe then loop over the results array and assemble a list of results with the titles in links and paragraphs with the abstract of the site. Notice the bracket notation for abstract as abstract is a reserved word in JavaScript2 :).\n\n} else {\n\t\tlist = document.createElement('p');\n\t\tlist.appendChild(document.createTextNode(config.noresults));\n\t}\n\tform.replaceChild(list,out);\n\tout = list;\n};      \n\nIf there aren\u2019t any results, we define a paragraph with the no results message as list. In any case we replace the old out (the loading message) with the list and re-define out as the list.  \n\nreturn{\n\t\tconfig:config,\n\t\tinit:init,\n\t\tfound:found\n\t};\n}();\n\nAll that is left to do is return the properties and methods we want to make public. In this case found needs to be public as it is accessed by the API return. We return init to make it accessible and config to allow implementers to override any of the properties.\n\nUsing the script\n\n\nIn order to use this script, all you need to do is to add it after the form in the document, override the API key with your own and call init():\n\n<form id=\"customsearch\" action=\"http://search.yahoo.com/search\">\n\t<div>\n\t\t<label for=\"p\">Search this site:</label>\n\t\t<input type=\"text\" name=\"p\" id=\"term\">\n\t\t<input type=\"hidden\" name=\"vs\" id=\"site\" value=\"bbc.co.uk\">\n\t\t<input type=\"submit\" value=\"go\">\n\t</div>\n</form>\n<script type=\"text/javascript\" src=\"boss-site-search.js\"></script>\n<script type=\"text/javascript\">\n\tSITESEARCH.config.appID = 'copy-the-id-you-know-to-get-where';\n\tSITESEARCH.init();\n</script>\n\nWhere to go from here\n\nThis is just a very simple example of what you can do with BOSS. You can define languages and regions, retrieve and display images and news and mix the results with other data sources before displaying them. One very cool feature is that by adding a view=keyterms parameter to the URL you can get the keywords of each of the results to drill deeper into the search. An example for this written in PHP is available on the YDN blog. For JavaScript solutions there is a handy wrapper called yboss available to help you go nuts.", "year": "2008", "author": "Christian Heilmann", "author_slug": "chrisheilmann", "published": "2008-12-04T00:00:00+00:00", "url": "https://24ways.org/2008/sitewide-search-on-a-shoestring/", "topic": "code"}
{"rowid": 139, "title": "Flickr Photos On Demand with getFlickr", "contents": "In case you don\u2019t know it yet, Flickr is great. It is a lot of fun to upload, tag and caption photos and it is really handy to get a vast network of contacts through it. \n\nUsing Flickr photos outside of it is a bit of a problem though. There is a Flickr API, and you can get almost every page as an RSS feed, but in general it is a bit tricky to use Flickr photos inside your blog posts or web sites. You might not want to get into the whole API game or use a server side proxy script as you cannot retrieve RSS with Ajax because of the cross-domain security settings.\n\nHowever, Flickr also provides an undocumented JSON  output, that can be used to hack your own solutions in JavaScript without having to use a server side script.\n\n\n\tIf you enter the URL http://flickr.com/photos/tags/panda you get to the flickr page with photos tagged \u201cpanda\u201d.\n\tIf you enter the URL http://api.flickr.com/services/feeds/photos_public.gne?tags=panda&format=rss_200 you get the same page as an RSS feed.\n\tIf you enter the URL http://api.flickr.com/services/feeds/photos_public.gne?tags=panda&format=json you get a JavaScript function called jsonFlickrFeed with a parameter that contains the same data in JSON format\n\n\nYou can use this to easily hack together your own output by just providing a function with the same name. I wanted to make it easier for you, which is why I created the helper getFlickr for you to download and use.\n\ngetFlickr for Non-Scripters\n\nSimply include the javascript file getflickr.js and the style getflickr.css in the head of your document:\n\n<script type=\"text/javascript\" src=\"getflickr.js\"></script>\n<link rel=\"stylesheet\" href=\"getflickr.css\" type=\"text/css\">\n\nOnce this is done you can add links to Flickr pages anywhere in your document, and when you give them the CSS class getflickrphotos they get turned into gallery links. When a visitor clicks these links they turn into loading messages and show a \u201cpopup\u201d gallery with the connected photos once they were loaded. As the JSON returned is very small it won\u2019t take long. You can close the gallery, or click any of the thumbnails to view a photo. Clicking the photo makes it disappear and go back to the thumbnails.\n\nCheck out the example page and click the different gallery links to see the results.\n\nNotice that getFlickr works with Unobtrusive JavaScript as when scripting is disabled the links still get to the photos on Flickr.\n\ngetFlickr for JavaScript Hackers\n\nIf you want to use getFlickr with your own JavaScripts you can use its main method leech():\n\ngetFlickr.leech(sTag, sCallback);\n\n \n\tsTag\n\tthe tag you are looking for\n\tsCallback\n\tan optional function to call when the data was retrieved.\n \n\nAfter you called the leech() method you have two strings to use:\n\n \n\tgetFlickr.html[sTag]\n\tcontains an HTML list (without the outer UL element) of all the images linked to the correct pages at flickr. The images are the medium size, you can easily change that by replacing _m.jpg with _s.jpg for thumbnails.\n\tgetFlickr.tags[sTag]\n\tcontains a string of all the other tags flickr users added with the tag you searched for(space separated)\n \n\nYou can call getFlickr.leech() several times when the page has loaded to cache several result feeds before the page gets loaded. This\u2019ll make the photos quicker for the end user to show up. If you want to offer a form for people to search for flickr photos and display them immediately you can use the following HTML:\n\n<form onsubmit=\"getFlickr.leech(document.getElementById('tag').value, 'populate');return false\">\n  <label for=\"tag\">Enter Tag</label>\n  <input type=\"text\" id=\"tag\" name=\"tag\" />\n  <input type=\"submit\" value=\"energize\" />\n  <h3>Tags:</h3><div id=\"tags\"></div>\n  <h3>Photos:</h3><ul id=\"photos\"></ul>\n</form>\n\nAll the JavaScript you\u2019ll need (for a basic display) is this:\n\nfunction populate(){\n  var tag = document.getElementById('tag').value;\n  document.getElementById('photos').innerHTML = getFlickr.html[tag].replace(/_m\\.jpg/g,'_s.jpg');\n  document.getElementById('tags').innerHTML = getFlickr.tags[tag];\n  return false;\n}\n\nEasy as pie, enjoy!\n\nCheck out the example page and try the form to see the results.", "year": "2006", "author": "Christian Heilmann", "author_slug": "chrisheilmann", "published": "2006-12-03T00:00:00+00:00", "url": "https://24ways.org/2006/flickr-photos-on-demand/", "topic": "code"}
{"rowid": 161, "title": "Keeping JavaScript Dependencies At Bay", "contents": "As we are writing more and more complex JavaScript applications we run into issues that have hitherto (god I love that word) not been an issue. The first decision we have to make is what to do when planning our app: one big massive JS file or a lot of smaller, specialised files separated by task. \n\nPersonally, I tend to favour the latter, mainly because it allows you to work on components in parallel with other developers without lots of clashes in your version control. It also means that your application will be more lightweight as you only include components on demand.\n\nStarting with a global object\n\nThis is why it is a good plan to start your app with one single object that also becomes the namespace for the whole application, say for example myAwesomeApp:\n\nvar myAwesomeApp = {};\n\nYou can nest any necessary components into this one and also make sure that you check for dependencies like DOM support right up front.\n\nAdding the components\n\nThe other thing to add to this main object is a components object, which defines all the components that are there and their file names.\n\nvar myAwesomeApp = {\n\tcomponents :{\n\t\tformcheck:{\n\t\t\turl:'formcheck.js',\n\t\t\tloaded:false\n\t\t},\n\t\tdynamicnav:{\n\t\t\turl:'dynamicnav.js',\n\t\t\tloaded:false\n\t\t},\n\t\tgallery:{\n\t\t\turl:'gallery.js',\n\t\t\tloaded:false\n\t\t},\n\t\tlightbox:{\n\t\t\turl:'lightbox.js',\n\t\t\tloaded:false\n\t\t}\n\t}\n};\n\nTechnically you can also omit the loaded properties, but it is cleaner this way. The next thing to add is an addComponent function that can load your components on demand by adding new SCRIPT elements to the head of the documents when they are needed.\n\nvar myAwesomeApp = {\n\tcomponents :{\n\t\tformcheck:{\n\t\t\turl:'formcheck.js',\n\t\t\tloaded:false\n\t\t},\n\t\tdynamicnav:{\n\t\t\turl:'dynamicnav.js',\n\t\t\tloaded:false\n\t\t},\n\t\tgallery:{\n\t\t\turl:'gallery.js',\n\t\t\tloaded:false\n\t\t},\n\t\tlightbox:{\n\t\t\turl:'lightbox.js',\n\t\t\tloaded:false\n\t\t}\n\t},\n\taddComponent:function(component){\n\t\tvar c = this.components[component];\n\t\tif(c && c.loaded === false){\n\t\t\tvar s = document.createElement('script');\n\t\t\ts.setAttribute('type', 'text/javascript');\n\t\t\ts.setAttribute('src',c.url);\n\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t}\n\t}\n};\n\nThis allows you to add new components on the fly when they are not defined:\n\nif(!myAwesomeApp.components.gallery.loaded){\n\tmyAwesomeApp.addComponent('gallery');\n};\n\nVerifying that components have been loaded\n\nHowever, this is not safe as the file might not be available. To make the dynamic adding of components safer each of the components should have a callback at the end of them that notifies the main object that they indeed have been loaded:\n\nvar myAwesomeApp = {\n\tcomponents :{\n\t\tformcheck:{\n\t\t\turl:'formcheck.js',\n\t\t\tloaded:false\n\t\t},\n\t\tdynamicnav:{\n\t\t\turl:'dynamicnav.js',\n\t\t\tloaded:false\n\t\t},\n\t\tgallery:{\n\t\t\turl:'gallery.js',\n\t\t\tloaded:false\n\t\t},\n\t\tlightbox:{\n\t\t\turl:'lightbox.js',\n\t\t\tloaded:false\n\t\t}\n\t},\n\taddComponent:function(component){\n\t\tvar c = this.components[component];\n\t\tif(c && c.loaded === false){\n\t\t\tvar s = document.createElement('script');\n\t\t\ts.setAttribute('type', 'text/javascript');\n\t\t\ts.setAttribute('src',c.url);\n\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t}\n\t},\n\tcomponentAvailable:function(component){\n\t\tthis.components[component].loaded = true;\n\t}\n}\n\nFor example the gallery.js file should call this notification as a last line:\n\nmyAwesomeApp.gallery = function(){\n\t// [... other code ...]\n}();\nmyAwesomeApp.componentAvailable('gallery');\n\nTelling the implementers when components are available\n\nThe last thing to add (actually as a courtesy measure for debugging and implementers) is to offer a listener function that gets notified when the component has been loaded:\n\nvar myAwesomeApp = {\n\tcomponents :{\n\t\tformcheck:{\n\t\t\turl:'formcheck.js',\n\t\t\tloaded:false\n\t\t},\n\t\tdynamicnav:{\n\t\t\turl:'dynamicnav.js',\n\t\t\tloaded:false\n\t\t},\n\t\tgallery:{\n\t\t\turl:'gallery.js',\n\t\t\tloaded:false\n\t\t},\n\t\tlightbox:{\n\t\t\turl:'lightbox.js',\n\t\t\tloaded:false\n\t\t}\n\t},\n\taddComponent:function(component){\n\t\tvar c = this.components[component];\n\t\tif(c && c.loaded === false){\n\t\t\tvar s = document.createElement('script');\n\t\t\ts.setAttribute('type', 'text/javascript');\n\t\t\ts.setAttribute('src',c.url);\n\t\t\tdocument.getElementsByTagName('head')[0].appendChild(s);\n\t\t}\n\t},\n\tcomponentAvailable:function(component){\n\t\tthis.components[component].loaded = true;\n\t\tif(this.listener){\n\t\t\tthis.listener(component);\n\t\t};\n\t}\n};\n\nThis allows you to write a main listener function that acts when certain components have been loaded, for example:\n\nmyAwesomeApp.listener = function(component){\n\tif(component === 'gallery'){\n\t   showGallery();\n\t}\n};\n\nExtending with other components\n\nAs the main object is public, other developers can extend the components object with own components and use the listener function to load dependent components. Say you have a bespoke component with data and labels in extra files:\n\nmyAwesomeApp.listener = function(component){\n\tif(component === 'bespokecomponent'){\n\t\tmyAwesomeApp.addComponent('bespokelabels');\n\t};\n\tif(component === 'bespokelabels'){\n\t\tmyAwesomeApp.addComponent('bespokedata');\n\t};\n\tif(component === 'bespokedata'){\n\t\tmyAwesomeApp,bespokecomponent.init();\n\t};\n};\nmyAwesomeApp.components.bespokecomponent = {\n\turl:'bespoke.js',\n\tloaded:false\n};\nmyAwesomeApp.components.bespokelabels = {\n\turl:'bespokelabels.js',\n\tloaded:false\n};\nmyAwesomeApp.components.bespokedata = {\n\turl:'bespokedata.js',\n\tloaded:false\n};\nmyAwesomeApp.addComponent('bespokecomponent');\n\nFollowing this practice you can write pretty complex apps and still have full control over what is available when. You can also extend this to allow for CSS files to be added on demand.\n\nInfluences\n\nIf you like this idea and wondered if someone already uses it, take a look at the Yahoo! User Interface library, and especially at the YAHOO_config option of the global YAHOO.js object.", "year": "2007", "author": "Christian Heilmann", "author_slug": "chrisheilmann", "published": "2007-12-18T00:00:00+00:00", "url": "https://24ways.org/2007/keeping-javascript-dependencies-at-bay/", "topic": "code"}
{"rowid": 186, "title": "The Web Is Your CMS", "contents": "It is amazing what you can do these days with the services offered on the web. Flickr stores terabytes of photos for us and converts them automatically to all kind of sizes, finds people in them and even allows us to edit them online. YouTube does almost the same complete job with videos, LinkedIn allows us to maintain our CV, Delicious our bookmarks and so on.\n\nWe don\u2019t have to do these tasks ourselves any more, as all of these systems also come with ways to use the data in the form of Application Programming Interfaces, or APIs for short. APIs give us raw data when we send requests telling the system what we want to get back.\n\nThe problem is that every API has a different idea of what is a simple way of accessing this data and in which format to give it back.\n\nMaking it easier to access APIs\n\nWhat we need is a way to abstract the pains of different data formats and authentication formats away from the developer \u2014 and this is the purpose of the Yahoo Query Language, or YQL for short. \n\nLibraries like jQuery and YUI make it easy and reliable to use JavaScript in browsers (yes, even IE6) and YQL allows us to access web services and even the data embedded in web documents in a simple fashion \u2013 SQL style.\n\nSelect * from the web and filter it the way I want\n\nYQL is a web service that takes a few inputs itself:\n\n\n\tA query that tells it what to get, update or access\n\tAn output format \u2013 XML, JSON, JSON-P or JSON-P-X\n\tA callback function (if you defined JSON-P or JSON-P-X)\n\n\nYou can try it out yourself \u2013 check out this link to get back Flickr photos for the search term \u2018santa\u2019*%20from%20flickr.photos.search%20where%20text%3D%22santa%22&format=xml in XML format. The YQL query for this is \n\nselect * from flickr.photos.search where text=\"santa\"\n\nThe easiest way to take your first steps with YQL is to look at the console. There you get sample queries, access to all the data sources available to you and you can easily put together complex queries. In this article, however, let\u2019s use PHP to put together a web page that pulls in Flickr photos, blog posts, Videos from YouTube and latest bookmarks from Delicious.\n\nCheck out the demo and get the source code on GitHub.\n\n<?php\n  /* YouTube RSS */\n  $query = 'select description from rss(5) where url=\"http://gdata.youtube.com/feeds/base/users/chrisheilmann/uploads?alt=rss&v=2&orderby=published&client=ytapi-youtube-profile\";';\n  /* Flickr search by user id */\n  $query .= 'select farm,id,owner,secret,server,title from flickr.photos.search where user_id=\"11414938@N00\";';\n  /* Delicious RSS */\n  $query .= 'select title,link from rss where url=\"http://feeds.delicious.com/v2/rss/codepo8?count=10\";';\n  /* Blog RSS */\n  $query .= 'select title,link from rss where url=\"http://feeds.feedburner.com/wait-till-i/gwZf\"';\n  /* The YQL web service root with JSON as the output */\n  $root = 'http://query.yahooapis.com/v1/public/yql?format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys';\n  /* Assemble the query */\n  $query = \"select * from query.multi where queries='\".$query.\"'\";\n  $url = $root . '&q=' . urlencode($query);\n  /* Do the curl call (access the data just like a browser would) */\n  $ch = curl_init(); \n  curl_setopt($ch, CURLOPT_URL, $url); \n  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); \n  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);\n  curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);\n  $output = curl_exec($ch); \n  curl_close($ch);\n  $data = json_decode($output);\n  $results = $data->query->results->results;\n  /* YouTube output */\n  $youtube = '<ul id=\"youtube\">';\n  foreach($results[0]->item as $r){\n\t$cleanHTML = undoYouTubeMarkupCrimes($r->description);\n\t$youtube .= '<li>'.$cleanHTML.'</li>';\n  }\n  $youtube .= '</ul>';\n  /* Flickr output */\n  $flickr = '<ul id=\"flickr\">';\n  foreach($results[1]->photo as $r){\n\t$flickr .= '<li>'.\n\t\t\t   '<a href=\"http://www.flickr.com/photos/codepo8/'.$r->id.'/\">'.\n\t\t\t   '<img src=\"http://farm' .$r->farm . '.static.flickr.com/'.\n\t\t\t   $r->server . '/' . $r->id . '_' . $r->secret . \n\t\t\t   '_s.jpg\" alt=\"'.$r->title.'\"></a></li>';\n  }\n  $flickr .= '</ul>';\n  /* Delicious output */\n  $delicious = '<ul id=\"delicious\">';\n  foreach($results[2]->item as $r){\n\t$delicious .= '<li><a href=\"'.$r->link.'\">'.$r->title.'</a></li>';\n  }\n  $delicious .= '</ul>';\n  /* Blog output */\n  $blog = '<ul id=\"blog\">';\n  foreach($results[3]->item as $r){\n\t$blog .= '<li><a href=\"'.$r->link.'\">'.$r->title.'</a></li>';\n  }\n  $blog .= '</ul>';\n  function undoYouTubeMarkupCrimes($str){\n\t$cleaner = preg_replace('/555px/','100%',$str);\n\t$cleaner = preg_replace('/width=\"[^\"]+\"/','',$cleaner);\n\t$cleaner = preg_replace('/<tbody>/','<colgroup><col width=\"20%\"><col width=\"50%\"><col width=\"30%\"></colgroup><tbody>',$cleaner);\n\treturn $cleaner;\n  }\n?>\n\nWhat we are doing here is create a few different YQL statements and queue them together with the query.multi table. Each of these can be run inside YQL itself. Check out the YouTube, Flickr, Delicious and Blog example in the console if you don\u2019t believe me. The benefit of using this table is that we don\u2019t make individual requests for each query but we get all the data in one single request \u2013 which means a much better performing solution as the YQL server farm is faster on the web than our servers.\n\nWe point the query to the YQL web service end point and get the resulting data using cURL. All that we need to do then is to convert the returned data to HTML lists that can be printed out inside an HTML template.\n\nMixing, matching and using HTML as a data source\n\nThis was a simple example of what YQL can do for you. Where it gets really powerful however is by mixing and matching different APIs. YQL is also a good tool to get information from HTML documents. By using the html table you can load the content of an HTML document (which gets fixed automatically by HTMLTidy) and use XPATH to filter down results to what you need. Take the following example which takes headlines from the news.bbc.co.uk homepage and runs the results through Yahoo\u2019s Term Extractor API to give you a list of currently hot topics.\n\nselect * from search.termextract where context in (\n  select content from html where url=\"http://news.bbc.co.uk\" and xpath=\"//table[@width=800]//a\"\n)\n\nTry it out in the console or see the results here. In English, this means:\n\n\n\tGo to http://news.bbc.co.uk and get me the HTML\n\tRun it through HTML Tidy to clean it up.\n\tGet me only the links inside the table with an attribute of width and the value 800\n\tGet only the content of the link and for each of the links\n\t\n\t\tTake the content and send it as context to the Yahoo Term Extractor API\n\t\n\t\n\nIf we choose JSON-P as the output format we can use the outcome directly in JavaScript (see this demo or see its source):\n\n<ul id=\"hottopics\"></ul>\n<script type=\"text/javascript\">\nfunction hottopics(o){\n  var res = o.query.results.Result,\n\t  all = res.length,\n\t  topics = {},\n\t  out = [],\n\t  html = '',\n\t  i=0;\n  /* create hash from topics to prevent repetition */\t \n  for(i=0;i<all;i++){\n\ttopics[res[i]] = res[i];\n  };\n  for(i in topics){\n\tout.push(i);\n  };\n  html = '<li>' + out.join('</li><li>') + '</li>';\n  document.getElementById('hottopics').innerHTML = html;\n};\n</script>\n<script type=\"text/javascript\" src=\"http://query.yahooapis.com/v1/public/yql?q=select%20content%20from%20search.termextract%20where %20context%20in%20(select%20content%20from%20html%20where%20url%3D%22http%3A%2F%2Fnews.bbc.co.uk%22%20and%20xpath%3D%22%2F%2Ftable%5B%40width%3D800%5D%2F%2Fa%22)&format=json&callback=hottopics\"></script>\n\nUsing JSON, we can also use PHP which means the demo works for everybody \u2013 not only those with JavaScript enabled (see this demo or see its source):\n\n<ul id=\"hottopics\"><li>\n<?php\n$url = 'http://query.yahooapis.com/v1/public/yql?q=select%20content'.\n\t   '%20from%20search.termextract%20where%20context%20in'.\n\t   '%20(select%20content%20from%20html%20where%20url%3D%22'.\n\t   'http%3A%2F%2Fnews.bbc.co.uk%22%20and%20xpath%3D%22%2F%2F'.\n\t   'table%5B%40width%3D800%5D%2F%2Fa%22)&format=json';\n$ch = curl_init(); \ncurl_setopt($ch, CURLOPT_URL, $url); \ncurl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); \ncurl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);\ncurl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);\n$output = curl_exec($ch); \ncurl_close($ch);\n$data = json_decode($output);\n$topics = array_unique($data->query->results->Result);\necho join('</li><li>',$topics);\n?>\n</li></ul>\n\nSummary\n\nThis article could only scratch the surface of YQL. You have not only read access to the web but you can also write to web services. For example you can update Twitter, post to your WordPress blog or shorten a URL with bit.ly. Using Open Tables you can add any web service to the YQL interface and you can even run server-side JavaScript which is for example useful to return Flickr photos as HTML or get the HTML content from a document that needs POST data.\n\nThe web of data is already here, and using YQL you don\u2019t have to be a web services expert to use it and be part of it.", "year": "2009", "author": "Christian Heilmann", "author_slug": "chrisheilmann", "published": "2009-12-17T00:00:00+00:00", "url": "https://24ways.org/2009/the-web-is-your-cms/", "topic": "code"}
{"rowid": 233, "title": "Wrapping Things Nicely with HTML5 Local Storage", "contents": "HTML5 is here to turn the web from a web of hacks into a web of applications \u2013 and we are well on the way to this goal. The coming year will be totally and utterly awesome if you are excited about web technologies.\n\nThis year the HTML5 revolution started and there is no stopping it. For the first time all the browser vendors are rallying together to make a technology work. The new browser war is fought over implementation of the HTML5 standard and not over random additions. We live in exciting times.\n\nStarting with a bang\n\nAs with every revolution there is a lot of noise with bangs and explosions, and that\u2019s the stage we\u2019re at right now. HTML5 showcases are often CSS3 showcases, web font playgrounds, or video and canvas examples.\n\nThis is great, as it gets people excited and it gives the media something to show. There is much more to HTML5, though. Let\u2019s take a look at one of the less sexy, but amazingly useful features of HTML5 (it was in the HTML5 specs, but grew at such an alarming rate that it warranted its own spec): storing information on the client-side.\n\nWhy store data on the client-side?\n\nStoring information in people\u2019s browsers affords us a few options that every application should have:\n\n\n\tYou can retain the state of an application \u2013 when the user comes back after closing the browser, everything will be as she left it. That\u2019s how \u2018real\u2019 applications work and this is how the web ones should, too.\n\tYou can cache data \u2013 if something doesn\u2019t change then there is no point in loading it over the Internet if local access is so much faster\n\tYou can store user preferences \u2013 without needing to keep that data on your server at all.\n\n\nIn the past, storing local data wasn\u2019t much fun.\n\nThe pain of hacky browser solutions\n\nIn the past, all we had were cookies. I don\u2019t mean the yummy things you get with your coffee, endorsed by the blue, furry junkie in Sesame Street, but the other, digital ones. Cookies suck \u2013 it isn\u2019t fun to have an unencrypted HTTP overhead on every server request for storing four kilobytes of data in a cryptic format. It was OK for 1994, but really neither an easy nor a beautiful solution for the task of storing data on the client.\n\nThen came a plethora of solutions by different vendors \u2013 from Microsoft\u2019s userdata to Flash\u2019s LSO, and from Silverlight isolated storage to Google\u2019s Gears. If you want to know just how many crazy and convoluted ways there are to store a bit of information, check out Samy\u2019s evercookie.\n\nClearly, we needed an easier and standardised way of storing local data.\n\nKeeping it simple \u2013 local storage\n\nAnd, lo and behold, we have one. The local storage API (or session storage, with the only difference being that session data is lost when the window is closed) is ridiculously easy to use. All you do is call a few methods on the window.localStorage object \u2013 or even just set the properties directly using the square bracket notation:\n\nif('localStorage' in window && window['localStorage'] !== null){\n\tvar store = window.localStorage;\n\n\t// valid, API way\n\t\tstore.setItem(\u2018cow\u2019,\u2018moo\u2019);\n\t\tconsole.log(\n\t\tstore.getItem(\u2018cow\u2019)\n\t); // => \u2018moo\u2019\n\n\t// shorthand, breaks at keys with spaces\n\t\tstore.sheep = \u2018baa\u2019 \n\t\tconsole.log(\n\t\tstore.sheep \n\t); // \u2018baa\u2019\n\n\t// shorthand for all\n\t\tstore[\u2018dog\u2019] = \u2018bark\u2019\n\t\tconsole.log(\n\t\tstore[\u2018dog\u2019]\n\t); // => \u2018bark\u2019\n\n}\n\nBrowser support is actually pretty good: Chrome 4+; Firefox 3.5+; IE8+; Opera 10.5+; Safari 4+; plus iPhone 2.0+; and Android 2.0+. That should cover most of your needs. Of course, you should check for support first (or use a wrapper library like YUI Storage Utility or YUI Storage Lite).\n\nThe data is stored on a per domain basis and you can store up to five megabytes of data in localStorage for each domain.\n\nStrings attached\n\nBy default, localStorage only supports strings as storage formats. You can\u2019t store results of JavaScript computations that are arrays or objects, and every number is stored as a string. This means that long, floating point numbers eat into the available memory much more quickly than if they were stored as numbers.\n\nvar cowdesc = \"the cow is of the bovine ilk, \"+\n\t\t\"one end is for the moo, the \"+\n\t\t\"other for the milk\";\n\nvar cowdef = {\n\tilk\u201cbovine\u201d,\n\tlegs,\n\tudders,\n\tpurposes\n\t\tfront\u201cmoo\u201d,\n\t\tend\u201cmilk\u201d\n\t}\n};\n\nwindow.localStorage.setItem(\u2018describecow\u2019,cowdesc);\nconsole.log(\n\twindow.localStorage.getItem(\u2018describecow\u2019)\n); // => the cow is of the bovine\u2026 \n\nwindow.localStorage.setItem(\u2018definecow\u2019,cowdef);\nconsole.log(\n\twindow.localStorage.getItem(\u2018definecow\u2019)\n); // => [object Object] = bad!\n\nThis limits what you can store quite heavily, which is why it makes sense to use JSON to encode and decode the data you store:\n\nvar cowdef = {\n\t\"ilk\":\"bovine\",\n\t\"legs\":4,\n\t\"udders\":4,\n\t\"purposes\":{\n\t\"front\":\"moo\",\n\t\"end\":\"milk\"\n\t}\n};\n\nwindow.localStorage.setItem(\u2018describecow\u2019,JSON.stringify(cowdef));\nconsole.log(\n\tJSON.parse(\n\t\twindow.localStorage.getItem(\u2018describecow\u2019)\n\t)\n); // =>  Object { ilk=\u201cbovine\u201d,  more\u2026}\n\nYou can also come up with your own formatting solutions like CSV, or pipe | or tilde ~ separated formats, but JSON is very terse and has native browser support.\n\nSome use case examples\n\nThe simplest use of localStorage is, of course, storing some data: the current state of a game; how far through a multi-form sign-up process a user is; and other things we traditionally stored in cookies. Using JSON, though, we can do cooler things.\n\nSpeeding up web service use and avoiding exceeding the quota\n\nA lot of web services only allow you a certain amount of hits per hour or day, and can be very slow. By using localStorage with a time stamp, you can cache results of web services locally and only access them after a certain time to refresh the data.\n\nI used this technique in my An Event Apart 10K entry, World Info, to only load the massive dataset of all the world information once, and allow for much faster subsequent visits to the site. The following screencast shows the difference:\n\n\n\nFor use with YQL (remember last year\u2019s 24 ways entry?), I\u2019ve built a small script called YQL localcache that wraps localStorage around the YQL data call. An example would be the following:\n\nyqlcache.get({\n\tyql: 'select * from flickr.photos.search where text=\"santa\"',\n\tid: 'myphotos',\n\tcacheage: ( 60*60*1000 ),\n\tcallback: function(data) {\n\t\tconsole.log(data);\n\t}\n});\n\nThis loads photos of Santa from Flickr and stores them for an hour in the key myphotos of localStorage. If you call the function at various times, you receive an object back with the YQL results in a data property and a type property which defines where the data came from \u2013 live is live data, cached means it comes from cache, and freshcache indicates that it was called for the first time and a new cache was primed. The cache will work for an hour (60\u00d760\u00d71,000 milliseconds) and then be refreshed. So, instead of hitting the YQL endpoint over and over again, you hit it once per hour.\n\nCaching a full interface\n\nAnother use case I found was to retain the state of a whole interface of an application by caching the innerHTML once it has been rendered. I use this in the Yahoo Firehose search interface, and you can get the full story about local storage and how it is used in this screencast:\n\n\n\nThe stripped down code is incredibly simple (JavaScript with PHP embed):\n\n// test for localStorage support\nif(('localStorage' in window) && window['localStorage'] !== null){\n\nvar f = document.getElementById(\u2018mainform\u2019);\n// test with PHP if the form was sent (the submit button has the name \u201csent\u201d)\n\n\n\t// get the HTML of the form and cache it in the property \u201cstate\u201d\n\tlocalStorage.setItem(\u2018state\u2019,f.innerHTML);\n\n// if the form hasn\u2019t been sent\u2026\n\n\n\t// check if a state property exists and write back the HTML cache\n\tif(\u2018state\u2019 in localStorage){\n\t\tf.innerHTML = localStorage.getItem(\u2018state\u2019);  \n\t}\n\n\t\n\n}\n\nOther ideas\n\nIn essence, you can use local storage every time you need to speed up access. For example, you could store image sprites in base-64 encoded datasets instead of loading them from a server. Or you could store CSS and JavaScript libraries on the client. Anything goes \u2013 have a play.\n\nIssues with local and session storage\n\nOf course, not all is rainbows and unicorns with the localStorage API. There are a few niggles that need ironing out. As with anything, this needs people to use the technology and raise issues. Here are some of the problems:\n\n\n\tInadequate information about storage quota \u2013 if you try to add more content to an already full store, you get a QUOTA_EXCEEDED_ERR and that\u2019s it. There\u2019s a great explanation and test suite for localStorage quota available.\n\tLack of automatically expiring storage \u2013 a feature that cookies came with. Pamela Fox has a solution (also available as a demo and source code)\n\tLack of encrypted storage \u2013 right now, everything is stored in readable strings in the browser.\n\n\nBigger, better, faster, more!\n\nAs cool as the local and session storage APIs are, they are not quite ready for extensive adoption \u2013 the storage limits might get in your way, and if you really want to go to town with accessing, filtering and sorting data, real databases are what you\u2019ll need. And, as we live in a world of client-side development, people are moving from heavy server-side databases like MySQL to NoSQL environments.\n\nOn the web, there is also a lot of work going on, with Ian Hickson of Google proposing the Web SQL database, and Nikunj Mehta, Jonas Sicking (Mozilla), Eliot Graff (Microsoft) and Andrei Popescu (Google) taking the idea beyond simply replicating MySQL and instead offering Indexed DB as an even faster alternative.\n\nOn the mobile front, a really important feature is to be able to store data to use when you are offline (mobile coverage and roaming data plans anybody?) and you can use the Offline Webapps API for that.\n\nAs I mentioned at the beginning, we have a very exciting time ahead \u2013 let\u2019s make this web work faster and more reliably by using what browsers offer us. For more on local storage, check out the chapter on Dive into HTML5.", "year": "2010", "author": "Christian Heilmann", "author_slug": "chrisheilmann", "published": "2010-12-06T00:00:00+00:00", "url": "https://24ways.org/2010/html5-local-storage/", "topic": "code"}