In this tutorial we are going to look at building a simple Django application that integrates with theYahoo BOSS search framework. More specifically we're going to be using theBOSS Mashup Framework.

First, lets address the most pressing question:What the hell is Yahoo BOSS?BOSS isBuild Your Own Search Serviceand presents us with a fairly low level interface with Yahoo's search engine, not just to search our own site, but to search pretty much anything. The BOSS Mashup Framework, which is what we are going to be using, is open for any developers and has very few restrictions.

Fussy Details

First lets get all the little configuration stuff out of the way. There is a fair bit, but none of it is very difficult. As a warning, I'll point out that theBOSS Mashup Framework requires Python 2.5, and won't work with previous versions without some changes¹.

1. Sign up for a BOSS App ID.

2. Create a new Django project, lets call itmy_search.

   django-admin.py startproject my_search

3. Create a Django app insidemy_search, lets name ityahoo_search.

   python2.5 manage.py startapp yahoo_search

4. Download the Python library for controlling BOSS.

5. Unzip it into themy_search/yahoo_searchfolder, and rename it toboss.

   unzip boss_mashup_framework_0.1.zip rm boss_mashup_framework_0.1.zip mv boss_mashup_framework_0.1 boss

6. Yahoo didn't do a great job of packaging something that just works, so we have to go through a few steps to build the framework. (Although, these sub-instructions here are lifted almost directly from the includedREADMEfile, so its not that they didn't document it, just that its a bit of a pain to get working.) In Yahoo's defense, I think the reason they did a 'bad' job of packaging is that they probably ran into some incompatable licenses.

   1. Install Simple JSON if you don't have it installed. You can check if you have it installed by entering a Python2.5 prompt and typing

      importsimplejson

      If that didn't work,download Simple JSON. And then install it.

      python2.5 setup.py build python2.5 setup.py install

   2. Create the foldermy_search/yahoo_search/boss/deps/.

   3. Downloaddict2xmlandxml2dict, and extract them into the deps folder, remove the.tgzfiles, and return to theboss目录中。

      tar -xzvf dict2xml.tgz tar -xzvf xml2dict.tgz rm *.tgzcd..

   4. Now we can finally build the framework.

      python2.5 setup.py build python2.5 setup.py install

   5. Next, we have to update the settings inboss/config.json. I only changed the first three settings:appid,email, andorg. Theappidis the one you were given uponsigning up for BOSS.

   6. Check that it all worked by running (from within thebossdirectory):

      python2.5 examples/ex3.py

   7. From here on things are going to deviate from theREADME一点,我们要行动exampleandyosinto ouryahoo_searchdirectory, moveconfig.jsoninto ourmy_searchdirectory and get rid of everything else (well, you might want to keep theexamplesfolder for your own benefit).

      mv config.json ../../ mv yos ../ mv examples ../cd.. rm -r boss

Okay, now we're all done with the setup, and are ready to move on to putting together a simple Django application that uses the BOSS Mashup Framework.

Defining our App

Now that we have all the setup out of the way, we need to decide exactly what our app is going to do. To begin with (however, fear not, this is posed to turn into a multi-part series where we gradually put together a more interesting app) we're going to do something really simple:search Yahoo News based on the results of a posted form.

Yep. As simple as you can get. We'll make it more interesting afterwards, when we have something that works.

URLs

First lets edit our project'surls.pyto include urls from ouryahoo_search应用程序。my_search/urls.pyis should look like this:

fromdjango.conf.urls.defaultsimport*urlpatterns=patterns('',(r'^',include('my_search.yahoo_search.urls')),)

However, we haven't actually createdmy_search/yahoo_search/urls.pyyet, so lets do that real quick.

fromdjango.conf.urls.defaultsimport*urlpatterns=patterns('',(r'^$','my_search.yahoo_search.views.index'),)

As you can see by looking aturlpatternswe're only going to have one viewindex, and it is going to be handling everything for us.

Theindexview

Now we're going to write theindexview, which will be handling everything for us. Start out by openingmy_search/yahoo_search/views.py. Lets start out with all the imports we're going to need.

fromdjango.shortcutsimportrender_to_responsefromdjangoimportnewformsasformsfromyos.bossimportysearchfromyos.yqlimportdb

We're going to userender_to_responseto render templates,newformsto query our user for their search term,ysearchfor retrieving data from BOSS, anddbto format those retrieved results into something a bit more managable.

Writing thesearchfunction

Now lets write a simple search function we'll use for querying BOSS.

defsearch(str):data=ysearch.search(str,vertical="news",count=10)news=db.create(data=data)returnnews.rows

Brief Aside

If you wanted to search from Yahoo's web results instead of their news, you'd simply change the line

data=ysearch.search(str,vertical="news",count=10)

to

data=ysearch.search(str,count=10)

The data returned by thesearchfunction is a list of dictionaries that look like this:

{u'sourceurl':u'http://www.channelweb.com/',u'language':u'en english',u'title':u'Google Works With eBay And PayPal To Curtail Phishing',u'url':u'http://www.crn.com/security/208808698?cid=ChannelWebBreakingNews',u'abstract':u'Google Gmail requires eBay and PayPal to use DomainKeys to authenticate mail in an anti-phish effort',u'clickurl':u'http://www.crn.com/security/208808698?cid=ChannelWebBreakingNews',u'source':u'ChannelWeb',u'time':u'22:26:08',u'date':u'2008/07/11'}

Thesearchfunction is very basic, but will be enough for this initial version of the application. Lets move forward.

A simplenewform

Next we need to create a (very) simplenewformthat we will use for querying our users' for their search terms.

classSearchForm(forms.Form):search_terms=forms.CharField(max_length=200)

Thats all we'll need for now, carry on. (I said it was simple.)

Actually implementing theindexview

Okay, now lets stop for a moment and consider what theindexview needs to accomplish.

1. It needs to check if there are any incoming POST parameters.
2. If there are POST parameters, it needs to validate them usingSearchForm, and then usesearchto put together the results.
3. It needs to userender_to_responseto render a template contain aSearchForm, and any search results (if applicable).

Okay, translating that into Python we get ourindexfunction:

defindex(request):results=Noneifrequest.method=="POST":form=SearchForm(request.POST)ifform.is_valid():search_terms=form.cleaned_data[“search_term”]results=search(search_terms)else:form=SearchForm()returnrender_to_response(“yahoo_search/index.html',{'form':form,'results':results})

Admittedly we haven't written theindex.htmltemplate yet, that will be our next task. Beyond that, this is a pretty standard Django view.

Filling in theindex.htmltemplate

First, we need to create the template directory for ouryahoo_search应用程序。Frominsidethemy_search/yahoo_searchdirectory:

mkdir templates mkdir templates/yahoo_search

And then create the filetemplates/yahoo_search/index.hml, and open it up in your editor. This is going to be a simple template, containing only an input box for searching, and a listing of the results.

It'll look like this:

My Search
action="/"method="post"><表>{ {form}}type="submit"value="Search">
{%ifresults%}
{%forresultinresults%}
{%comment%}Notice we are using { { result.clickurl }} instead of{ { result.url }}. You might wonder why we are doingthat, and the answer is pretty simple: because thatswhat Yahoo is asking us to.http://developer.yahoo.com/search/boss/boss_guide/univer_api_query.html#url_vs_clickurl{%endcomment%}class="title">href="{ {result.clickurl}}">{ {result.title}}class="date">{ {result.date}}class="time">{ {result.time}}class="source">href="{ {result.sourceurl}}">{ {result.source}}class="abstract">{ {result.abstract}}
{%endfor%}
{%endif%}

Download Zip of Files

If you haven't been keeping up, or if your code is behaving strangely,you can grab a zip of all these files. Just unzip these somewhere, fill in the first three entries (your BOSSappid,email, andorg) inmy_search/config.json, and you'll be ready to take a look at the app in the next step.

Update 7/12/2008: Unfortunately, the way the BOSS library has been built it isn't enough to simply copy overyosfolder, and instead you will need to follow the installation steps for the BOSS Framework listed above (step #6). Specifically, you need to work through those steps and finish with:

python2.5 setup.py build python2.5 setup.py install

Its a bit of a pain, and I'll see if I can clean things up to make it simpler.

Seeing it work

Now we've finished building the app, lets fire it up.

python2.5 manage.py runserver

Navigate over tohttp://127.0.0.1:8000/, and you'll see a friendly search box waiting for you. Type in a search term, hit enter, and voila, you'll see a list of your results. I searched foriPhoneand got a page of results like this:

One gotcha I'll point out is that the helper library Yahoo has supplied relies onconfig.jsonbeing in the base directory where the Python is being run from. This will be true for your development setup, but won't necessarily be the case on your deployment server. I believe the best solution here would be to add the contents ofconfig.jsonto your project'ssettings.pyfile and tweak theyos/boss/ysearch.pyfile to load the settings usingdjango.conf.settingsinstead of from disk.

Let me know if you have any questions, and I'll try to answer them. Time permitting, I'll continue with another segment or two working on building a slightly more compelling search service than what we have created so far.

Update 7/12Thanks to Wayne's comments I was able to simplify thesearchfunction quite a bit. Specifically, he pointed out that I was using the library to prependynews$to all the dictionaries' keys, then getting upset it was there and removing it manually. Woops.