all posts

WhatIWantMost: Looking at third-party search

Published to Blog on 3 Jan 2007

Since we last talked... well, since we last talked about my progress on the project, I knocked out a few more simple tasks and then I started on a detour away from the important user stories. Hey, it's my project I can do whatever I want!

After doing the "Add an item to the wishlist" thing, it was simple enough to finish up "edit an item" and "delete an item". As simple as doing a couple of things like this:

WishlistItem wli = GetUpdateUIValues(WIService.GetByID(int.Parse(lblUpdateItemID.Text)));

and this:


To be honest, I did have to create some UI elements and pass values in from the form, but really - is that any work at all?


Then came the sidetrack. I started looking at web services from third parties like Live, Amazon, and Ebay because I eventually want to incorporate some of that stuff into the site. I had never really worked with any of them so I thought I would take a peek to see what I had in store for me in the future.

Because I want to pull back search results that are products, naturally the first place I looked was Froogle. But after doing a little research it seems that Google has not implemented an API for Froogle yet. I'm not looking to do a bunch of screen-scraping so I quickly moved on. My next stop was Live Search. Live has a new "beta" product search, but I found that after working with their API that while you can search for images, news, classifieds and a bunch of other stuff, their product search is not part of the API yet. Strike two.

But since I was already knee deep into looking at Live, I figured I would go ahead and implement it anyway. Live offers WSDL so you simply add a web reference to the URL and away you go. The Live Search API is fairly easy to work with - do a request, get a response and then for my prototyping purposes I chose to bind the results to a datagrid and prettied it up a tiny bit. The code looks something like this:

SearchRequest searchRequest = new SearchRequest();
searchRequest.AppID = "myappid";
searchRequest.Query = "( OR OR pampers";
SourceRequest source = new SourceRequest();
searchRequest.Requests = source;
MSNSearchService searchService = new MSNSearchService();
SearchResponse searchResponse = searchService.Search(searchRequest);
dg.DataSource = searchResponse.Responses[0].Results;

You can see the results - nothing fancy. For all my searches I chose to use "Pampers" as my search term so I would be comparing apples. One of the configuration options for Live Search, as well as others, is that you can limit results to certain domains. In the above case I chose Amazon, Ebay and NewEgg, but I could easily put together a larger list of sites I'd like to search.

The next search API that I investigated was Yahoo. Implementing that was fairly easy as well. I downloaded their C# API from their site (which even came with unit tests!), built it, added a reference to my project and then added code similar to the Live example:

YahooSearchService yahoo = new YahooSearchService();
ResultSet resultSet = yahoo.productSearch("myyahooid", "pampers");
List<OfferType> offers = new List<OfferType>();
List<CatalogType> catalogitems = new List<CatalogType>();
foreach (ResultType result in resultSet.Result){
       if (result.Item.ToString() == "Yahoo.API.ProductSearch.OfferType"){
          OfferType offer = (OfferType) result.Item;
          CatalogType catalogitem = (CatalogType) result.Item;
dgCatalog.DataSource = catalogitems;
dgOffers.DataSource = offers;


Looks a little prettier with Yahoo, right? They have images where Live does not. The only other major difference between Yahoo and Live is that Yahoo's returned items be one of two product types. Rather than combine the items into one list, for the time being I chose to display them in two different grids. The good thing about both the Live Search API and the Yahoo API is that the search results are strongly typed and implement IEnumerable - so its easy to add them to a grid, repeater or your weapon of choice. Not all the other APIs I investigated played so nicely.

Next came the Amazon product search. Amazon offers some API documentation and such on their developer site and one of the things it pointed to was the open source C# AmazonECS.NET library. I downloaded it, compiled it, and added a reference to it. By the way, the example site in the source is very well done indeed. If you get a chance, check it out.

Implementing an Amazon search using the library was almost as simple as using Live Search and Yahoo:

SearchRequest searchRequest = new SearchRequest();
searchRequest.Keywords = "pampers";
searchRequest.SubscriptionId = "myAmazonID";
Items items = searchRequest.Search(true);

However the Items collection does not implement IEnumerable so rather than being able to set it as my datagrid's datasource there was a bunch of additional code involving... no I can't say it... a DataSet, DataColumns, and DataRows. I will of course refactor that later if I choose to follow the Amazon path, but for my prototype it worked. 

My next stop was Google. I already reported that Froogle was out of the picture, but I decided to check out what Google had to offer anyway. I had also recently read that some big changes were made recently (for the worse) to what Google was offering. I found their Google AJAX API and implemented that. It is not a traditional API in the same terms as the ones we've already looked at, but is more like a drop-in Javascript control that offers a configurable Google search on your site, similar to what Live recently came up with. The configuration for the Google control is all Javascript-based. It's not too tough to work with and they offer plenty of examples on their site. Best of all, it looks very professional. You can see in the image below that I have created customized tabs for a couple of different domains as well as web, local, news, etc.

So far so good. Looking at all of the above search services took about one evening's worth of work. Looking into the Ebay API took an additional two. The Ebay API is huge. All I wanted was a simple product search, but because there are hooks in the API for everything from managing your Ebay Store to adding items to your favorites list it takes forever to learn. It took me at least 30 minutes to figure out how to display an image for an item because the documentation says something like this: if it is a gallery item then use the galleryurl, if it is a xxx item then check the yyy field and try the default category image, otherwise use the zzz field, ... The good news is that the documentation is really thorough. I downloaded the .NET API and played around with the examples, which are pretty good. I did notice that the unit tests seemed to be old, though, and about half of them failed.

Ebay does have a REST API and seems to be moving more in that direction. I played with that for a bit but chose to go with the API library to give me strongly typed objects. Along with being so big, the Ebay API seems a bit overcomplicated to me. Things like bid count and url are in the ListingDetails object of the Item, but the CurrentPrice is in the SellingStatus object of the Item, and other things like Currency and BuyItNow price are direct properties of the Item. Finally, you need three separate IDs assigned by Ebay (and another set of three for their live site) as well as a user account, password, and authorization token to make each call. Maybe I'm just a little skewed because all I want is a simple product search, but it just took too long for me to muck around in all the details. Nevertheless, everything you could ever want or need to do with Ebay is in there, you just need to find it. The code for my product search follows:

ApiAccount apiacct = new ApiAccount("MyDevid", "MyAppid", "MyCertid");
eBayAccount ebayacct = new eBayAccount("MyUserID", "MyPassword");
ApiCredential cred = new ApiCredential();
cred.ApiAccount = apiacct;
cred.eBayAccount = ebayacct;
cred.eBayToken = "MyUserToken";
ApiContext context = new ApiContext();
context.ApiCredential = cred;
context.EPSServerUrl = "";
context.SignInUrl = "";
context.SoapApiServerUrl = "";
context.XmlApiServerUrl = "";
eBay.Service.Call.GetSearchResultsCall call = new eBay.Service.Call.GetSearchResultsCall(context);
call.Version = "485";
SearchResultItemTypeCollection items = call.GetSearchResults("pampers");

See what I mean about it being complicated compared to the other APIs demonstrated before? And because Items is a collection of Item objects that are themselves just containers for a bunch of other objects they don't play nice with datagrids, repeaters, etc. It takes some parsing of those objects to get the data that you need.  I'll quit whining now. The end result looks good.

As I mentioned, the pages that I currently have in place and demonstrated are just prototypes. What I'd like to eventually do is choose one or two of the searches to implement and integrate into the site. For example, when looking at an item in someone's wish list you could click a link to see a popup (for lack of a better word) with search results to easily take you to a site to purchase that item. The Google search is probably the nicest looking so I may incorporate it somewhere in the site, however the Ebay and Amazon searches offer affiliate bonuses so they are the strongest options (Yahoo might, too. I'll check into that).

Earlier I pointed out the Live Search Box. I have been working with that recently on a client site and it works well, too - very similar to the Google AJAX Search.

My only real problem with using more than one of these search services is that they each have their own interfaces. I could end up with web references for one search, a referenced assembly or two for another, javascript (or some type of user/server control) for another, and hard-coded REST queries for another. That could really get messy. Perhaps I will write my own library that abstracts the details to each one and simply returns a list of products. Sounds fun, but that's a task for another day - for now I need to get back on track and start writing some code to get some more of those user stories implemented and crossed off my list.

Dan Hounshell
Web geek, nerd, amateur maker. Likes: apis, node, mobile, motorcycles, watches, food, Nashville, Savannah, Cincinnati and family. Dislikes: mean people
Dan Hounshell on Twitter