Check out this code! (Part I)

Sign in to queue

Description

Are you the type that forgets to return books on time?  Wish you could just keep an open tab at the local library?  I know that I do!  Instead of waiting for that email notice that arrives a week after the fact, this code can help you keep on top of what's checked out, and what's due.
Arian's Blog

Difficulty: Intermediate
Time Required: 1-3 hours
Cost: Free
Software: Visual Basic or Visual C# Express Editions
Hardware: None
Download:

Introduction

I love our local library.  For myself, I get countless books that I wouldn't want to buy since I'll never read them again.  For my two kids, we have an immense selection of children's books to rotate.  The problem, is getting them back on time!  We don't have one single library day, so often our library bookshelf at home contains books that all go back different times, and things like movies are always due back sooner anyway.  What would be great, I decided, would be an application sitting in the system tray to show me what's coming coming up, or already overdue.  Being able to have one view for all library accounts in our family is a great thing!

Part I of this article highlights creating a reusable class library (DLL) to communicate with a specific library back-end server, Horizon Information Portal, in use in hundreds of libraries.  A simple test application will show how to use this code to search the card catalog and to see what books you have checked out.  Concepts include XML consumption, HTTP requests, and a clean object model abstracted away from any user interface.  A very simple test harness application is included.

Part II will take that reusable library and create a desktop application to expose more of the available search features, and allow multiple libraries and patrons to be monitored for upcoming and overdue books.  Concepts will include using a notification icon (appearing in the system tray), context menus (right-click), data binding, and predicates.

The source code is available in both C# and VB (the download links are above the introduction).  If you didn't know yet, you can program for .NET without paying a single dime!  Just download one of the Express editions of Visual Studio, from this link: Visual Studio 2005 Express Edition.

Looking on the Horizon

Our local library uses Horizon Information Portal, thus my choice of back-end.  From Google searches, I find hundreds of libraries using the system so it must be pretty popular.  I contacted the SirsiDynix Corporation (creators of Horizon) for information on their API, but after a number of transfers and a promised callback that never materialized, I realized that they just don't have that side of things put together yet.  They do obviously support it though.  I learned that by appending &GetXML=true to a query string, every Horizon request returns well-formed XML.  It is this XML that enables an object model to be created.

The first thought with XML-based requests/responses is to use the built-in .NET XML serialization features.  I started with this approach, but quickly grew frustrated.  Without documentation or schema files, there was a lot of manual tweaking of data types.  It was also difficult to generate a schema since certain nodes only appeared in certain situations, so I was never sure that I had everything.  Worse still, half-way through my coding a new version was released and that actually changed the name of one of the elements.  After more searching, I saw that not all libraries always run the newest version, so data will vary depending on what library you connect to.  XML serialization just didn't seem flexible enough.

My solution was to just work with the XML manually.  If you haven't worked with XML yet, don't be afraid!  The basic concept is a tree of nodes.  Each node may or may not have attributes, and will either contain text or more nodes.  Accessing any data in an XML file is just figuring out the path to the node.

XML Messages

The Horizon Information Portal XML format relies on elements with text, not a single attribute that I've come across.  There are messages for basic search, advanced search, renewing materials, getting account information and a number of other things.  Requests are solely URL-based, and the responses are pure XML (similar to REST, but more monolithic).  Requests can get to be fairly long including all required parameters and the session ID.

Responses all begin with the same blocks of data.  Information regarding the server, client browser, and authentication status come back with every message.  The messages also include the complete set of search options (by title, by author), sort options, and search limitations (by library branch, by collection).  This is a large amount of information (10k or so), and it comes back with every response.  It's not very efficient, but it gets the job done.  Amazingly, the XML data also includes presentation information for the toolbar.  As it turns out, all of the redundant data is because the entire dataset is tailored for presentation.  Adding GetXML=true to your query string returns XML instead of HTML, but it's the same information.  If it wasn't for this option, you'd be stuck "scraping" the HTML response, and it would be very fragile indeed.  Unfortunately, it's still somewhat fragile.  If SirsiDynix does decide to make an API available directly, they'll need to separate their data a little better.

Sending most responses requires parameters called menu and aspect, and sometimes submenu as well.  All messages should include a session ID (generated by the server).  Once a user is authenticated, this session ID keeps you logged in.  Even for unauthenticated requests such as search, I suspect that passing a bogus session ID causes the server to create a new one each time.  There's no reason to make things difficult on the other side!

Sending Requests

To make it easier, I tried to factor out much of the communication-related code into the HorizonDataAccess class.  This contains methods for formatting the query string and retrieving data via GET and POST operations.  I also wrapped a number of methods for converting XML nodes into objects in the HorizonDataMapper.  I discovered that due to the presentation nature of the XML data, many fields have no set location in the response.  For example, when searching for a title, some libraries display author name, holding title (holding is the cool way of referring to books, CD's, DVD's, and other items the library checks out!), publisher, and publication date, while other libraries only may display the name and author.  It seems configurable at the library level, and unfortunately the data is very bound to the display. The worst part is that these customizable fields don't even have fixed node names.  You have to determine where fields will be from the itemheader block:

<itemheader>
  <col>
    <label>Location</label>
  </col>
  <col>
    <label>Collection</label>
  </col>
</itemheader>

Another library might use a named XML element for Location or Collection.  This lack of consistency is very frustrating!  To get around this problem, I read in the name of all dynamically-assigned fields:

Visual Basic

Friend Sub ReadDynamicFieldDefinitions(ByVal doc As XmlDocument)
        ' Clear it out...
        dynamicFields = New Dictionary(Of String, String)()

        ' Find header fields
        Dim labelNodes As XmlNodeList = doc.SelectNodes("//searchresults/header/col/label")
        For i As Integer = 0 To labelNodes.Count - 1

            dynamicFields.Add(labelNodes(i).InnerText, "cell[" & (i + 1) & "]/data/text")
        Next

        ' Find item header fields
        labelNodes = doc.SelectNodes("//searchresults/itemheader/col/label")
        For i As Integer = 0 To labelNodes.Count - 1
            dynamicFields.Add(labelNodes(i).InnerText, "item[0]/cell[" & (i + 1) & "]/data/text")
        Next
End Sub

C# Code

internal void ReadDynamicFieldDefinitions(XmlDocument doc)
{
    // Clear it out...
    dynamicFields = new Dictionary<string, string>();

    // Find header fields
    XmlNodeList labelNodes = doc.SelectNodes("//searchresults/header/col/label");

    for (int i = 0; i < labelNodes.Count; i++)
    {
        dynamicFields.Add(labelNodes[i].InnerText, "cell[" + (i + 1) + "]/data/text");
    }

    // Find item header fields
    labelNodes = doc.SelectNodes("//searchresults/itemheader/col/label");

    for (int i = 0; i < labelNodes.Count; i++)
    {
        dynamicFields.Add(labelNodes[i].InnerText, "item[0]/cell[" + (i + 1) + "]/data/text");
    }
}

It's a mess, I admit!  The dynamicFields object contains mappings from field names (like "Location" or "Status" or "Due Date") to XPath query.  If you haven't worked with XPath, it's the scheme for describing the location of an element within an XML document.  There's a great tutorial at W3Schools if you want to learn more.  Retrieving that field then is just a simple lookup, then a call to SelectSingleNode() on the XmlNode object.  The bad thing, is that because some libraries use this dynamic mapping, and others don't you need to check in two places for some fields.  For instance, many libraries expose the due date of a holding in a node named duedate, while other libraries use the dynamic method.  Lost yet?  I understand!  Thankfully you don't need to deal with it.  You can just take the classes in this library as a starting point and get it all for free.  It's great that SirsiDynix makes everything so configurable, but the data shouldn't change -- just the presentation of it.

The Object Model

Though the data is all XML, I didn't want to deal with XML everywhere.  As you've seen, it's messy, but it's also too much to keep track of in an application.  My next step was to create objects to encapsulate a library, patron, a holding, and a number of other concepts.  Objects are setup from the XML data, but the application only sees the end objects.  Simplicity!

The starting point is the Library class.  This is where you define the URL for the server and give it a descriptive name.  Any other actions will start with calls to this class.  Create a Library object, set the BaseUrl property, and call the Init() method.

Visual Basic

Dim myLibrary as New Library
myLibrary.Name = "Hometown Library"
myLibrary.BaseUrl = "http://hometown.lib.us/ipac20/ipac.jsp"

C# Code

Library myLibrary = new Library();
myLibrary.Name = "Hometown Library";
myLibrary.BaseUrl = http://hometown.lib.us/ipac20/ipac.jsp;

Notice that URL.  Every single Horizon Information Portal installation that I've come across ends with "ipac20/ipac.jsp".  Just entering the host name isn't enough.  The next step is to create a ServerRequest object.  This contains the various fields that you set to indicate the search term and how to perform the search.  Invoking the search is done by calling Search on the Library object.  This will return a collection of Holding objects, each being a book, DVD, audiobook, etc.

Image 1 - Search Results from your local library

Visual Basic

Dim req As New ServerRequest("search", "", "basic_search")
req.SearchTerm = "programming"
Dim results As List(Of Holding) results = myLibrary.Search(req)

For Each h As Holding In results
    Console.WriteLine(h.Name)
End For

C# Code

ServerRequest req = new ServerRequest("search", "", "basic_search");
req.SearchTerm = "programming";
List<Holding> results = myLibrary.Search(req);

foreach(Holding h in results)
{
    Console.WriteLine(h.Name);
}

For authenticated messages (books checked out, patron name/address) you will need to create an instance of Patron.  Invoke the Login method of the Library class, and it will authenticate the user and associate the Library and Patron together into a session (CurrentSession property).  You can invoke methods like ReadPatronDetails to fill in the various user fields, items checked out, fines, holds, etc.  If you don't call Login, or the session expires, the user will be logged in automatically.  Note that there's no encryption on any libraries that I've tested on so passwords will be sent in plaintext.

Figure 2 - Checked out items

Final Note

This was created by analyzing the available XML, but it's clear that each library heavily customizes their search pages.  Since the XML returns reflects the display, not the underlying data, one library might return the title when you search by title, but not by subject, while another library returns all fields in all searches.  It can be frustrating, but if you needed to, you could tweak things a bit for a given library.  Of course, better would be to add more resilience by adding any enhancements to the common code.  Try not to code for one library.  See if there's some flag somewhere that determines the behavior that you see.  If so, responding to the flag would allow the enhancement to work for other libraries.

Next Steps

At this point, the code is ready to drop into another project.  Once thing that occurred to me is that a Web-based application could create RSS feeds of checked out books, overdue, holds coming in, that sort of thing.  Also, just RSS feeds of matching items for a given string -- possibly useful to monitor for new books added to the library.  Maybe someone else has already implemented another library's API and a mega library application could result!  My next column will discuss an application to take advantage of this code with data binding, the system tray, and more.

Conclusion

This was a really fun project for me.  Such a feeling of excitement to figure out to use that XML data for new things!  That's the beauty of XML -- once you have structured data you can do so much with it.  I hope to hear that this code is put to use in interesting new projects.  I'm glad to answer questions about the or address any (gulp!) bugs that are found!  Download Visual Studio Express and get started!


Arian Kulp is an independent software developer and writer working in the Midwest.  He has been coding since the fifth grade on various platforms, and also enjoys photography, nature, and spending time with his family.  Arian can be reached through his web site at http://www.ariankulp.com.

Tags:

utility, Windows

The Discussion

  • User profile image
    Christopher Steen

    telerik radEditor for Office SharePoint Server 2007 has been officially released! + Telerik for MCMS...

  • User profile image
    Nate

    Where do you find the link for your local library?  Or the backend URL?  E-Mail: jackson_n@lionsairsoft.org

  • User profile image
    Clint

    Arian Kulp created a way to keep on top of what you checked out from the library and will tell you when

  • User profile image
    Mike Stramba

    Neat!

    I plugged in my local library and it worked great (Toront Public Library)  http://hip.tpl.toronto.on.ca/ipac20/ipac.jsp

    One ting I found lacking especially on dialup was some sort of feedback when a button is clicked.

    I started to modify the testform and added a new textbox, which just displays "testing" when the Update button is clicked.  I also added a MessageBox call, both of them are physically placed in the code before the "_patron = ..." code.  The Messagebox pops up immediately while the textbox doesn't get filled until seemingly the rest of the "try" block gets executed.  Why is that?

    Also, how could I display the "raw" xml messages coming back from the server?

    Mike

    ==========================

    MessageBox.Show("Test Message");

    textBox1.Text = "Testing now";

    _patron = new Horizon.Patron(this.userTextBox.Text, this.passwordTextBox.Text);

    lib.ReadPatronDetails(_patron);

  • User profile image
    Shane S

    For Nate:

    To find your library's url, Google 'Horizon Information Portal' and scroll through the listings until your library site is found.  If '#Focus' appears in the url, you will need to remove it.  I have tried it with three libraries and have found the url for the library.

    For Arian Kulp:

    Can you try adding a feature to store libraries in the program so you don't have to enter it every time?

  • User profile image
    huse

    how could I display the "raw" xml messages coming back from the server?

    I have made an application taht display raw messages at server!

    link:

    http://huseyincakir.freehostia.com/output.PNG

  • User profile image
    Lous St-Amour

    Argh. I was all ready to make an iPhone application for my local library (Toronto Public Library, as mentioned above) when I saw that they've upgraded to some new system, probably what I've seen references to as Horizon 8. The URLs are horrendous: seemingly random bits of session info. They've eliminated the query strings at the end, sure, but what they have now isn't much better. Library catalogues should just use the library barcode as the URL, with an addition for output format (e.g. xml) and another for branch. Session info belongs in a cookie, or at the very end of the URL, so it's easy to remove if linking to something. Right now I don't know if the link I make will work an hour from now, not to mention any APIs I might want to create through screen scraping. (Now the only option available to the general public.) Apparently, however, for library staff, the new system is much more open and easier to access, reducing the need to use XML internally to create a new interface. York University Libraries also uses this new system. Quite frustrating, really. I thought Open API meant publicly accessible, at least for developers ... but I can't find any docs on the new stuff, except that "most display functions can now produce XML" in some PowerPoint presentation. Sigh. Maybe I'll contact the TPL later to try and write something officially.

Add Your 2 Cents