Coding4Fun Articles

Aggregating Syndicated Content

Description

  In this article, it is shown how to extend basic RSS user control to work with more than one content feed. Combining syndicated content from various sources, known most simply as aggregation, widens the breadth of information available on the web site and allows relating information in uniquely relevant ways for users.
3Leaf Development

Difficulty: Easy
Time Required: 1-3 hours
Cost: Free
Software: Visual Basic or Visual C# Express Editions
Hardware:
Download:

Building on my last installment, in this article I'll show you how to extend our basic RSS user control to work with more than one content feed. Combining syndicated content from various sources, known most simply as aggregation, widens the breadth of information available on your web site and lets you relate information in uniquely relevant ways for users.

One interesting background item I'd like to mention is Kent Sharkey's MSDN article E Pluriblog Unum: Merging RSS Feeds, which demonstrates some nifty techniques for merging RSS feeds into a single file. While the article is based on version 1.1 of the .NET Framework, you can use the source code that accompanies the article to jump start your own projects. I didn't do that myself, but I did borrow some of the ideas about how to approach aggregating syndicated content into a comprehensive view.

The first thing I considered was how to manage a list of individual feeds. While you can use a number of different techniques for tracking a list of source feeds, such as an Outline Processor Markup Language (OPML) file, I decided to take advantage of SQL Server 2005 Express Edition and the enhanced ASP.NET 2.0 data binding model to implement a basic RSS feed manager. Besides the ease of use ASP.NET 2.0 provides for accessing data, my main reason for choosing this method is that it sets the stage for leveraging the personalization services available using the Web Parts framework. After all, the ultimate goal of this type of application is to allow individual users to choose their own list of feeds. But let's not get ahead of ourselves; my goal in this article is to show how to manage the feed list itself.

In a fresh Visual Studio Web Developer Express project, I added a new SQL Database. I created a single table in the database and named it feeds. Using the integrated table designer, I configured three columns in the table: an identity column and two string-based columns for storing the title and location of the feed.

Click here for larger image

(click image to zoom)

Next, I added a new Web User Control to the project. From the Toolbox, I dragged a SqlDataSource control to the design surface of the Web User Control. This control is the cornerstone of the enhanced data access model in ASP.NET 2.0. It provides a direct connection between user interface controls and an ADO.NET managed provider (such as the SQL database I use in this application). From the smart tag menu, I selected "Configure Data Source..." to launch the data source configuration wizard.

Click here for larger image

(click image to zoom)

I followed the steps in the wizard to create a new connection string for the database, making sure that I selected the option to save the connection string information to the application configuration file. The wizard provides a page for building the default SQL SELECT statement. You can enter a custom SQL statement or stored procedure, or do what I did and work with the available table in the database to build the query statement directly.

Once you select a table, the available columns appear in the Columns list. As you select individual columns, the text of the SELECT statement displayed on the page gets updated accordingly. The next page in the wizard allows you to test the query. That wasn't much help in this case because I didn't have any data in the database.

After completing the wizard, I went back to the Toolbox and added a GridView control to the Web User Control page. The smart tag menu for the GridView control immediately appeared and I used the "Choose Data Source" dropdown menu to set the DataSourceID property to the SqlDataSource control.

Click here for larger image

(click image to zoom)

Next, I selected the "Edit Columns" item from the smart tag menu to edit the GridView's columns. I used the editor to make the identity column (feedId) invisible and updated the header text for the title and url columns so they would look better when displayed on the web page.

I also updated the appearance of the GridView using the Auto Format editor (accessible from the smart tag menu). I then moved over to the Properties window for the GridView control. To allow users to update or delete an item directly inside the list, I changed the AutoGenerateDeleteButton and AutoGenerateEditButton properties to true. This automatically altered the appearance of the GridView in the designer. As shown below, Edit and Delete links now appeared on the left side of the grid.

Click here for larger image

(click image to zoom)

The ADO.NET 2.0 object model is very simple, with the SqlDataSource control responsible for executing CRUD operations against a data source on behalf of its bound server controls. In this case, the SqlDataSource control is responsible for executing not only the SELECT statement to retrieve the data, but also the DELETE and UPDATE commands. Using the Properties window for the SqlDataSource control, I opened the Command and Property Editor for the DeleteQuery property and typed in the necessary SQL statement to perform a delete.

Notice that the parameter I used is written out as @original_feedId. The SqlDataSource control has another property titled OldValuesParameterFormatString that specifies how an input parameter ought to appear in a query. The default for this property is original_{0}, where {0} is a placeholder for the actual value of the parameter. Also note that I changed the Parameter source value to Control and the ControlID value to point to the GridView control. I followed the same procedure to add an UPDATE SQL statement in the UpdateQuery property of the SqlDataSource control.

Well, all that is well and good. However, I found that the GridView does not provide an easy way to add new records to a database. You can solve this problem in a number of different ways (including creative use of the footer template of the GridView). My solution was to use the DetailsView control. Like the GridView, this is another versatile tool for data access. It lets you navigate over a set of data, displaying one record at a time, update or delete the current record, or add a new record. You can also use this control in conjunction with a GridView to implement a classic master/detail representation of data. In that scenario, the GridView simply displays a list of selectable items, which when selected appear in detail in the DetailsView control. The flexibility of these controls gives you a lot of choices in providing the best user interface for your Web pages. For my purposes, I liked the idea of providing direct edit and delete in the GridView, while using the DetailsView for adding new records.

Click here for larger image

(click image to zoom)

After adding a DetailsView control directly beneath the GridView, I set its DataSourceID property to the SqlDataSource control and made sure that only its AutoGenerateInsertButton property was set to true. To boost performance, I changed the EnabledViewState property to false. With that done, I added the necessary SQL text for performing a database INSERT in the InsertQuery property of the SqlDataSource control.

The only difference between the configuration of the InsertQuery property and the other query-related properties of the SqlDataSource control was that the ControlID for InsertQuery was set to the DetailsView control (rather than the GridView).

Remarkably, I implemented all the logic for managing the feed list without having to write a single line of code. Of course, downloading and merging the feeds is another matter, but it is certainly worth pausing here to marvel at the power and simplicity of the ASP.NET 2.0 data binding architecture.

In order to display the aggregated RSS 2.0 feeds, the first hurdle is implementing a common data format. Because I only needed to display a few standard data items (item, description and link), I decided to use a strongly-typed DataSet to store the data gathered from each feed. The main advantage of this approach is that I could easily bind the resulting data to a user interface control. After adding a new DataSet to the project, I opened it in the designer and created a single DataTable with four columns (see below).

Click here for larger image

(click image to zoom)

In addition to the three basic item attributes, I also included a column called Feed to store the name of the source RSS feed so I could display this information on the Web page.

Next, I added two private methods to the FeedManager control's code file. The first method, MergeRssFeeds, is called from the control's Load event handler. The first line of code creates a class-level instance of the strongly-typed AggregateRSS dataset used to hold the data extracted from the RSS feeds. The code then iterates through the rows of the GridView control, which represent each of the configured RSS feeds. Within the For loop, the code calls a second method, AddRssFeed, passing in the name of the RSS Feed and the URL address of the RSS feed.

Visual Basic

Private Sub MergeRssFeeds()
RssData = New AggregatedRSS
If Me.GridView1.Rows.Count > 0 Then
For i As Integer = 0 To Me.GridView1.Rows.Count - 1
AddRssFeed(Me.GridView1.Rows(i).Cells(2).Text, _
Me.GridView1.Rows(i).Cells(3).Text)
Next i
End If
Me.Repeater1.DataSource = RssData
Me.Repeater1.DataBind()
End Sub

Visual C#

private void MergeRssFeeds()
{
RssData = new AggregatedRSS();
try
{
if (this.GridView1.Rows.Count > 0)
{
for (int i = 0; i < this.GridView1.Rows.Count; i++)
{
AddRssFeed(this.GridView1.Rows[i].Cells[2].Text,
this.GridView1.Rows[i].Cells[3].Text);
}
}
this.Repeater1.DataSource = RssData;
this.Repeater1.DataBind();
}
catch (WebException wx)
{
Literal1.Text = String.Format(
"<h3>Error</h3><p>A Web exception occurred while merging the" +
" RSS feeds.<br/>Make sure you have a valid connection to the " +
"internet.</p><p><em>{0}<em></p>", wx.Message);
Literal1.Visible = true;
}
}

At the end of the loop, the code sets the DataSource property of the Repeater control to the populated DataSet and then calls the DataBind method of the Repeater to bind the data to the control.

The helper method AddRssFeed performs most of the heavy lifting in this application. The code first fetches the RSS feed using the WebRequest class and loads the data into an XmlTextReader. The code then sets up a While loop to iterate over the XML stream. To make the code easier to explain, I've included the first few lines of code below.

Visual Basic

Private Sub AddRssFeed(ByVal feedTitle As String, ByVal feedUrl As String)
Dim rssReader As XmlTextReader = Nothing
Dim itemCount As Integer = 0

Try
Dim rssFeed As WebRequest = WebRequest.Create(feedUrl)
rssReader = New XmlTextReader(rssFeed.GetResponse().GetResponseStream())
While rssReader.Read
[see detail below]
End While
Finally
If Not rssReader Is Nothing Then rssReader.Close()
End Try
End Sub

Visual C#

private void AddRssFeed(string feedTitle, string feedUrl)
{
XmlTextReader rssReader = null;
int itemCount = 0;

try
{
WebRequest rssFeed = WebRequest.Create(feedUrl);
rssReader =
new xmlTextReader(rssFeed.GetResponse().GetResponseStream());
while (rssReader.Read())
{
[See the detail below]
}
}
finally
{
if (itemReader != null) itemReader.Close();
}
}

Inside the outside While loop, the code looks for elements named "item". This indicates that an RSS feed item is the current node. When this happens, the code creates a new RssItemRow to store the item data and then creates a new XmlReader object using the ReadSubTree method of the parent XmlReader. This reader (called itemReader) holds the current RSS item data. The code calls the Read method of the inner XmlReader to find each of the data items of interest. When found, the code stores the data value in the appropriate cell of the RssItemRow. At the end of the inside While loop, the code calls the strongly-typed AddRssItemRow method of the DataTable to add the row to the DataSet.

Visual Basic

While rssReader.Read
If rssReader.IsStartElement And ("item" = rssReader.LocalName) Then
Dim itemReader As XmlReader = Nothing
Try
Dim newRow As AggregatedRSS.RssItemRow = RssData.RssItem.NewRssItemRow()
itemReader = rssReader.ReadSubtree()
newRow.Feed = feedTitle
While itemReader.Read
If itemReader.IsStartElement Then
If ("title" = itemReader.LocalName) Then
newRow.Title = itemReader.ReadString
ElseIf ("description" = itemReader.LocalName) Then
Dim newDescription As String = itemReader.ReadString
' Truncate description to 100 characters
If newDescription.Length > 100 Then
newDescription = newDescription.Substring(0, 100) + " ..."
End If
newRow.Description = newDescription
ElseIf ("link" = itemReader.LocalName) Then
newRow.Link = itemReader.ReadString
End If
End If
End While
RssData.RssItem.AddRssItemRow(newRow)
itemCount = itemCount + 1
Finally
If Not itemReader Is Nothing Then itemReader.Close()
End Try
End If
' Add only the first five items from each feed
If itemCount >= 5 Then Exit While
End While

Visual C#

while (rssReader.Read())
{
if ((rssReader.IsStartElement()) && ("item" == rssReader.LocalName))
{
XmlReader itemReader = null;
try
{
AggregatedRSS.RssItemRow newRow = RssData.RssItem.NewRssItemRow();
itemReader = rssReader.ReadSubtree();
newRow.Feed = feedTitle;
while (itemReader.Read())
{
if (itemReader.IsStartElement())
{
if ("title" == itemReader.LocalName)
newRow.Title = itemReader.ReadString();
else if ("description" == itemReader.LocalName)
{
string newDescription = itemReader.ReadString();
//Truncate description to 100 characters
if (newDescription.Length > 100)
newDescription = newDescription.Substring(0, 100) +
" ...";
newRow.Description = newDescription;
}
else if ("link" == itemReader.LocalName)
newRow.Link = itemReader.ReadString();
}
}
RssData.RssItem.AddRssItemRow(newRow);
itemCount++;
}
finally
{
if (itemReader != null) itemReader.Close();
}
// Add up to 5 of the items returns (this can be set in a
// configuration file)
if (itemCount >= 5) break;
}
}

That's all the code I need to implement the RSS aggregation. As you can see, the hairiest part of the application was setting up the XmlReader and iterating through them to extract the necessary feed data. But once that task was accomplished, ASP.NET's superior data binding capabilities made building this application very easy.

Click here for larger image

(click image to zoom)

Looking ahead, the next step in the evolution of this project is to separate out the feed manager and feed display components into separate controls and to add personalization services ASP.NET 2.0 Web Parts framework. Until next time.

The Discussion

  • faimace

    This is a informative tutorial but i have a problem with the Edit &  Delete of the title and location url. Can i have some assistance on this?

Comments closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums, or Contact Us and let us know.