Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

TwitterDrive – The Revolution in Cloud Storage

 tDrive In this article, Brian Peek will provide a technical overview of TwitterDrive, an application used to store and retrieve files via the Twitter messaging service.
ASPSOFT, Inc.

Difficulty: Intermediate
Time Required: 2-3 hours
Cost: Free
Software: Visual C# Express 2008 SP1, Visual Basic Express 2008 SP1, or Visual Studio 2008 SP1, .NET Framework 3.5 SP1
Hardware: None
Download: Application, Source Code
Discussion Forum: Forum

 

Introduction

As you've probably guessed by now, TwitterDrive is my April Fool's Day application for Coding4Fun this year.  While the application does actually work, the limitations of Twitter make it functionally useless.  That said, I'll still take you through the important parts as there is some value in the code with regard to the Twitter API, LINQ to XML, threading, and a couple other things.

How It Works

Not very well, actually.  The concept is we take a file, compress it, uuencode/base64 encode it (that is, make a text representation of the binary contents), and then upload it 140 characters at a time to Twitter as status messages.  When the file upload is complete, an index is written of every file that is currently stored on Twitter with some information that can be used to later retrieve the file.

Twitter

Let's start with the Twitter API.  Full documentation on this API can be found at:

http://apiwiki.twitter.com/

TwitterDrive only requires a very small subset of this functionality to work.  We need the ability to post a new status message, download a user timeline, destroy a status, and verify the user's credentials.  All of these things are handled in the TwitterService class.

The heart of this class consists of two methods:  GetTwitter and PostTwitterGetTwitter will make a GET request to Twitter of the specified URL, and PostTwitter will make a POST request to Twitter of the specified URL with the supplied data.

C#

private XDocument GetTwitter(string url)
{
WebClient wc = new WebClient();

// only authenticate if we have a password
if(!string.IsNullOrEmpty(Password))
wc.Credentials = new NetworkCredential(Username, Password);

Stream s = wc.OpenRead(url);

// return an XDocument for LINQ
XmlTextReader xmlReader = new XmlTextReader(s);
XDocument xdoc = XDocument.Load(xmlReader);
xmlReader.Close();
return xdoc;
}

private XDocument PostTwitter(string url, string data)
{
byte[] bytes = Encoding.ASCII.GetBytes(data);

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";

// if we're writing, we need to authenticate
request.Credentials = new NetworkCredential(Username, Password);

// if this is 'true', Twitter breaks
request.ServicePoint.Expect100Continue = false;
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = bytes.Length;

Stream reqStream = request.GetRequestStream();
reqStream.Write(bytes, 0, bytes.Length);

// turn the response into an XDocument for use with LINQ
HttpWebResponse resp = (HttpWebResponse)request.GetResponse();
XmlReader xmlReader = XmlReader.Create(resp.GetResponseStream());
XDocument xdoc = XDocument.Load(xmlReader);
xmlReader.Close();
return xdoc;
}

VB

Private Function GetTwitter(ByVal url As String) As XDocument
Dim wc As New WebClient()

' only authenticate if we have a password
If (Not String.IsNullOrEmpty(Password)) Then
wc.Credentials = New NetworkCredential(Username, Password)
End If

Dim s As Stream = wc.OpenRead(url)

' return an XDocument for LINQ
Dim xmlReader As New XmlTextReader(s)
Dim xdoc As XDocument = XDocument.Load(xmlReader)
xmlReader.Close()
Return xdoc
End Function

Private Function PostTwitter(ByVal url As String, ByVal data As String) As XDocument
Dim bytes() As Byte = Encoding.ASCII.GetBytes(data)

Dim request As HttpWebRequest = CType(WebRequest.Create(url), HttpWebRequest)
request.Method = "POST"

' if we're writing, we need to authenticate
request.Credentials = New NetworkCredential(Username, Password)

' if this is 'true', Twitter breaks
request.ServicePoint.Expect100Continue = False
request.ContentType = "application/x-www-form-urlencoded"
request.ContentLength = bytes.Length

Dim reqStream As Stream = request.GetRequestStream()
reqStream.Write(bytes, 0, bytes.Length)

' turn the response into an XDocument for use with LINQ
Dim resp As HttpWebResponse = CType(request.GetResponse(), HttpWebResponse)
Dim xmlReader As XmlReader = XmlReader.Create(resp.GetResponseStream())
Dim xdoc As XDocument = XDocument.Load(xmlReader)
xmlReader.Close()
Return xdoc
End Function

Each of these methods will use the NetworkCredentials object to authenticate to Twitter for reading or writing, as required.  Twitter API methods return simple XML objects.  Both GetTwitter and PostTwitter take these XML documents and turn them into XDocument objects that can be easily queried with LINQ to XML later.

We only need to call two Twitter “get” methods: user_timeline and verify_credentials.  There are two overloaded methods which call the user_timeline API:

C#

public IList<Status> GetUserTimeline(int page, int count)
{
XDocument xdoc = GetTwitter(string.Format("http://twitter.com/statuses/user_timeline.xml?count={0}&page={1}&id={2}", count, page, Username));
IList<Status> statuses = ParseStatuses(xdoc);
return statuses;
}

public IList<Status> GetUserTimeline(int since_id, int max_id, int page, int count)
{
XDocument xdoc = GetTwitter(string.Format("http://twitter.com/statuses/user_timeline.xml?since_id={0}&max_id={1}&count={2}&page={3}&id={4}", since_id, max_id, count, page, Username));
IList<Status> statuses = ParseStatuses(xdoc);
return statuses;
}

VB

Public Function GetUserTimeline(ByVal page As Integer, ByVal count As Integer) As IList(Of Status)
Dim xdoc As XDocument = GetTwitter(String.Format("http://twitter.com/statuses/user_timeline.xml?count={0}&page={1}&id={2}", count, page, Username))
Dim statuses As IList(Of Status) = ParseStatuses(xdoc)
Return statuses
End Function

Public Function GetUserTimeline(ByVal since_id As Integer, ByVal max_id As Integer, ByVal page As Integer, ByVal count As Integer) As IList(Of Status)
Dim xdoc As XDocument = GetTwitter(String.Format("http://twitter.com/statuses/user_timeline.xml?since_id={0}&max_id={1}&count={2}&page={3}&id={4}", since_id, max_id, count, page, Username))
Dim statuses As IList(Of Status) = ParseStatuses(xdoc)
Return statuses
End Function

Each of these methods creates a URL with the appropriate query string arguments (see the Twitter API docs for the full list) and then calls our ParseStatuses method with the returned XDocument, which will give us a List of Status objects. The Twitter-returned status contains a variety of data, such as:

<status>
<created_at>Mon Mar 30 07:20:57 +0000 2009</created_at>
<id>1234567123</id>
<text>Status text</text>
<source>web</source>
<truncated>false</truncated>
<in_reply_to_status_id/>
<in_reply_to_user_id/>
<favorited>false</favorited>

<user>
<id>12345678</id>
<name>Some Person</name>
<screen_name>myscreenname</screen_name>
<description/>
<location/>

<profile_image_url>
http://video.ch9.ms/ecn/c4fcontent/migration/9525376/default_profile_normal.png
</profile_image_url>
<url/>
<protected>false</protected>
<followers_count>1</followers_count>
</user>
</status>

We only care about a few elements and only parse those, namely id, text, user (which is in itself an XML chunk), and created_at.

C#

private IList<Status> ParseStatuses(XContainer container)
{
// return a list of Status objects
var query = from status in container.Descendants("statuses").Descendants("status")
select ParseStatus(status);
return query.ToList();
}

private Status ParseStatus(XDocument xdoc)
{
// create a Status object from the returned XML
var query = from status in xdoc.Descendants("status")
select ParseStatus(status);

return query.SingleOrDefault();
}

private Status ParseStatus(XElement xelement)
{
Status s = new Status()
{
ID = (int)xelement.Element("id"),
Text = (string)xelement.Element("text"),
UserInformation = ParseUserInformation(xelement.Element("user")),
// HTTP-formatted date
CreatedAt = DateTime.ParseExact(xelement.Element("created_at").Value,
"ddd MMM dd HH:mm:ss zzzz yyyy",
CultureInfo.GetCultureInfoByIetfLanguageTag("en-us"),
DateTimeStyles.AllowWhiteSpaces)
};

return s;
}

VB

Private Function ParseStatuses(ByVal container As XContainer) As IList(Of Status)
' return a list of Status objects
Dim query = _
From status In container.Descendants("statuses").Descendants("status") _
Select ParseStatus(status)
Return query.ToList()
End Function

Private Function ParseStatus(ByVal xdoc As XDocument) As Status
' create a Status object from the returned XML
Dim query = _
From status In xdoc.Descendants("status") _
Select ParseStatus(status)

Return query.SingleOrDefault()
End Function

Private Function ParseStatus(ByVal xelement As XElement) As Status
Dim s As New Status() With { _
.ID = CInt(xelement.Element("id")), _
.Text = CStr(xelement.Element("text")), _
.UserInformation = ParseUserInformation(xelement.Element("user")), _
.CreatedAt = DateTime.ParseExact(xelement.Element("created_at").Value, _
"ddd MMM dd HH:mm:ss zzzz yyyy", CultureInfo.GetCultureInfoByIetfLanguageTag("en-us"), DateTimeStyles.AllowWhiteSpaces) }
Return s
End Function

These methods use LINQ to XML to drill down into the XML document to parse out individual status objects and return them as a list.  You will notice the user element contains a user_information XML chunk, and the ParseUserInformation method parses that data

C#

private UserInformation ParseUserInformation(XContainer container)
{
// parse and return a UserInformation object
return new UserInformation
{
ID = (int)container.Element("id"),
Name = (string)container.Element("name"),
ScreenName = (string)container.Element("screen_name")
};
}

VB

Private Function ParseUserInformation(ByVal container As XContainer) As UserInformation
' parse and return a UserInformation object
Dim ui As New UserInformation() With { _
.ID = CInt(container.Element("id")), _
.Name = CStr(container.Element("name")), _
.ScreenName = CStr(container.Element("screen_name")) }

Return ui
End Function

The verify_credentials API is called in a similar manner, but in this case we will “listen” for a 401 (Unauthorized) exception and return true or false appropriately:

C#

public bool VerifyTwitterCredentials(string username, string password)
{
try
{
WebClient wc = new WebClient();
wc.Credentials = new NetworkCredential(username, password);
Stream s = wc.OpenRead("http://twitter.com/account/verify_credentials.xml");
s.Close();
}
catch(WebException we)
{
if((we.Response as HttpWebResponse).StatusCode == HttpStatusCode.Unauthorized)
return false;
throw;
}

return true;
}

With the “get” methods out of the way, we can write our “post” methods.  We only need two methods here as well:  update and destroy.  Update is used to write a new tweet to Twitter, and destroy is used to remove an existing tweet.

C#

public Status UpdateStatus(string status)
{
XDocument doc = PostTwitter("http://twitter.com/statuses/update.xml", "status=" + status);
Status s = ParseStatus(doc);

// if we don't get the text we sent, we hit the rate limit...
if(s.Text != HttpUtility.UrlDecode(status))
throw new TwitterRateLimitException("Twitter upload limit reached.");
return s;
}

public Status Destroy(int status)
{
XDocument doc = PostTwitter("http://twitter.com/statuses/destroy/" + status + ".xml", "id=" + status);
return ParseStatus(doc);
}

 

VB

Public Function VerifyTwitterCredentials(ByVal username As String, ByVal password As String) As Boolean
Try
Dim wc As New WebClient()
wc.Credentials = New NetworkCredential(username, password)
Dim s As Stream = wc.OpenRead("http://twitter.com/account/verify_credentials.xml")
s.Close()
Catch we As WebException
If (TryCast(we.Response, HttpWebResponse)).StatusCode = HttpStatusCode.Unauthorized Then
Return False
End If
Throw
End Try

Return True
End Function

The UpdateStatus method has a bit of code to determine whether we hit Twitter's upload rate limit.  Twitter will limit you to about 100 tweets per hour.  The only way to determine if you've hit the upload limit (that I found at least) is to compare the text from status element returned from the update to the text that was sent.  If the upload limit is hit, the last valid status element will be returned, and therefore the text blocks will not match.  If this happens, we throw our own custom TwitterRateLimitException up the stack for the UI to handle.

TwitterDrive

Now that we can talk to Twitter, the next step is to write the code that uses these methods to store and retrieve our file data.  This code is located in the TwitterDrive class.

One of the goals I had was to make this application thread-happy to provide a responsive UI while files are uploaded and downloaded.  Therefore, this class sets up three events which can be hooked by the UI to receive periodic status information:

C#

public class ChunkEventArgs : EventArgs
{
public Status Status;
public int ChunkLength;
public int Total;

public ChunkEventArgs(Status status, int length, int total)
{
Status = status;
ChunkLength = length;
Total = total;
}
}

...

// event handlers for async file transfer
public event EventHandler<ChunkEventArgs> ChunkUpload;
public event EventHandler<ChunkEventArgs> ChunkDownload;
public event EventHandler<EventArgs> TransferComplete;

 

VB

Public Class ChunkEventArgs
Inherits EventArgs
Public Status As Status
Public ChunkLength As Integer
Public Total As Integer

Public Sub New(ByVal status As Status, ByVal length As Integer, ByVal total As Integer)
Status = status
ChunkLength = length
Me.Total = total
End Sub
End Class

These events will be called at the appropriate times and provide information to allow the UI to draw a progress bar and other status information.

The UploadFile method does exactly what you think: it uploads the file specified.  The file contents are loaded into memory, compressed, base64 encoded, and finally URL encoded.  Then, this large string is cut into 140 character chunks and sent to Twitter, one chunk at a time.  After each chunk is uploaded, the ChunkUpload event is called.  Finally, when all chunks are uploaded, a new FileEntry object is created, added to the in-memory file index, and that index is finally written to the top of the Twitter status list.

C# 

public void UploadFile(string path)
{
Status s = null;
int startID = 0;
int length = 0;
int chunkLength = 140;

// encode the file
string file = EncodeFile(path);

// upload the chunks
for(int i = 0; i < file.Length; i+= chunkLength)
{
// get the proper length (i.e. don't get full 140 if we're at the end)
string chunk = file.Substring(i, Math.Min(chunkLength, file.Length-i));

// handle the case where we chopped a chunk in the middle of an encoded character
// in this case, chop off the encoded data and reset the counter
if(chunk.EndsWith("%2"))
{
chunk = chunk.Substring(0, chunk.Length-2);
i -= 2;
}

if(chunk.EndsWith("%"))
{
chunk = chunk.Substring(0, chunk.Length-1);
i -= 1;
}

try
{
// upload the chunk
s = _twitter.UpdateStatus(chunk);

// notify listeners that we uploaded
if(ChunkUpload != null)
ChunkUpload(this, new ChunkEventArgs(s, chunk.Length, file.Length));
}
catch(TwitterRateLimitException)
{
throw new TwitterDriveException("Twitter upload limit reached. Please try again later.");
}

if(i == 0)
startID = s.ID;

length++;
}

// create a new FileEntry for this file
FileEntry fe = new FileEntry()
{
Filename = Path.GetFileName(path),
StartStatus = startID,
EndStatus = s.ID,
Length = length,
FileIndex = GetNextIndex()
};

// update the index
UpdateFileIndex(fe);

// notify we're done
if(TransferComplete != null)
TransferComplete(this, null);
}

VB

Public Function UploadFile(ByVal filepath As String)
Dim s As Status = Nothing
Dim startID As Integer = 0
Dim length As Integer = 0
Dim chunkLength As Integer = 140

' encode the file
Dim file As String = EncodeFile(filepath)

' upload the chunks
For i As Integer = 0 To file.Length - 1 Step chunkLength
' get the proper length (i.e. don't get full 140 if we're at the end)
Dim chunk As String = file.Substring(i, Math.Min(chunkLength, file.Length-i))

' handle the case where we chopped a chunk in the middle of an encoded character
' in this case, chop off the encoded data and reset the counter
If chunk.EndsWith("%2") Then
chunk = chunk.Substring(0, chunk.Length-2)
i -= 2
End If

If chunk.EndsWith("%") Then
chunk = chunk.Substring(0, chunk.Length-1)
i -= 1
End If

Try
' upload the chunk
s = _twitter.UpdateStatus(chunk)

' notify listeners that we uploaded
RaiseEvent ChunkUpload(Me, New ChunkEventArgs(s, chunk.Length, file.Length))
Catch e1 As TwitterRateLimitException
Throw New TwitterDriveException("Twitter upload limit reached. Please try again later.")
End Try

If i = 0 Then
startID = s.ID
End If

length += 1
Next i

' create a new FileEntry for this file
Dim fe As New FileEntry() With {.Filename = Path.GetFileName(filepath), .StartStatus = startID, .EndStatus = s.ID, .Length = length, .FileIndex = GetNextIndex()}

' update the index
UpdateFileIndex(fe)

' notify we're done
RaiseEvent TransferComplete(Me, Nothing)
End Function

Let's take a closer look at the EncodeFile method:

C#

private string EncodeFile(string path)
{
// load the file
byte[] buff = File.ReadAllBytes(path);

// compress
MemoryStream ms = new MemoryStream();
GZipStream gs = new GZipStream(ms, CompressionMode.Compress);
gs.Write(buff, 0, buff.Length);
gs.Close();

byte[] buffCompressed = ms.ToArray();

// base64, urlencode
return HttpUtility.UrlEncode(Convert.ToBase64String(buffCompressed));
}

VB

Private Function EncodeFile(ByVal path As String) As String
' load the file
Dim buff() As Byte = File.ReadAllBytes(path)

' compress
Dim ms As New MemoryStream()
Dim gs As New GZipStream(ms, CompressionMode.Compress)
gs.Write(buff, 0, buff.Length)
gs.Close()

Dim buffCompressed() As Byte = ms.ToArray()

' base64, urlencode
Return HttpUtility.UrlEncode(Convert.ToBase64String(buffCompressed))
End Function

The code here reads the bytes of a file into a byte array.  Then, a MemoryStream object is created and tied to a GZipStream (compression) object.  Next, the byte array is written to the stream which will compress the data on the fly.  Once the stream is closed, calling ToArray on the MemoryStream will return a byte array of the compressed data.  The byte array is base64 encoded (turned into text) and finally URL encoded so it can be sent to Twitter.

The file index is written at the end of each file upload.  The index is comprised of a delimited string which contains the data in the FileEntry object.  Each file is uploaded as one tweet, with an end marker to denote where the file index ends.

C#

private void WriteFileEntries()
{
// write an end marker (write this first since they come out in reverse order later)
_twitter.UpdateStatus(FileEntryEnd);

for(int i = 0; i < _fileEntries.Count; i++)
WriteFileEntry(_fileEntries[i]);
}

private void WriteFileEntry(FileEntry fe)
{
// simple delimited list of data
string entry = string.Format(FileEntryHeader +
"{0}" + FileEntrySeparator +
"{1}" + FileEntrySeparator +
"{2}" + FileEntrySeparator +
"{3}" + FileEntrySeparator +
"{4}",
fe.Filename, fe.StartStatus, fe.EndStatus, fe.Length, fe.FileIndex);
_twitter.UpdateStatus(entry);
}

VB

Private Sub WriteFileEntries()
' write an end marker (write this first since they come out in reverse order later)
_twitter.UpdateStatus(FileEntryEnd)

Dim i As Integer = 0
Do While i < _fileEntries.Count
WriteFileEntry(_fileEntries(i))
i += 1
Loop
End Sub

Private Sub WriteFileEntry(ByVal fe As FileEntry)
' simple delimited list of data
Dim entry As String = String.Format(FileEntryHeader & "{0}" & FileEntrySeparator & "{1}" & FileEntrySeparator & "{2}" & FileEntrySeparator & "{3}" & FileEntrySeparator & "{4}", fe.Filename, fe.StartStatus, fe.EndStatus, fe.Length, fe.FileIndex)
_twitter.UpdateStatus(entry)
End Sub

A file list can be retrieved and parsed out with the following code:

C#

public IList<FileEntry> GetFileIndex()
{
bool end = false;
int page = 1;

// the index should be the last things tweeted, but if we have a failed upload, it may not be
while(!end && page < 5)
{
IList<Status> indexes = _twitter.GetUserTimeline(page, 200);

_fileEntries.Clear();

// parase out the file entries
foreach(Status index in indexes)
{
FileEntry fe = ParseFileIndexString(index.Text);
if(fe != null)
{
fe.IndexStatusId = index.ID;
_fileEntries.Add(fe);
}

if(index.Text.StartsWith(FileEntryEnd))
{
end = true;
break;
}
}
page++;
}
return _fileEntries;
}

private FileEntry ParseFileIndexString(string index)
{
if(!index.StartsWith(FileEntryHeader))
return null;

string[] fileEntry = index.Split(Convert.ToChar(FileEntrySeparator));
FileEntry fe = new FileEntry()
{
Filename = fileEntry[0].Replace(FileEntryHeader, string.Empty),
StartStatus = int.Parse(fileEntry[1]),
EndStatus = int.Parse(fileEntry[2]),
Length = int.Parse(fileEntry[3]),
FileIndex = int.Parse(fileEntry[4])
};
return fe;
}

VB

Public Function GetFileIndex() As IList(Of FileEntry)
Dim [end] As Boolean = False
Dim page As Integer = 1

' the index should be the last things tweeted, but if we have a failed upload, it may not be
Do While (Not [end]) AndAlso page < 5
Dim indexes As IList(Of Status) = _twitter.GetUserTimeline(page, 200)

_fileEntries.Clear()

' parase out the file entries
For Each index As Status In indexes
Dim fe As FileEntry = ParseFileIndexString(index.Text)
If fe IsNot Nothing Then
fe.IndexStatusId = index.ID
_fileEntries.Add(fe)
End If

If index.Text.StartsWith(FileEntryEnd) Then
[end] = True
Exit For
End If
Next index
page += 1
Loop
Return _fileEntries
End Function

Private Function ParseFileIndexString(ByVal index As String) As FileEntry
If (Not index.StartsWith(FileEntryHeader)) Then
Return Nothing
End If

Dim fileEntry() As String = index.Split(Convert.ToChar(FileEntrySeparator))
Dim fe As New FileEntry() With {.Filename = fileEntry(0).Replace(FileEntryHeader, String.Empty), .StartStatus = Integer.Parse(fileEntry(1)), .EndStatus = Integer.Parse(fileEntry(2)), .Length = Integer.Parse(fileEntry(3)), .FileIndex = Integer.Parse(fileEntry(4))}
Return fe
End Function

This code will retrieve the user's timeline, loop through the tweets, looking for file index entries (those that start with the proper delimiter).  When one is found, the data is parsed back into a FileEntry object and added to the in-memory cache.  You will note that the GetFileIndex method will read up to 1000 tweets looking for the end marker before giving up.

Now that we can upload files and build the index, we need to be able to download, decode and save a file.  This is done in the obviously named DownloadFile method:

C#

public void DownloadFile(FileEntry fe, string path)
{
StringBuilder sb = new StringBuilder(fe.Length * 140);

// get statuses for this user starting/ending at the IDs for this file
IList<Status> chunks = _twitter.GetUserTimeline(fe.StartStatus-1, fe.EndStatus, 1, 200);

// order them from oldest to newest
var orderedChunks = from chunk in chunks
orderby chunk.ID ascending
select chunk;

foreach(Status chunk in orderedChunks)
{
// trim off any extra characters
string newChunk = chunk.Text.TrimEnd('.').Trim();
sb.Append(newChunk);

// notify listeners that we downloaded a chunk
if(ChunkDownload != null)
ChunkDownload(this, new ChunkEventArgs(chunk, newChunk.Length, fe.Length * 140));
}

// decode and write the file
byte[] buff = DecodeFile(sb.ToString());
File.WriteAllBytes(Path.Combine(path, fe.Filename), buff);

// notify that we're done
if(TransferComplete != null)
TransferComplete(this, null);
}

 VB

Public Function DownloadFile(ByVal fe As FileEntry, ByVal filepath As String)
Dim sb As New StringBuilder(fe.Length * 140)

' get statuses for this user starting/ending at the IDs for this file
Dim chunks As IList(Of Status) = _twitter.GetUserTimeline(fe.StartStatus-1, fe.EndStatus, 1, 200)

' order them from oldest to newest
Dim orderedChunks = _
From chunk In chunks _
Order By chunk.ID Ascending _
Select chunk

For Each chunk As Status In orderedChunks
' trim off any extra characters
Dim newChunk As String = chunk.Text.TrimEnd("."c).Trim()
sb.Append(newChunk)

' notify listeners that we downloaded a chunk
RaiseEvent ChunkDownload(Me, New ChunkEventArgs(chunk, newChunk.Length, fe.Length * 140))
Next chunk

' decode and write the file
Dim buff() As Byte = DecodeFile(sb.ToString())
File.WriteAllBytes(Path.Combine(filepath, fe.Filename), buff)

' notify that we're done
RaiseEvent TransferComplete(Me, Nothing)
End Function

This method returns the user's timeline starting and ending at the tweets specified in the FileEntry object.  The status list is ordered in ascending order (i.e. oldest to newest).  Each status is appended to the previous using a StringBuilder object.  When all chunks are processed, the file is decoded and saved to the chosen location.

C#

public byte[] DecodeFile(string data)
{
// base64 to binary
byte[] buff = Convert.FromBase64String(data);

// decompress
MemoryStream ms = new MemoryStream(buff);
GZipStream gs = new GZipStream(ms, CompressionMode.Decompress, false);

// original
byte[] decompressed = ReadAllBytes(gs);
return decompressed;
}

VB

Public Function DecodeFile(ByVal data As String) As Byte()
' base64 to binary
Dim buff() As Byte = Convert.FromBase64String(data)

' decompress
Dim ms As New MemoryStream(buff)
Dim gs As New GZipStream(ms, CompressionMode.Decompress, False)

' original
Dim decompressed() As Byte = ReadAllBytes(gs)
Return decompressed
End Function

To decode the file, we do the opposite of what we did before: the file is converted to bytes from the base64 encoded string, a GZipStream is used to decompress the data, and the final, uncompressed file contents are returned to the caller to be saved to the disk.

Be sure to look through the entire TwitterDrive.cs/vb file for the full picture, but these are the important parts.

User Interface

The final piece is the user interface.

4-1-2009 4-01-42 AM

A very simple UI for uploading, downloading and deleting files.  At the start of the application, the three TwitterDrive events are hooked: ChunkUpload, ChunkDownload, and TransferComplete.  These three event handlers are used to update the progress bar on a dialog box that pops up during the file transfer.

When upload or download is selected, the file is selected, and a new thread is created to start the actual process.  Here is what the download process looks like:

C#

// start a new thread to grab the file
Thread t = new Thread(() => _twitterDrive.DownloadFile(fe, fbd.SelectedPath));
t.Start();

if(_progress.ShowDialog() == DialogResult.Cancel)
t.Abort();
else
MessageBox.Show("File download complete.", "TwitterDrive", MessageBoxButtons.OK, MessageBoxIcon.Information);

VB

' start a new thread to grab the file
Dim t As New Thread(CType(Function() _twitterDrive.DownloadFile(fe, fbd.SelectedPath), ThreadStart))
t.Start()

If _progress.ShowDialog() = System.Windows.Forms.DialogResult.Cancel Then
t.Abort()
Else
MessageBox.Show("File download complete.", "TwitterDrive", MessageBoxButtons.OK, MessageBoxIcon.Information)
End If

A new thread is created, whose ThreadStart parameter is the DownloadFile method of the TwitterDrive class.  The thread is started, and the progress dialog box is shown.  If the progress box is cancelled, the thread is aborted, otherwise the dialog box is closed in response to the TransferComplete event sent by the TwitterDrive class.

Using the Application

  1. Create a new Twitter account for use in TwitterDrive.
  2. Enter the credentials for this new account in the provided textboxes and click Refresh.  Or, if you want to download files from someone else, enter just the username and click Refresh.
  3. Upload and download data at will.

Conclusion

And there we have it.  A functionally useless application that actually works.  Give it a try and find out what the limitations are.  Just be sure to use a Twitter account that isn't your main feed otherwise you'll have some pretty angry followers.

Thanks

A special thanks to Mark Zaugg, Dan Fernandez, and Giovanni Montrone for helping me test this application.  Giovanni is also partly responsible for this horrible idea after joking that I should Twitter him a file.  So blame him.

Also thanks to Clint Rutkas for throwing together the icon (it's a combo of the SkyDrive icon + the Twitter icon).

Bio

Brian is a Microsoft C# MVP who has been actively developing in .NET since its early betas in 2000, and who has been developing solutions using Microsoft technologies and platforms for even longer. Along with .NET, Brian is particularly skilled in the languages of C, C++ and assembly language for a variety of CPUs. He is also well-versed in a wide variety of technologies including web development, document imaging, GIS, graphics, game development, and hardware interfacing. Brian has a strong background in developing applications for the health-care industry, as well as developing solutions for portable devices, such as tablet PCs and PDAs. Additionally, Brian has co-authored the book "Coding4Fun: 10 .NET Programming Projects for Wiimote, YouTube, World of Warcraft, and More" published O'Reilly. He previously co-authored the book "Debugging ASP.NET" published by New Riders.  Brian is also an author for MSDN's Coding4Fun website.

Tags:

Follow the Discussion

  • slyislyi

    This code could be converted to use Azure Queues for storage Wink

    http://msdn.microsoft.com/en-us/library/dd573356.aspx

  • Alexei PshenichnyiAlexei Pshenichnyi

    Good idea and good article at all, Brian!

    But things like

    "string entry = string.Format(FileEntryHeader + "{0}" + FileEntrySeparator + "{1}" + FileEntrySeparator + "{2}" + FileEntrySeparator + "{3}" + FileEntrySeparator + "{4}", fe.Filename, fe.StartStatus, fe.EndStatus, fe.Length, fe.FileIndex);"

    and

    "TransferComplete(this, null);" written by Microsoft C# MVP look doubly bad.

  • Clint RutkasClint I'm a "developer"

    @Alexei,

    Absolutely no part of the TwitterDrive code should be judged for quality or completeness.  It was a joke hacked out in a couple days for April Fool's Day.  It's buggy and awful, but it kinda' works.  Smiley

Remove this comment

Remove this thread

close

Comments Closed

Comments have been closed since this content was published more than 30 days ago, but if you'd like to continue the conversation, please create a new thread in our Forums,
or Contact Us and let us know.