You know, I'm doing something similar myself. If you bug me enough, I might just zip up and post my XML-to-object graph code, though I make to guarantees about its security, performance, or bugs.
Given something like
<Button Text="Click Me">
It will give you what you expect. Just pass in an XmlNode and you get an object back.
To do the scripting bit, you can probably just make a set of scripting objects and have all the text parsing done there.