I don't think the ASP.NET authorization system ever sees the .wiki part.
A typical FlexWiki URL would be, for example, http://channel9.msdn.com/wiki/default.aspx/Channel9.ProductFeedback
The way I think it works is that IIS's URL parser follows the whole path until it reaches the 'default.aspx' in the middle. It sees that default.aspx is a file, not a directory, so it looks up the aspx extension in the metabase and sees that it's associated with ASP.NET. ASP.NET then determines if the user has access to default.aspx. If so, it runs the page.
The remainder of the path, in this case /Channel9.ProductFeedback, is available to FlexWiki's default.aspx in the HttpRequest object's PathInfo property. FlexWiki uses that, and the NamespaceMap.xml file, to determine which file contains the raw wiki data. Here it would be ProductFeedback.wiki in the WikiBase folder for the Channel9 namespace; archives are in ProductFeedback(timestamp-user).awiki files.
Basically, you need some form of authorization mechanism within FlexWiki itself, and at the moment, I don't think there is one. Feel free to suggest it on the SourceForge tracker.