Sitewide recent comments
You can get sitewide recent comments fast, without taxing your MySQL database.
This is a plugin that grabs new comments and inserts it into a sitewide_comments table. Once the sitewide_comments is populated, then you can retrieve the comments with a simple select of one table.
What about existing comments?
You can use the UNION statement to query all comments from all blogs, export them as CSV, and then import them into the sitewide_comments table.
Advantages are:
It's fast, since you are querying just one table.
Disadvantages are:
You'll be using twice as much storage space for your comments. However, how much storage does your comments currently use?
Every time someones adds/edits/deletes a comment, the same action takes place on the sitewide_comments table. However, in most cases, adding/editing/deleting comments will incur resources spread over a period of time, so this will be less noticeable. The other approach of grabbing all comments from separate tables at once and displaying them will happen in one operation (huge spike in resource), and it'll happen as often as you query for sitewide recent comments. If it's everytime someone refreshes your index site page, then it'll be a huge resource hog. So duplicating the comments into one table is a reasonable approach.
This is a solution with the assumption that disk is cheap, but your users' time isn't.