How To Load Bundled Images In A Prism Webapp

One of the features in the Prism Webapp Bundle for Google Wave is a toaster pop-up notification of unread waves using the window.platform.showNotification() method.  The third parameter is named aImageURI, and is described by the nsIPlatformGlue IDL as, “The URI of an image to use in alert. Can be null for no image.”

Which is great, except … what URI scheme and path does one use?  Every example I could find always passed null for the image, so after giving up on the web I joined the Prism mailing list and posted a question.  The first response was to use the inline data scheme, with a base64-encoded image.  It was ugly, but it worked.

A later response, however, yielded the proper way:

resource://webapp/path/to/image.png

A Prism Webapp Bundle For Google Wave

Though Prism and Google Wave go great together simply creating a web app from the Prism Firefox add-on, Prism supports some script extensions that allow for more desktop-like integration of apps running inside it.  For example, you can call the window.platform.showNotification() method to cause a little toaster pop-up with the number of unread Waves.

I’ve created a webapp bundle that does just that.  Unfortunately, such bundles at present only work with the stand-alone version of Prism.  The Firefox add-on is really a better way to run Prism, but if you’re using it you’ll need to do a little manual mucking in your webapp profile to use this bundle.

Stand-Alone Bundle

So, if you just want the bundle, here you go.  Note that I haven’t really tested it on the stand-alone version, so please let me know if something is broken.

Hack Your Webapp

As I said, if you’re using the add-on version, you’ll need to do a little manual hacking.  After you create the webapp, as described in my earlier post, open up Explorer and navigate to your Prism webapp bundle cache.  On Windows, this is in %APPDATA%\WebApps (something like C:\Users\Brian\AppData\Roaming\WebApps); on Linux, it is ~/.webapps.  You should see your Google Wave webapp in that directory.  Add the webapp.js script to that directory, and also add in images/google-wave-52×32.png.  Now you should get a toaster pop-up and task bar notification when there are new waves.

It would be nice if Google were to add a <link rel=”webapp”> to Wave, referencing an appropriate bundle.  If anybody there sees this and cares to use my code as a crude starting point, I am releasing this code under an MIT license.

Interpreting the Imperial Network

A guest post by ferrix.

Windows Installer: As anyone who has done .msi development knows, you will never find a more wretched hive of scum and villainy.

Visual Studio 2005 valiantly tries to make things easier by offering “Setup and Deployment” projects.  This thing magically binds together the build outputs of other projects and burps out a plausible .msi file.  Hoora……waitaminute, something’s not quite right here.  Yeah.. it turns out that if you want anything but the barest minimum of shoving files and registry keys onto target machines, you’re going to have to do some post-processing, son.  Fortunately, Microsoft provides a handy COM API for torturing the .msi SQL database until it agrees to do your bidding.

What?  Oh, sorry…you didn’t know that an .msi was basically a demented relational database crammed into a file?  Congratulations, now you can share my nightmares.

But I come here not to complain about the .msi file format, nor Visual Studio.  The main course of today’s rant will be the installer engine itself, msiexec.  Specifically Windows Installer 4, which led me on a merry chase today.  I accidentally missed a dependency for one of my custom actions, and got the following lovely error:

The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2869

I’ve been hacking on these beasties for a couple years, this was not my first dance with 2869.  In fact, the internet is filled with stories about it.  This isn’t actually an error about what went wrong during the install, it’s an error about [what went wrong when the installer was trying to tell you [what went wrong during the install] ].  This is what we call a masking error, meaning “Your installer is so broken, I can’t even tell you about it properly”.  A detailed install log offers up:

DEBUG: Error 2869:  The dialog ErrorDialog has the error style bit set, but is not an error dialog

Most forum threads about this error are from hapless end users trying to get their program download to work, and vendors supplying fixed versions.  Everyone addresses the root cause, i.e. the error actually being thrown first, that eventually leads to the 2869.  Often it has to do with impersonation problems.  Well and good, but I already knew what was wrong with my custom action.  What I wanted, and couldn’t find anywhere, was someone who understood why the error reporting mechanism itself was failing.  (Spoilers: eventually found several right answers.  It’s easier to find them in retrospect once I knew what was wrong.)

What could be wrong with the ErrorDialog?  This guy came right off the truck from Visual Studio, and my tweaks never touched it.  Nevertheless, I spent about an hour poring over the documentation, trying to find any possible detail that was different between my .msi and the spec.  It all checked out totally fine.  But no matter what I tried, not an error dialog.  Not an error dialog.  My kingdom for an error dialog!

It’s such a pointed error, you see, and there are so many subtle requirements, I thought I must be missing something.  Was it the phase of the moon or something about the feng shui orientation of my laptop?  In casting about for a solution I happened upon this note.  It talked about adding an entry to the Error table, which is advice I hadn’t seen before:

In order to see the actual error, open the MSI with ORCA and add the following entry to the “Error” table.

1001 | “Error[1]:  [2]“

My logs never showed a 1001 error code, and a missing entry in the error table doesn’t have any relevance to the properties of the error dialog being correct.  And yet, and yet…  The page referred to 2869.  With nothing to lose, I tried adding the entry.  As if by magic, the error reporting immediately began working just fine.  Total changes needed to the error dialog: zero.  Total time wasted on this: one afternoon.

What happened?  In this case not only is the 2869 masking the underlying error, but the windows installer engine itself was lying about the nature of the masking error, and as a side effect of the problem, hiding the real error code (1001) to boot!  Why 2869 and not something like, “So listen, I see there’s no format string in the Error table for #1001… so regrettably I must now poop myself.”

I can totally imagine how it went down.  The developer needs to implement a new error formatting behavior in version 4, but in the event that the .msi has broken error handling, he has to tell the user about it somehow.  Adding a brand new error code would require changes up and down the source tree.  It’s almost deadline already, and hey look: this error 2869 is pretty close, it’s about error dialogs not working.  Surely anyone who gets that error will quickly understand what was meant.  One line of code, and under the wire home free.

In addition to venting, my goal in writing up this harrowing experience is to provide more link love for pages that show how to fix Windows Installer error 2869 when the log says your error dialog “has the error style bit set, but is not an error dialog” but really it is, and it’s a phony error hiding error 1001 instead.

These aren’t the errors you’re looking for.  Move along.

Multi-Threading with VFS

One of the new features in the BagIt Library will be multi-threading CPU-intensive bag processing operations, such as bag creation and verification.  Modern processors are all multi-core, but because the current version of the BagIt Library is not utilizing those cores, bag operations take longer than they should.  The new version of BIL should create and verify bags significantly faster than the old version.  Of course, as we add CPUs, we shift the bottleneck to the hard disk and IO bus, but it’s an improvement nonetheless.

Writing proper multi-threaded code is a tricky proposition, though.  Threading is a notorious minefield of subtle errors and difficult-to-reproduce bugs.  When we turned on multi-threading in our tests, we ran into some interesting issues with the Apache Commons VFS library we use to keep track of file locations.  It turns out that VFS is not really designed to be thread-safe.  Some recent list traffic seems to indicate that this might be fixed sometime in the future, but it’s certainly not the case now.

Now, we don’t want to lose VFS – it’s a huge boon.  Its support for various serialization formats and virtual files makes modeling serialized and holey bags a lot easier.  So we had to figure out how to make VFS work cleanly across multiple threads.

The FileSystemManager is the root of one’s access to the VFS API.  It does a lot of caching internally, and the child objects coming from its methods often hold links back to each other via the FileSystemManager.  If you can isolate a FileSystemManager object per-thread, then you should be good to go.

The usual way of obtaining a VFS is through the VFS.getManager() method,which returns a singleton FileSystemManager object.  Our solution was to replace the singleton call with a ThreadLocal variable, with the initialValue() method overloaded to create and initialize a new StandardFileSystemManager.  The code for that looks like this.

private static final ThreadLocal fileSystemManager = new ThreadLocal() { @Override protected FileSystemManager initialValue() { StandardFileSystemManager mgr = new StandardFileSystemManager(); mgr.setLogger(LogFactory.getLog(VFS.class)); try { mgr.init(); } catch (FileSystemException e) { log.fatal("Could not initialize thread-local FileSystemManager.", e); } return mgr; } };

The downside is that we lose the internal VFS caching that the manager does (although it still caches inside of a thread).  But that’s a small price to pay for it working.

Useful PDF ImageMagick Recipes

It turns out that ImageMagick is really quite good at reading, writing, re-arranging, and otherwise mucking with PDFs.  Unfortunately, you need to know the proper incantation, which can take much trial and error to figure out.  So, for my own future reference:

Split A PDF Into Parts

$ convert -quality 100 -density 300x300 multipage.pdf single%d.jpg

The quality parameter is the quality of the written JPEGs, and the density is the DPI (in this case, 300 DPI in both X and Y).

Join JPEG Parts Into A PDF

$ convert -adjoin file*.jpg doc.pdf

Rotate a PDF

$ convert -rotate 270 -density 300x300 -compress lzw in.pdf out.pdf

This assumes a TIFF-backed PDF. The density parameter is important because otherwise ImageMagick down-samples the image (for some reason). Adding in the compression option helps keep the overall size of the PDF smaller, with no loss in quality.

Now, if I can just figure out how to make future me remember to look here…

Fun With C# 2.0 Iterators

In the little free time that I have, I have been messing around with writing a .NET program to help me with the large amount of photo metadata editing I want to do on all the photos I’ve been uploading to my Flickr photostream.

It’s been fun doing some .NET again. It’s easy to forget how nice of a language C# is especially with all of the fancy new features 2.0 brings. Last night, I came up with a fun little method of using iterators.

Here’s the scenario: I need to collect photo IDs for editing. As a simple solution, I have a very small dialog box that contains a text field, an “Add Another” button, a “Done” button, and a “Cancel” button. When the user clicks “Add Another,” I want to save the entered photo ID and re-display the dialog. If they click “Done,” the currently entered ID should be saved, and the dialog should vanish. Finally, if they click cancel, the current ID does not get saved, and the dialog disappears.

I have set up the DialogResult of the “Done” buttton to return with DialogResult.OK, and “Add Another” to return with DialogResult.Yes. Here’s the code in the UI controller responsible for coordinating this:

public IEnumerable GetPhotoIds()
{
AddPhotoDialog dialog = new AddPhotoDialog();

result = dialog.ShowDialog(this.parentWindow);
}
}


Then, in the top-level UI controller, I can simply do:

AddPhotoUIController controller = new AddPhotoUIController(this.mainForm);

foreach (string photoId in controller.GetPhotoIds())
{
PhotoInfo info = this.FlickrApi.PhotosGetInfo(photoId);
this.CurrentBatch.Containers.Add(new SinglePhoto(info));
}


Pretty slick. I wish they had made this feature more general, though, and implemented full-on continuations.

Subversion Dump File Splitter

I recently had to do some mangling of a dump of my personal Subversion repository. Basically, I had to modify some paths and revision copy numbers before re-importing to a clean repository. However, the dump was in one huge 300 MiB file, making it really difficult for to open for editing.

Normally, the solution would simply be to re-dump the repository using the --revision option to the svnadmin dump command. Unfortunately, in a flash of stupidity, I deted my old repository before I had the new one working. So I wrote a little Perl script to split the dumpfile into seperate files.


#!/usr/bin/perl

print REV $line;
}


Ah…short and sweet, just the way a Perl script should be. Normally, I would immediately delete any Perl that I might happen to write, as I think the language is too flexible to be properly maintained over any length of time. However, since I can’t find anything else like this out on Teh Intarweb, I figure I’ll leave it here for posterity.

Updated Mantis Basic Authentication Code

I’ve updated my post on Making Mantis with Basic Authentication Not Suck to fix an oversight I discovered in the original version.

Basically, direct links into Mantis would not work, since most of the Mantis pages redirect to the login page when a user has not yet authenticated. The modification was simply to modify the login page to detect basic authentication and redirect to the previously modified login script.

Making Mantis with Basic Authentication Not Suck

At work, I’ve been doing a lot of system setup and administration. While it’s definitely beyond my job description as a Software Architect, there’s nobody else than can do it right on my team – and thus it falls to me.

When administrating a large number of servers, it becomes quickly apparant that a centralized directory of credentials is absolutely necessary. The obvious solution is OpenLDAP. After creating a directory, though, it becomes necessary to integrate the various applications into the directory so that the users may use the same credentials across all applications. (I’m not even talking about single sign-on. That’s a whole other kettle of fish.)

In some cases, it’s built right in. For example, our Linux servers use pam_ldap and nss_ldap to tie in the operating system. For Apache, the modules mod_ldap and mod_auth_ldap provide the necessary integration; and some web applications, such as Subversion fall in line quite naturally.

Many web applications, though, support their own authentication via a web page, and those applications often do not fit in quite the way you would hope. Two such applications we use are MediaWiki and MantisBT. In the case of MediaWiki, I found a series of modifications that pretty much worked. I had to tweak a few things, and while I’m not going to write about them here, feel free to contact me if you’d like to know what I did.

Mantis, though, ostensibly supports basic authentication. The authentication section in the manual has a specific BASIC_AUTH option. The problem is that it doesn’t work quite right.

I am protecting my Mantis directory with an Apache authentication directive. In the .htaccess in the Mantis directory, I require basic authentication, and the authenticated user must be a member of the cn=Mantis group in the directory.


SSLRequireSSL


order deny,allow
deny from all



The Mantis configuration directive to use basic authentication is set in the config_inc.php file.


# --- authentication settings ---
$g_login_method = BASIC_AUTH;


So in this scenario, you would never be able to get to the login page without authenticating with the web server. Mantis, rather than treating the basic auth as authoratitive, it takes the username and password received and then tries to log in using them (at least in version 0.19.3). I’m not sure the when that would be the right thing to do, but it certainly isn’t in this case because it requires me to keep the password in the directory synchronized with the password in the Mantis database.

So our goal is to make Mantis ignore its own password and simply think it has successfully authenticated when g_login_method is set to BASIC_AUTH.

As a nice feature, the core/authentication_api.php file has support for auto-creation of a user if it doesn’t exist when logging in via basic authentication. We want to keep that functionality. However, it inserts the basic authentication password into the database. While it really shouldn’t hurt anything, I’d like to keep it seperate for clarity. So on line 77, I’ve modified the call to user_create() in the auth_attempt_login() to generate a random password and insert that into the database.


# Modified to generate a random password.
# Since the basic authentication should be authoratative, then
# this password is just a dummy password, and should never be used.
# -- BCV
$t_cookie_string = user_create( $p_username, auth_generate_random_password() );


The next step is to make auth_attempt_login() succeed when basic authentication has been used. Modifying the if block around the auth_does_password_match(), we use some short-circuiting to skip the check (and therefore always succeed) if the login method equals BASIC_AUTH.


# Since basic auth is authoratative, and assuming that this page
# cannot be viewed unless it has already succeeded, then don't
# bother checking a password, and just do a valid login.
# -- BCV
if (BASIC_AUTH != $t_login_method && !auth_does_password_match( $t_user_id, $p_password ) ) {
user_increment_failed_login_count( $t_user_id );
return false;
}


The final step is to redirect to the login.php script at certain appropriate points. The first is in the default page, named index.php. Normally, it redirects to the login page if the user is not authenticated. We’re going to modify that behavior when we’re using basic authentication so that it skips right to the login script instead. That way, it will automatically attempt to log in the user.


if ( auth_is_user_authenticated() ) {
print_header_redirect( 'main_page.php' );
} else if (BASIC_AUTH == config_get( 'login_method')) {
print_header_redirect( 'login.php' );
} else {
print_header_redirect( 'login_page.php' );
}


Finally, we don’t really want them to be able to log out, since that would require a login again. The modification I’ve chosen does a logout, and then immediately attempts to log in again.


auth_logout();

if ( BASIC_AUTH == config_get( 'login_method' ) ) {
print_header_redirect( 'index.php' );
} else {
print_header_redirect( config_get( 'logout_redirect_page' ) );
}


And that’s it! Remember, the entire premise of these modifications is that the web server authentication is fully trusted, and that access to the underlying PHP scripts cannot occur unless the user has already successfully authenticated. If that’s no the case in your scenario, then these modifications won’t be secure.

Feel free to take these modifications for yourself. I’m going to see about submitting them as a patch to the official Mantis codebase. Oh, and if you find anything that I’ve missed, or anything that’s insecure, please let me know.

Making Old Links Work with mod_rewrite

So one of the big concerns with moving from an old blog installation to new software is the possibility of breaking links from around the Internet to content that I have created. At first, one might think that this kind of thing might generate a massive SEP field, but there’s more at stake than just a broken link.

What is that, you ask? PageRank. The content on my site has garnered a non-trivial score in Google’s view of the world, and such a score is valued, as it determines not only how high your own sites appear in any given Google query, but also how high people to whom you link appear. It is valued so much, in fact, that people often resort to comment spamming even mildly popular sites in order to increase their own search results.

So making sure the old link /article.php?story=20040727104257410 correctly maps to the new link /failed_sandbox_peer_launches_or_something.html is an important migration step.

So what’s the tool of choice? Well mod_rewrite, of course. The mod_rewrite package is an Apache module that performs some black magic of rewriting URLs from one form into another. In my particular case, the requirement was to map a Geeklog story ID (SID) into a Drupal node ID (NID). Surprisingly, that’s relatively easy to do.

The .htaccess file in this application’s directory contains the following declarations. The first line examines the incoming query string for a variable called story, and it captures it using a regular expression. The second line looks for a request beginning with article.php, and then maps it to a value pulled from a mapping called geekmap. Notice the %1, which is a backreference to the SID captured in the previous line. The question mark at the end strips off any other query strings. Finally, the L stops all further mod_rewrite processing, and the R causes the new URL to be returned as an HTTP 301 response code, meaning a permanent redirect.

The two-part rewriting is required because mod_rewrite doesn’t support examination of query strings inside of a RewriteRule.


# Do Geeklog to alias mapping.
RewriteCond %{QUERY_STRING} story=(\d+)
RewriteRule ^article.php /${geekmap:%1|}? [L,R=301]


So that begs the question: Where is that map called geekmap defined? It can be found In the httpd.conf.


# Add a rewrite map for Geeklog to Drupal
RewriteMap geekmap txt:/etc/apache2/sid-to-alias-rewrite-map.txt


That’s pretty self-explanatory, except for that txt part. It just specifies the simple text-file format for a rewrite map. There are other formats, that can be found in the documentation.

The map file is also pretty simple. It’s just a series of tab seperated lines. The left column is the key, and the right column is the value.


20040727104257410 failed_sandbox_peer_launches_or_something.html
20040808110934404 three_tequila_night.html
20040809092923819 i_dont_even_know_her.html
...


And that’s it! I do similar mappings for the old static page links, as well as the old RSS feed, but those are all much simpler cases. Finally, there’s some other Drupal magic on the backend to convert those long underscored URLs into node ID URLs (such as node/136), as well as further mod_rewrite magic to make the Drupal URLs pretty – but that’s beyond the scope of all this.