Asynchronous Accessibility?

Our current gecko accessibility engine (GAE) happily serves chrome and content information through synchronous desktop accessibility API such as MSAA/IA2. For example a screen reader makes a synchronous IPC call over COM via MSAA/IA2 into our process where we (GAE) grab the information requested and return it (pass it back). Firefox will almost certainly be moving web content rendering out of the chrome process. Communication between chrome and content processes will be done asynchronously because users like their browsers to be responsive. Desktop accessibility API is implicitly synchronous. In addition to the chrome (browser UI) process, content processes contain a lot of juicy information (i.e DOM) that needs to be served through desktop accessibility API. What to do? Hypothesis 1: Forward messages between the chrome and content processes. H1 conclusion: Fail. Google tried this and it is too slow. Hypothesis 2: Cache all the chrome and content information in one process. H2 conclusion: Google is now doing this with some success. If a screen reader is detected, they cache the accessible DOM tree(s) in the main browser process. Cache latency can lead to screen readers getting stale information sometimes, but it is expected that firing desktop update events will mitigate this. Hypothesis 3: Create asynchronous desktop API. H3 conclusion: Yeah, this might be a pipe dream. I am currently thinking about something like H2, but that would allow assistive technology to transition to H3. In terms of pictures, here is my thinking: This minimalist figure shows boxes for chrome, and content processes with asynchronous connections to box that represents an accessibility process called "Async Accessibility" (needs a better name). This box butts up to another box that essentially represents our current gecko accessibility engine, which in turn has a synchronous connection to assistive technology (over desktop API). If we then begin providing desktop access to "Async Accessibility", my hope is that progressive AT could begin using this API when they are ready. As I write this I admit I have serious doubts this would actually happen, but I want to get this thinking out there. This figure shows AT using asynchronous API to access "Async Accessibility" through some as yet undefined API and usage pattern. In some ways H2 and H3 are similar, with H2 being more straightforward. I haven't attempted to estimate the engineering effort for either approach but there is a good chance we'll end up with something closer to H2. It all depends on resourcing and time, and how quickly our desktop firefox moves to multi-process content and how long we might support a single process mode. I'd like to thank Josh Matthews (jdm) for sanity checking this before I posted it. Feedback is welcome. I'm hoping to iterate on this and to ultimately develop a solid implementation plan by mid December. Thanks for reading.

Firefox 4 beta - AT Vender Relief

Success. Earlier this month I posted an alert about the temporary Windows screen reader bustage with our Firefox 4 betas. Since then Marco posted a status update about our temporary and long term fixes. We worked together, along with the screen reader developers to provide 3 solutions. The first is a temporary hack, and the other two are long term fixes and probably generally useful anyway:

1. The "emulation" hack. For known AT that require them, we resurrected the windows that were recently removed from Firefox. We intend to only create these windows for legacy AT so that users don't necessarily need to pay for upgrades for their software to use Firefox 4.
2. The speedy document accessors fix. We added quick methods to get parent and child documents, and indexed access to documents.
3. The QueryService fix. Provide a special service ID for getting the accessible object that represents the browser tab content document that contains the accessible object queried. Here's some sample code to illustrate how you might use this service:

static const GUID SID_IAccessibleContentDocument = {0xa5d8e1f3,0x3571,0x4d8f,0x95,0x21,0x07,0xed,0x28,0xfb,0x07,0x2e};

// Assuming you have a CComPtr<IAccessible> accessible
CComQIPtr<IServiceProvider> serviceProvider = accessible;
CComPtr<IAccessible> accessibleDoc;
serviceProvider->QueryService(SID_IAccessibleContentDocument, IID_IAccessible, ( void**) &accessibleDoc);

If the 'accessible' is in web content then 'accessibleDoc' represents the main content document that contains 'accessible'. It works in all cases we tested, including special documents like about:addons, as well as documents with iframes where we still always return the main content document. This support is in our nightly builds and will come with FF4 beta7. I hope this will make it easier to manage your cache and context switching.

Thanks for reading.

Firefox 4 beta - AT Vendor Alert

Hello friendly accessibility technology vendors and developers.

As of the Firefox 4 beta 5, the child windows associated with browser tabs have been removed thereby breaking the expectations of most windows screen readers. Regretfully, the timing of this change and the substantial impact it has on AT caught the Mozilla accessibility team by surprise. Unfortunately this change is a critical step in our browser's technical roadmap. The good news is that this issue was quickly discovered during our a recent beta cycle and now we can work together on a fix!

Let's fix this the right way. We probably shouldn't have AT relying on window classes to indicate where browser tabs are, however convenient it might have been years ago. I think, depending on how this per browser tab separation has been used in your code, we can provide an even better solution perhaps even allowing some hack removal on your side. We've very recently been in contact with most of the main AT vendors, and some have what sounds like a fix already in hand. We want to make sure we protect all our users from potential bustage, so we want to hear from all accessibility technology developers that use Windows accessibility API and have been relying on per browser tab window classes.

Something to look for... anywhere you have code checking for a specific window class "MozillaContentWindowClass" your code will almost certainly now be broken for FF4 beta5+. Note this class used to correspond to the HWND of the child content window that received focus whenever the top level window gained focus. To see how things look now I recommend Microsoft's UISpy tool. Be sure to run this against the most recent FF 4 beta or nightly.

We have time but must move swiftly. Please contact us directly or on our accessibility community mailing list.

Thanks for reading.

Wanted: Mozilla Linux Accessibility Developer

The Mozilla Corporation is seeking a software developer for an exciting opportunity to hack in our Linux accessibility layer. The successful candidate will make core code additions and improvements to the accessibility engine used by the award winning Firefox browser, and other gecko based applications.

Requirements:
C, C++
Team player with good communication habits

Bonus:
Atk, AT-SPI
Familiarity or fascination with assistive technology (AT)

Scope of work:
6 month contract. Rate negotiable.

Details of work:
Your primary duties will consist of working with the Mozilla team to fix gecko Linux accessibility bugs. This may include performance profiling of Firefox, Thunderbird, at-spi, and AT such as the Orca screen reader against cases where there is user perceivable slowness. You will work with the Mozilla accessibility team and the Linux accessibility community (primarily the Orca team) to prioritize and fix the most critical bugs. You may also be able to help bring the existing Orca + Firefox automated tests to an acceptable level and design and assemble an automated nightly test setup. You may also have the opportunity to collaboratively make improvements to the Linux accessibility infrastructure (at-spi2).

If interested please send a plain text résumé to: the first letter of my first name joined to my entire last name, at mozilla dot com.

Self Voicing Apps and Screen Readers could live together

Screen readers are applications that interrogate the desktop, applications, and events, in order to provide users with an aural rendering.

Self voicing applications already render aurally; and as a user interacts with the application, they control what is spoken.

These worlds can collide, and traditionally folks have been leery of self voicing applications. I think of this in the same way as an application might not follow the visual conventions of a given platform/desktop.

But.

What if there was a standard way for screen readers to hook into the TTS system and find out what text is being queued for speech? Then, if the text came from a self voicing application, the screen reader could pull it from the queue and incorporate it as it sees fit.

So like an application should pay attention to "desktop integration", so too an aural application could pay attention to "screen reader integration" except by somehow allowing screen readers to intercept.

Thoughts?

Thanks for reading.

Mozilla at CSUN 2010

First of all, check out my colleague Marco's recap of CSUN 2010. I'll try add to his report, while avoiding redundancy.

Uhm. Wow. That was rough. Three days of people loving Mozilla.

Meet ups

I landed at CSUN on Thursday afternoon and left on Saturday morning. In the short time I was there I took quite a few notes and looking over them now I see they comprise a long to-do list. I won't bore you with that here. I met with accessibility hackers from such places as GNOME, RIM, Adobe, Novell, Deque Systems, GW Micro, IBM, The Paciello Group, and of course my colleagues from Mozilla, Marco and Alexander. Cheeky Brad Taylor used beer to lure me into an interview. The rest of the to-do list came from interactions with users and suppliers. It was amusing to see how they reacted when I asked them questions like "how can we do better"; gathering it was sort of a refreshing change of pace from the booths that are selling product.

Cloud
No, this bit isn't about accessibility in the cloud. Compared to last year's CSUN, I felt a bit of a dark cloud unfortunately. Dreams of a full open source stack for accessibility have taken a hit recently, with Oracle closing Will Walker's position as GNOME Accessibility Lead, and other funding sources tightening up. Promising Caribou creator/hacker Ben Konrath has been an innocent victim of frozen grant funds within the group that contracted him. This is happening as the GNOME accessibility infrastructure is being ported to d-bus and other exciting and hairy issues loom, such as GNOME 3.0, and GNOME Shell. Novell has been doing more than its share in accessibility, but will that continue? I'd really like to see a coordinated funding strategy among the distros. I can dream can't I?

Sunshine
I am keenly aware of how lucky I am to work at Mozilla. More than ever I am resolved to make the most of it, to seize the days, and to help move the accessibility needle as much as we can. We have a tough road ahead of us with the web moving ever more to visual media, we'll see more usage of canvas, and (hopefully) SVG. We see people excited about video, and captioning. We should continue to support accessibility innovation in web video, to leverage things like javascript access to annotation tracks and sync information. Could be wicked times.

Memorable Moments
In his blog Marco mentioned speaking with a humorous gentleman through a sign language interpreter. Witnessing this, I was deeply moved for some reason.

Dinner with Joanie, the current Orca (Linux screen reader) maintainer, followed by debugging a pesky Orca+Firefox bug.

Finally meeting Charles McCathieNevile, Matt May, Flavio Percoco Premoli, Bryen Yunashko, Mike Gorse, and many others. I hope to see them all again.

Chatting with Steve Faulkner about some shared W3C pain.

Sharing a flight back to Toronto with Greg Fields (RIM accessibility) and planning a UI meetup.

OK that's enough! Sorry. Thanks for reading.

Accessibility Diagnostic Jetpack

I put together a little jetpack that tells you if Firefox's accessibility service is actually running in your browser. It is basically David Baron's "about:accessibilityenabled" extension but as a jetpack. If you want to give it a whirl, strap it on.

You'll need to grab Jetpack if you don't have it already. Alternatively you can just use David Baron's handy about:accessibilityenabled extension.

Aside: we've noticed the gecko accessibility service is being used in some unusual places, for example, by some desktop anti-spyware; leading to unnecessary performance degradation. If you discover that our accessibility service is unexpectedly running in your Firefox session, please consider contacting me so we can figure out why...

Thanks for reading.

Accessible SVG is closer than you think

[Firstly, my apologies for not blogging more often (blame twitter). I have a number of unpublished blogs that I've let get stale so I'm going try not to save unpublished blogs anymore. We'll see how that goes. Anyways, here's one I probably should have written late last year...]

Sometime in early December, Aaron pinged me to discuss the topic of SVG accessibility. This led to a quick Firefox experiment (tweeted here) using the cutting edge HTML5 parser and WAI-ARIA. Note this demo is especially uninteresting if you don't use a screen reader. Long story short, it works and will work even better with a little gecko love. Web developers that have already invested in learning ARIA can apply this knowledge to SVG elements, add some JavaScript, and have a pretty striking and accessible application. Note: please don't look to my experiment for "striking" visuals... that's just not my thing... you have to use your imagination here.

I wonder, given the momentum HTML5 canvas currently has in the web developer playground, if the ability to inline SVG in HTML will breathe new life into SVG? As far as accessibility goes, it is certainly something for which I can hope.

Oh and if someone wants to fix up my demo, feel free, and please drop me a line... and I'll link to yours. I'm a total SVG noob.