Podcast Covering Web Accessibility

I recorded this Accessibility Podcast a few weeks ago for the Google Developer Blog. It focuses on the overall topic of Web Accessibility and covers some of what I have been building with respect to custom clients for Google Services, alongside a broad range of issues around developing usable applications.

Emacspeak Video Demo: Looking Up The Weather

In the first of a sequence of posts demonstrating live emacspeak user interaction, I'll give an overview of what it feels like to use Emacspeak's powerful Web Command Line provided by its rich collection of web gadgets made up of search wizards and smart url templates

Here are a few things to note about this and all the subsequent video demos in this series:

• All demos were recorded using recordmydesktop on an Ubuntu laptop running Linux (Ubuntu Dapper).
• The text-to-speech is produced by the IBM ViaVoice engine, this can now be purchased from a couple of sources in the US and Europe.
• All demos are recorded with the speech rate at which I use Emacspeak. These demos are primarily intended to show the efficiency of the environment, and slowing down the speech would detract significantly from that goal.
• Frequent users of text-to-speech will still find the output comprehensible. For the rest, you can match it up with what you see on the screen ;-)
• Careful listeners will notice some jitter/clipping in the audio --- this appears to be an artifact of using recordmydesktop that I was unable to eliminate.
• In each of these articles, I'll enumerate the actions I take, and their effect. I'll enumearate actual keystrokes, but I suggest readers focus more on the atomic commands being executed, rather than individual keypresses.

Looking Up The Weather

Looking up current weather is a sufficiently common action where browsing to a weather site, typing in ones zip-code and reviewing the resulting page is three steps too many. This is also why most modern desktop environments provide gadgets that display such live information with minimal user intervention. This first video shows Emacspeak smart url RSS Weather From Wundergroundin action.

Here is the RSS Weather Demo. This is an OGG/Theora video and should play with tools like mplayer. The table below shows a brief description of each user action and its effect.

Action	Key	Effect
Access Gadgets	C-e u	Prompts for gadget
Pick Weather Gadget	rss tab	partial input completes to RSS Weather From Wunderground
Pick Default Location	return	CA/SanJose
		Speaks weather forecast
Quit	q	Stops speech, dismisses weather forecast, and speaks current context. In this case the shell becomes current and you hear the relevant information.

A Transcript Of What You Hear

Users who could see the video half of this demo but were not frequent text-to-speech users reported that it would be useful to see a transcript of what is being spoken. I'm revising this entry to have an annotated transcript. Note that the transcript may not be accurate to the last word, since I'm typing it in by hand; however it should give one a sense of what information emacspeak chooses to speak.

Access Gadgets

The spoken prompt is Resource:.

Pick Gadget

You hear me type RS and then you hear Emacspeak completing my input to RSSWeather. Half way through that utterance, I know it's found the right gadget, so I hit enter, and that stops the utterance RSSWeather half-way through and speaks the next prompt. Notice that you also hear auditory icons to indicate that a prompt input area just opened.

Location

With the gadget now selected, you hear the City/State prompt generated by the weather gadget. It also produces a default location of CA/San_Jose. In the demo, you hear the State/City prompt, along with the default value. I hit enter midway through that utterance, again the speech flushes immediately. The gadget now has enough information to do its work; it pulls the RSS feed from Wunderground, converts it to XHTML (using XSLT) and displays the page using Emacs/W3. Once displayed, it starts speak the forecast you see on the screen.

Forecast

Here is a rough transcript of what you hear as the weather forecast is spoken. The forecast starts with the words: Conditions 73 degrees F Partly cloudy 4:50pm PDT August 18 ... This is in a lower-pitched (deeper) voice since it was generated from the title of the corresponding RSS item. Emacspeak uses Aural CSS to produce such audio formatted information. The contents of the item are spoken in the default voice starting with the utterance Temperature 73 degrees F 23 degrees C ... Conditions partly cloudy ... When I've heard enough, I hit q to quit.

Quit

When I quit the weather gadget, a number of things happen.

• You hear auditory icon close-object to indicate the weather forecast being dismissed.
• You hear context information indicating that the Emacs shell buffer has now become current.
• For this context information, you hear it speak the working directory of the shell buffer and the word shell

Emacspeak And GMail

See this article by my Google colleague Srinivas Annam that outlines the availability of GMail Filters from the basic HTML interface. This was the final piece that remained to convince me to use my GMail account for email --- now, keeping the GMail Inbox clean and free of clutter has become a snap.

To go with this, I've added a few smart URL templates to Emacspeak's Web Command Line. Once you've signed in, you can use template GMail Search to type a search term, and find matching mesages. Note that GMail uses CSS class msg to tag the actual contents of a message. You can use this to advantage by hitting e c on a message link, and specifying msg as the class value to filter the message.

At some point I'll add a couple of Emacspeak wizards for creating filters; the present HTML interface is still a bit too click intensive for my liking. But cudos to Srinivas for doing the hard work that lets me discover the pain points in the HTML interface; until now these were completely invisible to me since I couldn't use GMail from the Emacspeak environment.

Emacs/W3 note: Note that signing in to GMail from the main GMail screen defeats W3. An easy work-around, and something that is more efficient anyway is to use the glogin.xml form found in Emacspeak --- use C-e ?/ in Emacspeak to pick that form. Once you're signed in to Google, you can:

• Open your inbox
• Perform searches to find the message you want

AMixer And Emacspeak: Controlling The Sound Card

In the days of OSS, Emacspeak had a nice Emacs-interface to aumix and it is a piece of functionality that I have missed even more under ALSA, given that one can do many more sophisticated tweaks to one's sound-card. Tool alsamixer --- a full-screen terminal application is bewilderingly confusing (at least to me), and amixer though usable required me to go find out how it worked each time I needed to do something new. The final straw came last weekend when I tried to record some Emacspeak demos using recordmydesktop and needed to configure ALSA so that it would capture sound directly from the PCM output, rather than the microphone.

To cut a short story even shorter, I ended up writing an Emacs wrapper around amixer called --- well, you guessed it, amixer. The code is checked in as amixer.el. The Emacspeak keybinding C-e ( formerly used to manipulate aumix is now ALSA-aware and will intelligently default to using the new amixer tool if /usr/bin/amixer is available.

The Web The Way You Want

While working on miscellaneous Web related things including:

• Working on the W3C TAG in trying to understand how HTML TagSoup and well-formed markup might co-exist on the Web --- without the bad perennially driving out the good,
• Thinking about the Web Command Line, and placing various Web gadget/widget technologies in perspective,
• Building Emacs-G-Client --- an Emacs client for Google services,
• Connecting Emacs and Firefox via MozREPL to get the best of both worlds ,
• Refactoring some of the Web-related code in Emacspeak to better reflect the underlying ideas,
• Adding additional Web goodies to Emacspeak including Google Suggest in the Emacs minibuffer,
• And items too numerous to fit in this margin ...

I also wrote a draft chapter on specialized Web browsing. Given the set of things I have been working on, the end result is to point out that given the architecture and underlying design principles of the Web as embodied by HTTP, URIs and HTML, specialized Web browsing is in fact not so specialized after all.

Google Suggest: Minibuffer Completion When Googling

Google Suggest has been available until now as a Firefox extension --- it displays a dynamically generated list of completions as one does Google searches in Firefox. The prefered way of Googling on the Emacspeak audio desktop --- using Emacspeak's Websearch facility available via C-e ? is now Google-Suggest enabled. This means that when doing Google searches via C-e ?g, you can type a partial query, and hit TAB to get a list of possible completions for the query.

Emacspeak WebMarks: Online Bookmarks Using Google

Emacspeak module emacspeak-webmarks adds support for adding, viewing and finding Google Bookmarks. Google Bookmarks allows you to store your bookmarks at Google; this module adds support similar to that provided by Firefox Bookmarklets for Google Bookmarks implemented in JavaScript. Note that this module though relatively small was one of the motivators for the code refactoring described in Web Interaction in Emacspeak.

Usage Tips

• To use Webmarks, you need to be signed into Google.
• Signing in to Google is easily done using the Webform provided as xml-forms/glogin.html in the Emacspeak distribution --- invoke it via key C-e?/.
• In addition, the very first time you use Google Bookmarks, add your first bookmark using form xml-forms/bookmark-add.html. Fill out that form, hit submit, and extract the value the zx parameter returned by the server.
• Use emacs Custom to save the value of the zx parameter as Emacs option emacspeak-webmarks-key.

Web Interaction In Emacspeak

Users running out of SVN will have noticed that the emacspeak codebase has seen a significant number of updates over the last couple of weeks. During this time, I've refactored the Web interaction code in Emacspeak to meet the following goals:

• All Emacspeak Web smarts have now been re-factored to be independent of any given Web browser. In practice, this avoids code duplication between W3 and W3M support.
• The interface to XSLT has been significantly improved.
• When using W3, Emacspeak automatically builds a cache of CSS classnames and ids occuring on the page. These are then used to provide interactive completion when invoking commands emacspeak-extract-by-id (bound to e i in Web pages) and emacspeak-we-extract-by-class (bound to e c in Web pages).
• Module emacspeak-we holds all code related to editing Web pages before they are displayed.
• Module emacspeak-webutils holds all Web utility code.
• Atom and RSS feeds are now pulled using Emacs-22 built-in library url rather than via libxslt. This has the advantage that the feeds are pulled with any cookies the browser may have set during the session.
• All url templates, search wizards and related Web utilities should now be capable of working similarly under W3 and W3M.
• Module emacspeak-moz provides a basic level of integration with Firefox --- see my related blog post on FireBox.
• Module emacspeak-w3m still needs work to take advantage of all of these changes; I myself do not use W3M much, so those updates will happen as and when W3M users contribute the necessary time and patches.

Emacs-G-Client: Leveraging New Picasa API Features

The Google Picasa team announced a set of useful additions to the Picasa Web API yesterday. I've added support for most of these new features in module gphoto that is part of my Emacs G-Client package.

You can get the latest version of package Emacs-G-Client via SVN. Note that this development version of package Emacs-G-Client also includes a light-weight client for finding and playing YouTube videos.

Emacspeak And Beautiful Code

In the fall of 2006, I was invited by O'Reilly Media to participate in an innovative book project called Beautiful Code. The project put together a set of chapters that focused on capturing the collected insight from creating real software. A particularly attractive aspect of this book was that it focused on code --- unlike many book projects in the field of software engineering, here the goal was to explicitly focus on real code and how it could be made beautiful.

The final book is now available in print. The chapter on Emacspeak is being published on this Web site under a Creative Common Licence --- this HTML version includes Chapter 31: Emacspeak --- The Complete Audio Desktop and the Afterword section from the book.

Searching The Emacspeak Knowledge Base

Information about Emacspeak and its use is available from a collection of Web sites, and being able to restrict the search to these authoritative sources is a good way of quickly finding the right document, without being distracted by the numerous hits one finds when doing a search across the whole Web. You can now search the Emacspeak Knowledge Base by using search form emacspeak-search.html; access it via Emacspeak command emacspeak-websearch and specify /to bring up the list of available forms. This form is also the default search form on the Emacspeak Web site.

HowTo: Implement Emacspeak Knowledge Base Search

Implementing the above using CSE is trivial --- all I needed to do was:

1. Checked in an HTML file into the Emacspeak repository at Knowledge Base.
2. Create a CSE that uses the above document.

Emacs G-Client, Reader, And CSE: Searching Past Articles From Google Reader

So I use module greader (part of package G-Client to read a large number of ATOM and RSS feeds. I have long missed the ability to search for articles I remember having read a few weeks or months ago; though I typically find it with an appropriately phrased Google Search, I've always wanted to have the ability to restrict the search to the feeds I subscribe to --- this makes formulating the query much easier.

The advent of CSEs --- see my earlier blog post entitled On The Fly Custom Search combined with Google Reader's ability to export ones subscription list as an OPML file gives me exactly what I needed.

HowTo: Enable Searching Of Past Articles From Google Reader

• Since you need to be signed in to access the subscription list, the CSE magic cannot directly retrieve your subscription list at present. as a consequence, you need to retrieve and save your list of feeds to a location that is publicly accessible. Feature Request: Would be nice if Reader did this automatically by talking to CSE on behalf of the signed-in user --- that would obviate the next manual step.
• Export your subscription list using command greader-opml. You'll need to do an svn update to get the above command; alternatively, sign in to Google and retrieve your subscription list.
• Finally, customize emacspeak option emacspeak-url-template-reading-list-opml to the location of the saved OPML file.

Now, you can use Emacspeak url template reader subscription search from the Emacspeak Web Command Line to search articles you remember having seen in your Google Reader. This is also an excellent means of finding articles of interest that one might have missed in the past. As an example, I recently became interested in Selenium --- an extremely powerful Web application testing framework. Finding articles from the past that I ought to have read but hadn't was a snap using the feature describe here.

Google Books In Emacspeak

To coincide with today's announcement of gull-text access to public domain works from Google Books, I've updated the corresponding emacspeak Web Command-Line wizard. Use C-e u to invoke URL templates and type google books to access it.

Making Search Fly: On-The-Fly Custom Search Engines

The Custom Search Engine team at Google recently released CSE On The Fly a truly amazing feature. Incidentally Google Custom Search is the same piece of magic that brought us Accessible Search last year.

So in the spirit of continuing to enhance the Web Command Line in Emacspeak for every smart Web tool that becomes available, I've checked in two new url templates that demonstrate how one can leverage this to be smart and selective about what one reads.

Searching Favorite Feeds

So I read a lot of Blogs, my current Blog Reader is Google Reader via --- you guessed it -- Emacs module GReader (part of the Emacs G-Client package). But I often feel the need to search my favorite feeds. URL-template Reader Subscriptions lets you do this; what's more it's not specific to Google Reader. All you need do is to publish an OPML file listing your favorite feeds and customize Emacs variable emacspeak-url-template-reading-list-opml to point to that location.

Official GoogleBlog Search

Google has a large number of Google-specific Blogs --- I usually read them through this aggregated feed: All GoogleBlog Stream Emacspeak wizard Official GoogleBlog Search builds a CSE from this feed to let you search articles from all of Google's blogs.

Eventually, I'll also add a meta search wizard that lets one construct any CSE on the fly --- with Lisp such meta-programming is a snap!

Later yesterday evening, I checked in a third url-template called On The Fly CSE that prompts for a search term and the URL for the feed of feeds that specifies the content to search.

Emacs-G-Client: Uploading Photos To PicasaWeb

The SVN version of package Emacs-G-Client at g-client now includes a new module gphoto.el that can create albums and upload photos to Google's PicasaWeb site --- as an example, see my photo gallery. Emacspeak users are likely to find the RSS and ATOM feeds for photo albums more useful in general; package gphoto provides easy access to previewing these feeds as well as viewing/editting the metadata that goes along with the pictures. As an example of such a feed, here is Hubbell Labrador's Graduation album.

Multilingual Dictionary Lookup Via Google

From the every useful Google tool deserves a Web Command -line equivalent ...

I just checked in an Emacspeak url-template for accessing multilingual dictionary lookup via Google. Invoke it like any other url-template using C-e u and type mult tab. Specify the word to look up, and the source-target language pair using two letter codes --- this is analogous to how the Emacspeak Translation Via Google tool works. And remember, if there is something you find yourself doing often on the Web, there is most likely an Emacspeak Web Command Line gadget for it --- well, at least that is true for the things I find myself doing often;-)

FireBox: Put The Fox In The Box

I've finally found the right development environment for myself for writing and debugging Web Applications that use JavaScript to implement client-side interaction. It turns out that it wasn't just me who found the thought of programming inside the Web browser a painful experience --- pleasant though the final end-user interaction that those results deliver might be for the final user. I discovered MozRepl --- a read-eval-print loop for Firefox. MozRepl is a Firefox extension that allows you to open a connection to a running Firefox session and gain access to a JavaScript interpreter context that can access all aspects of the Firefox runtime.

This is quite neat, I can now use the power of Emacs to write and debug end-user JavaScript applications. But wait, there is more. So in general, as someone who doesn't need to suffer from the hit on cycles and memory that running an X environment involves, I usually dont start GDM --- the graphical desktop --- on my Linux box. Believe me, running just at the console, especially with the LCD turned off makes my laptop run a lot longer. So challenge: How do you take the fox's head off Firefox? How do you run a headless Firefox?

Turns out that the original X Windows developers didn't always have access to all the displays that they were developing X applications for --- so they created XVFB --- the X Virtual Frame Buffer server. Like all good things in the Open Source world, XVFB continues to survive --- even though today, X developers hardly if ever resort to XVFB. But in the fine UNIX tradition of Get out of my way or I'll turn you into a shell script XVFB also turns out to be just what I needed in order to run FireFox as a headless application.

So in summary: I'm typing this blog on the shuttle bus riding home, with the monitor turned off, and Firefox running headless as I debug some of the code I've been writing. If you want to put the fox's head in a box yourself, here is a pointer to FireBox -- share and enjoy!

Emacspeak And Beautiful Code

Beautiful Code is a collection of essays on software design, with all proceeds going to Amnesty International. It includes a chapter on the use of Lisp advice to speech-enable Emacs --- AKA Emacspeak. I'll eventually publish an HTML version of my article on the emacspeak Web site. In the meantime, I highly recommend the complete book --- which if you need an accessible version can probably be obtained from organizations like BookShare.

Google Group For Package G-Client

I've created a Google Group for package G-Client here: Emacs-G-Client. If you are using package G-Client you can use this group to discuss your experiences with other users. Note that the codebase for this package is evolving actively under SVN at lisp/g-client .

An Essay On Eyes-Free Computing

I just posted an essay on eyes-free computing to my MathZomeblog. This essay highlights the relevance of ZomeTool in teaching mathematical concepts to students who are visually impaired. More generally, it describes my experiences as a mathematician who cannot see. I'm posting the abstract here; the complete essay can be found on my Web site.

The experiences described in this essay have influenced the software I have built and use on a daily basis; it should be of interest to:

• Emacspeak users wishing to understand why things look like the way they do in Emacspeak.
• Students with visual impairments who are entering the field of mathematics.
• Teachers working with visually impaired students.
• And the generally curious mathematician who wishes to view the world from a different perspective.

Abstract

This essay outlines some of my experiences as a mathematician who cannot see. Note that I transitioned to being a Computer Scientist during Graduate School. However I strongly believe in the edict Once a mathematician, always a mathematician! — my training in mathematics continues to influence the way I think.

I've been unable to see since the age of 14, which means that I've studied and practiced mathematics predominantly in an eyes-free environment. This essay is my first conscious attempt at asking the question What is involved in doing mathematics when you cannot see? I hope that some of the experiences outlined here will prove insightful to mathematicians at large. At its heart, mathematics is about understanding the underlying structure inherent in a given area of interest — and where no such structure exists — to define the minimal structure that is needed to make forward progress.

The general perception that mathematics might be hard to do in an eyes-free environment probably traces itself to the common view of mathematics as a field where one performs copious calculations on paper. I'll illustrate some of the habits and abilities one evolves over time to compensate for the lack of ready access to scratch memory provided by pencil and paper when working in an eyes-free environment. In this essay, I hope to demonstrate that mathematics in its essence is something far bigger. By being bigger than calculations on paper, not being able to see rarely if ever proves an obstacle when it comes to doing mathematics; the challenges one needs to overcome are primarily centered around gaining access to mathematical material, and communicating ones insights with fellow mathematicians. Thus, a large portion of this essay focuses on solutions to the challenges inherent in mathematical communication.