W4A Keynote: Cloud Computing And Equal Access For All

I'll be giving the opening keynote at the upcoming W4A 2008 conference in Beijing on April 21. You can find an online version of the paper along with the slides here: Cloud Computing And Equal Access For All. Coincidentally, another excellent Web 2.0 accessibility event is happening on the same day in London --- see Accessibility 2.0 --- it's unfortunate one cannot be in multiple places at opposite corners of the globe at the same time!

Emacspeak Goes Social

Leveraging The Benefits Of Free Speech!

For Immediate Release:
April 1, 2008

Live From San Jose ... Emacspeak Goes Social!

Investors and users alike welcomed today's announcement that Emacspeak (NASDOG:ESPK) would be going social --- Going Social is better than Going Postal!

As a pioneer in the space of eyes-free information access, and a firm believer in free speech and free software, Emacspeak will now help users go social speech-free --- all users need do is to use the system. When in use, the free-social features of Emacspeak will talk to others on your behalf, answer inane questions, and contribute to the community by in its turn asking even more inane questions of everyone else. In a repeat of the network effect that has led to the resounding success of systems like the World Wide Web and The Blogosphere, these viral features in Emacspeak are expected to win ones running instance many social connections. The longer one uses these features, the deeper one's social graph --- going forward, the information encapsulated in these social graphs will be converted to ever-increasing stacks of small pieces of green paper.

Coming Soon!

As these features are launched over the next few weeks, expect Emacspeak generated conversation streams to show up everywhere ranging from Twitter streams to random email messages that you can usefully use to forward to spammers. This innovative approach to communication finally adds value to spam --- and is being hailed as the next biggest business model to hit the ether. By making such content available on the Internet, the system will foster the long term human goal of organizing and searching all the world's ignorance to make it universally accessible --- thereby bringing ignorance on par with knowledge!

My Web-2.0 Application Is Feeling Accessible

If you feel up to Web hackery and want to win a cool T-shirt in the bargain, see My Web-2.0 Application Is Feeling Accessible!. You can see examples of what you can achieve with this framework in the AxsJAX showcase.

Emacspeak WebSpace --- Interaction-Free Information Access

A few months ago, I started an Emacspeak module called emacspeak-webspace that is now ready for wider use. The goal of this module is to unobtrusively fetch useful information from the Web and communicate it at those times that one is context-switching among tasks. I gave a talk on user interaction at the last Hackers Conference in November; in the same session, there was another talk whose gist was a plea for less human-computer interaction --- motivation: User Interfaces are nice, but wouldn't it be nice if one didn't have to explicitly interact with the machine to get information? The speaker coined the term Zen interfaces in that context, something that stuck in memory long after the talk.

I built that thought into module emacspeak-webspace. Conceptually, it consists of smart fetchers that fetch information asynchronously from the Web, and smart displayers that communicate this information at appropriate times. These are detailed below.

Fetchers

There are two fetchers at present:

Weather

Fetches current weather conditions for your location.

News

Fetches headlines from a customizable collection of ATOM and RSS feeds.

Note that this module is not intended to be an RSS or ATOM feed-reader; for that, use module greader --- an API-based Google Reader client that is bundled with Emacspeak.

Communicating Useful Information Usefully

With the information in hand, the next question is how does one communicate this information usefully, and what does at the appropriate time mean? Things to avoid:

Interaction-Free

Do not require explicit user action to hear the information.

Avoid Chatter

Avoid creating an auditory user interface that chatters at the user all the time.

These are conflicting constraints. Notice that in a visual interface, one can meet the interaction-free requirement by displaying the information in a toolbar or sidebar and allow the user to ignore or absorb the information at will.

Emacspeak uses Emacs' header-line to display the continuously updating information. This meets the interaction-free requirement. The header line updates every time Emacs updates its display, and automatically speaking it would produce too much feedback. But Emacspeak doesn't automatically speak the header-line; it only speaks it when there is a context-switch.

How To Use

Here is how I am using emacspeak-webspace at present:

Weather

Activate weather display in the calendar and scratch buffers.

News

Activate feed headlines in selected shell buffers.

You hear the updated information when switching to buffers where the webspace display is active.

Activating WebSpace Displays

Webspace displays are activated via the following commands; all Webspace displays will be placed by default on prefix key hyper-space

• emacspeak-webspace-headlines: hyper-space h
• emacspeak-webspace-weather hyper-space w

Share And Enjoy, And May The Source Be With You!

Announcing: The Coming Of Piglets To The Emacspeak Desktop

This is to announce a new emacspeak module called Piglets that brings together Emacs and Firefox to create a powerful framework for authoring Web interaction wizards.

Why Piglets?

You might well ask Why Piglets?, and might conjecture that PIGLETS might stand for Powerful Internet Gadgets for a Light-Weight Talking System. You might conjecture that the Emacspeak mascot likes pig-ears; or you might even think of attributing it to the fact that my friend and colleague Charles Chen and creator of Fire Vox was born in the year of the pig. But you'd be mostly wrong in all of the above.

Piglets on the Emacspeak desktop are the result of having two large (and powerful) software pigs connect over a socket. A few months ago, I blogged here about MozREPL and how it allows me to Put The Fox In A Box. Piglets mark the completion of the Emacs/Firefox integration that started with Firebox. Once you install Fire Vox, the free self-voicing extension for Firefox, piglets become a versatile means to leverage the self-voicing Fire-Vox/Firefox DOM from the comfort of the emacspeak environment.

What You Need

Caveat: ALL of this is early experimental software --- and you'll need to tweak things for your environment to get things working.

• A version of Emacspeak from the subversion repository.
• Check the installation of the servers/python files in your Emacspeak installation.
• Confirm that the HTTP wrapper for the TTS servers works. You can most easily do this by running:

  # Start the HTTPspeech server 
  # from emacspeak/servers/python
  python HTTPSpeaker.py  outloud 2222 &
  # you can replace outloud with dtk-exp
  # but the bindings to other TTS  servers is not defined for now
  # Run wget to test the speech server:
  wget --post-data='speak:hello world' localhost:2222
  

  If all is well, you should here the TTS engine say Hello World

  Do not proceed if the above does not work.

• Install Fire-Vox and configure it to use the Emacspeak TTS server. You can do this with ORCA providing the feedback. Alternatively, once you have installed Fire-Vox, shutdown Firefox and then edit your ~/.mozilla/firefox/default/prefs.js

  user_pref("firevox.LastWorkingTTS", 4);
  

• The above sets up Fire-Vox to use the running HTTP speech server you started earlier.

Loading And Running Piglets

The Piglets framework is implemented in module emacspeak-piglets.el. There is a Fire-Vox binding in module emacspeak-firevox.el and a binding to the JawBreaker game in emacspeak-jawbreaker.el.

How Does It Work?

When you get the various pieces configured and working, here is how things work:

• Piglets place you in a special interaction buffer in Emacs.
• Typing keys in this buffer go to Firefox.
• Control keys send commands to Firefox using MozREPL.
• The latter is most useful in conjunction with Web 2.0 applications that have been AxsJAXed.
• Additional commands give access to FireFox features such as the URL bar. As an example, hit C-w and type the phrase you'd like to go into the address bar --- either a URL or a search term. As an example, try typing XKCDComic. This will automatically do a Google Lucky Search (thanks to FireFox magic) and open the XKCD site.
• But wait, there's more! Because XKCD has been AxsJAXed, you will hear Fire-Vox automatically speak the comic strip and its associated transcript. Hit ? in the FireFox interaction buffer to hear the available keystrokes for this AxsJAXed application; in general, ? brings up keyboard help for AxsJAXed applications.

ToDos:

These are some todos that I plan to get to eventually --- if you have coding cycles to contribute, feel free to work on these.

• Create an HTTP binding to the TTS servers using TCL and the TCL HTTP libraries. This will eliminate the dependency on the Python wrapper I originally wrote for ORCA in fall 2005.
• Write more Piglets.
• Make installation and configuration simpler.
• Test installation and configuration of the various pieces.

Web Accessibility And Usability: Coming Back To The Basics

The Nielsen Norman Group has made available a detailed report on accessibility, which includes the results of several usability tests --- see Going Beyond Alt Text. It's a very good read, and though its conclusions might be depressing to people coming to this area from the outside --- they should be no surprize to users who have been trying to use the Web via spoken output over the last 10 years. From the perspective of the Emacspeak user who lives in a specialized browsing environment that is optimized for performing oft-repeated tasks, there are several interesting take-aways from this report:

• Though so-called Web Accessibility Standards have attempted to focus on the behavior observed when using screenreaders with mainstream browsers, that thread of work appears to be achieving little with respect to the real metric of task completion. As technologists, we would all do well to remember that users come to the Web, and Web Access solutions not to use the browser but rather to complete one or other task.
• As described in Specialized Browsers and The Web The Way You Want It, task-oriented access and specialized user-optimized web tools have been around since the inception of the Web.
• Though the Nielsen study asked users to carry out each of the given tasks by going to a given Web site, it would be interesting to see how such tasks work out in the Emacspeak environment. It's a given that an emacspeak user trying to buy a music CD at an online store would run into a brick wall fairly quickly (see, even online stores are made of brick and mortar;-)). However, as an Emacspeak user I'd never do that I'd either go to Amazon's highly efficient Amazon Accessible Store or faster yet, type an appropriate query at Google and click on the relevant Ad that sports a Google Checkout badge.
• And more interestingly, it would be interesting to carry out a follow-up user study to compare the rate of task completion as well as observed efficiencies/inefficiencies between users of Emacspeak and generic browser/screenreader combinations for tasks such as:
  1. Play NPR news from the last hour.
  2. Play your local NPR station.
  3. Play BBC News from the last hour.
  4. Skim the top stories from CNN.
  5. Look up today's stock market numbers for the major indices.
  6. And items too numerous to enumerate in this margin.

Directions Using Public Transport From Google Maps

In the spirit of adding a smart URL template to enhance the Emacspeak Web Command Line for useful tools I discover ...

I just checked in a Public Transit Via Google Maps tool into the Emacspeak SVN repository. You specify trip details in the form start to destination e.g., 2715 La Terrace Circle 95123 to San Jose Airport and get back a filtered view that shows the information you want. Note that clicking on the link in this blog will give you the entire page, which is fairly easy to navigate. But having a smart filter in Emacspeak makes it that much more efficient to use.

Share And Enjoy,

Announcing Emacspeak 27.0 AKA FastDog

For Immediate Release

San Jose, CA, (Nov. 24, 2007)
Emacspeak: --- Bringing Cutteng-Edge Access For Sharp Users
--Zero cost of upgrade/downgrades makes priceless software affordable!

Downloads	Reference	Installation	Usage	Tips	Tools	Support
About the author

Emacspeak Inc (NASDOG: ESPK) announces the immediate world-wide availability of Emacspeak-27 --a powerful audio desktop for leveraging today's evolving data and service-oriented semantic Web.

Major Enhancements

1. emacspeak-web: Updated Web interaction.
2. emacspeak-ess: Speech-enables Emacs Statistics Interface
3. Header Line Support:
4. Windows Key Now Stops Speech
5. Smarter mode-line output:
6. Google Suggest provides completion for search queries.
7. Integrated Support For Google Services
8. emacspeak-moz: Firefox integration.
9. Emacspeak-webmarks: Online bookmarks.

And others too numerous for this margin ...

AxsJAX, Speech-Enabled Games And Auditory User Interfaces

This is not part of Emacspeak, but is relevant to emacspeak users given that:

• Speech-enabled games have proven to be a successful means of discovering AUI techniques that work, beginning with the speech-enabled version of Tetris.
• The emerging Emacs to Firefox connection using module emacspeak-moz.el and mozrepl
• The ability to run Fire Vox with the Emacspeak speech servers --- look for a blog post on that soon.

Charles Chen and I recently released a JavaScript powered framework for access-enabling AJAX applications --- see AxsJAX. Along with access-enabling useful applications such as Google WebSearch and Google Reader, we also access-enabled JawBreaker, a popular game much in the spirit of Tetris --- but without a ticking clock. See AxsJAX showcase for pointer to this and other Web-2.0 applications that have been AxsJaxed.

The Web The Way You Want

I gave a talk last week at University of Washington entitled The Web The Way You Want; it should be of general interest to users interested in flexible access to the Web.

Podcast Covering Web Accessibility

I recorded this Accessibility Podcast a few weeks ago for the Google Developer Blog. It focuses on the overall topic of Web Accessibility and covers some of what I have been building with respect to custom clients for Google Services, alongside a broad range of issues around developing usable applications.

Emacspeak Video Demo: Looking Up The Weather

In the first of a sequence of posts demonstrating live emacspeak user interaction, I'll give an overview of what it feels like to use Emacspeak's powerful Web Command Line provided by its rich collection of web gadgets made up of search wizards and smart url templates

Here are a few things to note about this and all the subsequent video demos in this series:

• All demos were recorded using recordmydesktop on an Ubuntu laptop running Linux (Ubuntu Dapper).
• The text-to-speech is produced by the IBM ViaVoice engine, this can now be purchased from a couple of sources in the US and Europe.
• All demos are recorded with the speech rate at which I use Emacspeak. These demos are primarily intended to show the efficiency of the environment, and slowing down the speech would detract significantly from that goal.
• Frequent users of text-to-speech will still find the output comprehensible. For the rest, you can match it up with what you see on the screen ;-)
• Careful listeners will notice some jitter/clipping in the audio --- this appears to be an artifact of using recordmydesktop that I was unable to eliminate.
• In each of these articles, I'll enumerate the actions I take, and their effect. I'll enumearate actual keystrokes, but I suggest readers focus more on the atomic commands being executed, rather than individual keypresses.

Looking Up The Weather

Looking up current weather is a sufficiently common action where browsing to a weather site, typing in ones zip-code and reviewing the resulting page is three steps too many. This is also why most modern desktop environments provide gadgets that display such live information with minimal user intervention. This first video shows Emacspeak smart url RSS Weather From Wundergroundin action.

Here is the RSS Weather Demo. This is an OGG/Theora video and should play with tools like mplayer. The table below shows a brief description of each user action and its effect.

Action	Key	Effect
Access Gadgets	C-e u	Prompts for gadget
Pick Weather Gadget	rss tab	partial input completes to RSS Weather From Wunderground
Pick Default Location	return	CA/SanJose
		Speaks weather forecast
Quit	q	Stops speech, dismisses weather forecast, and speaks current context. In this case the shell becomes current and you hear the relevant information.

A Transcript Of What You Hear

Users who could see the video half of this demo but were not frequent text-to-speech users reported that it would be useful to see a transcript of what is being spoken. I'm revising this entry to have an annotated transcript. Note that the transcript may not be accurate to the last word, since I'm typing it in by hand; however it should give one a sense of what information emacspeak chooses to speak.

Access Gadgets

The spoken prompt is Resource:.

Pick Gadget

You hear me type RS and then you hear Emacspeak completing my input to RSSWeather. Half way through that utterance, I know it's found the right gadget, so I hit enter, and that stops the utterance RSSWeather half-way through and speaks the next prompt. Notice that you also hear auditory icons to indicate that a prompt input area just opened.

Location

With the gadget now selected, you hear the City/State prompt generated by the weather gadget. It also produces a default location of CA/San_Jose. In the demo, you hear the State/City prompt, along with the default value. I hit enter midway through that utterance, again the speech flushes immediately. The gadget now has enough information to do its work; it pulls the RSS feed from Wunderground, converts it to XHTML (using XSLT) and displays the page using Emacs/W3. Once displayed, it starts speak the forecast you see on the screen.

Forecast

Here is a rough transcript of what you hear as the weather forecast is spoken. The forecast starts with the words: Conditions 73 degrees F Partly cloudy 4:50pm PDT August 18 ... This is in a lower-pitched (deeper) voice since it was generated from the title of the corresponding RSS item. Emacspeak uses Aural CSS to produce such audio formatted information. The contents of the item are spoken in the default voice starting with the utterance Temperature 73 degrees F 23 degrees C ... Conditions partly cloudy ... When I've heard enough, I hit q to quit.

Quit

When I quit the weather gadget, a number of things happen.

• You hear auditory icon close-object to indicate the weather forecast being dismissed.
• You hear context information indicating that the Emacs shell buffer has now become current.
• For this context information, you hear it speak the working directory of the shell buffer and the word shell

Emacspeak And GMail

See this article by my Google colleague Srinivas Annam that outlines the availability of GMail Filters from the basic HTML interface. This was the final piece that remained to convince me to use my GMail account for email --- now, keeping the GMail Inbox clean and free of clutter has become a snap.

To go with this, I've added a few smart URL templates to Emacspeak's Web Command Line. Once you've signed in, you can use template GMail Search to type a search term, and find matching mesages. Note that GMail uses CSS class msg to tag the actual contents of a message. You can use this to advantage by hitting e c on a message link, and specifying msg as the class value to filter the message.

At some point I'll add a couple of Emacspeak wizards for creating filters; the present HTML interface is still a bit too click intensive for my liking. But cudos to Srinivas for doing the hard work that lets me discover the pain points in the HTML interface; until now these were completely invisible to me since I couldn't use GMail from the Emacspeak environment.

Emacs/W3 note: Note that signing in to GMail from the main GMail screen defeats W3. An easy work-around, and something that is more efficient anyway is to use the glogin.xml form found in Emacspeak --- use C-e ?/ in Emacspeak to pick that form. Once you're signed in to Google, you can:

• Open your inbox
• Perform searches to find the message you want

AMixer And Emacspeak: Controlling The Sound Card

In the days of OSS, Emacspeak had a nice Emacs-interface to aumix and it is a piece of functionality that I have missed even more under ALSA, given that one can do many more sophisticated tweaks to one's sound-card. Tool alsamixer --- a full-screen terminal application is bewilderingly confusing (at least to me), and amixer though usable required me to go find out how it worked each time I needed to do something new. The final straw came last weekend when I tried to record some Emacspeak demos using recordmydesktop and needed to configure ALSA so that it would capture sound directly from the PCM output, rather than the microphone.

To cut a short story even shorter, I ended up writing an Emacs wrapper around amixer called --- well, you guessed it, amixer. The code is checked in as amixer.el. The Emacspeak keybinding C-e ( formerly used to manipulate aumix is now ALSA-aware and will intelligently default to using the new amixer tool if /usr/bin/amixer is available.

The Web The Way You Want

While working on miscellaneous Web related things including:

• Working on the W3C TAG in trying to understand how HTML TagSoup and well-formed markup might co-exist on the Web --- without the bad perennially driving out the good,
• Thinking about the Web Command Line, and placing various Web gadget/widget technologies in perspective,
• Building Emacs-G-Client --- an Emacs client for Google services,
• Connecting Emacs and Firefox via MozREPL to get the best of both worlds ,
• Refactoring some of the Web-related code in Emacspeak to better reflect the underlying ideas,
• Adding additional Web goodies to Emacspeak including Google Suggest in the Emacs minibuffer,
• And items too numerous to fit in this margin ...

I also wrote a draft chapter on specialized Web browsing. Given the set of things I have been working on, the end result is to point out that given the architecture and underlying design principles of the Web as embodied by HTTP, URIs and HTML, specialized Web browsing is in fact not so specialized after all.

Google Suggest: Minibuffer Completion When Googling

Google Suggest has been available until now as a Firefox extension --- it displays a dynamically generated list of completions as one does Google searches in Firefox. The prefered way of Googling on the Emacspeak audio desktop --- using Emacspeak's Websearch facility available via C-e ? is now Google-Suggest enabled. This means that when doing Google searches via C-e ?g, you can type a partial query, and hit TAB to get a list of possible completions for the query.

Emacspeak WebMarks: Online Bookmarks Using Google

Emacspeak module emacspeak-webmarks adds support for adding, viewing and finding Google Bookmarks. Google Bookmarks allows you to store your bookmarks at Google; this module adds support similar to that provided by Firefox Bookmarklets for Google Bookmarks implemented in JavaScript. Note that this module though relatively small was one of the motivators for the code refactoring described in Web Interaction in Emacspeak.

Usage Tips

• To use Webmarks, you need to be signed into Google.
• Signing in to Google is easily done using the Webform provided as xml-forms/glogin.html in the Emacspeak distribution --- invoke it via key C-e?/.
• In addition, the very first time you use Google Bookmarks, add your first bookmark using form xml-forms/bookmark-add.html. Fill out that form, hit submit, and extract the value the zx parameter returned by the server.
• Use emacs Custom to save the value of the zx parameter as Emacs option emacspeak-webmarks-key.

Web Interaction In Emacspeak

Users running out of SVN will have noticed that the emacspeak codebase has seen a significant number of updates over the last couple of weeks. During this time, I've refactored the Web interaction code in Emacspeak to meet the following goals:

• All Emacspeak Web smarts have now been re-factored to be independent of any given Web browser. In practice, this avoids code duplication between W3 and W3M support.
• The interface to XSLT has been significantly improved.
• When using W3, Emacspeak automatically builds a cache of CSS classnames and ids occuring on the page. These are then used to provide interactive completion when invoking commands emacspeak-extract-by-id (bound to e i in Web pages) and emacspeak-we-extract-by-class (bound to e c in Web pages).
• Module emacspeak-we holds all code related to editing Web pages before they are displayed.
• Module emacspeak-webutils holds all Web utility code.
• Atom and RSS feeds are now pulled using Emacs-22 built-in library url rather than via libxslt. This has the advantage that the feeds are pulled with any cookies the browser may have set during the session.
• All url templates, search wizards and related Web utilities should now be capable of working similarly under W3 and W3M.
• Module emacspeak-moz provides a basic level of integration with Firefox --- see my related blog post on FireBox.
• Module emacspeak-w3m still needs work to take advantage of all of these changes; I myself do not use W3M much, so those updates will happen as and when W3M users contribute the necessary time and patches.

Emacs-G-Client: Leveraging New Picasa API Features

The Google Picasa team announced a set of useful additions to the Picasa Web API yesterday. I've added support for most of these new features in module gphoto that is part of my Emacs G-Client package.

You can get the latest version of package Emacs-G-Client via SVN. Note that this development version of package Emacs-G-Client also includes a light-weight client for finding and playing YouTube videos.

Emacspeak And Beautiful Code

In the fall of 2006, I was invited by O'Reilly Media to participate in an innovative book project called Beautiful Code. The project put together a set of chapters that focused on capturing the collected insight from creating real software. A particularly attractive aspect of this book was that it focused on code --- unlike many book projects in the field of software engineering, here the goal was to explicitly focus on real code and how it could be made beautiful.

The final book is now available in print. The chapter on Emacspeak is being published on this Web site under a Creative Common Licence --- this HTML version includes Chapter 31: Emacspeak --- The Complete Audio Desktop and the Afterword section from the book.