# Audio Deja Vu: Audio Formatted Math On The Emacspeak Desktop

## 1 Overview

This article previews a new feature in the next Emacspeak release —

audio-formatted Mathematics using Aural CSS. Volker Sorge worked

at Google as a Visiting Scientist from Sep 2012 to August 2013, when

we implemented math

access in ChromeVox — see this brief overview. Since leaving

Google, Volker has refactored and extended his work to create an Open

Source Speech-Rule-Engine implemented using NodeJS. This

speech-rule-engine can be used in many different environments;

Emacspeak leverages that work to enable audio-formatting and

interactive browsing of math content.

## 2 Overview Of Functionality

Math access on the Emacspeak desktop is implemented via module

emacspeak-maths.el — see js/node/Readme.org in the Emacspeak GitHub

repository for setup instructions.

Once loaded, module `emacspeak-maths`

provides a *Math Navigator* that

implements the user interface for sending Math expressions to the

Speech-Rule-Engine, and for interactively browsing the resulting

structure. At each step of the interaction, Emacspeak receives math

expressions that have been annotated with Aural CSS and produces

audio-formatted output. The audio-formatted text can itself be

navigated in a special * Spoken Math* emacs buffer.

Module `emacspeak-maths.el`

implements various affordances for

dispatching mathematical content to the Speech-Rule-Engine — see

usage examples in the next section.

## 3 Usage Examples

### 3.1 The Emacspeak Maths Navigator

- The
*maths navigator*can be invoked by pressing S-SPC (hold

down Windows key and press SPC) — this runs the command emacspeak-maths-navigator/body. - Once invoked, the /Maths Navigator can be used to enter an

expression to read. - Pressing SPC again prompts for the LaTeX math expression.
- Pressing RET guesses the expression to read from the current context.
- The arrow keys navigate the expression being read.
- Pressing o switches to the
buffer and exits the**Spoken Math**

navigator.

See the relevant chapter in the online Emacspeak manual for details.

### 3.2 Math Content In LaTeX Documents

- Open a LaTeX document containing math content.
- Move point to a line containing mathematical markup.
- Press S-SPC RET to have that expression audio-formatted.
- Use arrow keys to navigate the resulting structure.
- Press any other key to exit the navigator.

### 3.3 Math Content On Wikipedia

- Open a Wikipedia page in the Emacs Web Wowser (EWW) that has

mathematical content. - Wikipedia displays math as images, with the alt-text giving the

LaTeX representation. - Navigate to some math content on the page, then press S-SPC

a to speak that content — a is for alt. - As an example, navigate to Wikipedia Math Example, locate math expressions on that page, then

press S-SPC a.

### 3.4 Math Content From The Emacs Calculator

- The built-in Emacs Calculator (
`calc`

) provides many complex

math functions including symbolic algebra. - For my personal
`calc`

setup, see tvr/calc-prepare.el in the

Emacspeak GitHub repo. - This setting below sets up the Emacs Calculator to output results

as LaTeX: (setq calc-language 'tex) - With the above setting in effect, launch the emacs Calculator by

pressing M-##. - Press ' — to use algebraic mode — and enter
`sin(x)`

. - Press a t to get the Taylor series expansion of the above

expression, and press x when prompted for the variable. - This displays the Taylor Series expansion up to the desired

number of terms — try 7 terms. - Now, with Calc having shown the results as TeX, press S-SPC

RET to browse this expression using the*Maths Navigator*.

## 4 And The Best Is Yet To Come

This is intentionally called an *early preview* because there is still

much that can be improved:

- Enhance the rule engine to infer and convey more semantics.
- Improved audio formatting rules to better present the available information.
- Update/tune the use of Aural CSS properties to best leverage

today's TTS engines. - Integrate math-reading functionality into more usage contexts in

addition to the ones enumerated in this article.

## 5 References

- Youtube Video from early 2013 demonstrating Math Access in Chrome
- AllThings Digital outlining math access — published June 2013.
- Assets 2016 publication describing this work.
- js/node/aster-math-examples.tex Collection of math examples in

LaTeX from AsTeR. Used to progressively improve speech-rules and

the resulting audio-formatted output - Speech-Rule-Engine on github.
- Speech-Rule-Engine in action: Accessible Maths in all browsers

Date: 2017-02-08 Wed 00:00