Saturday, April 22, 2017

Mail On The emacspeak Audio Desktop

Email On The Emacspeak Audio Desktop

1 Overview

This question comes up every few months on the emacspeak mailing
list. In general, see
Emacspeak Tools to quickly discover available speech-enabled
applications. This article outlines some of the available email setups
given the wide degree of variance in this space.

2 Background

How one puts together an email environment is a function of the

  1. How email is retrieved.
  2. How email is stored (if storing locally).
  3. How email is sent.

Here is an overview of what is available as viewed from the world of
Linux in general and Emacs in particular:

2.1 Email Retrieval

Email can be retrieved in a number of ways:

  • IMap via Emacs This is implemented well in GNUS, and poorly in
    Emacs/VM. Note that Emacs is single-threaded, and fetching large
    volumes of email via IMap is painful.
  • Batch Retrieval: IMap Tools like fetchmail, offlineimap and friends that live
    outside of Emacs can be used to batch-retrieve email in the
    background. The retrieved mail gets delivered locally as in the past.
  • Mail Filtering: UNIX procmail enables filtering of locally
    delivered email into separate folders for automatically organizing
    incoming email.

2.2 Sending Email

Sending email involves:

  1. Composing email — typically invoked via key-sequence C-x m
    (command: compose-mail). Emacs email packages implement
    specific versions of this command, e.g. vm-mail from package
    emacs/vm, message-mail from the message package etc.
  2. Sending email: This is specific to the email provider being used,
    e.g., GMail. In the past, UNIX machines could talk SMTP to
    the Mail Gateway, but this has mostly disappeared over time. For
    an example of how to configure Emacs to send email via GMail
    using SMTP , see file tvr/gm-smtp.el in the emacspeak repository.

2.3 Local Storage Format

  • UNIX Mail: An email folder is a file of messages. This
    format is used by clients like Emacs/VM, UNIX Mail etc.
  • Maildir: A mail folder is a directory, with
    individual email messages living in files of their
    own. Sample clients include MH-E (UNIX MH), MU4E.
  • RMail This is Emacs' original email format.

3 Putting It All Together

The next sections show my present email setup put together using the
building blocks described above.

  1. I use Linux on all my machines, and Android on my phone.
  2. I mostly limit email usage on my phone to get a quick overview of email that might require immediate attention — toward this end, I have a to-mobile GMail label that collects urgent messages.
  3. Linux is where I handle email in volume.
  4. I use my Inbox as

my ToDo list — which means that I leave little or no email in my
Inbox unless I'm on vacation and disconnected from email.

3.1 Desktop: Batch Retrieval And Emacs/VM

This is the email setup on my workstation. See next section for the
email setup while mobile.

  1. I batch-retrieve email using fetchmail.
  2. This email gets filtered through procmail and auto-filed into
    several folders based on a set of procmail rules. Typical rules
    include separating out various email lists into their respective folders.
  3. Note that this does not preclude using IMap via GNUS to read
    email while online.
  4. Email that is not filtered into separate folders e.g. email that
    is sent directly to me, email regarding projects that need
    immediate attention etc., land up in folder ~/mbox.
  5. So when I launch emacs/vm on my desktop, the above is all I
    need to deal with at any given moment.
  6. I typically read Auto-filed mailing lists using emacs/vm about once a day or
    less — I use package mspools to get a quick overview of the
    state of those mail folders.

3.2 Mobile AccessOn Laptop: GNUS And IMap

See gnus-prepare.el for my gnus configuration for accessing GMail
via imap. That configuration is setup to access multiple GMail accounts.

  1. I see each GMail label as a separate group in GNUS.
  2. I only sync high-priority labels — this works well even
    over slow WIFI connections while on the road. As an example, the
    afore-mentioned to-mobile GMail label is a high-priority group.
  3. Module gm-nnir defines a GNUS/GMail extension that enables
    one to search GMail using GMail's search operators — that is my
    prefered means of quickly finding email messages using
    search. This is very fast since the search happens server-side,
    and only email headers are retrieved when displaying the search
  4. Note that this solution is not laptop/mobile specific — I use
    this setup for searching GMail from my desktop as well.

3.3 Composing And Sending EMail

  1. I use compose-mail to compose email.
  2. I optionally activate orgtbl-mode and/or orgstruct-mode if
    editing structured content within the email body.
  3. I send email out using the setup in gm-smtp.el.

4 Conclusion

  1. Email in Linux/Emacs is composed of a set of
    independent building blocks — this gives maximal flexibility.
  2. That flexibility allows one to put together different email
    workflows depending on the connectivity environment in use.

Wednesday, March 01, 2017

Emacs: Check Interactive Call For Emacspeak

Emacs: Check Interactive Call For Emacspeak

1 Background

Emacspeak uses advice as the means to speech-enable Emacs.
Emacspeak's advice forms need to check if the function being
speech-enabled is being called interactively — otherwise one would
get a lot of chatter as these functions get called from within elisp
programs, e.g. functions like forward-sexp or kill-sexp, that play
the dual role of both an interactive command, as well as a convenient
elisp function.

Until Emacs 24, the solution used was to write code that did the
following check:

(when (interactive-p) ...

In Emacs-24, interactive-p was made obsolete and replaced with

(called-interactively-p 'interactive)

Emacspeak initially used the above form to perform the equivalent
check. However, around the same time, Emacs' advice implementation
went through some changes, and there was an attempt to replace
advice.el with nadvice.el.

At the end of that round of changes, some problems emerged with the
new called-interactively-p implementation; specifically, calling
:called-interactively-p_ within around advice forms resulted in hard
to debug errors, including one case of infinite recursion involving
library smie.el when invoked from within ruby-mode.

After studying the problem in depth in 2014, I decided to create an
Emacspeak-specific implementation of the is-interactive check.

The resulting implementation has worked well for the last 30 months;
this article is here mostly to document how it works, and the reason
for its existence. Note that Emacspeak uses this custom predicate
only within advice forms. Further, this predicate has been coded
to only work within advice forms created by emacspeak. This
constraint can likely be relaxed, but the tighter implementation is
less risky.

2 Implementation — ems-interactive-p

2.1 Overview

Within an advice forms defined by Emacspeak, detect if the enclosing
function call is the result of explicit user interaction, i.e. by
pressing a key, or via an explicit call to
call-interactively. Emacspeak produces auditory feedback only if
this predicate returns t.

We first introduce a flag that will be used to record if the enclosing
(containing) function has an Emacspeak-defined advice on it and is
called interactively — these are the only cases that our predicate
needs to test.

(defvar ems-called-interactively-p nil
  "Flag that records if containing function was called interactively."

Next, we define a function that checks if interactive calls to a
function should be recorded. We're only interested in functions that
have an advice form defined by Emacspeak — all Emacspeak-defined
advice forms have the name emacspeak.

(defun ems-record-interactive-p (f)
  "Predicate to test if we need to record interactive calls of
this function. Memoizes result for future use by placing a
property 'emacspeak on the function symbol."
   ((not (symbolp f)) nil)
   ((get f 'emacspeak) t) ; already memoized
   ((ad-find-some-advice f 'any  "emacspeak") ; there is an emacspeak advice
    (put f 'emacspeak t)) ; memoize for future and return true
   (t nil)))

This is a memoized function that remembers earlier invocations by
setting property emacspeak on the function symbol.

All advice forms created by Emacspeak are named emacspeak, so we
can test for the presence of such advice forms using the test:

(ad-find-some-advice f 'any  "emacspeak")

If this test returns T, we memoize the result and return it.

Next, we advice function call-interactively to check
if the function being called interactively is one of the functions
that has been adviced by Emacspeak. If so, we record the fact in the
previously declared global flag

(defadvice call-interactively (around emacspeak  pre act comp)
  "Set emacspeak  interactive flag if there is an Emacspeak advice 
on the function being called."
  (let ((ems-called-interactively-p ems-called-interactively-p)) ; preserve enclosing state
    (when (ems-record-interactive-p (ad-get-arg 0))
      (setq ems-called-interactively-p (ad-get-arg 0)))

We define an equivalent advice form on function
funcall-interactively as well. Now, whenever any function that has
been adviced by Emacspeak is called interactively, that interactive
call gets recorded in the global flag. In the custom Emacspeak
predicate we define, we check the value of this flag, and if
set, consume it, i.e. unset the flag and return T.

(defsubst ems-interactive-p ()
  "Check our interactive flag.
Return T if set and we are called from the advice for the current
interactive command. Turn off the flag once used."
  (when ems-called-interactively-p                 ; interactive call
    (let ((caller (cl-second (backtrace-frame 1))) ; name of containing function
          (caller-advice  ;advice generated wrapper
           (ad-get-advice-info-field ems-called-interactively-p  'advicefunname))
          (result nil))
      (setq result
            (or (eq caller caller-advice) ; called from our advice
                (eq ems-called-interactively-p caller))) ; called from advice wrapper
      (when result
        (setq ems-called-interactively-p nil) ; turn off now that we used  it

The only fragile part of the above predicate is the call to
backtrace-frame which we use to discover the name of the enclosing
function. Notice however that this is no more fragile than the current
implementation of called-interactively-p — which also uses
backtrace-frame; If there are changes in the byte-compiler, this
form may need to be updated. The implementation above has the
advantage of working correctly for Emacspeak's specific use-case.

Friday, February 10, 2017

Audio Deja Vu: Audio Formatted Math On The Emacspeak Desktop

Audio Deja Vu: Audio Formatted Math On The Emacspeak Desktop

1 Overview

This article previews a new feature in the next Emacspeak release —
audio-formatted Mathematics using Aural CSS. Volker Sorge worked
at Google as a Visiting Scientist from Sep 2012 to August 2013, when
we implemented math
access in ChromeVox
— see this brief overview. Since leaving
Google, Volker has refactored and extended his work to create an Open
Source Speech-Rule-Engine implemented using NodeJS. This
speech-rule-engine can be used in many different environments;
Emacspeak leverages that work to enable audio-formatting and
interactive browsing of math content.

2 Overview Of Functionality

Math access on the Emacspeak desktop is implemented via module
emacspeak-maths.el — see js/node/ in the Emacspeak GitHub
repository for setup instructions.

Once loaded, module emacspeak-maths provides a Math Navigator that
implements the user interface for sending Math expressions to the
Speech-Rule-Engine, and for interactively browsing the resulting
structure. At each step of the interaction, Emacspeak receives math
expressions that have been annotated with Aural CSS and produces
audio-formatted output. The audio-formatted text can itself be
navigated in a special Spoken Math emacs buffer.

Module emacspeak-maths.el implements various affordances for
dispatching mathematical content to the Speech-Rule-Engine — see
usage examples in the next section.

3 Usage Examples

3.1 The Emacspeak Maths Navigator

  • The maths navigator can be invoked by pressing S-SPC (hold
    down Windows key and press SPC) — this runs the command emacspeak-maths-navigator/body.
  • Once invoked, the /Maths Navigator can be used to enter an
    expression to read.
  • Pressing SPC again prompts for the LaTeX math expression.
  • Pressing RET guesses the expression to read from the current context.
  • The arrow keys navigate the expression being read.
  • Pressing o switches to the Spoken Math buffer and exits the

See the relevant chapter in the online Emacspeak manual for details.

3.2 Math Content In LaTeX Documents

  1. Open a LaTeX document containing math content.
  2. Move point to a line containing mathematical markup.
  3. Press S-SPC RET to have that expression audio-formatted.
  4. Use arrow keys to navigate the resulting structure.
  5. Press any other key to exit the navigator.

3.3 Math Content On Wikipedia

  1. Open a Wikipedia page in the Emacs Web Wowser (EWW) that has
    mathematical content.
  2. Wikipedia displays math as images, with the alt-text giving the
    LaTeX representation.
  3. Navigate to some math content on the page, then press S-SPC
    a to speak that content — a is for alt.
  4. As an example, navigate to Wikipedia Math Example, locate math expressions on that page, then
    press S-SPC a.

3.4 Math Content From The Emacs Calculator

  1. The built-in Emacs Calculator (calc) provides many complex
    math functions including symbolic algebra.
  2. For my personal calc setup, see tvr/calc-prepare.el in the
    Emacspeak GitHub repo.
  3. This setting below sets up the Emacs Calculator to output results
    as LaTeX: (setq calc-language 'tex)
  4. With the above setting in effect, launch the emacs Calculator by
    pressing M-##.
  5. Press ' — to use algebraic mode — and enter sin(x).
  6. Press a t to get the Taylor series expansion of the above
    expression, and press x when prompted for the variable.
  7. This displays the Taylor Series expansion up to the desired
    number of terms — try 7 terms.
  8. Now, with Calc having shown the results as TeX, press S-SPC
    RET to browse this expression using the Maths Navigator.

4 And The Best Is Yet To Come

This is intentionally called an early preview because there is still
much that can be improved:

  1. Enhance the rule engine to infer and convey more semantics.
  2. Improved audio formatting rules to better present the available information.
  3. Update/tune the use of Aural CSS properties to best leverage
    today's TTS engines.
  4. Integrate math-reading functionality into more usage contexts in
    addition to the ones enumerated in this article.

5 References

  1. Youtube Video from early 2013 demonstrating Math Access in Chrome
  2. AllThings Digital outlining math access — published June 2013.
  3. Assets 2016 publication describing this work.
  4. js/node/aster-math-examples.tex Collection of math examples in
    LaTeX from AsTeR. Used to progressively improve speech-rules and
    the resulting audio-formatted output
  5. Speech-Rule-Engine on github.
  6. Speech-Rule-Engine in action: Accessible Maths in all browsers

Date: 2017-02-08 Wed 00:00

Author: T.V Raman

Wednesday, January 04, 2017

Fun With TTS (Voxin) And Ladspa

Fun With TTS (Voxin) And Ladspa

1 Executive Summary

Voxin 1.6 — AKA ViaVoice Outloud — no longer requires that the
Emacspeak TTS server be built as a 32-bit binary. This means that
installing Voxin on 64-bit systems is now significantly easier since
you no longer need to install 32-bit versions of TCL, TCLX, and the
dependencies needed by library In addition to
easing the installation process, not needing 32-bit binaries means
that the Emacspeak Outloud server can now take advantage of audio
processing such as that provided by LADSPA.

2 Going 64-Bit: Upgrading To Voxin 1.6

  1. Install Voxin-1.6 or later from Voxin.
  2. Update Emacspeak from GitHub (this will be part of the next
    public release).
  3. Rebuild the binary in the servers/linux-outloud
cd servers/linux-outloud && make clean && make

If all goes well, you'll now have a 64-bit version of
You can now run the Outloud server as servers/outloud.
In about a year's time, servers/32-outloud will move to
servers/obsolete, as will the associated servers/32-speech-server
and servers/ssh-32-outloud.

3 Applying LADSPA Effects Processing To TTS

With a 64-bit build of in place, we can now call on
installed LADSPA plugins to apply digital sound processing to TTS
output. To experiment with the possibilities, see some of the
virtual sound devices defined in servers/linux-outloud/asoundrc.
Copy over that file to your ~/.asoundrc after updating it to match
your sound setup — you'll likely need to change the default
sound-card to match your setup.
You can now set environment variable ALSA_DEFAULT to one of the
tts_<effect> virtual devices — and have the Outloud server apply
the specified LADSPA effect to the generated TTS. Here is an example:

cd servers 
(export ALSA_DEFAULT=tts_reverb; ./outloud)

4 The Best Is Yet To Come …

The possibilities are endless — ALSA with LADSPA provides a rich
suite of audio processing possibilities.

5 Acknowledgements

I'd like to acknowledge Gilles Casse for his work over the years on
ensuring that Linux users have access to good quality TTS. Outloud
would have been dead a long time ago if it weren't for his continued
efforts toward keeping the lights on. His newest creation, libvoxin
that forms the crux of Voxin-1.6 is an excellent piece of engineering
that is likely to help Outloud survive for the future on modern Linux
distros. Note that Gilles is also the primary author of the Emacspeak
ESpeak server.