\title{Backslash---Mathematical Activity}
\author[Jonathan Fine]{Jonathan Fine\\\texttt{J.Fine@uk.ac.cam.pmms}}

\begin{Article}

\noindent 
First an apology.  I did not allow time to proof my last article and
so did notice many small errors.  None were \TeX{}nically important,
except the solution to exercise 3.  I thank David Carlise for
pointing out that although \verb"\par" and \verb"\xyz" might have the
same meaning, only \verb"\long" macros will accept a \verb"\par" in
their parameter text.  I guess I also forgot to mention that when used
as a macro parameter delimiter, the meaning of a control sequence has
no bearing but the name is everything.  And in the middle of page~17,
right column, the line
\begin{quote}\quad\verb!\spaceit \endspaceit!\end{quote}
\noindent should be deleted.

The themes of this \BV\ issue are mathematics and tables. Siep
Kroonenberg's article on tables is excellent.  Here is a little
trick for use within mathematics.  It involves active characters.  The
sort of thing one might wish to do is have, say, \verb"[[" act as a
sort of ligature for a compound math character, such as $[\![$.

For every character code 0--255 there is a mathcode, which controls
just how that character should be typeset, when in mathematics.  More
exactly, it gives the class or part of mathematical speech, the font
family to use, and the location with the font family.

A little known and little used feature of \TeX{} is,
\begin{quote}
A \verb"\mathcode" can also have the special value \verb+"8000"+,
which causes the character to behave as if it has catcode 13
(active).  Appendix~B uses this feature to make \verb"'" expand to
\verb"^{\prime}" in a slightly tricky way.
\end{quote}
Knuth writes on [155] (this means page 155 of the {\em \TeX book}). 
This feature is not used by \verb"plain" for any other purpose.  This
remark is flagged as a `double dangerous bend' and so this article
may not be suitable for all readers.

As a result of this magic value for mathcode, a character can be made
to act as if it were active when it is in mathematics mode, but not
in text mode.  This is done without changing the catcodes, and so even
if ordinary letters are so made special, formation of control
sequence names proceeds as usual.  Moreover, math macros such as
\verb"\matrix" and \verb"\eqalign" read their text as a parameter, and
this fixes the category codes.  (The \verb"plain" footnote macro goes
to some length to avoid this [363], so as to allow category code
changes to occur within the text of the footnote.  This enable
verbatim text to there appear.)

Knuth [48] ``discourage[s] people from making extensive use of
\verb"\catcode" changes except in unusual circumstances'' precisely
because ``when the arguments to a macro are first scanned \ldots{}
their categories are fixed once and for all at that time.'' A
\verb"\matrix" may contain math and ordinary text, or may itself be
the argument to another macro (this is why verbatim does not work
properly within \LaTeX{} section titles).  Thus, to achieve a smart
\verb"[[" by category code changes would be difficult, and create
many unwelcome side effects.

However, mathcode \verb+"8000+ does not have these problems, because
it is not a category code change.  To understand the use of this
unusual mathcode, let us change the math code of \verb"[" to this new
value in such a way that ordinary documents will process exactly as
before.  The line
\begin{verbatim}
\mathcode '\[ "8000
\end{verbatim}
will change the mathcode to the magic value, but the previous
value---which controlled its conduct---is now lost.  So we shall
first save it.  In fact the code below
\begin{verbatim}
\ifnum \mathcode`\[ = "8000
\else
  \begingroup
    \catcode `\[ =13
    \global \mathchardef [ \mathcode`\[
  \endgroup
  \mathcode `\[ = "8000
\fi
\end{verbatim}
will first test that we haven't monkeyed with it before, and if safe
to do so, will \verb"\mathchardef" an {\em active\/} \verb"[" to the
original mathcode value, and finally set the mathcode of \verb"[" to
the magic value.

These changes (unless \verb"[" already has an active meaning, say for
use within ordinary text) should have no effect whatsoever on the
processing of manuscripts.  So what have we gained?

Previously the mathcode of \verb"[" caused the appropiate character
to be looked up from the appropiate font, and used as a mathematical
part of speech of the appropiate class.  Now the mathcode of \verb"["
will cause the meaning of active character \verb"[" to be looked up. 
The current value of this meaning is a mathchar which causes the
previous appropiate action.  {\em This meaning can now be changed},
to produce new behaviour.  This is the gain.

Our example is that we wanted \verb"[[" to produce $[\![$.  This
compound symbol fragment was produced using \verb"$[\![$", where
\verb"\!" gives a negative thin space.  To obtain this same result,
but using \verb"[[" as input, we must reset the value for active
\verb"[".

Here's how.  Active \verb"[" must inspect the next token.  To avoid
\verb"\futurelet" complications, I will assume is not a brace or a
space, and so can be read as a parameter.  If it is another \verb"["
we produce the compound symbol, otherwise we produce a single $[$ and
restore the parameter to the input stream.
\begin{verbatim}
\def \next #1%
{%
  \ifx #1[%
    \lbrack@\!\lbrack@
  \else
    \lbrack@
    \expandafter #1%
  \fi
}
\end{verbatim}

\begin{verbatim}
\begingroup
  \catcode`\[=13  % active
  \global \let [ \next
\endgroup
\end{verbatim}
The control sequence \verb"\next" is used to hold the value until we
change the catcode of \verb"[" to access active \verb"[".  If we
tried to make the definition all at once, we would find that we would
no longer have access to regular \verb"[".  The command
\verb"\lbrack@" has been freshly introduced, to hold the customary
mathcode of \verb"[".  This could have been obtained via
\begin{verbatim}
\mathchardef \lbrack@ \mathcode`[
\end{verbatim}
if we had though to {\em before\/} we started changing things.  As it
now is, we can use
\begin{verbatim}
\mathchardef \lbrack@ "405B
\end{verbatim}
which value comes from \verb"plain.tex" (see [344]).

The \verb"\expandafter" in the above definition is to prevent code
such as
\begin{verbatim}
$ [ \mathmacro { argument } ] $
\end{verbatim}
producing a disaster, where \verb"\mathmacro" takes a single
parameter.  Stepping through the above code for active \verb"["
we will we get
\begin{verbatim}
\lbrack@ \expandafter 
\mathmacro \fi { argument }
\end{verbatim}
as an intermediate result.  Without the \verb"\expandafter", the
\verb"\mathmacro" would get \verb"\fi" as its argument, and that is
totally wrong.  As it is, the \verb"\expandafter" causes the token
{\em after\/} the \verb"\mathmacro", which is the \verb"\fi",  to be
expanded {\em before} \verb"\mathmacro" does its piece.  When
\verb"\fi" is expanded [213], 
\begin{quote}
\TeX{} reads to the end of any text that ought to be skipped.  The
``expansion'' of a conditional is empty.
\end{quote}
and this is just what we want.  The \verb"\fi" is gone, and so now
\verb"\mathmacro" gets its proper argument.

This device, which I call {\em active mathematical characters\/} makes
all sort of dirty trickery possible.  Mathematicians have a wide
range of complicated symbols, diagrams, matrices, and so forth. 
Perhaps use of this device will allow for improved input syntax for
at least some of these devices.

Finally, problems and solutions.  Problem~5 from last issue has a
short solution (six lines of 80~column code) but seems to require a
long explanation.  The solution to Problem~6 will be given in the
next issue.  There are two new problems for this issue.  The solution
to Problem~7 is in the {\em \TeX{}book}.  Problem~8 asks a question
about possible \verb"\mathchar" values.

\noindent
{\bf Solution 5.}
{\em The problem was to write a macro which will trim the leading and
trailing spaces from user supplied text.}
Assume that 
\verb"\text" is a macro whose expansion is the user-supplied text,
such as
\begin{verbatim}
\def \text { apples and oranges }
\end{verbatim}
and that \verb"\trim" \verb"\text" is to redefine \verb"\text" as
\begin{verbatim}
\def \text {apples and oranges}
\end{verbatim}
which is as before, but without leading and trailing spaces. 
However, the original value of \verb"\text" may contain macros,
nested braces, and perhaps even conditionals.

Here is the solution, with comments as we go along.
\begin{verbatim}
\catcode`\@=11  % @ is a letter
\def\trim #1{%
  \expandafter\trim@
  \expandafter{#1 }%
  #1%
}
\end{verbatim}
If \verb"\text" is the argument to \verb"\trim", the expansion of
\verb"\trim" will result in \verb"\trim@" being called with two
parameters.  The first will be, enclosed in braces, the user
supplied text {\em but with an additional trailing space} (the
reason for which will be given later) and the second the name of the
control sequence (\verb"\text") whose redefinition is sought.

We now set things up to remove the leading space, if any.  We use
\verb"@" as a private delimiter, for it cannot occur {\em with
category code 11\/} in user supplied text.  The expansion of
\begin{verbatim}
\def\trim@ #1{\trim@@ @#1 @ #1 @ @@}
\end{verbatim}
will cause \verb"\trim@@" to see before it {\em two\/} copies of the
user-supplied text, both with (another) additional trailing space,
the first copy without and the second with an additional leading space.
This whole mess is closed with \verb"@@", which functions as a
delimiter.

The trick now is to have \verb"\trim@@" look for text delimited on
both left and right by the pair \verb*"@ " of tokens (being an \verb"@"
followed by a space).  If the user-text has a leading space, such
occurs around the first copy.  If not, around the second copy.  The
parameter delimiters of
\begin{verbatim}
\def\trim@@ #1@ #2@ #3@@{%
  \trim@@@\empty #2 @%
}
\end{verbatim}
select the appropiate copy of the user-text to be parameter 
\verb"#2".  The rest of the arguments can be thrown away,
all the way up to the \verb"@@" delimiter.  The parameter \verb"#2"
will be the user-text, with a trailing space added twice
(by \verb"trim" and by \verb"trim@" also), and with the leading space
(if present) stripped.  

We are nearly done now.  The purpose of the \verb"\empty" (a macro
which expands to nothing) will be explained later.  We copy the user
supplied text with yet another trailing space (that's the third time
we've done this) and call \verb"\trim@@@" with \verb"@" as a
delimiter.

Here come the final and amusing macro.  We wish to strip the trailing
space, if present.  Perversely, we have three times added a trailing
space.  Now {\em in regular user defined text, by virtue of \TeX's
reading rules [37], it is impossible for user supplied text to
contain two successive explicit space characters}.  So we use two
successive spaces characters as a delimiter, to strip trailing
spaces.  This is why we have been so assiduously been building them
up at the end.  

We need a helper macro
\begin{verbatim}
\def\unbrace#1{#1}
\end{verbatim}
to allow the construction of the final macro
\begin{verbatim}
\unbrace{\def\trim@@@ #1 } #2@#3
{%
  \expandafter\def
  \expandafter #3%
  \expandafter {%
  #1}%
}
\catcode`\@=12  % restore @
\end{verbatim}
whose first parameter \verb"#1" is delimited by {\em two space
characters}.  This strips the trailing space, and we discard any
other spaces there may be, up to the trailing \verb"@".  The third
parameter \verb"#3" is the control sequence (\verb"\text" in our
case) whose stripped redefinition we seek.

By now, \verb"#1" is stripped of leading and trailing space, and has
an \verb"\empty" prepended.  This is `stripped' via the
\verb"\expandafter"'s.  The macro is finished.

Some further explanations are required.  A trailing space is added
{\em three\/} times when it might seem that twice is enough, to cover
the case that \verb"\text" is empty.  In that situation the first
added {\em trailing\/} space will also be a {\em leading\/} space,
and will be treated as such.  The purpose of the \verb"\empty" is to
forestall \TeX's (usually helpful) custom of stripping ``the
outermost braces enclosing the argument'' [204].  Without this sweet
nothing, the macros produce from
\begin{verbatim}
\def\text {{well wrapped}}
\end{verbatim}
the new value
\begin{verbatim}
\def\text {well wrapped}
\end{verbatim}
which is wrong!  Earlier in the expansion the trailing space(s)
stopped this happening.

Finally, an acknowledgement. The basic ideas for dealing with the
leading space are due to Donald Arseneau, but the trailing double
space trick is all my own work.

\noindent
{\bf Exercise 7.}
What reason does Knuth give for choosing \verb"$" as the math
bracket.  Hint:  mathematics and tables are known as `penalty work'
because they will attract an extra charge from the typsetter.  The
solution in on [127].

\noindent
{\bf Exercise 8.}
Why should the value \verb+"8000+ be forbidden [155] as mathchar
(rather then mathcode) value?  And why not?

\end{Article}
\endinput








\catcode`\@=11

\ifnum \mathcode`\[ = "8000
\else
  \begingroup
    \catcode `\[ =13
    \global \mathchardef [ \mathcode`\[
  \endgroup
  \mathcode `\[ = "8000
\fi
\def \next #1%
{%
  \ifx #1[%
    \lbrack@\!\lbrack@
  \else
    \lbrack@
    \expandafter #1%
  \fi
}
\begingroup
  \catcode`\[=13  % active
  \global \let [ \next
\endgroup
\mathchardef \lbrack@ "405B


\catcode`\@=12
\end{Article}
\endinput