Historical errata for Language Implementation Patterns

The link to the StringTemplate site is incomplete, it is missing the .org TLD.

The sibling node of fontsize should be 14 and not =

The parsers we’re working within this book

should be:

The parsers we’re working with in this book

A more powerful parsing strategy
means it’s easier to parsers. You
don’t need to factor your parser is
much to suit week or parsing decisions.
===>
A more powerful parsing strategy
means it’s easier to parsers (“to parse”? “for parsers”? not sure which you intended). You
don’t need to factor your parser as
much to suit week or parsing decisions.

This is precisely
what they backtracking
parser does.
==>
This is precisely
what the backtracking
parser does.

“Grammar to Recursive-Descent Parser Mapping” is a distactingly bad name, especially for the first pattern. It’s a noun with a six-word adjective, and many different ways of mentally grouping those words: “(Grammar to Recursive-Descent) Parser Mapping”, or “Grammar to (Recursive-Descent Parser Mapping)”. A shorter name would be better: “Grammar drives Parser” or even “Grammar begets Parser”. Or if the current words are required, how about rearrange to: “Mapping of Grammar to Recursive-Descent Parser”

“To avoid building parse trees…” This mistakenly suggests that somehow at this juncture our objective is to avoid building parse trees. This seems beside the point. The point seems to be more to point out that the call sequence follows the path of what amounts to a parse tree, and consequently we can attach actions to those calls to get work done, if that fits the task at hand.

Light gray tone on the X is too light in the PDF

Morse code “dashes” are printed as underscores and all run together. Maybe try M-dashes, or put spaces between the underscores? (Then the intercharacter space will need increase too.)

2nd last line: “Lexers are also called a scanner..” plural-singular mismatch

Implementation section P43-46. EOF_TYPE and EOF appear in a number of places. It would be helpful to provide a mention of what the point is of these two things. Are they different types? Is one a TYPE and the other a character? Why do they both seem to be part of the same series of consts? (p45).

“The parser should attempt the alternative that can start with the current lookahead token.”
Should that be alternativeS ?

“…here is a rule whose parsing decision is deterministic because the single lookahead token uniquely predicts which alternative to choose.” This appears to be saying “this rule is able to make a choice because it is able to make a choice”. Or is there more to it?

“The two alternatives begin with the same ID token.” Probably means “the same token, which is ID.”

“Next we need…” Suggests that match() does the compare, and consume() does the consuming. However, it turns out that, counter expectations from the function name, match() does a comparison AND consumes. (The code comment does note this.) This is a crucial thing to understand in reading ANTLR-generated source code.

“they backtracking” —> the backtracking

“Homogeneous nodes represent specific children by position within a list rather than with named node fields. We call that a normalized child list.”
I’m pretty sure that “homogeneous nodes” means the nodes are of the same data type, while “normalized” means they are accessible only by position (index) in the parent’s child list, and not by name (as member fields of parent). See also page 80.

The word “normalized” is used over and over, but not defined. See also page 18 where it seems to be defined probably incorrectly.

Given that Parser.consume() resets the index (p) to 0 when doing an actual (rather than speculative/backtracking parse), the memoize/BacktrackingParser.java should clear entries at the same time.

Token indices in list_memo wouldn’t correctly apply to parsing of subsequent input lines.

(This doesn’t show up in the example, as only a single line, a single memoization of a ‘list()’ attempt occurs)

“The one caveat to action gating is that there are some actions we must execute during backtracking. Tracking variable and method names is critical…”
Not mentioned — what to conclude from this. Does “we must” mean “ANTLR’s code does this” or is this instead saying that backtracking can’t be combined with actions that set state that semantic predicates refer to?

We we

“tale two” —> “take two”

“Although there is only a single node data type, we don’t need to use the type system of the implementation language to distinguish between node”.
Why “although”? Seems that if you are using a single node type, you clearly are not using the type system of the implementation language.

“It doesn’t worked so well”

Table text extends beyond bottom of page, cutting off everything after “ing for patterns, then, we need”.

“This implementation has a number of advantages over an embedded walker.” … and following.
I’m pretty sure this para is comparing “visitor switching on node type” versus GHJV-style visitor, and NOT to “embedded walker” per se. “call every method visit() appears to refer back to page 121, right? (Also, ”call…method" here means “name…method”, but is ambiguous with “execute…method”.

“ANTLR generates ” plural/singular inconsistent

“it’s a bit tricky” repeated.

PDF page 15, last paragraph: “This book identifies ### n common
language implementation patterns.” PDF page 9 says: “There are
23 patterns in this book.” If the number is finalized at 23,
then both locations should use the same number, if, however, the
number of patterns in the book has not been finalized, then both
should use the marker (?) “### n”.

Replace photo of Ter insulting Brazilians :-)

Diagram at top of page 16:
Because of the use of the word “OR” in the ‘Semantic Analyzer’
box, the diagram would imply (for example) that an “Interpreter”
could be something that recognizes & builds an IR, then annotates
that IR. This conflicts with definition of “Interpreter” on page
15: “An interpreter reads, decodes, AND executes instructions.”

A nit on pg 16:
“Pattern 1, Grammar to Recursive-Descent Parser Mapping, on page
37 shows us how to convert grammars to handbuilt
parsers. ANTLR (or any other parser generator) can do this
conversion automatically for us …”

yacc is a “parser generator”, but produces neither
Recursive-Descent nor hand-built parsers.

Page 14, Section 1.1 “The big picture” (Big Picture) starts using
the term “sentence”, which has not been introduced. This appears
to be the formal language grammar use of the word (which is very
nicely defined in Section 4.1, pg 72 of “The Definitive ANTLR
Reference”). An unfamiliar reader may assume the natural
language meaning of “sentence”.

Awkward sentence: “They include a record of the rules
used to recognize the input not just the input.”
Perhaps?: “They include a record of the rules
used to recognize the input - not just the input itself.”

Plural/singular mismatch: “The semantic rules of your language
dictates which symbol table pattern to use.”

Sentence at top of page - a comma would be helpful:
“… implementation patterns including …” ->
“… implementation patterns, including …” (rewrite rule?)

Parenthesized phrase to complete sentence:
“… to another language (at the extreme, compilers translate high
level programs down to machine code).” ->
“… to another language. (At the extreme, compilers translate high
level programs down to machine code.)”

I’m not english so I’m not beyond making a mistake, but i think there is a litle mistake when it says :

“Finally, we’ll can look…”

I’m not sure we can use will + can.

“Upon error, we want to thrown exception” should be “Upon error, we want to throw an exception”.

Same with “phrase” (see #39366)

Diagram at top of page: label in “Reader” box looks like
“TR” (bottom of ‘I’ chopped off?)

A quick note about some of the reasons ‘cfront’ was later abandoned
might be instructive here, to show that there can be drawbacks to this
approach, and to emphasize that this is an historical “overall C
application pipeline”.

Associate acronym “DSL” with “Domain Specific Language(s)”
somewhere early on.

Singular/plural mismatch in second paragraph of section 2.3:
“The if-then-else code template in the parser become a …”

“Rule expr might look a little funny to you and it should.” ->
“Rule expr might look a little funny to you, and it should.”

“Say we want to recognize list of names” ->
“Say we want to recognize A list of names” OR
“Say we want to recognize listS of names”

Second to last paragraph:
“The only difference lies in the type of their input symbols,
characters or tokens.”
“Only” is a little strong, but I guess it depends on what you’re
trying to emphasize here. A newbie might take it too literally
though. Perhaps “primary”?

Consider rewording:
“Other than left recursive rules, there are other grammar constructs” ->
“Besides left recursive rules, there are other grammar constructs”
(reads better than repeating “other”)

In “public class G {”: <> instead of <> ?

Second to last sentence: This is the first use of the term “subrule”.
Short definition for the layperson, i.e., how does it differ from
“rule”: a rule invoked by another rule? a rule contained within
another rule?

(Also pg 41)
“Pattern 2, LL (1) Recursive-descent Lexer, on the following page …”
capitalization of “Recursive-Descent” should be consistent.

Paragraph beginning with “In this case, …”
Enclose all occurences of the semicolon token in single quotes as done
later in the paragraph, or perhaps a more distinctive font? This
single character blends into the surrounding explanatory text too much
- at least in this PDF. Regardless, the method to separate token
character(s) from explanatory text should be consistent.

Last paragraph: “… rather than a token variable.” ->
“… rather than a Token variable.” (capitalized - if referring to
the type of the variable ‘lookahead’)

“… for which we’ll need more a powerful recognition pattern.” ->
“… for which we’ll need a more powerful recognition pattern.”

“There lots of similar situations that occur in real grammars.” ->
“There are lots of similar situations that occur in real grammars.”

“… tests the first two tokens lookahead
LA (1) and LA (2) to predict the …” ->
“… tests the first two lookahead tokens,
LA (1) and LA (2), to predict the …”

“… it’s easy to get lost will all of the scanning ahead
and rewinding.” ->
“… it’s easy to get lost with all of the scanning ahead
and rewinding.”

Second to last paragraph, last sentence:
“Using a syntactic predicate in an ANTLR rule, we’d say:”
Syntatic predicates have not been introduced at this point.
Only semantic predicates have been mentioned so far,
on pg 57.

“The first alternative will fail, causing a backtracking parser to
rewind the input and attempts the second alternative.” ->
“The first alternative will fail, causing a backtracking parser to
rewind the input and attempt the second alternative.”
(attempts -> attempt)

“When to apply” for Pattern 4:
“A more powerful parsing strategy means it’s easier to parsers.”
If parsers are code, what is meant by “easier to (for?) parsers”?
Less machine instructions to execute? Did you mean “easier to
write grammars”?

“When to apply” for Pattern 4:
“You don’t need to factor your parser is much to suit week or parsing
decisions.” Huh?!

Last sentence on the page:
“Now that we’ve got the basic idea behind these advanced parsing
strategies, let’s jump into their pattern definitions.”

It’s not really clear what “these” is referring to - just patterns
5 & 7 above? or “advanced parsing strategies” in general?
The four patterns just discussed above on the page seem to be mixing
summary and introduction. Two have already been covered, so won’t be
“jumped into”. I found this transition somewhat unnatural.

“when backtracking, a parser does not report errors”
Syntax errors? Hopefully it still reports fatal exceptions
like “out of memory” errors, etc.

“Our parser can start out with a small fixed buffer size, n. Like
Pattern 4, LL (k) Recursive-Descent Parser, on page 52, …”
It might help to explain that the earlier circular buffer approach
works best for LL (k), so that the user will better understand why the
different token buffer scheme (shift left?) used for the backtracking
parser case.

(illustrations top of page)
Though not critical to the point being made, I found myself
wondering why the ‘p’ position was shifted left 2 instead of
4 or 1. Put another way, why hold onto 2 consumed tokens vs.
3 or 0?

(bottom of page)
comment refers to LT (i), code references LT (1)

(top of page)
“The integers ZERO and above indicate a previously successful
parse. The integer also records the token index ONE PAST where the
parsing method finished previously.”
… so when would the integer be 0 ?

(code for ‘memoize’ method)
“int stopTokenIndex = failed?FAILED:index();” ->
“int stopTokenIndex = failed ? FAILED : index();”
(spaces around the ‘?’ and ‘:’ make this line more readable)

(first code sample)
“while ( LA (1)CONST || LA(1)VOLATILE || LA (1)VOLATILE ||" -> "while ( LA(1)CONST || LA (1)==VOLATILE ||”
(VOLATILE test repeated - also in 3rd code sample)

Chapter titles should use consistent capitalization.
(Suggest all caps except for minor words: and, for, to, etc.)

(first paragraph)
“… because they can generate input as soon as they recognize a
construct.” ->
“… because they can generate output as soon as they recognize a
construct.” (input -> output)

(Section 4.1, first paragraph)
“… we know ‘Rebecca’ is the subject, ‘runs’ is the verb, and ‘to the
park’ is the object.”
Technically, “to the park” is a prepositional phrase, modifying the
verb, “runs”. A better example of a (direct) object is “thee” in the
footnote, where “with this ring” and “this way” are also prepositional
phrases. How about?: “Terence (subject) wrote (verb) a great
parser generator (object).” ;-)

The code (page 45) declares EOF = 0.

The text (page 46) reads EOF (–1).

I think the text is right.

(code comment at bottom of page)
“External visitors need unique int per node for id purposes …”
But this int won’t be “unique” on a “per node” basis.
Doesn’t that mean each node in the tree would return a different int
for its node type? Maybe you meant “per token”?

“The following toString() generates trees of the form:
(root child1 child2 …).” ->
“The following toStringTree() generates trees of the form:
(root child1 child2 …).”

(since it was just pointed out that toString() only prints
individual nodes.)

“Track expression type (integer or vector) for each expr nod.” ->
“Track expression type (integer or vector) for each expr node.”
(nod -> node)

“if ( exprType !=tINVALID ) {”
inconsitent spacing around != affect readability

“(exprType==tINTEGER?”tINTEGER“:”tVECTOR“)+”>“;”
Spaces don’t show up well in PDF. Lack of spaces makes this code
less readable IMHO. Also, why italic font?

“public class AddNode extends ExprNode {
public AddNode(ExprNode left, … ”

Since “add” is a verb, this class name resembles a method name (e.g.,
for “adding” a node to a tree). Perhaps “AdditionNode” or “PlusNode”?

week -> weak

In the last line of the Purpose section, “a week underlying parsing strategy” should be “a weak underlying parsing strategy”.

(code at top of page)
In ‘toStringTree’ method, the two cases:
left null, right != null and left != null, right null
aren’t handled well. Both of these are allowed by AddNode constructor
above. (Imbalanced ‘add’ trees are permitted but not printed.

Perhaps robustness isn’t a concern, given that this code is for
instructional purposes only? Or are we to assume that such a
check would be done at the parser level? If the latter, then
the first test could just as easily be written as:

// Parser guarantees both children are either null or non-null
if ( left==null ) return this.toString();

(assuming unary plus is a different token type, of course).

( in body of print(AddNode n) )
“System.out.print(token);” ->
“System.out.print(n.token);”

(second to last paragraph)
“Rule expr makes it more clear what the trees look like, though." What is "expr” ?

( in body of print(AddNode n) )
“System.out.print(token);” ->
“System.out.print(n.token);”
See comment #39534 (pg 109)

“There are number of nice tools …” ->
“There are a number of nice tools …”

System.out.print(token!=null?token.toString():“”);

(same as comment #39509, pg 100) No spaces, ‘null’ in italic font.

“Here is the start of a PrintVisitor with the code for
visiting AddNode:”
…
public class PrintVisitor implements VecMathVisitor {
public void visit(AssignNode n) {

The code is actually for visiting an AssignNode, not
an AddNode.

“This implementation does not need an interface at all and has a
double-dispatch mechanism that is completely encapsulated within the
visitor.”
Can we still call it a “double-dispatch”? Or would it be more
accurate to say “… emulates a double-dispatch mechanism that is
completely …” ?

“… about getting a language up and running fast and we do about
runtime efficiency.” ->
“… about getting a language up and running fast THAN we do about
runtime efficiency.”

(towards the top of page)
Extra whitespace and sentence fragment:
“more than a single node data type and with an irregular child list
representation.”

“^(‘print’ {print(”print “);} expr) {print(”\
“);}”
This gets a tad confusing with the 2 definitions of “print”.
Perhaps the ‘print’ member function could be named something
else in this case, e.g., “log”, “output”, etc.?

(towards the top of page)
Extra whitespace and sentence fragment:
“as it encounters tree patterns of interest.”

“Wildcard (dot) matches a node or subtree.”
It might be helpful to explain ‘.’ (dot) as a wildcard earlier on page
126 for the grammar example at the bottom of the page: “x=.”

(top of page)
“… we only needed to apply one rule for given subtree.” ->
“… we only needed to apply one rule for a given subtree.”

(middle of page)
“… we match an integer multiply subtrees and use a …” ->
“… we match an integer multiply subtree and use a …” OR
“… we match integer multiply subtrees and use a …”

(output from running ‘java Test2 < u1’)
Point out that ‘#’ comments are not part of the actual output.
(First output comment is just a repeat of the input comment, which
actually could be entered harmlessly as part of the input on Unix.)

Text is incomplete; after the table the text continues with “an intermediate…”

Extra whitespace & sentence fragment:
“phrases and sentences in the language specified by the grammar.”
(see also #39621, pg 101 & #39623, pg 125)
A quick scan shows this is also a problem on pages:
41, 46, 51, 63, 71, 96, 101, 116, 118, 122, 125, 142, 146, 167,
184, 192, 200, 207, 234

Still confused by the “or” in the ‘Semantic Analyzer’ box. I take
that to mean that we could replace the entire phase with any ONE of:
“collect info”, “annotate IR”, “rewrite IR”, or “execute”. So we
could have a Reader that builds the IR, followed by a Semantic
Analyzer that JUST collects info (and does nothing else), and still
the combination would be an Interpreter? The text, on the other hand,
seems to imply that an Interpreter has to “execute” something.

Feel free to disregard if you think I’m being too literal here. (I
know that I do that a lot and that it can be annoying :-) I did
understand what the diagram was meant to convey. Just don’t know if
everyone will.

“… here’s the class that represents represents built-in types like
int and float:”
(* 2 ‘represents’)

“To represent a scope, we’ll us an interface” ->
“To represent a scope, we’ll use an interface” (us -> use)

Just below the first paragraph, the words “flat scope” appear by themselves.

The text preceding the ‘varDeclaration’ rule refers to it by a different name (‘localDeclaration’).

“Upon: Variable declaration x:
Actions: ref x’s type. def x in the current scope.”

Though implied in the preceding description of Cymbol, it may be
helpful to explicitly state that there are no user-definable types in
this example, i.e., that x’s type has to be one of the Builtin types
already def’ed. (This is stated more explicitly later on pg. 147:
“This version of Cymbol has only two kinds of symbols: variables and
built-in types”)

(code example towards the bottom of page)
“…$start.getLine()…”
Some might wonder where “$start” came from.

[Download symtab/monolithic/Cymbol.g]
primary
: ID // reference variable in an expression

You’d have to download the code to see where the rule ‘primary’
comes from.

“We’ll need to more Symbol subclasses to handle …” ->
“We’ll need to add/create/? more Symbol subclasses to handle …”

“Upon/Action(s)” table:
May want to either point out that ‘f’ & ‘x’ are not referring
specifically to the ‘f’ & ‘x’ in the previous code/tree example, or
use different letters (Greek/italic letters maybe?).

“… ref float, def f, push f, ref int, def x, …” ->
“… ref float, def f, push method scope of f, ref int, def x, …”

This is how the function scope of f is referred to in the next
paragraph, and it seems much clearer than “push f”.

“… def x, ref floatdef y, push local scope of f, …” ->
“… def x, ref float, def y, push local scope of f, …”
(floatdef -> float, def)

(bottom of page)
“Pattern , Symbol table for classes, on page we’ll separate
them into …” ->
“In Pattern , Symbol table for classes, on page we’ll separate
them into …” OR
“Pattern , Symbol table for classes, on page will separate
them into …”

(bottom of page)
“… two tree walking phases to support forward references
(methods can call methods defined later in the class).” ->
"… two tree walking phases to support forward references.
(Methods can call methods defined later in the class.)

Parenthetical complete sentences which are not internal to an
enclosing sentence should be written as separate complete
sentences w/ period inside the parentheses. See:
grammar.ccc.commnet.edu/GRAMMAR/marks/parentheses.htm
www.butte.edu/departments/cas/tipsheets/punctuation/parentheses.html
thegrammarexchange.infopop.cc/eve/forums/a/tpc/f/340600179/m/3366050132

The paragraph after this one (pg 154) also has a couple of cases of
non-internal parenthetical complete sentences: “Actually, we’re
ignoring …” and “Note that the y reference …”
(See also #39370 from Beta 1 errata - now on page 21.)

(code sample at bottom of page)
See #39662, pg. 148 ($start not explained)

(top of page)
“The test rig in the source code directory build an AST …” ->
“The test rig in the source code directory builds an AST …”

(second to last line of Test output)
Although the label is “args:”, the argument in
method f’s scope, x:int, is not printed (unlike
the “locals:” and “globals:” symbols).

Reminder: In backtracking parser pattern, use speculate / backtrack term properly.

Preface: “If you’ve have done some work…”

Second sentence on the page reads “To distinguish what kind statement is coming down the road […]”. It is missing the word “of” between “kind” and “statement”.

(first paragraph)
“Because there are two pointers emanating from each
ClassSymbol nodes, classes actually …” ->
“Because there are two pointers emanating from each
ClassSymbol node, classes actually …” (nodes -> node)

(last paragraph)
“To recognize illegal forward references, we can use a trick involving
token indexes. If a symbol reference resolves to a local or global
symbol, the reference’s token index must come after the definition’s
token index.”

Suggest a parenthetical remark or footnote to point out that this
scheme won’t work with the LL (k) circular token buffer or the
“shift-left” token buffer introduced earlier (pgs. 54 & 67
respectively), where the token index can reset to 0. In such cases
one would have to use a separate, strictly monotonically increasing
token counter.

To the degree that the book targets non-specialists, it may be helpful to say more about what sentences mean in this context. You mentioned the term in the introduction without much clarification and it appears again here. “Sentence” and “phrase” are great shorthand if the reader knows the longhand.

Do you mean “translators and the like” or “translators and their ilk”? “Translators and the ilk” seems wrong.

(End of page) The semicolon terminating the example statement disappears when the reader considers the tree, possibly leading to confusion. The example works fine without the confusing semicolon in the text.

The phrase “To parse trees” near the end of the page threw me off (modifier attachment ambiguity). I read “parse trees” rather than “to parse.” I’d suggest “To traverse parse trees …”

The best way to do ANYTHING is to work backwards — to “start with the end in mind.” :-)

In bug finder part deux, it would be great to connect the dots for the reader back to the source file. Seeing that “#2” is duplicated doesn’t say which expression in the source is at fault. The reader may wonder how to get back to this information. (These examples are great, btw).

It may confuse the reader to see both a relational expression (x<0) and an integer (0) classified as an expression at this point in the evolving plot of the book.

You draw the analogy to parts of speech in natural language. It would be helpful to include a figure like 2.1 (or the inline ones) for a short English sentence, including both the “sentence diagram” and tree structures.

The notion of a 2D structure appears at the top of the page without much motivation. It’s not revisited again until p.83, where it helps understanding. Maybe provide a cross reference to p.83? Not mention 2D structure on p.33? Give a little more context on p.33?

Related to earlier comments about sentences and phrases, section 2.2 starts off with a brief mention of what a language is. I’m imagining a one-stop section earlier in the book that informally discusses these formal terms. Maybe one of those sidebar boxes that appear so frequently in Prag Prog titles? Cross references back to this discussion as appropropriate would be helpful for newbies.

Opening of section 2.4. Lose the “clearly”

(Top of page). Calling a grammer “terse” seems almost pejorative. Concise? Compact? Clear?

Under “Converting Rules,” the phrase “rule embedded within another rule surrounded by parens” is confusing. There needs to be an earlier definition of this syntactic convention.

You assume the reader is familiar with the ()?, ()*, and ()+ subrules. Is this the page that is intended to introduce those types of subrules? It would be helpful to have introduced these constructs (including parents, Kleene thingies, alternation, etc.) earlier.

“… but there are number of ways to implement the actions.” ->
“… but there are a number of ways to implement the actions.”

“If you want, you can leave the sample implementation until
you’re trying to implement your own language.”

Not sure I understand this. Are you saying the following is
only important if one is implementing their own language?

“You can look at Cymbol.g in the source code directory to look at the
details.”
Wording?: “You can look at … to look at …”

(bottom of page)
“/ For a.b, only look in a only to resolve b, not up scope tree */” ->
“/ For a.b, only look in a to resolve b, not up scope tree */” OR
“/ For a.b, look only in a to resolve b, not up scope tree */”
(one too many only’s)

(first paragraph)
" … and then they resolution pass in Ref.g." ->
" … and then the resolution pass in Ref.g." (they -> the)

“Related patterns”
“This pattern builds on Pattern 17, ‘Symbol table for nested scopes’, on
page 150. Pattern 19, ‘Symbol table for classes’ extends this pattern to
support classes.”

Doesn’t pattern 19 build on Pattern 18 (as well as 17) ?

“Object-oriented languages like Cymbol also support forward references
to symbol defined later in the file (methods can refer to methods and
fields defined later).” ->
“Object-oriented languages like Cymbol also support forward references
to symbols defined later in the file. (Methods can refer to methods and
fields defined later.)”
(symbol -> symbols; stand-alone parenthetical sentence)

“The logical place to stash information is an AST representation of
the input (it’s much more efficient to walk an AST multiple times than
the original token stream).” ->
“The logical place to stash information is an AST representation of
the input. (It’s much more efficient to walk an AST multiple times
than the original token stream.)”
(stand-alone parenthetical sentence)

(Class declaration C)
“Set t.symbol to sym. Set C’s super-
Class field to sym.” ->
“Set t.symbol to sym. Set C’s super-
class field to sym.”
(‘c’ in superclass not capitalized)

(Method declaration f)
“Set the type field of the MethodSymbol for f’s to sym.” ->
“Set the type field of the MethodSymbol for f to sym.”

(bottom of page)
“/ Where to look next for symbols; super class or enclosing scope */” ->
“/ Where to look next for symbols; superclass or enclosing scope */”
(superclass one word)

The sentence (mid-page) “To use this lexer pattern, we create an instance of a lexer an input string or stream reader” seems to be missing something. “… a lexer AND an input string …?”

(Bottom of page) Need to say what a “test rig” is.

(Variable reference x)
“Set x.symbol (x’s ID node) to sym’s symbol field to sym.”
?? Reads as: x.symbol = sym.symbol = sym

(fig. 7.10 caption)
“Figure 7.10: How resolve() walks up the scope tree classes” ->
“Figure 7.10: How resolve() walks up the scope tree for classes”

(the two methods: ‘public Scope getParentScope()’)
ScopedSymbol.getParentScope() uses the getter, getEnclosingScope().
ClassSymbol.getParentScope() returns the ‘enclosingScope’ field
directly. (both work, but inconsistent - getter preferred?)

(comments in ‘enterClass’ action code)
“currentScope.define(cs); // def struct in current scope
currentScope = cs; // set current scope to struct scope” ->
“currentScope.define(cs); // def class in current scope
currentScope = cs; // set current scope to class scope”
(struct -> class)

“… it doubly links the class names ID AST node with the …” ->
“… it doubly links the class name’s ID AST node with the …”
(missing apostrophe)

“These the dashed lines pointing at the int symbol … ” ->
“These are the dashed lines pointing at the int symbol … ”

(bottom of page)
“… but the tree patternmatcher does not try to match expr against
subtrees (we don’t mention rule expr in the topdown or downup rules).”
"… but the tree pattern matcher does not try to match expr against
subtrees. (We don’t mention rule expr in the topdown or downup
rules.)
(stand-alone parenthetical sentence; ‘pattern matcher’ two words -
font/fill makes it look like there’s a space, cut & paste and/or
search show one word)

Full parse of subtrees within the pattern matching - very cool
stuff!!

(inherit.cymbol code)
Should be some indication of what the numbered icons refer to.
Scopes?

(summary of chapters/patterns)
Though Cymbol is understandably a simplified subset of C, tailored
to introducing these concepts, it would be very interesting to know
briefly if/how one would extend these patterns to handle C
namespaces, Java packages, etc.

we’ll us an interface

should be

we’ll use an interface

(Near top) Should read “recognizing English phrase” should read “recognizing the English phrase.”

Bogus comment opener in first public void walk()

(semantics/types/Types.g: @after rule)
$a.start - first use of .start, but not explained until
following page in a code comment:
“// $expr.start is root of tree matched by expr rule”

(first paragraph)
“… it’s a good idea to tuck them out of the way has methods in
another class):” ->
“… it’s a good idea to tuck them out of the way as methods in
another class):”
(has -> as ?)

“public Type arrayIndex(CymbolAST id, CymbolAST index) {”
Why is the index parameter even necessary if it isn’t
used for anything?

(Test.java)
‘TokenRewriteStream’ is used here and on the following page
in the ‘showTypes’ method, but the explanation of what it is
doesn’t come until page 204.

(second paragraph)
“It records all of the insertion command and then ”executes" them …" ->
“It records all of the insertion commands and then ”executes" them …"

(semantics/types/Test.java - showTypes method)
Use of ‘printf’ here creates a dependency on Java 1.5 for the example
code, which, if I recall correctly, is still not the case for ANTLR
itself. Maybe this should be noted in the section “Languages and
tools used in this book” or in a README file in the tpdsl-code
distribution? (As a reformed C programmer, I love being able to
use ‘printf’ again too.)

“The next section describes exactly how functions ‘resultType’ and
‘promoteFromTo’ work and how to …”

But in the next section ‘promoteFromTo’ appears nowhere as a function,
having been abandoned in favor of the 2-dimensional lookup table of
the same name. Also the ‘resultType’ function seems to have been
renamed to ‘getResultType’ in the following sections.

“… calling them Parser Expression Grammars (PEGs) ‘Parsing
expression grammars: a recognition-based syntactic foundation
[For04]’.” ->
“… calling them Parsing Expression Grammars (PEGs) ‘Parsing
Expression Grammars: A Recognition-Based Syntactic Foundation
[For04]’.”
(Parser -> Parsing ; fix capitalization - also needed in
Bibliography.)

(first paragraph)
“… we’d write (float)((int)‘a’+3)4.2. (our Cymbol C+ subset doesn’t
have doubles so we’ll use float).” ->
“… we’d write (float)((int)‘a’+3)4.2. (Our Cymbol C+ subset doesn’t
have doubles so we’ll use float.)”
(stand-alone parenthetical sentence)

“… the tables make it really easy see the valid operand types and
promotions.” ->
“… the tables make it really easy to see the valid operand types and
promotions.”

“Computers only cares about operators and operands.” -> “Computers only care about operators and operands.”

(Near the bottom of the page.)

“Let’s assume P p; and Q q.” ->
“Let’s assume P *p; and Q *q”."

“… to the Cymbol language we’ve used throughout this chapter (we’ve
ignored pointers until now to keep things simple).” ->
“… to the Cymbol language we’ve used throughout this chapter. (We’ve
ignored pointers until now to keep things simple.)”
(stand-alone parenthetical sentence)

“This way, the definition and type safety tree passes don’t have to
worry about the new syntax (the AST only has pointer arithmetic).” ->
“This way, the definition and type safety tree passes don’t have to
worry about the new syntax. (The AST only has pointer arithmetic.)”
(stand-alone parenthetical sentence)

“The following method answers whether we can copy a pointer
up one type to a destination type.” ->
“The following method answers whether we can copy a pointer
of one type to a destination type.”
(up -> of)

(semantics/oo/PointerType.java: javadoc comment for ‘canAssignTo’)
“dest must be pointer and to same type …” ->
“destType must be pointer and to same type …” (dest -> destType)

(code comment at bottom of page)
“// not comparing object pointers; types we point at must be the same
// For example: int *p; int *q; p = q;”

I’m confused by the above comment. Given int *p; int *q; (C) then:
p = q is pointer assignment
*p = *q is assignment of objects pointed at
p q is pointer comparison *p *q is comparison of objects pointed at
typeid(p) typeid(q) is pointer type comparison typeid(*p) typeid(*q) is comparison of types pointed at

so shouldn’t comment be something more like:
“// For example: int *p; int *q; typeid(*p) == typeid(*q);” ?
(which is probably more C++ness than anyone wants to get into ;^)

(bottom of page)
“But, we can’t add to pointers together.” ->
“But, we can’t add two pointers together.” (to -> two)

“So far, we’ve learned how recognize sentences, construct AST …” ->
“So far, we’ve learned how to recognize sentences, construct AST …”

Lexers are also called a scanners, lexical analyzers, or tokenizers. -> Lexers are also called scanners, lexical analyzers, or tokenizers.

Memoizing allows us to skip the second parse of list. -> “Memorizing” allows us to skip the second parse of list.

(last paragraph) “The fields in a C struct are visible …”

Second paragraph, “As we exit a scope, we pop it off the stack…” Parallel construction with “push it on the stack.”

In the scope stack diagram, depth = 1 is labeled on the left with “function f,” but the two stacks on the right show scope for function g.

Comment on function g refers to function f (copy-paste typo).

(paragraph 4) “see two different scopes stacks” scopes -> scope.

The final comment in the resolve function is misleading. By the time the recursion “tops out” (?) there is no scope above.

“… a sequence of instructions followed by a period (whitespace is
not meaningful as it is in Python):” ->
“… a sequence of instructions followed by a period on a line by
itself, otherwise whitespace is not meaningful (as it is in Python):”

I believe newline (‘\
’) is usually considered whitespace, and isn’t
newline being used in the following to prevent the parser from
seeing “c = a.y”? :

struct A {x, y}
a = new A
def f(z):
b = 3
c = a
.
y = 5

“High-level interpreters directly execute source code instructions or
the AST equivalent (low-level interpreters execute instructions called
bytecodes that are close to CPU machine instructions).” ->
“High-level interpreters directly execute source code instructions or
the AST equivalent. (Low-level interpreters execute instructions called
bytecodes that are close to CPU machine instructions.)” OR
“High-level interpreters directly execute source code instructions or
the AST equivalent. Low-level interpreters execute instructions called
bytecodes that are close to CPU machine instructions.”
(stand-alone parenthetical sentence - or remove from parentheses?)

“In the next section, we’ll figure out what kind of symbol table we
need for interpreter.” ->
“In the next section, we’ll figure out what kind of symbol table we
need for an interpreter.” OR
“In the next section, we’ll figure out what kind of symbol table we
need for interpreters.”

“The processor for Pattern 25, Tree-based interpreter,
on page 239 is also pretty heavyweight (Pattern 13, External tree
visitor, on page 120).”

Suggest some context for the reference to Pattern 13, perhaps:

"The processor for Pattern 25, Tree-based interpreter, on page 239 is
also pretty heavyweight. (It uses Pattern 13, External tree visitor,
on page 120.)

To walk the entire tree looking for patterns, then, we need … This sentence is cut off and not completed on the following page.

(near bottom) The text says “bytecode and up to three parameters,” but the definition shows operands [1 ..n]. How about the following?

op [operand1 [operand2 [operand3]]]

(near bottom) “access any element in the register array where is a stack” should read “…array whereas a stack”

For consistency, both “case BR” and “case ICONST” should have a final “break” (or neither should).

In figure 10.2, it looks like the entries in the constant pool are backwards. The disassembly has the string at location #0, but the diagram shows it at location #1. Similarly for the function info.

In figure 10.3, the indexes into the constant pool are goofy. Why does CALL #0 point to constant pool entry #1? I must be mussing something here.

(second last paragraph) “an associated obsession” ??

(last paragraph) “table” and “management” are run together.

(figure 10.7) register subscripts wrong for all three groups of ternary instructions: “rk = rj op rj” should be “rk = ri op rj”

(last paragraph) What is ‘@kw’? Also on p. 277.

(first sentence) “array then we can” -> “array that we can”

(figure 10.8) Individual instructions are run together. At fist glance, “fload ri , rj , nfstore ri , rj , n” looked liked there was an “fload” instruction and an “nfstore” instruction. Here and in other instruction figures, there needs to be more separation between instructions - each on separate row? Semicolons between? Couple of quads of space?

“To implement Pattern 10, Normalized Heterogeneous AST, on
page 100, for example, C would have to cut-and-paste the normalized
children definition into each node struct definition.” - not quite correct.

There is no need to copy and paste common part of the struct. Following technique can be used:

struct AST
{
AST* root;
AST* children;
AST* next_sibling;
Token* token;
};

struct ExprNode
{
AST node;
int expr_type;
};

ExprNode* can be safely converted to AST*, because AST struct is the first member of ExprNode:

AST* createAST(Token* token)
{
AST* ast = malloc(sizeof(AST));
memset(ast, 0, sizeof(AST));
ast->token = token;
return ast;
}

AST* createExprNode(Token* token)
{
ExprNode* ast = malloc(sizeof(ExprNode));
memset(ast, 0, sizeof(ExprNode));
ast->ast.token = token;
return (AST*)ast; // OK, since ExprNode begins with AST struct
}

void visitAST(AST* ast)
{
if(ast == 0) return;

if(asttoken>type == EXPR)
{
ExprNode* expr = (ExprNode*)ast; // OK, after AST there expected extra data for ExprNode
printf(“expression type %d\
”, expr->expr_type);
}

visitAST(ast->children); // continue with children list
visitAST(ast->next_sibling); // then continue with next sibling
}

This is widely used and I suppose the same approach was used by first C to C translator, before C compilers were created.

“Everything happens in the parser because a source-level interpreter
doesn’t create an AST nor translate the source code to bytecodes or
machine code.” ->
“Everything happens in the parser because a source-level interpreter
doesn’t create an AST OR translate the source code to bytecodes or
machine code.” OR
“Everything happens in the parser because a source-level interpreter
doesn’t create an AST, nor does it translate the source code to
bytecodes or machine code.”

(bottom of page)
“To call a method, we look up the method name and get the list of
formal parameters.”

Since there are no classes in Pie, it is probably best to stick
with the term ‘function’ instead of ‘method’.

“Since there is no scope tree, we don’t interconnect these objects all.” ->
“Since there is no scope tree, we don’t interconnect these objects at all.”
(?)

“… needs three things: the name, the formal list of parameters
($ids), and the body’s starting token index ($slist.start):” ->
“… needs three things: the name, the formal list of parameters
($args), and the body’s starting token index ($slist.start):”
($ids -> $args)

“Then we assign the actual parameters to the formal parameter
names in a newly created function memory (FunctionSpace), …” ->
“Then we assign the actual parameters to the formal parameter
names in a newly created function memory space (FunctionSpace), …”
(“a … memory” -> “a … memory space”)

“You can ignore the warning message from ANTLR; it’s the standard
if-then-else ambiguity.”

Suggest inserting a reference to the relevant section in [Par07] for
those who want to know more about this warning, what it means, and why
it can be ignored.

(bottom of page, top of pg 241)
“… this pattern doesn’t define function and struct symbols as it
executes code (this interpreter doesn’t have def() and aggr() action
methods).” ->
“… this pattern doesn’t define function and struct symbols as it
executes code. (This interpreter doesn’t have def() and aggr() action
methods.)”

“To call function we first grab the function name …” ->
“To call a function we first grab the function name …”

“They are not too hard to build and very flexible (we can add new
instructions without much trouble).” ->
“They are not too hard to build and very flexible. (We can add new
instructions without much trouble.)”

“Then, we can execute those bytecodes on an efficient interpreter
called a virtual machine (VM) (we’re conjuring up an imaginary
machine).” ->
“Then, we can execute those bytecodes on an efficient interpreter
called a virtual machine (VM). (We’re conjuring up an imaginary
machine.)”

“Bytecode interpreters have a number of useful characteristics
including portability (machine code is specific to a CPU whereas
bytecodes run on any computer with a compatible interpreter).” ->
“Bytecode interpreters have a number of useful characteristics
including portability. (Machine code is specific to a CPU whereas
bytecodes run on any computer with a compatible interpreter.)”

“… in a register-based interpreter (we can assume there are
an infinite number registers).” ->
“… in a register-based interpreter. (We can assume there are
an infinite number of registers.)”
(“number” -> “number of” ; stand-alone parenthetical sentence)

“… works like the old HP calculators that used Reverse Polish
Notation (RPN) notation.” ->
“… works like the old HP calculators that used Reverse Polish
Notation (RPN).”
(not necessary to repeat “notation”)

“… languages like Python so we’d use generic …” ->
“… languages like Python, so we’d use generic …”

Suggest quick note somewhere that semicolon ‘;’ thru EOL is a comment
(for those Java programmers uninitiated to the wonders of
Assembly language ;-)

“If the instruction has an operand (which is always right next to the
bytecode), the code fragment pulls the operand from code memory and
bumps the instruction pointer by four bytes.”

Exception: the BR instruction, immediately preceding this paragraph,
doesn’t do this. Perhaps more generally: 4*number-of-operands bytes.

(Figure 10.3)
Also, FunctionSymbol for “main” has addr=6; should be 0.

“The stacks grow to the right and indicates the state before each
instruction executes.” ->
“The stacks grow to the right and indicate the state before each
instruction executes.”
(indicates -> indicate)

“Unfortunately, there’s a lot more going on in a general-purpose
programming-language interpreter than what we’ve discussed here (these
bytecode interpreters rely heavily on the Java VM).” ->
“Unfortunately, there’s a lot more going on in a general-purpose
programming language interpreter than what we’ve discussed here.
(These bytecode interpreters rely heavily on the Java VM.)”
(“programming language” not hyphenated ; stand-alone parenthetical
sentence)

“The next section gives you a taste of what’s involved in building in
more general and more efficient interpreter.” ->
“The next section gives you a taste of what’s involved in building a
more general and more efficient interpreter.”
(second “in” -> “a”)

it’s easy to confuse resolving a variable with loading it value -> it’s easy to confuse resolving a variable with loading its value

“Knowledge of computer architecture such as cache memory and CPU
pipelines is essential.” ->
“Knowledge of computer architecture, such as cache memory and CPU
pipelines, is essential.”
(commas)

“… squeeze every last drop of efficiency out the phone’s ARM8 CPU
and flash memory (flash memory is much slower than dynamic memory
memory but doesn’t get amnesia when you lose power).”
“… squeeze every last drop of efficiency out of the phone’s ARM8 CPU
and flash memory. (Flash memory is much slower than dynamic memory
memory but doesn’t get amnesia when you lose power.)”
(“out” -> “out of” ; stand-alone parenthetical sentence)

“The Dalvik VM executes many fewer instructions to achieve the same
result (it can reuse registers to avoid unnecessary operand stack
pushes and pops).” ->
“The Dalvik VM executes many fewer instructions to achieve the same
result. (It can reuse registers to avoid unnecessary operand stack
pushes and pops.)”

“… but you can learn more by looking at the Dalvik VM source code (you
can also learn some of the underlying principles by doing a web search
for ‘threaded interpreter’).” ->
“… but you can learn more by looking at the Dalvik VM source code.
(You can also learn some of the underlying principles by doing a web
search for ‘threaded interpreter’.)”

(Pattern 27 - “When to apply”)
“It suitable for efficiently executing DSLs and general purpose
programming languages.” ->
“It’s suitable for efficiently executing DSLs and general purpose
programming languages.”
(It -> It’s)

“We can isolate the method implementations in a subclass (this is a
generically useful pattern whereby we separate functionality from
language syntax.)” ->
“We can isolate the method implementations in a subclass. (This is a
generically useful pattern whereby we separate functionality from
language syntax.)”

“We can store characters as integers, but we have to store strings,
floats, and function symbols in the constant pool (we’ll look at
function symbols in the next section).” ->
“We can store characters as integers, but we have to store strings,
floats, and function symbols in the constant pool. (We’ll look at
function symbols in the next section.)”

“… object array field, parameters first call knows how much
space to create …” ->
“… object array field, parameters first. call knows how much
space to create …”
(two sentences? missing period)

(Figure 10.5: Stack-based general bytecode instructions)
“fload f, fstore a” (load field f, store field a ??)

Also, I found the use of ‘f’ for both “function” and “field” in this
table somewhat confusing.

(bottom of page; code comment)
“; print f(1,2)” -> “; print f(10,20)”

(second to last paragraph)
“… to assemble programs before execution (BytecodeDefinition defines
the instruction set).” ->
“… to assemble programs before execution. (BytecodeDefinition defines
the instruction set.)”

(code for INSTR_FADD)
All previous discussion of floating point types indicates that they
are stored in the constant pool, yet this code appears to be getting
them directly off the stack.

In particular here’s the discussion on pg 255:

“The float type in Java is 32 bits and could fit in code memory no
problem. Unfortunately, we can’t fool Java (our implementation
language) into treating a 4-byte integer as a floating-point number
without truncating it. That means we have to store floating-point
values in the constant pool as well as strings.”

(Figure 10.7: Register-based arithmetic bytecode instructions)
“fadd ri, rj, rk fsub ri, rj, rk fmul ri, rj, rk
Arithmetic operators for integers. rk=rj op rj” ->
“fadd ri, rj, rk fsub ri, rj, rk fmul ri, rj, rk
Arithmetic operators for floats. rk=rj op rj”
(integers -> floats)

“That image shows the registers for a function definition with p
parameters and t locals and temporaries (n=1+p+t):”

In that case the final r subscript in image above this should be
p+t, and not t

(code in function f())
“iconst r4,30 ; get int 2 into a reg” ->
“iconst r4,30 ; get int 30 into a reg”

(Figure 10.8: Register-based general bytecode instructions)
Descriptions for [fg]load and [fg]store:
Figure 10.5 uses “and reverse” for the “store” instructions, whereas
here the description of the instruction is given explicitly. I think
the latter is preferable.

(Figure 10.9: Register transfers and state of call stack at ret in g())
FunctionSymbol for “f”:
“locals=4” -> “locals=3”

(comment in code for call())
“// copy him arguments from calling stack frame to new stack frame”
Copy “him arguments” ?

The production for expr is

expr returns [AST tr] // expr returns a subtree
: a=multExpr {$tr = $a.tr;}
( ‘+’ b=multExpr {$tr = new AddNode($a.tr, $b.tr);} )*
;

I believe the first argument to AddNode should be $tr. Otherwise, “1 + 2 + 3” would result in the same tree as “1 + 3”.

In ‘When to apply’ column for Pattern 4: “You don’t need to factor the parser as much to suit weaker parsing decisions”
Shouldn’t that read: “You don’t need to factor the grammar as much…” ?

Application-specific parser actions can have side effects we can’t undo like “launch missiles.”
Memorably dramatic example - yes, but given the state of this sad old world perhaps we could have something a little less militaristic… “deliver pizza”, “divert flight to Amsterdam” :-)

“It turns out we need to know what kind of statement we fed in, just not what the grammar calls it. We can drop it”
I stumbled on this. How about something like…
“We can drop this too because we only need to know what kind of statement we fed in, not what the grammar calls it.”

For the lazy reader, like me, it would be nice to have the pattern numbers prefix the relevant sections in the table of contents (e.g. as viewed in the sidebar of Preview)

Word “languages” is repeated:

For non-
object-oriented languages languages like C, we’d apply Patter n 22,

“to switch interpreter actions on off in the parser” should be “to switch interpreter actions on and off in the parser”.

For safety, AST.getNodeType should check for token == null

Pedantry I know, but…

AST.toString should check for null token

In AST.toStringTree the condition children != null in the for loop is redundant because this was checked at the start of the method.

The comment for variable j refers to i (as the comment to function g refers to f:

Í { int j; } // define variable i in another nested local scope
}
Î void g() { // define function f in global scope

(Second to last line) “only one deep” begs for a unit; “one level deep?”

(End of first paragraph in 11.2) “ANTL-RMorph” breaks badly across lines.

(end of page) Bad page break — example table continues two pages later, following figures 11.9 and 11.10.

(first text paragraph) Although “unmolested” is great slang, it might be offensive to some readers; “unchanged” is clear and not emotionally charged.

(first paragraph after “Defining the translation rules”) “A grammar” doesn’t need the capital “A” after a colon

The word “tail” is kind of ambiguous. It’s used to refer to the trailer of the overall document and its a fragment rule that snags the tail of the line. I’d suggest renaming the tail() function to trailer(). This would also be consistent with the description in the third-last paragraph and makes nice parallelism with header().

(TABLE fragment) As defined, does this allow a row to start on the same line as one that starts with two dashes? It seems like the pattern should be (‘\
—\
’ ROW)*

This is the only place in the book that uses LA() explicitly. Based on the recent antlr-interest posting regarding parsing Perl regular expressions, where the author went kind of nuts with this, is a brief caveat in order here?

(first paragraph) I suspect that “using Jasmin, a Pattern 26, Bytecode Assembler” resulted from the use of Prag Prog’s “excellent proprietary DSL” as extended. But when it’s expanded in print, it reads awkwardly. Maybe “using Jasmin, which employs Pattern 26, …” or something like that.

The pattern’s purpose statement has an extra full stop.

(second last paragraph) Feels like “DSL” should be “DSLs”

Table too large to fit on page. Bad page break on previous page — too much white space.

(second paragraph, first sentence)
“In a computer translator, to ”understand" a phrase means syntactically
and semantically analyze it." ->
“In a computer translator, to ”understand" a phrase means to syntactically
and semantically analyze it."

There is an example saying:

So, let’s add some other rules that invoke optional_init:
decl: ‘int’ ID optional_init ‘;’ ;
func_call: ID ‘(’ arg ‘)’ ;
arg : ‘int’ ID optional_init ;

The “func_call” rule doesn’t seem to apply to anything in this page. Not sure if it should be there.

(second last paragraph) “(ST)” doesn’t need the parens.

In the example for conditional includes, it doesn’t seem like “” should be there.

(beginning of second paragraph) Insert “is” in “A template hierarchy the dual …”

(end of page) “from->to” makes a bad break across pages

(second last paragraph) “and inserts into the overall file template” ??

(paragraph following operator(o), third last line) “the parser create a tree” (plural “creates”)

Sample code on the previous page does not match Figure 9.2. Figure 9.2 member ‘a’ should point to 1st StructInstance box and member ‘b’ the 2nd StructInstance

(first text paragraph) “… hide any x arguments of the same because …”

(last paragraph of 12.5) “… build an object-relational database mapping.” The word “mapping” sounds strange here; although it will do mapping, what’s built is more like a “mapper” or a “mapping engine” or something.

Why isn’t the ID of the Person table set to use AUTO_INCREMENT as is the Person_roles table?

(last sentence) “then includes subordinate tables…” rather than “then includes subordinate tables…”

(first sentence of “Java objects” box) “… figure out where how to get information …”
Not sure what you intended, but it’s not this.

Second line from bottom of page: “It’ll teaches the basics…”

Missing the “TO” in:

In order print out a tree using a text-based encoding…

In this part: The following toString( ) generates strings of the form: (root child1 child2 …)

It seems this should be the toStringTree(), and not the toString().

Can’t read the text at the end of this page. The text goes below the footer.

I feel that the title of the book is misleading. The content of the book is about Language Implementation Patterns. It explains the various tricks you can use to implement DSLs.

From the title I expected the content to be about designing languages. What kinds of syntaxes and structures are most effective for various purposes? When is it a good idea to include typing? What are various alternatives and tradeoffs for specifying types? What might a configuration language look like, or a custom scripting language, or some document description language? I would think that Terence would have a wealth of experience designing a variety of types of languages, and that was the experience I was hoping to consume in this book.

So while the title of the book is compelling, I feel that it’s misleading as-is. I don’t know what’s involved with changing the title of a book that’s already at this stage, but I worry that this title will hit the wrong audience and miss the intended audience.

(second paragraph of main text) template name is wrong, should be objectTables instead of objectTable

As on page 342, if AUTO_INCREMENT is desired, this ST needs to change.

(near middle of page) “The main program decides which target generate …”

(second text paragraph) “… little bit of Java code common …” doesn’t read well. Do you mean “… of common Java code …” ??

Enforcing Static Typing Rules
Chapter 8 page
Line 10 is: “no array index out of bounds.”
should probably be: “array index out of bounds.”

“Syntax {«expr»}? is a semantic predicate that … ” ->
“The syntax {«expr»}? is a semantic predicate that … ”

(top of page)
“Their job is simply emit text in the right format not perform
a translation.” ->
“Their job is to simply emit text in the right format, not perform
a translation.”

(code samples)
Shouldn’t appending comma & newline/space be moved up in these
loops?, i.e.:

for (Column c : columns) {
if (i>0) buf.append(“,\
”);
buf.append(" " ); // indent a bit
buf.append©;
i;
}

for (String a : attrs) {
if (i>0) attrBuf.append(“, ”);
attrBuf.append(a);
i;
}

(top of page)
“Now that we have some idea what a template looks like and how to
execute it, …” ->
“Now that we have some idea of what a template looks like and how to
execute it, …”

expr definition should read
expr : ‘x’ ‘+’ ‘1’ // used by returnstat
rather than
expr : ‘x’ ‘+’ ‘0’

(top of page)
“Dynamic scoping means that method can reference file’s attribute, such
as the filename attribute.” ->
“Dynamic scoping means that method can reference file’s attributes, such
as the filename attribute.”
(‘file’ probably has more than one attribute)

“… and inject attributes in any order we want without fear
of premature evaluation.”

I think there’s a pill for that problem ;^)

“Piecing together a string buffer isn’t the most flexible solution
either, as we saw the last chapter.” ->
“Piecing together a string buffer isn’t the most flexible solution
either, as we saw in the last chapter.”

“DOT needs different tree nodes to have unique names. If we have two
x nodes in the tree, for example, DOT would think they are the same
node. To prevent that, we have to define a different node (but with the
same text label).”

It would be nice if the example contained an illustration of the
above point, e.g.,

(VAR int x (+ 3 3)): “If we have two ‘three’ nodes in the tree …”
(VAR int x (+ 3 (+ 4 5)): “If we have two ‘plus’ nodes in the tree …”

“void f(int *a, int *x) { // ‘int a[]’ becomes ‘int *a’”

but “int x” should not become “int *x”

“In this case, it sets attribute decls to the list
we collect while matching the trees for this rule.”

but the grammar fragment above calls the attribute
“defs”, not “decls”:

“-> file(defs={$d})”

(top of page)
“@after {$st = %operator(o={$start.getText()});}”

why using “%operator” here and not just “operator” ?

I think there’s a missing “the” before “expression” in this:

In C, expression T (6) is either a function call…

I’d go for “such as” and not “like” in:

…side effects we can’t undo like “launch missiles.”

What you call a “fetch-execute-cycle” I think is more often called a “fetch-decode-execute cycle.”

In addition to, or maybe even instead of, the venerable dragon book, these days I’d recommend the pair: Appel’s Modern Compiler Implementation in Java (also available in other language flavors) and Muchnick’s book Advanced Compiler Design and Implementation.

In regard to the following:

“Descent” refers to its top-down nature and “recursive”
refers to the fact that its functions are potentially recursive.

That definition is itself a bit recursive. If you don’t want to assume the reader knows what that means (unclear to me) then maybe say something like, “…refers to the fact that its functions potentially call themselves” or something along those lines.

In the code example, the first comment uses the number 1, but everything else in the code refers to 0. But the text above also refers to 1. Probably a good idea to make it all consistent.

I dunno if I’d use the word “grok.” I think I’d go for the less Heinleiny turn of phrase “fully understand.”

With regard to this:

we have to squirrel away (memoize) the result of
invoking the various parsing methods.

Since “memoize” is effectively being defined here, it’s weird that the term itself lands in parentheses. I’d say something like, “…we have to memoize — that is, squirrel away — the result…”

“For example, the GCC C compiler, adds a number of extensions beyond C.”

I think that second comma is superfluous.

“That means we have to verify it syntax and
verify that it makes sense” should be “That means we have to verify its syntax and verify that it makes sense”

“The basic idea in an object-to-relational database is to map classes
to tables and fields to columns in the corresponding table.” ->
“The basic idea in an object-to-relational-database mapping is to
translate classes to tables and fields to columns in the corresponding
table.”
(The database itself is not “object-to-relational”.)

“… has two roles (mgr and coder) and no vacation (there’s no entry
in table Person_vacation for column Person_ID1)." -> "... has two roles (mgr and coder) and no vacation. (There's no entry in table Person_vacation for column Person_ID1.)”

“… we can stuff it into the overall objectTable template via method
genSchema():” ->
“… we can stuff it into the overall objectTables template via method
genSchema():”
(The template name in the code below is plural"

(second to last paragraph)
“… let’s extract data from the input model and inject into the
overall root template.” ->
“… let’s extract data from the input model and inject it into the
overall root template.”

(top of page)
output(class, fields, arrayFields, nonPrimitiveTypes)
::= “<objectTables()>”

Why no ellipses (as w/ ‘call’ at top of pg. 340)? ; e.g.:
output(class, fields, arrayFields, nonPrimitiveTypes)
::= “<objectTables(…)>”

(Same w/ “<serializerClass()>”)
Now I’m confused as to when to use ellipses and when not to.

“which target to generate according to the commandline argument
(-java or sql):">
”which target to generate according to the command line argument
(-java or -sql):"
(“commandline” -> “command line” or “command-line”)

“Method filterFields() is the same as before except that computes
the extra nonPrimitiveTypes set of fields.” ->
“Method filterFields() is the same as before except that it computes
the extra nonPrimitiveTypes set of fields.”

“Now it’s time to figure out how to create the template hierarchy for
the serializer and deserialize methods.”

Noun vs. verb inconsistency between “serializeR” and
“deserialize”.

“The <! … !> comment is an ST comment inside the template that
doesn’t appear the output.” ->
“The <! … !> comment is an ST comment inside the template that
doesn’t appear in the output.”

(top of page)
“… elements of a PreparedStatement (it’s like a built-in SQL
template language that uses question marks … ” ->
“… elements of a PreparedStatement (which is like a built-in SQL
template language that uses question marks … ”

“We nearing the end of our adventure.” ->
“We’re nearing the end of our adventure.”

(bottom of page)
“For example, there are DSLs for describing DNA/RNA sequences,
chemical molecules, molecular formulas (e.g., H20), and so on.” ->
“For example, there are DSLs for describing DNA/RNA sequences,
chemical molecules, molecular formulas (e.g., H2O), and so on.”
(H20 -> H₂O ; H-twenty -> H-two(subscript)-Oh)

(chapter title)
“Putting it all together” -> “Putting It All Together”
(capitalization to match other chapters)

(first paragraph)
“In ”The language of genes" [?], Searls says …"

Searls, David B., “The language of genes”,
The journal “nature” (www.nature.com/nature), Vol 420,
November 14, 2002
Nature Publishing Group, November 2002

“… BOX described by Merijn de Jonge in ‘Pretty-Printing
for Software Reengineering’ [?].”

de Jonge, Merijn,
“Pretty-Printing for Software Reengineering” in
International Conference on Software Maintenance (ICSM 2002),
pages 550-559, IEEE Computer Society Press, October 2002

“For example, I personal use a few versions … ” ->
“For example, I personally use a few versions … ”

You might want to do a search-and-replace of “StringBuffer” with “StringBuilder”.

Maybe this is splitting hairs too much, but to the extent that a method is a function, Java does allow functions within functions, when that inner function is defined inside a class.

In this:

“Method resolve( ) does nothing more than look up «symbol-name» in a hash table dictionary.”

A hash table would be an implementation detail. I’d just say “a dictionary.”

A minor error, but on page 12 you mention that there are 31 patterns and on page 19 you say there are 28.

I just bought this eBook and am excited to read it, it’s exactly what I needed after reading your Definitive ANTLR Reference.

Second paragraph, third sentance partially repeats. “Fortunately, it’s not as hard as hard as it seems at first glance”

Also, third paragraph, fourth sentance reads: “the most complicated applications need patterns from I, II, and II or I, II, and IV.” That third “II” should probably be “III”

Microsoft Research link in the footnote runs off the right side of the page.

The very last sentance on this page appears to be half missing.

The program symbol :: novel character analogy in this section isn’t really fleshed out, which makes the sentence “In a novel, we might have an elf character (the category) called Sri (the name) that becomes a soldier (the type)” seem quite out of place. Doubly so due to the sudden mention of elves.

In the parenthetical note “(assuming to language we’re interpreting has functions)” the word “to” is probably meant to be “the”.

EPUB-Version on iPhone (Stanza) : Chapter 4 page 1 and chapter 5, section “Identifying phase structure”, page 1 - some text is being printed on top of the regular text (“Missing …”) and makes it unreadable

$ java cp ../../antlr-3.2.jar org.antlr.Tool Graphics.g should be
$ java -cp ../../antlr-3.2.jar org.antlr.Tool Graphics.g

Grammatical woe in “To obtained a different traversals”

“When to apply” section for Tree Pattern matcher seems to abruptly end with “To walk the entire tree look- ing for patterns, then, we need”

This appears to actually be a PDF rendering problem, rather than something missing in the source text.

The = operator tells us (everthing -> everYthing) we need to know

The paragraph starting with “The most basic reader component combines” seems to be split unintendently.

as if the key where a property

should be (where -> were):

as if the key were a property

Line too long. Line 7 (… Some programming languages present) protrudes into the right margin.

Comma missing in: “…really a maximum not the…”; I suggest:
“…really a maximum, not the…”

first line: “it’s easier use” should be “it’s easier to use”

Wrong brake in ExprN-ode

Last entry in table ends with an incomplete sentence.

“It’s no fair” (third of the way down the page) should, I think, be “It’s not fair”. unless you’re from Scotland.

public Type uminus(CymbolAST a) {
if ( !(a.evalType_int && a.evalType_float) ) {
listener.error(text(a)" must have int/float type in "
text((CymbolAST)a.getParent()));
return _void;
}
return a.evalType;
}

The && on the second line above should presumably be a ||, or the == expressions should be changed to != and the outer ! removed.

In interp/stack/Interpreter.java on p282 of the text, there is the following excerpt:

public void exec() throws Exception {
// SIMULATE “call main()”; set up stack as if we’d called main()
if ( mainFunction==null ) {
mainFunction = new FunctionSymbol(“main” , 0, 0, 0);
}
StackFrame f = new StackFrame(mainFunction, ip);
calls[++fp] = f;
ip = mainFunction.address;
cpu();
}

This is just sample code, but I thought it worth mentioning because I found it confusing, and that interfered with my absorption of the text.

The line “StackFrame f = new StackFrame(mainFunction, ip);” concerns me. I believe the top-level stack frame return address is being implicitly set to 0 (the default initialization value for “int ip”), or even a floating value if exec() is repeatedly called. I believe that implicit in this is the assumption that the assembler for the “main” routine (explicitly declared or starting at instruction 0) must end with “halt”, not “ret”, as ret would dereference the floating ip value in the stack frame to try to return to main’s non-existent caller. If main was implicit (starting at 0) and contains a “ret” this could be an infinite loop.

I think it might be worth explicitly replacing the exec() line above with something like

// return address guaranteed to be outside code_size to terminate cpu() loop on “ret”
StackFrame f = new StackFrame(mainFunction, Integer.MAX_VALUE);

Making it MAXINT ensures a “ret” from main will still terminate as then (ip > codeSize), so the cpu() loop will terminate. Even if “ret” from main is illegal in your assembler grammar, this code won’t cause questions to be raised like “what is the value of ip at this point?”.

The link fourth link in the footnotes contains whitespace in the URL, after “.puc-”, where there should be no whitespace.
(http omitted, because of the website restrictions of including links in errata)
4.\twww.tecgraf.puc- rio.br/~lhf/ftp/doc/jucs05.pdf

This sentence has an extra comma:
So, do static bug analyzers such as FindBugs1 and Coverity.2

There is an extra comma that makes the sentence below incorrect.

“He described the amazing gymnastics he and the VM team went through to squeeze every last drop of efficiency out of the phone’s ARM, CPU, and flash memory.”

I think it should read:

“He described the amazing gymnastics he and the VM team went through to squeeze every last drop of efficiency out of the phone’s ARM CPU, and flash memory.”

The phone probably has a cpu with an arm instruction set, which dalvik is targetting. It does not have an ARM and a seperate CPU.

This of course is my guess. A number of phones have more than one processor.

anintermediate (missing a space)

buildabstract syntax
trees (ASTs) — missing a space

First paragraph after “Who Should Read This Book”. “…programming languages.” gets split up into three different lines: first “pro-”, then a newline saying only “gramming” and a new newline saying “languages.”

Second paragraph after “Languages and Tools Used In This Book”. “…we’ll use aparser generator…” should be “…we’ll use a parser generator…”

For example, C function definitions and declarations are
identical until the parser sees ; or {:

{. instead of {:

or instead of and, right?

public Type uminus(CymbolAST a) {
if ( !(a.evalType_int || a.evalType_float) ) {

public Type uminus(CymbolAST a) {
if ( !(a.evalType_int && a.evalType_float) ) {

line 5 of first paragraph of Bytecode Interpreter,
“We call the instructionsbytecodes because…” missing a space

line 3 of the last paragraph
“… into vocabulary symbols calledtokens.” — missing a space

first line
“words, he built a C to C translator calledcfront.” — missing a space

The penultimate line
"In our case, the tokenizer (orlexer) yields…) — missing a space

Just 2 lines of class HeteroAST? It would be better if the full source provided.

I was a bit wary of your implementation of getResultType (..), because it promotes arguments to a type based on the result type of the expression. This happens to work for arithmetic expressions, but not for the other two categories, because then always the result type will be boolean.
For example, if I write

if (2 == ‘2’) …
then I expect a promotion for the ‘2’ as would be the case
for the arithmetic expression 2 + ‘2’. Your implementation (which I am glad say, compiled without any problems) supported my intuition that I would be missing some promotions: it does not add (int) to ‘2’.

Or am I missing something here?

best regards,
Jur

epub bug: Linked sections in the PDF read as “Section 1.2”, but in the epub they read as “the section 1.2”, which is really jarring when placed after a period. (So the last paragraph on PDF page 14 from the epub format reads:

The chapters within the different parts proceed in the order you’d follow to implement a language. the section A Tour of the Patterns describes how all the patterns fit together.

)

This problem is all throughout this book, and made reading the epub feel like it was littered full of errors. (I mean, I knew darn well it was just a matter of fixing some processing steps, but it still feels like errors, you know?)

Token.java uses ListLexer.tokenNames[type] directly rather than using the ListLexer::getTokenName(int) method provided in ListLexer.java. (Perhaps the tokenNames[] should have been declared private to ask the compiler to enforce the abstraction.)

“(which we can ignore it for our purposes here)” should read “(which we can ignore for our purposes here)” (remove the ‘it’) or “(we can ignore it for our purposes here)” (remove the ‘which’).

‘parse tree clearly identifies the parts of speech in sentence the cat runs quickly should read ’parse tree clearly identifies the parts of speech in the sentence the cat runs quickly. (Add ’the’ in front of ‘sentence’.)

The table showing Patterns vs When to Apply on page 130 of the PDF is useless in the epub version on the Sony PRS-505 reader. (It looks good enough in fbreader though.) Putting it in prose would have been useful.

The figures differ between the PDF and epub versions. The PDF is correct, the epub version is incorrect. (The first tree is incorrectly copied from the second tree.)

“(a tree of Scope’s)” should be “(a tree of Scopes)”

I laughed when I saw this:

To make things more concrete, here’s an implementation preview:
// create new scope whose enclosing scope is the current scope
currentScope = new LocalScope(currentScope); // push new scope

The ‘concrete’ bit is one comment line, one text line, and no supporting code structures such as class LocalScope::LocalScope(LocalScope parent) or something like that. It just seemed funny that a paragraph of prose resorts to 50% commented Java for ’concrete’ness but lacks the datastructures to make it happen. :) (I hope you can see the humour in it.)

In the epub version, C is incorrectly broken on my Sony PRS-505. (This also happens later in the text, too.)

The table of “upon”, “action(s)” on this page of the PDF renders poorly in the epub version. On the Sony PRS-505, each table cell is individually centered in its column, which makes for very difficult reading. No idea what to do to make it palatable. (In fbreader, it is more legible once you know it is a flattened table. It’d be super-confusing if I didn’t already know what it was.)

“Classes are like structs that they can inherit members” should read “Classes are like structs except they can inherit members”

SymbolTable.java includes the string “must be have struct type”, should read “must have struct type”.

“If resolves to a local variable” should read “If x resolves to a local variable”.

public Object exec(PieAST t) is inconsistent: half the statements use “method(t); break;” and the other half uses “return method(t);”. Since there is no code after the switch table, it’s hard to see why half would continue after the switch and half wouldn’t continue. If there’s a reason, please explain, and if there isn’t a reason, please standardize. :)

“works like the old HP calculators that used Reverse Polish Notation (RPN).” I object! :) You can buy brand-new HP calculators with SD card slots and USB mass storage support that run RPN. They’re still awesome. :)

“terminating <\\ul> tag.” should be "terminating

tag."

“In the previous chapter, we’ll see how language patterns apply to some interesting problems.” Probably meant to say “In this chapter”.

On p. 162 it says “The sequence of actions for t.cymbol starts like this: push global scope, def int, def float, ref int, def i, ref float, def f, push f, ref int, def x, ref float, def y, push local scope of f, ref float, def i, and so on.”.

On the other hand, on p. 163, it says upon start of file, actions “push a GlobalScope. def BuiltInType objects for int, float, void.” should be performed.

Isn’t a “def void” missing on p. 162 then? I would guess that otherwise the “ref void” for the return type of the second method would not yield anything?

Some code is clearly missing. Second line from the bottom of the page initiates parsing with the rule “downup” but there is no such rule in the grammar. Also, the two patters folloing this, patters 18 and 19 have exactly the same problem. Without this rule, the code doesn’t execute and the patterns don’t work.

In section 2.3, first paragraph, Line 5: A space is missing between the words “called” and “grammars”.

In the first sentence on the page, toward the end, you need a space between the words “the” and “semantics”.

In the first sentence on the page, toward the end, you need a space between the words “the” and “semantics”.

existing
Unfortunately, this constructor cannot enforce proper structure. We
could pass in an empty list, but the grammar requires a least one vector

Should read
…but the grammar requires at least one vector

Bottom of page example says “is the core of fieldload()” but it’s fieldstore().

Figure 4.2 last sentence: r^ … Rule r should must a single node, not a subtree

I think the code to promote nodes in getResultType() is wrong. It promotes ta to ta and tb to tb. I guess it should promote ta to tb and tb to ta. Otherwise the results will always be null. So, the could should be

a.promoteToType = promoteFromTo[ta, tb];
b.promoteToType = promoteFromTo[tb, ta];

End of line 7 should be “to sym”.

“computing expression value result types” that should be removed right before the example because the code does not have that method.

public class AddNode extends AST should extend ExprNode

“This implementation has a number of advantages over an embedded walker.” should reference traditional visitor not embedded Walker or perhaps double dispatch visitor.

PrintVisitor.java should be IndependentPrintVisitor.java on this page and the previous. Line counts might be off by one also.

“pop arg scope” in the comment of the code for DefRef.g should probably be " pop method scope".

BuiltInType should be BuiltInTypeSymbol

“The next two chapters catalog the most common patterns for interpreting and translating programs.” should be about the next two parts about not chapters.

“Killing off a character means there should be no further references to it.”
May not be an accurate analogy. In a story, a character killed /can/ be referenced later as having been killed (but not as still alive). If by ‘killing’ you mean the character being ‘simply dropped’ from the plot — as in a sequel — then, that’s another story (pun intended).

Possible fix: “Killing off a character means there should be no further references to it [as being alive].”

What a wonderful book, though! If I had the means, I would nominate Prof Terence Parr for writing what I think is easily the book of the century on an otherwise very intimidating topic (thanks to the Dragon book).

at the end of the second paragraph “skip tree writing” should be “skip tree rewriting”

“the return address and space for parameters”

“only 7 out of the 26 bytes” should be 27 bytes.

Method gen(): “return Method(m.id, gen(m.bodyBlock));” should have “new” in there.

In the following two sentences describing Pattern 4, the word “parser” (but not parsing“) should be replaced with ”grammar":

A more powerful parsing strategy means
it’s easier to build the parser. You don’t need to
factor the parser as much to suit weaker parsing
decisions.

The type promotion in getResultType is wrong since it currently looks up using using promoteFromTo[ta][ta] and promoteFromTo[tb][tb] which for both cases is always null. The correct promotion promotes to the result type.

Like this:
int tresult = result.getTypeIndex();
a.promoteToType = promoteFromTo[ta][tresult];
b.promoteToType = promoteFromTo[tb][tresult];

“If resolves to a local variable or parameter,” should probably start with “If x resolves…”

missing space in ‘it discovers’ in the first sentence of the second paragraph.

The text describes the PropertyFileBaseListener generated by ANTLR but the example code shows the PropertyFileBaseVisitor instead.

There is no a call of `consume` in the constructor of `ListParser`. Thus `lookahead` remains uninitialized and the program crashes with `NullPointerException`.

You can forget about the previous erratum. It is not mentioned that the constructor of `Parser` class should call `consume`. It is worth adding an implementation of `Parser` constructor in the book.

semantics/promote/SymbolTable.java: the printed book and pdf do show wrong code.

public Type getResultType(Type[][] typeTable, CymbolAST a, CymbolAST b) {
int ta = a.evalType.getTypeIndex(); // type index of left operand
int tb = b.evalType.getTypeIndex(); // type index of right operand
Type result = typeTable[ta][tb]; // operation result type
// promote operand types to result type
a.promoteToType = promoteFromTo[ta][result]; <—— ooops
b.promoteToType = promoteFromTo[tb][result]; <—— ooops
return result;
}

should be result.getTypeIndex().

On pattern 25, Tree-Based Interpreter I noticed something odd in the Pie.g grammar definition. Note this error is in the source code that accompanies the book. I am putting page 233 as a reference to the Implementation section in the pattern.

Not sure why getEnclosingScope is called twice. All the examples work as expected so worry there.

From interp/tree/Pie.g

functionDefinition
\t:\t’def’ ID
\t\t{
FunctionSymbol fs = new FunctionSymbol($ID.text,currentScope);
currentScope.define(fs); // def method in globals
currentScope = fs; // set current scope to method scope
}
‘(’ (vardef (‘,’ vardef)* )? ‘)’
\t\t{currentScope = new LocalScope(fs);}
\t\tslist
\t\t{
\t\tfs.blockAST = $slist.tree;
\t\tcurrentScope = currentScope.getEnclosingScope();
\t\tcurrentScope = currentScope.getEnclosingScope();
\t\t}
\t\t-> // pass nothing to interpreter
\t;

slist
\t:\t’:’ NL statement+ ‘.’ NL\t-> ^(BLOCK statement+)
\t|\tstatement\t\t\t\t\t-> ^(BLOCK statement)
\t;

In the consume method for the BacktrackingParser:

public void consume() {
p;
// have we hit end of buffer when not backtracking?
if ( p==lookahead.size() && !isSpeculating() ) {
// if so, it’s an opportunity to start filling at index 0 again
p = 0;
lookahead.clear(); // size goes to 0, but retains memory
}

sync(1); // get another to replace consumed token
}

This if statement that should clear out the lookahead and reset the token pointer to 0 will never be called since every time we consume a token we sync one past the pointer. If we want this to happen we need to add a check around the call to sync that looks like this:

if(p >= lookahead.size()) {
sync(1); // get another to replace consumed token
}

EPUB version: Please include higher resolution bitmaps in the epub version. Some readers are better at scaling images than others (some don’t appear to scale images at all, rendering some diagrams unreadable), but in any case deliver blurred results at best.